Data Source Connector – Amazon S3

Data Source Connector – Amazon S3

Our platform enables you to quickly and easily connect your data sources to data from a File Key/Location in Amazon S3 Storage.

This enables a one-way synchronization, where any changes in the Amazon S3 Storage File Key/Location are regularly downloaded into your data source.

Your data source will be automatically updated in this way until you remove the connector or an error occurs (such as losing authorization to access the file).

Connection

You need to have configured a Connection to your Amazon S3 Storage account before adding a Connector.

Adding

Data Source Settings

  • Navigate the side menu to Data Hub > Data Sources
  • Hover over a data source and click the “Settings” icon link ()
  • In Settings, hit the “Add Connector” button
  • Select the relevant option to add the connector

This will refresh the page with the newly added connector ready for configuration.

Nothing has been saved at this stage, so save your connector after making any changes to enable or update it.

Alternatively, if you’re viewing the rows of a data source, you can navigate to the Settings page using the option under the page’s title.

Configuring

Next, configure the following:

  • Bucket Name
    Enter the name of the bucket in which the file is stored.
  • File Key/Location
    Specify the location that uniquely identifies the file within the S3 bucket.
    For example:
    Development/Projects1.xls
    Finance/statement1.csv
  • Refresh Frequency
    The frequency at which this connector pulls data.

File Format Requirements

Please ensure your data is in .csv or .xlsx format and does not include any formulas. The spreadsheet should be stored in your connected Amazon S3 Storage account.

When you save your changes, the connector will be fired for the first time, and after that, it will run at the refresh frequency you specified in the connector settings.

Wait a minute or so, or click the three-dot menu (  ) and access the RUN NOW to manually trigger the connector and check the Rows in your data source.

If the rows were not downloaded, there may be an error with the file’s data or permissions to access it. To identify and resolve the issue, you can review the logs by clicking the three-dot menu (  ) and selecting VIEW LOGS.

Once resolved, you can run the connector again and confirm that the rows have been successfully downloaded.

    • Related Articles

    • Amazon S3 - Data Source Connector

      Our platform enables you to quickly and easily connect your data sources to data from a File Key/Location in Amazon S3 Storage. This enables a one-way synchronization to occur - where any changes in the Amazon S3 Storage File Key/Location are ...
    • Amazon S3

      Our platform enables you to quickly and easily connect your data sources to data from a File Key/Location in Amazon S3 Storage. This enables a one-way synchronization to occur - where any changes in the Amazon S3 Storage File Key/Location are ...
    • Connecting to Amazon S3

      In order to push/pull data from your Amazon S3 storage buckets or S3 API-compliant third-party services, you must first configure an S3 Connection. Adding a Connection When logged into your Appenate web portal, navigate to Data Hub > Connections > ...
    • Data Source Connector – Dropbox

      Our platform enables you to quickly and easily connect your data sources to data from a file in both Dropbox Personal as well as Dropbox for Business. This enables a one-way synchronization to occur – where any changes in the DropBox file are ...
    • Data Source Connector – Box

      Our platform enables you to quickly and easily connect your data sources to data from a file in Box. This enables a one-way synchronization, where changes in the Box file are regularly downloaded into your data source. Your data source will be ...