Your data source will be automatically updated in this way until you remove the connector or an error occurs (such as losing authorization to access the database).

 You must ensure that you have a valid Google Cloud Platform Connection set up in your organization before adding a Google BigQuery Connector. You also need to set up a few basic things in the GCP console.

Adding

Data Source Settings

• Navigate the side menu to Data Hub > Data Sources
• Hover over a data source and click the “Settings” icon link ()
• In Settings, hit the “Add Connector” button
• Select the relevant option to add the connector

This will refresh the page with the newly added connector ready for configuration.

Nothing has been saved at this stage, so save your connector after making any changes to enable or update it.

Alternatively, if you’re viewing the rows of a data source, you can navigate to the Settings page using the option under the page’s title.

Configuring a Google BigQuery Data Source Connector

Next, configure the following:

• Refresh Frequency
  Specify the time interval for which data is pulled and refreshed by this connector.
• Project Id
  Specify the Google Cloud Platform Project ID that contains your target BigQuery dataset.
• Dataset Name
  Specify the Dataset Name where the table is located.
• Table Name
  The BigQuery table from which rows will be retrieved.
• Date Format
  Specify the format for converting date values to text when retrieving new rows.
• Ignore Internal Columns
  If enabled, this connector will ignore all columns where the column name starts with an underscore, e.g., _lastupdated
  If your table has a column named _identity and internal columns are not ignored, this column will always be the first column of the data source. Other internal columns are added to the end.

When you save your changes, the connector will fire for the first time and run at the scheduled interval. Wait about a minute, or click the three-dot menu (  ) and select Run Now to trigger it manually. Then, check the Rows in your data source.

If rows are not downloaded, there may be an error with the Google BigQuery API connection. To troubleshoot the issue, click the three-dot menu (  ) and select View Logs to review them.