Create a Fivetran metadata collector

Create a collector to import metadata from Fivetran.

Before you begin

Before you begin, verify the following:

• All per-requisite tasks are completed. For more information, see Prepare to run the Fivetran collector.

• If you plan to run the collector on-premise, a MID Server is setup for the collector. For more information, see MID Server for metadata collectors.

• Role required: connection-admin

Procedure

1. Navigate to All > Workflow Data Fabric > Workflow Data Fabric Home.

2. Select the Connect Hub [Omitted image "wdf-connect-hub-icon.png"] Alt text: Connect Hub icon icon in the left sidebar.

3. Select Create > Metadata collector.

4. From the System list, select Fivetran.

5. From the Connection type list, select one of the following:

   1. Select New connection to configure a new connection.

   2. Select Existing connection to reuse an existing connection and select an existing connection from the Connections list.

      The configuration form is filled with details from the existing connection. The name is appended with the word Copy and sensitive details like password aren't copied.

6. On the form, fill in the fields.

   Field Description Connection name Unique identifier for the connection. This field can't be modified once the connection is established. Short description Purpose and details of the connection.

7. Configure the connection options.

<table id=“table_s3_collector_props”><thead><tr><th>

Field

</th><th>

Description

</th></tr></thead><tbody><tr><td>

Use MID server

</td><td>

Option to connect to the source system through a MID Server. When selected, the MID selection options appear.

</td></tr><tr><td>

MID selection

</td><td>

MID Server routing option for the collector job. This field appears only when Use MID server is selected.

• Auto-Select MID Server — The system selects an available MID Server at job execution time. Default selection.

• Specific MID Server — The collector job runs on the selected MID Server. If the MID Server is offline at job execution time, the job fails with an error identifying the MID Server by name.

• Specific MID Cluster — The collector job runs on any available MID Server within the selected cluster.

</td></tr><tr><td>

MID Server

</td><td>

MID Server to use for the collector job. This field appears only when Specific MID Server is selected. The selected MID Server is saved on the collector record and used for all subsequent collection runs until changed.

</td></tr><tr><td>

MID Cluster

</td><td>

MID cluster to use for the collector job. This field appears only when Specific MID Cluster is selected. The selected cluster is saved on the collector record; any available MID Server within the cluster may serve the job.

</td></tr><tr><td>

Capabilities

</td><td>

One or more capability labels that restrict auto-selection to MID Servers advertising those capabilities. This field appears only when Auto-Select MID Server is selected.

</td></tr><tr><td>

MID application

</td><td>

MID Server application to use for the collector job. This field appears only when Auto-Select MID Server is selected.

</td></tr></tbody> </table>8. Enter the authentication details.

|Field|Description|
|-----|-----------|
|Fivetran API key|The Fivetran API key used to authenticate to the REST API.|
|Fivetran API secret|The Fivetran secret used to authenticate to the REST API.|

1. Enter the Fivetran configuration details.

   Field Description Exclude groups Exclude the specified Fivetran group's contents from the catalog. Include groups Include only the specified Fivetran group's contents in the catalog.

2. Configure the advanced options.

<table id=“table_ocr_gc4_33c”><thead><tr><th>

Field

</th><th>

Description

</th></tr></thead><tbody><tr><td>

Max retries

</td><td>

The number of times the system retries a failed API call.Default: 5

</td></tr><tr><td>

Retry delay

</td><td>

The number of seconds to wait between retry attempts for a failed API call.Default: 2 seconds

</td></tr></tbody> </table>11. Select Save.

Result

The metadata collector is created and appears on the Connectors page with a Configured status. It is now ready to connect to the source system and harvest metadata.

What to do next

After creating the collector, you can perform any of the following tasks:

• Run the collector manually to harvest metadata immediately. See Run metadata collectors manually.

• Automate metadata collection by scheduling regular collector runs. See Schedule metadata collector runs.

• Monitor execution status and troubleshoot issues by viewing the runtime logs. See View runtime logs for collector runs.

• Discover and evaluate the harvested data assets in the Data Catalog. See Governing the Data Catalog.

Parent Topic:Fivetran metadata collector