BigQuery

An overview of the BigQuery integration with Secoda

BigQuery Metadata Extracted

Getting Started with Big Query

There are three steps to connect Big Query with Secoda:

1. Enable the Big Query API

2. Create a service account for Secoda

3. Connect Big Query to Secoda

Enable Big Query API for GCP

Log in to your existing project on GCP and enable the BigQuery API. Once you’ve done so, you should see BigQuery in the “Resources” section of Cloud Platform.

Create a service account for Secoda

To provide least privilege to Secoda for extracting Big Query metadata, you can create a new service account following the steps below. Refer to Google Cloud’s documentation about service accounts for more information.

1. From the Navigation panel on the left, go to IAM & admin > Service accounts

2. Click Create Service Account along the top

3. Enter a name (for example: “secoda”) and click Create

4. When assigning permissions, make sure to grant the following permissions:

a) If you're creating the service account via the GCP console add the following roles:

BigQuery Metadata Viewer
BigQuery Resource Viewer
BigQuery Data Viewer
BigQuery Job User

If you want to enable metadata push feature, add the following role:

BigQuery Data Editor

If you want to enable syncing policy tags, add the following roles:

Data Catalog Admin
Data Catalog Tag Editor
Tag User

b) If you're programatically creating the service account add the following roles:

roles/bigquery.metadataViewer
roles/bigquery.resourceViewer
roles/bigquery.dataViewer        
roles/bigquery.jobUser

If you want to enable metadata push feature, add the following role:

roles/bigquery.tables.update

If you want to enable syncing policy tags, add the following roles:

roles/datacatalog.admin
roles/datacatalog.tagEditor
roles/datacatalog.tagUser

5. Create a JSON key. The downloaded file will be used to create your warehouse in the next section.

Connect Big Query to Secoda

• Log into your Secoda profile at https://app.secoda.co

• From the Navigation panel on the left go Integrations > Add new integration

• Select Big Query

• Enter in the project name and paste the JSON key file contents that was downloaded

• Click "Connect"