BigQuery Connector

Install PipeRider with the BigQuery connector and connect to a data source.

BigQuery Connector


Ensure you have already installed the gcloud CLI, and then install PipeRider with the BigQuery connector.
pip install 'piperider[bigquery]'

Configure connection settings

Initialize a new PipeRider project using piperider init and when prompted select BigQuery as the data source.
The following information is required.
  • GCP Project ID
  • Authentication method (oauth or service-account)
  • BigQuery dataset name

Example initialization steps

$ piperider init
Initialize piperider to path /path/to/your/project/.piperider
[?] What is your project name? (alphanumeric only): dataproject
[?] Which data source would you like to connect to?: bigquery
> bigquery
[?] Authentication Methods:
> oauth
Oauth steps
Please enter the following fields for bigquery
[?] GCP Project ID: your-id
[?] Authentication Methods: oauth
> oauth
[?] Pick a dataset: {"project": "xx_projet-id_xx", "dataset": "yy_dataset_yy"}
If you don't see the expected project-id from the list, please modify ~/.config/gcloud/application_default_credentials.json and replace quota_project_id with **** the one you expect**.**
Service-account step
[?] The path of GCP Service Account Key File: /path/to/key/file
[?] The name of BigQuery DataSet: dataset-name

Test connection settings

After configuring your connection settings, ensure that PipeRider can connect to your BigQuery data source.
piperider diagnose

gcloud authentication

If you have not previously authenticated with gcloud, the output of diagnose will prompt you to execute a gcloud command to authenticate. This will open a new browser window and you will be prompted to authenticate with your Google account