BigQuery Connector

Install PipeRider with the BigQuery connector and connect to a data source.

Installation
pip install 'piperider[bigquery]'
Configuration (DBT)
Run the diagnose command in the dbt project
piperider diagnose
If you can successfully connect to BigQuery using dbt, PipeRider can also connect to BigQuery using the same profile settings. For details, please refer to the dbt BigQuery adapter documentation.
Configuration (Non-DBT)
non-dbt use case is deprecated since v0.25.0
Ensure you have already installed the gcloud CLI​
Configure connection settings
Initialize a new PipeRider project using piperider init and when prompted select BigQuery as the data source.
The following information is required.
GCP Project ID
Authentication method (oauth or service-account)
BigQuery dataset name
Example initialization steps
$ piperider init
Initialize piperider to path /path/to/your/project/.piperider
[?] What is your project name? (alphanumeric only): dataproject
[?] Which data source would you like to connect to?: bigquery
   snowflake
 > bigquery
   postgres
   sqlite
[?] Authentication Methods:
 > oauth
   service-account
Oauth steps
Please enter the following fields for bigquery
[?] GCP Project ID: your-id
[?] Authentication Methods: oauth
 > oauth
   service-account
​
[?] Pick a dataset: {"project": "xx_projet-id_xx", "dataset": "yy_dataset_yy"}
If you don't see the expected project-id from the list, please modify ~/.config/gcloud/application_default_credentials.json and replace quota_project_id with **** the one you expect**.**
Service-account step
[?] The path of GCP Service Account Key File: /path/to/key/file
[?] The name of BigQuery DataSet: dataset-name
Test connection settings
After configuring your connection settings, ensure that PipeRider can connect to your BigQuery data source.
piperider diagnose
gcloud authentication
If you have not previously authenticated with gcloud, the output of diagnose will prompt you to execute a gcloud command to authenticate. This will open a new browser window and you will be prompted to authenticate with your Google account

Snowflake Connector

Postgres Connector

Last modified 10d ago