BigQuery Connector
Install PipeRider with the BigQuery connector and connect to a data source.
pip install 'piperider[bigquery]'
Run the diagnose command in the dbt project
piperider diagnose
If you can successfully connect to BigQuery using dbt, PipeRider can also connect to BigQuery using the same profile settings. For details, please refer to the dbt BigQuery adapter documentation.
non-dbt use case is deprecated since v0.25.0
Initialize a new PipeRider project using
piperider init
and when prompted select BigQuery as the data source.The following information is required.
- GCP Project ID
- Authentication method (oauth or service-account)
- BigQuery dataset name
$ piperider init
Initialize piperider to path /path/to/your/project/.piperider
[?] What is your project name? (alphanumeric only): dataproject
[?] Which data source would you like to connect to?: bigquery
snowflake
> bigquery
postgres
sqlite
[?] Authentication Methods:
> oauth
service-account
Please enter the following fields for bigquery
[?] GCP Project ID: your-id
[?] Authentication Methods: oauth
> oauth
service-account
[?] Pick a dataset: {"project": "xx_projet-id_xx", "dataset": "yy_dataset_yy"}
If you don't see the expected project-id from the list, please modify
~/.config/gcloud/application_default_credentials.json
and replace quota_project_id with **** the one you expect**.**[?] The path of GCP Service Account Key File: /path/to/key/file
[?] The name of BigQuery DataSet: dataset-name
After configuring your connection settings, ensure that PipeRider can connect to your BigQuery data source.
piperider diagnose
If you have not previously authenticated with gcloud, the output of
diagnose
will prompt you to execute a gcloud command to authenticate. This will open a new browser window and you will be prompted to authenticate with your Google accountLast modified 10d ago