PipeRider
Search
⌃K

Quick Start

Integrate PipeRider with a dbt project in 5 mins
This guide illustrates the usage of PipeRider with a dbt project by using dbt's Jaffle Shop project as an example.

Prepare a dbt project

Clone the jaffle_shop repository, set up a profile, and make sure that you have installed dbt and related packages.

Clone Jaffle Shop

git clone https://github.com/dbt-labs/jaffle_shop.git
cd jaffle_shop

Create a dbt profile.yml

To quickly set up a profile, we provide the following example that uses DuckDB.
Save the following content directly to a new file name profiles.yml, and place it under the jaffle_shop directory.
# ./profiles.yml
jaffle_shop:
target: dev
outputs:
dev:
type: duckdb
path: jaffle_shop.duckdb

Install and run dbt

Install dbt-duckdb.
pip install dbt-duckdb
Ensure the project builds without error.
dbt build

Install PipeRider

Install PipeRider with the DuckDB connector.
pip install 'piperider[duckdb]'
PipeRider requires Python 3.7+

Tag models to profile

Use a config block to add the 'piperider tag to the following models:
  • models/customers.sql
  • models/orders.sql.
{{ config(tags=['piperider']) }}
Refer to Specify resources to profile for other methods of selecting resources.
If you don't specify models to profile, PipeRider will only be able to detect schema changes

Make a change to the project

Now that the dbt project is configured and PipeRider is installed, you can follow the normal practice of create a new branch to make your project changes on.

Create a new development branch

Create a new development branch to make changes to the project.
git checkout -b feature/add-average-value-per-order

Update a model

Edit models/customers.sql and add a new column to customers.
...
final as (
​
select
customers.customer_id,
customers.first_name,
customers.last_name,
customer_orders.first_order,
customer_orders.most_recent_order,
+ customer_payments.total_amount / customer_orders.number_of_orders as average_value_per_order,
customer_orders.number_of_orders,
customer_payments.total_amount as customer_lifetime_value
​
from customers
...

Build the project

Run dbt command again to ensure that the changes have been applied correctly and the build completes without error.
dbt build

Compare your branch with main

Using PipeRider's compare feature, you can now compare the state of you project before and and after making the recent model changes
Execute the following command to generate a comparison report for comparing the transformed models
piperider compare
PipeRider will output a comparison report link when the compssre process is finished. The report contains detailed data profiling statistics of your project before and after making the recent change.
PipeRider focuses on analyzing the data impact of code changes introduced in pull requests. By default, piperider compare employs a three-dot comparison method, which aligns with GitHub’s approach.
Please check the piperider run documentation for more information on running PipeRider and specifying resources.

What's next

Getting Started - Previous
Install PipeRider
Next - Getting Started
Run
Last modified 2mo ago