Specify resources to profile
How to specify which resources PipeRider should profile
To use the data profiling and comparison features of PipeRider, you will need to specify which resources PipeRider will run on. Without doing this, PipeRider will only detect schema change.
This can be done in the following ways:
- Tag resources
- Use the
- Use the
- Use the
--tableoption (deprecated, for non-dbt projects)
By default PipeRider looks for then 'piperider' tag when discovering resources tom profile. Tags can be added to dbt resources in various ways, such as tagging resources individually using a config block:
--selectoption can be used to select individual resources or groups of resources, and can be used with both
--selectoverrides the need for tagging resources
Run PipeRider on a single model.
piperider run --select path/to/model.sql
Run PipeRider on all models in the staging folder.
piperider compare --select 'models/staging/'
PipeRider supports all of the
--selectcriteria that you're used to from dbt, so you can pass resources to PipeRider with advanced commands like these.
Select all modified staging resources and children that are tagged with the 'piperider' tag:
piperider compare --select state:modified+, staging, tag:piperider
Select all modified resources that are materialized as tables:
piperider compare --select state:modified+, config.materialized:table
Select all modified staging resources:
piperider compare -s state:modified+, staging
--modifiedoption will run PipeRider on only nodes that have been modified, and their children. This is equivalent to the
E.g. compare modified models that are tagged with the 'piperider' tag:
piperider compare --modified
Override tags by pairing
E.g. compare modified staging models:
piperider compare --modified --select '/models/staging/'
--tableoption is used for profiling individual tables in non-dbt projects. It now deprecated in favor of the
--selectoption as PipeRider shifts focus to dbt projects.
Select an individual table to profile.
piperider run --table <table>