Comment on page
"Run" is a single execution of PipeRider on a dbt project. It generates observed results of the dbt project, such as table schema, profiling statistics, metric query results. They are used as the basis for comparison.
- Collect Metadata: To collect the column names and types for all models, sources, and seeds.
- Profile statistics: To obtain statistics for a model/seed/source and its columns, including row counts, null values, sum, average, text length, and other information, in order to gain insights into the data distribution of the model. By default, this feature is disabled since it can be resource-intensive. To enable profiling, you need to manually activate it. Please see the profiling document
- Query metric: A metric represents the computation of a specific column within a time interval, typically calculated on a daily or monthly basis. For example, daily revenue is a simple example of a metric. Metric queries allow you to perform basic queries on a DBT metric, such as retrieving the daily reports for the past 30 days or the monthly reports for the last 12 months. Similar to model profiling, you need to manually enable metric queries. Please see the metric document
To execute, use the
After the execution of a run, two artifacts are generated under the output directory
- JSON run result (
- HTML report (
The default output directory is located at
.piperider/outputs/<datasource>-<datetime>/. For ease of use, the latest run would also be sym-linked by
You can use the
--outputto change the output directory
piperider run --output /tmp/myrun
The following command would list the model you just modified.
dbt list -s tag:piperider
Afterwards, when running
piperider run, all models with the
piperidertag will be profiled by default.
In a dbt project, especially for analytics purpose project, it's common to have several metrics defined for visualization. (e.g. revenue, active users). PipeRider can query the metrics are visualize it in the run report.
To enable a metric query, there are two steps
Here is an metric example
- name: active_users
label: Active Users
description: "The active user"
time_grains: [day, week, month, year]
+ tags: ['piperider']
By default, PipeRider profiles models, sources, and seeds with
piperidertag, and query metrics with
piperidertag. However, you can also use additional options to select the specific resources that should be processed.
Use dbt list to select resources
dbt list --select <selector> | piperider run --dbt-list
select a model by file path
dbt list -s models/customers.sql| piperider run --dbt-list
select resource with tag
dbt list -s 'tag:piperider-dev' | piperider run --dbt-list
Select the resource to profile
You can also use
--tableto profile specific resource.
piperider run --table <resource_name>
piperider run --datasource <dbt-tareget>
piperider config list-datasource