Built-In Assertions

PipeRider's built-in set of data assertions with usage examples
Here are built-in assertions, there are two types of assertions, one takes no parameter, the other takes parameters.
Basic Assertions
assert_column_unique
Description: The values of column must be unique.
Assert: None
Tags:
world_city:
  columns:
    country_code:
      tests:
      - name: assert_column_unique
        tags:
          - dialing code
assert_column_not_null
Description: The values of the column must not be null.
Assert: None
Tags:
world_city:
  columns:
    name:
      tests:
      - name: assert_column_not_null
        tags:
          - city name
assert_column_value
Description: Assert the column value should be in the range.
Assert:
gte: the value should be greater than or equal to
gt: the value should be greater than
lte: the value should be less than or equal to
lt: the value should be less than
in: the value should belong to the set
The value should be between [0,10000)
world_city:
  columns:
    population:
      tests:
      - name: assert_column_value
        assert:
            gte: 0
            lt: 10000
The value of a datetime type column should be >= '2022-01-01'
world_city:
  columns:
    create_at:
      tests:
      - name: assert_column_value
        assert:
          gte: '2022-01-01;
The value of the column should belong to ["male", "female"] set
TITANIC:
  columns:
    Sex:
      tests:
      - name: assert_column_value
        assert:
          in: ["male", "female"]
Schema Assertions
assert_column_exist
Description: The column must exist.
Assert: None
Tags:
world_city:  #Table Name
  columns:
    country_code:
      tests:
      - name: assert_column_exist
        tags:
          - dialing code
assert_column_type
Description: The type of the column must match the specified type.
Assert:
type: numeric, string, datetime
Tags:
world_city:
  columns:
    name:
      tests:
      - name: assert_column_type
        assert:
          type: string
        tags:
          - city name
assert_column_schema_type
Description: The column schema type should match the specific schema type.
Assert:
schema_type: the schema type in data source. (e.g. TEXT, DATE, VARCHAR(128), ...)
world_city:
  columns:
    name:
      tests:
      - name: assert_column_schema_type
        assert:
          schema_type: TEXT
assert_column_in_types
Description: The type of the column must be contained in the list.
Assert:
types: [string, integer, numeric, datetime, boolean, other]
Tags:
world_city:  #Table Name
  columns:
    country_code:
      tests:
      - name: assert_column_in_types
        assert:
          types: [string]
        tags:
          - dialing code
Metric-based Assertions
You can have assertions against metrics generated by PipeRider directly with several assertion expressions.
Assertion expressions
Description: Metric-based assertions are assert the value of a metric.
Assert:
gte: the value should be greater than or equal to
gt: the value should be greater than
lte: the value should be less than or equal to
lt: the value should be less than
eq: the value should equal to
ne: the value should not equal to
The row count should be <= 1000000
world_city:
  tests:
  - metric: row_count
    assert:
      lte: 1000000
The missing percentage should be <= 0.01
world_city:
  columns:
    country_code:
      tests:
      - metric: nulls_p
        assert:
          lte: 0.01
The median should be between [10, 20]
world_city:
  columns:
    country_code:
      tests:
      - metric: p50
        assert:
          gte: 10
          lte: 20
Assertion Files
Custom Assertions
Last modified 1mo ago