Skip to content

Commit

Permalink
feat: add stddev macro
Browse files Browse the repository at this point in the history
  • Loading branch information
Dennis-UiPath committed Dec 20, 2023
1 parent 048e757 commit d1c872b
Show file tree
Hide file tree
Showing 5 changed files with 56 additions and 1 deletion.
7 changes: 7 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -31,6 +31,7 @@ This dbt package contains macros for SQL functions to run the dbt project on mul
- [date_add](#date_add-source)
- [date_from_timestamp](#date_from_timestamp-source)
- [datediff](#datediff-source)
- [stddev](#stddev-source)
- [string_agg](#string_agg-source)
- [timestamp_from_date](#timestamp_from_date-source)
- [timestamp_from_parts](#timestamp_from_parts-source)
Expand Down Expand Up @@ -111,6 +112,12 @@ This macro generates an id field that can be used as a column for the current mo
Usage:
`{{ pm_utils.id() }}`

#### stddev ([source](macros/multiple_databases/stddev.sql))
This macro computes the standard deviation of a set of values, `null` values are ignored in the calculation. This macro can only be used as an aggregate function. For SQL Server, at least one of the values provided should not be `null`.

Usage:
`{{ pm_utils.stddev('[expression]') }}`

#### string_agg ([source](macros/multiple_databases/string_agg.sql))
This macro aggregates string fields separated by the given delimiter. If no delimiter is specified, strings are separated by a comma followed by a space. This macro can only be used as an aggregate function. For SQL Server, the maximum supported length is 2000.

Expand Down
2 changes: 1 addition & 1 deletion dbt_project.yml
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
name: 'pm_utils'
version: '1.1.2'
version: '1.2.0'
config-version: 2

require-dbt-version: [">=1.0.0", "<2.0.0"]
9 changes: 9 additions & 0 deletions integration_tests/models/schema.yml
Original file line number Diff line number Diff line change
Expand Up @@ -90,3 +90,12 @@ models:
- equal_value:
actual: '`add_to_null_value`'
expected: '`add_to_null_value_expected`'

- name: test_stddev
tests:
- equal_value:
actual: '`stddev_actual`'
expected: '`stddev_expected`'
- equal_value:
actual: '`stddev_with_null_values_actual`'
expected: '`stddev_with_null_values_expected`'
30 changes: 30 additions & 0 deletions integration_tests/models/test_stddev.sql
Original file line number Diff line number Diff line change
@@ -0,0 +1,30 @@
with Input_data as (
select val
from (values (1),(2),(3),(4)) as Input(val)
),

Input_data_with_null as (
select val
from (values (1),(2),(null),(4)) as Input(val)
),

stddev_input_data as (
select
round({{ pm_utils.stddev('val') }}, 3) as `stddev_input_data`
from Input_data
),

stddev_input_data_null as (
select
round({{ pm_utils.stddev('val') }}, 3) as `stddev_input_data_null`
from Input_data_with_null
)

select
{{ pm_utils.to_varchar('stddev_input_data.`stddev_input_data`') }} as `stddev_actual`,
'1.291' as `stddev_expected`,

{{ pm_utils.to_varchar('stddev_input_data_null.`stddev_input_data_null`') }} as `stddev_with_null_values_actual`,
'1.528' as `stddev_with_null_values_expected`
from stddev_input_data
cross join stddev_input_data_null
9 changes: 9 additions & 0 deletions macros/multiple_databases/stddev.sql
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
{% macro stddev(field, relation) %}

{%- if target.type in ('databricks', 'snowflake') -%}
stddev({{ field }})
{%- elif target.type == 'sqlserver' -%}
stdev({{ field }})
{%- endif -%}

{% endmacro %}

0 comments on commit d1c872b

Please sign in to comment.