Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

initial suggestion minor tests completed sucessfully #183

Merged
merged 10 commits into from
Oct 8, 2024
Merged
Show file tree
Hide file tree
Changes from 9 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 0 additions & 2 deletions integration_tests/models/utils/cross_db/cross_db.yml
OscarSnowPlow marked this conversation as resolved.
Show resolved Hide resolved
Original file line number Diff line number Diff line change
Expand Up @@ -17,8 +17,6 @@ models:
tests:
- dbt_utils.equality:
compare_model: ref('expected_get_field_bq')
config:
+enabled: "{{ target.type in ['bigquery'] | as_bool() }}"
- name: test_indexed_unnest
tests:
- dbt_utils.equality:
Expand Down
13 changes: 9 additions & 4 deletions macros/utils/return_limits_from_model.sql
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ This program is licensed to you under the Snowplow Personal and Academic License
and you may not use this file except in compliance with the Snowplow Personal and Academic License Version 1.0.
You may obtain a copy of the Snowplow Personal and Academic License Version 1.0 at https://docs.snowplow.io/personal-and-academic-license-1.0/
#}
{% macro return_limits_from_model(model, lower_limit_col, upper_limit_col) -%}
{% macro return_limits_from_model(model, lower_limit_col, upper_limit_col, lower_output=False) -%}

{# In case of not execute just return empty strings to avoid hitting database #}
{% if not execute %}
Expand Down Expand Up @@ -35,9 +35,14 @@ You may obtain a copy of the Snowplow Personal and Academic License Version 1.0
{{ snowplow_utils.log_message("Snowplow Warning: *************") }}
{% do exceptions.warn("Snowplow Warning: No data in "~this~" for date range from variables, please modify your run variables to include data if this is not expected.") %}
{{ snowplow_utils.log_message("Snowplow Warning: *************") }}
{# This allows for bigquery to still run the same way the other warehouses do, but also ensures no data is processed #}
{% set lower_limit = snowplow_utils.cast_to_tstamp('9999-01-01 00:00:00') %}
{% set upper_limit = snowplow_utils.cast_to_tstamp('9999-01-02 00:00:00') %}
{% if lower_output %}
{% set lower_limit = snowplow_utils.cast_to_tstamp('0000-01-01 00:00:00') %}
{% set upper_limit = snowplow_utils.cast_to_tstamp('0000-01-02 00:00:00') %}
{%- else -%}
{# Default behaviour for incrementalisation. This allows for bigquery to still run the same way the other warehouses do, but also ensures no data is processed #}
OscarSnowPlow marked this conversation as resolved.
Show resolved Hide resolved
{% set lower_limit = snowplow_utils.cast_to_tstamp('9999-01-01 00:00:00') %}
{% set upper_limit = snowplow_utils.cast_to_tstamp('9999-01-02 00:00:00') %}
{% endif %}
{%- else -%}
{% set lower_limit = snowplow_utils.cast_to_tstamp(results.columns[0].values()[0]) %}
{% set upper_limit = snowplow_utils.cast_to_tstamp(results.columns[1].values()[0]) %}
Expand Down
3 changes: 3 additions & 0 deletions macros/utils/schema.yml
Original file line number Diff line number Diff line change
Expand Up @@ -92,6 +92,9 @@ macros:
- name: upper_limit_col
type: string
description: The column to take the `max` of to get the upper limit
- name: lower_output
type: boolean
description: This changes the output to a low set date if true and high set date if false, in the event the function cannot find value but needs to set a value
- name: set_query_tag
description: '{{ doc("macro_set_query_tag") }}'
arguments:
Expand Down
Loading