Skip to content

Commit

Permalink
Merge branch 'main' of github.com:dbt-labs/dbt-external-tables into p…
Browse files Browse the repository at this point in the history
…r/thomas-vl/230
  • Loading branch information
dataders committed Mar 11, 2024
2 parents 515fc4a + 383ee3c commit 112794f
Show file tree
Hide file tree
Showing 2 changed files with 13 additions and 1 deletion.
10 changes: 10 additions & 0 deletions integration_tests/models/plugins/bigquery/bigquery_external.yml
Original file line number Diff line number Diff line change
Expand Up @@ -67,6 +67,16 @@ sources:
columns: *cols-of-the-people
tests: *equal-to-the-people

- name: people_csv_with_max_staleness
external:
location: 'gs://dbt-external-tables-testing/csv/*'
options:
format: csv
skip_leading_rows: 1
max_staleness: INTERVAL 1 HOUR
columns: *cols-of-the-people
tests: *equal-to-the-people

- name: people_csv_schema_auto_detect_expiration
external:
hours_to_expiration: 24
Expand Down
4 changes: 3 additions & 1 deletion macros/plugins/bigquery/create_external_table.sql
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,8 @@
{%- set partitions = external.partitions -%}
{%- set options = external.options -%}
{%- set hours_to_expiration = external.get('hours_to_expiration') -%}
{%- set non_string_options = ['max_staleness', 'hours_to_expiration'] %}

{% if options is mapping and options.get('connection_name', none) %}
{% set connection_name = options.pop('connection_name') %}
{% endif %}
Expand Down Expand Up @@ -38,7 +40,7 @@
uris = [{%- for uri in uris -%} '{{uri}}' {{- "," if not loop.last}} {%- endfor -%}]
{%- if options is mapping -%}
{%- for key, value in options.items() if key != 'uris' %}
{%- if value is string -%}
{%- if value is string and key not in non_string_options -%}
, {{key}} = '{{value}}'
{%- else -%}
, {{key}} = {{value}}
Expand Down

0 comments on commit 112794f

Please sign in to comment.