Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[fix] br_ons_ #531

Merged
merged 1 commit into from
Apr 5, 2024
Merged

[fix] br_ons_ #531

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
24 changes: 10 additions & 14 deletions .github/workflows/elementary.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,9 +1,10 @@
---
name: Deploy Elementary Report
on:
push:
branches: [ "main", "master" ]
branches: [main, master]
schedule:
- cron: '00 22 * * 1-5'
- cron: 00 22 * * 1-5
workflow_dispatch:
jobs:
elementary:
Expand All @@ -14,19 +15,14 @@ jobs:
- name: Run Elementary
uses: elementary-data/[email protected]
with:
warehouse-type: bigquery
adapter-version: 1.5.9
profiles-yml: ${{ secrets.ELEMENTARY_PROFILES_YML }}
edr-command:
edr report
--file-path "report.html"
&&
edr send-report
--google-service-account-path "/tmp/gcs_keyfile.json"
--gcs-bucket-name "basedosdados"
--update-bucket-website "true"
warehouse-type: bigquery
adapter-version: 1.5.9
profiles-yml: ${{ secrets.ELEMENTARY_PROFILES_YML }}
edr-command: edr report --file-path "report.html" && edr send-report --google-service-account-path
"/tmp/gcs_keyfile.json" --gcs-bucket-name "basedosdados" --update-bucket-website
"true"
bigquery-keyfile: ${{ secrets.BIGQUERY_KEYFILE }}
gcs-keyfile: ${{ secrets.GCS_KEYFILE }}
gcs-keyfile: ${{ secrets.GCS_KEYFILE }}
- name: Upload report
uses: actions/upload-artifact@v3
with:
Expand Down
6 changes: 0 additions & 6 deletions dbt_project.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -255,15 +255,9 @@ models:
br_ons_avaliacao_operacao:
+materialized: table
+schema: br_ons_avaliacao_operacao
+post-hook:
- REVOKE `roles/bigquery.dataViewer` ON TABLE {{ this }} FROM "specialGroup:allUsers"
- GRANT `roles/bigquery.dataViewer` ON TABLE {{ this }} TO "group:[email protected]"
br_ons_estimativa_custos:
+materialized: table
+schema: br_ons_estimativa_custos
+post-hook:
- REVOKE `roles/bigquery.dataViewer` ON TABLE {{ this }} FROM "specialGroup:allUsers"
- GRANT `roles/bigquery.dataViewer` ON TABLE {{ this }} TO "group:[email protected]"
br_poder360_pesquisas:
+materialized: table
+schema: br_poder360_pesquisas
Expand Down
7 changes: 6 additions & 1 deletion models/br_bd_diretorios_brasil/schema.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -280,7 +280,12 @@ models:
- name: sigla_uf
description: Sigla do Estado
- name: situacao_setor
description: – ID de situação do setor - Situação urbana - IDs 1, 2 e 3; 1 - Área urbanizada de cidade ou vila 2 - Área não urbanizada de cidade ou vila 3 - Área urbana isolada - Situação rural – IDs 4, 5, 6, 7 e 8; 4 - Aglomerado rural de extensão urbana 5 - Aglomerado rural isolado, povoado 6 - Aglomerado rural isolado, núcleo 7 - Aglomerado rural isolado, outros aglomerados 8 - Zona rural, exclusive aglomerado rural
description: – ID de situação do setor - Situação urbana - IDs 1, 2 e 3; 1
- Área urbanizada de cidade ou vila 2 - Área não urbanizada de cidade ou
vila 3 - Área urbana isolada - Situação rural – IDs 4, 5, 6, 7 e 8; 4 -
Aglomerado rural de extensão urbana 5 - Aglomerado rural isolado, povoado
6 - Aglomerado rural isolado, núcleo 7 - Aglomerado rural isolado, outros
aglomerados 8 - Zona rural, exclusive aglomerado rural
- name: tipo_setor
description: Tipo de setor censitário
- name: id_microrregiao
Expand Down
11 changes: 4 additions & 7 deletions models/br_ibge_censo_2022/schema.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -870,12 +870,9 @@ models:
description: description
tests:
- dbt_utils.unique_combination_of_columns:
combination_of_columns:
- id_uf
- id_municipio
- id_setor_censitario
combination_of_columns: [id_uf, id_municipio, id_setor_censitario]
- not_null_proportion_multiple_columns:
at_least: 0.05
at_least: 0.05
columns:
- name: id_uf
description: ID da Unidade da Federação
Expand Down Expand Up @@ -905,9 +902,9 @@ models:
description: Total de Domicílios Coletivos (DCCM + DCSM)
- name: media_moradores_domicilios
description: Média de moradores em Domicílios Particulares Ocupados (Total
pessoas em Domicílios Particulares Ocupados / DPPO + DPIO)
pessoas em Domicílios Particulares Ocupados / DPPO + DPIO)
- name: porcentagem_domicilios_imputados
description: Percentual de Domicílios Particulares Ocupados Imputados (Total
DPO imputados / Total DPO)
DPO imputados / Total DPO)
- name: domicilios_particulares_ocupados
description: Total de Domicílios Particulares Ocupados (DPPO + DPIO)
2 changes: 1 addition & 1 deletion profiles.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -26,4 +26,4 @@ elementary:
dataset: elementary
method: service-account
keyfile: /tmp/bigquery_keyfile.json
threads: 2
threads: 2
Loading