-
Notifications
You must be signed in to change notification settings - Fork 1
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
- Loading branch information
Showing
444 changed files
with
34,159 additions
and
28,605 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1 +1 @@ | ||
*.sql linguist-detectable=true | ||
*.sql linguist-detectable=true |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,22 +1,18 @@ | ||
--- | ||
name: CD | ||
|
||
on: | ||
push: | ||
branches: | ||
- main | ||
|
||
branches: [main] | ||
env: | ||
REGISTRY: ghcr.io | ||
IMAGE_NAME: basedosdados/queries-basedosdados-dev/dbt-rpc | ||
|
||
jobs: | ||
build-container: | ||
name: Build, publish and register | ||
runs-on: ubuntu-latest | ||
steps: | ||
- name: Checkout | ||
uses: actions/checkout@v2 | ||
|
||
- name: Import Secrets | ||
id: import_secrets | ||
uses: hashicorp/[email protected] | ||
|
@@ -28,41 +24,35 @@ jobs: | |
secret/data/gcp_credentials/basedosdados-dev GCP_PROJECT_ID | GCP_PROJECT_ID ; | ||
secret/data/gcp_credentials/basedosdados-dev GKE_CLUSTER_NAME | GKE_CLUSTER_NAME ; | ||
secret/data/gcp_credentials/basedosdados-dev GKE_CLUSTER_ZONE | GKE_CLUSTER_ZONE ; | ||
- name: Setup Google Cloud CLI | ||
uses: google-github-actions/[email protected] | ||
with: | ||
service_account_key: ${{ steps.import_secrets.outputs.GCP_SA_KEY_BASE64 }} | ||
project_id: ${{ steps.import_secrets.outputs.GKE_PROJECT_ID }} | ||
export_default_credentials: true | ||
|
||
- name: Get GKE credentials | ||
uses: google-github-actions/[email protected] | ||
with: | ||
cluster_name: ${{ steps.import_secrets.outputs.GKE_CLUSTER_NAME }} | ||
location: ${{ steps.import_secrets.outputs.GKE_CLUSTER_ZONE }} | ||
credentials: ${{ steps.import_secrets.outputs.GCP_SA_KEY_BASE64 }} | ||
|
||
- name: Log in to the Container registry | ||
uses: docker/[email protected] | ||
with: | ||
registry: ${{ env.REGISTRY}} | ||
username: ${{ github.actor }} | ||
password: ${{ secrets.GITHUB_TOKEN }} | ||
|
||
- name: Build and push | ||
uses: docker/[email protected] | ||
with: | ||
context: . | ||
file: ./Dockerfile | ||
push: true | ||
tags: ${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}:${{ github.sha }} | ||
|
||
- name: Set up Kustomize | ||
run: |- | ||
curl -sfLo kustomize https://github.com/kubernetes-sigs/kustomize/releases/download/v3.1.0/kustomize_3.1.0_linux_amd64 | ||
chmod u+x ./kustomize | ||
- name: Deploy | ||
run: |- | ||
./kustomize edit set image gcr.io/PROJECT_ID/IMAGE_DBT:TAG=$REGISTRY/$IMAGE_NAME:$GITHUB_SHA | ||
|
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,18 +1,15 @@ | ||
--- | ||
name: Deploy docs | ||
|
||
on: | ||
push: | ||
branches: | ||
- main | ||
|
||
branches: [main] | ||
jobs: | ||
docs: | ||
name: Deploy docs | ||
runs-on: ubuntu-latest | ||
steps: | ||
- name: Checkout | ||
uses: actions/checkout@v2 | ||
|
||
- name: Import Secrets | ||
id: import_secrets | ||
uses: hashicorp/[email protected] | ||
|
@@ -21,33 +18,26 @@ jobs: | |
token: ${{ secrets.VAULT_TOKEN}} | ||
secrets: | | ||
secret/data/queries_credentials/<GCP_PROJECT_NAME> GCP_SA_KEY | GCP_SA_KEY; | ||
- name: Set up Python | ||
uses: actions/setup-python@v2 | ||
with: | ||
python-version: "3.9.x" | ||
|
||
python-version: 3.9.x | ||
- name: Upgrade pip | ||
run: | | ||
pip install -U pip | ||
- name: Install dependencies | ||
run: | | ||
pip3 install -r requirements-dev.txt -r requirements.txt | ||
- name: Setup credentials | ||
run: | | ||
echo ${{ steps.import_secrets.outputs.GCP_SA_KEY }} > dbt-sa.json | ||
python3 .github/workflows/scripts/modify_profiles.py | ||
- name: Generate docs | ||
run: | | ||
dbt docs generate --profiles-dir . | ||
- name: Delete credentials | ||
run: | | ||
rm dbt-sa.json | ||
- name: Deploy docs | ||
uses: JamesIves/[email protected] | ||
with: | ||
|
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,18 +1,15 @@ | ||
--- | ||
name: Metadata Automation | ||
|
||
on: | ||
push: | ||
branches: | ||
- '*' | ||
|
||
branches: ['*'] | ||
jobs: | ||
update-metadata: | ||
name: Update metadata from Google Sheets | ||
runs-on: ubuntu-latest | ||
steps: | ||
- name: Checkout | ||
uses: actions/checkout@v2 | ||
|
||
- name: Import Secrets | ||
id: import_secrets | ||
uses: hashicorp/[email protected] | ||
|
@@ -21,28 +18,23 @@ jobs: | |
token: ${{ secrets.VAULT_TOKEN}} | ||
secrets: | | ||
secret/data/queries_credentials/basedosdados-dev GCP_SA_KEY_BASE64 | GCP_SA_KEY_BASE64; | ||
- name: Setup Python version | ||
uses: actions/setup-python@v2 | ||
with: | ||
python-version: "3.9" | ||
|
||
python-version: '3.9' | ||
- name: Install Python dependencies for actions | ||
run: |- | ||
pip install -r requirements-dev.txt | ||
- name: Checkout to the head ref | ||
run: |- | ||
git fetch --depth=1 | ||
git checkout ${{ github.head_ref }} | ||
- name: Update metadata | ||
env: | ||
GCP_SA_KEY_BASE64: ${{ steps.import_secrets.outputs.GCP_SA_KEY_BASE64 }} | ||
run: |- | ||
python3 .github/workflows/scripts/metadata_automation.py | ||
- name: Commit changed files | ||
uses: stefanzweifel/git-auto-commit-action@v4 | ||
with: | ||
commit_message: 'Update metadata' | ||
commit_message: Update metadata |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -8,4 +8,4 @@ venv-dbt/ | |
.user.yml | ||
dbt_packages | ||
profiles.yml | ||
gists/test.py | ||
gists/test.py |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.