Skip to content

Commit

Permalink
chore: Setup cicd pipeline and ci scrips
Browse files Browse the repository at this point in the history
  • Loading branch information
Jo Lares authored and Jo Lares committed Feb 19, 2024
1 parent 9a4ae42 commit 03b0aa9
Show file tree
Hide file tree
Showing 3 changed files with 97 additions and 0 deletions.
37 changes: 37 additions & 0 deletions .github/workflows/cicd.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,37 @@
name: cicd

on: [push]

jobs:
elt:
runs-on: ubuntu-latest
env:
DBT_PROJECT_NAME: 'dbt_demo'
DBT_PROFILES_DIR: ''
steps:
- run: echo "${{ github.actor }} is testing out GitHub Actions 🚀."

- run: echo "🎉 The job was automatically triggered by a ${{ github.event_name }} event."

- run: echo "🐧 This job is now running on a ${{ runner.os }} server hosted by GitHub!"

- run: echo "🔎 The name of your branch is ${{ github.ref }} and your repository is ${{ github.repository }}."

- name: Check out repository code
uses: actions/checkout@v4

- run: echo "💡 The ${{ github.repository }} repository has been cloned to the runner."

- run: echo "🖥️ The workflow is now ready to test your code on the runner."

- name: List files in the repository
run: |
ls ${{ github.workspace }}
- run: echo "🍏 This job's status is ${{ job.status }}."

- run: sh setup.sh
working-directory: ${{ env.DBT_PROJECT_NAME }}

- run: sh run.sh
working-directory: ${{ env.DBT_PROJECT_NAME }}
3 changes: 3 additions & 0 deletions dbt_demo/run.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
### Run the elt pipeline using dbt

dbt run --profiles-dir="."
57 changes: 57 additions & 0 deletions dbt_demo/setup.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,57 @@

##
## Validate runner configuration
##

if [[ -z "${DBT_PROJECT_NAME}" ]]; then
echo "Environment variable DBT_PROJECT_NAME is not set"
exit 1
fi

# if [[ -z "${DBT_PROFILES_DIR}" ]]; then
# # Note: the value used by dbt when --profiles-dir is not is "/Users/<USER>/.dbt"
# echo "Environment variable DBT_PROFILES_DIR is not set"
# exit 1
# fi

##
## Install system dependencies
##

# Upgrade pip
python -m pip install --upgrade pip

pip --version

# Install virtualenv
# pip install virtualenv

# Create a virtual environment
python -m venv .venv

# Activate virtual environment
source .venv/bin/activate

##
## Install project dependencies
##

## Install dbt packages

# Install the dbt-<warehouse_provider> library using pip
pip install dbt-snowflake

# Confirm that the dbt installation was successful.
dbt --version

dbt init $DBT_PROJECT_NAME --profiles-dir=$DBT_PROFILES_DIR

##
## Setup dbt connection profile to data warehouse
##

# TODO
# ...

# Verify dbt connection profile to data warehouse
dbt debug

0 comments on commit 03b0aa9

Please sign in to comment.