Skip to content

Commit

Permalink
docs(README): add example YAML profiles for each warehouse
Browse files Browse the repository at this point in the history
  • Loading branch information
gwenwindflower authored Apr 24, 2024
1 parent 9b0fd08 commit 417f9f3
Showing 1 changed file with 68 additions and 5 deletions.
73 changes: 68 additions & 5 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -70,6 +70,23 @@ That's it! It's a single binary and has no dependencies on `dbt` itself, for max

Snowflake uses `externalbrowser` SSO authentication. It requires that you have SSO set up in your warehouse, it will then open a browser tab to authenticate and refresh a token in your **local** keychain. You'll be prompted to enter _your computer's user login_ (not your Snowflake password) to retrieve the token locally from your keychain.

Here's an example of a profile that works with `tbd` on Snowflake:

```yaml
lothlorien_labs:
target: dev
outputs:
dev:
type: snowflake
account: galadriel123
user: [email protected]
role: transformer
authenticator: externalbrowser
database: lothlorien
schema: mallorn_trees
threads: 16
```
### 🌯 BigQuery
BigQuery requires that you have the `gcloud` [CLI installed](https://cloud.google.com/sdk/docs/install) and authenticated for whatever projects you target.
Expand All @@ -78,6 +95,19 @@ BigQuery requires that you have the `gcloud` [CLI installed](https://cloud.googl
gcloud auth application-default login
```

Here's an example of a profile that works with `tbd` on BigQuery:
```yaml
moria_mining:
target: dev
outputs:
dev:
type: bigquery
method: oauth
project: moria-mining
dataset: dbt_gimli
threads: 16
```

I will likely bring in some other authentication options soon, but this is the easiest and most secure.

### 🦆 DuckDB
Expand All @@ -87,10 +117,43 @@ Using local DuckDB doesn't require authentication, just an existing DuckDB datab
> [!NOTE]
> I've built-in support for [MotherDuck](https://motherduck.com/), you just need to set an env var called `MOTHERDUCK_TOKEN` with your service token, then pass the path as `md:`, **but** until MotherDuck upgrades to v10 this requires you to use DuckDB 0.9.2 locally for compatibility. MotherDuck says the upgrade will happen any day now so hopefully this note will be removed soon!

Here's an example of a profile that works with `tbd` on DuckDB:

```yaml
legolas_analytics:
target: dev
outputs:
dev:
type: duckdb
path: /Users/winnie/dev/jaffle_shop_duckdb/mirkwood.duckdb
database: mirkwood
schema: archers
threads: 16
```


### 🐘 Postgres

Postgres setup is fairly normal! It just needs a host, port, user, password, and of course a database and schema that you want to model. The default values point to a localhost instance on port 5432, but you can change that to whatever you need and it should work with cloud-hosted Postgres instances as well (though I haven't yet tested this myself). Tests have been running on Postgres 16, but `tbd` makes very standard queries to `INFORMATION_SCHEMA` and doesn't do anything fancy with the connection, so it should work with any relatively modern Postgres version probably.

Here's an example of a profile that works on Postgres with `tbd`:

```yaml
gondor_security:
target: dev
outputs:
dev:
type: postgres
host: localhost
port: 5432
user: aragorn
password: arw3n1sBa3
database: gondor
schema: minas_tirith
threads: 8
connect_timeout: 10
```

### 🧱 Databricks

Databricks for now only works with Personal Access Token authentication. You can generate one in your Databricks account settings under the Developer tab. Put this in an environment variable called something like `DATABRICKS_TOKEN` (the default option). `tbd` will _not_ let you pass the value directly.
Expand All @@ -100,15 +163,15 @@ You'll need the information found in your SQL Warehouse's 'Connection details' t
If you're using a dbt profile, it will need to be configured for PAT authentication and referencing the token via an environment variable. The profile should look something like this:

```yaml
jaffle_shop_databricks:
gandalf_wizardy_co:
target: dev
outputs:
dev:
type: databricks
catalog: jaffle_shop
schema: dbt_winnie
host: 1234.cloud.databricks.com
http_path: /sql/1.0/warehouses/56789
catalog: maiar
schema: wizards
host: 1234.gandalf.databricks.com
http_path: /sql/1.0/warehouses/1234
token: "{{ env_var('DATABRICKS_TOKEN') }}"
threads: 16
```
Expand Down

0 comments on commit 417f9f3

Please sign in to comment.