Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support environment specific catalogs #3251

Open
mpcarter opened this issue Oct 14, 2024 · 0 comments
Open

Support environment specific catalogs #3251

mpcarter opened this issue Oct 14, 2024 · 0 comments
Labels
Improvement Improves existing functionality

Comments

@mpcarter
Copy link
Contributor

Add a new option for environment_suffix_target: catalog.

The desired behavior is that non-prod virtual environments would dynamically create new catalogs/databases. For example, sqlmesh plan dev results in:

mydb__dev.sushi.customers

The rationale is that this makes development with downstream consumers easier (e.g. reporting tools). Many tools will reference objects by schema.object. So, currently, if I want to test how a report or dataset behaves before promoting a model change to prod, I need to temporarily change the reference name to the schema or object inside the tool. If only the catalog/database name changed, then it is just a matter of temporarily updating the connection (which can usually be an environment variable) to use a different catalog.

Database engine limitations would mean only a subset of them could support this option. Primarily Snowflake, but also MotherDuck, Redshift, MSSQL, possibly Databricks.

@izeigerman izeigerman added the Improvement Improves existing functionality label Oct 14, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Improvement Improves existing functionality
Projects
None yet
Development

No branches or pull requests

2 participants