Skip to content

Commit

Permalink
Merge branch 'current' into auto-exposures-ga
Browse files Browse the repository at this point in the history
  • Loading branch information
runleonarun authored Dec 18, 2024
2 parents 940e828 + 388d2d7 commit 76fd57a
Show file tree
Hide file tree
Showing 10 changed files with 216 additions and 41 deletions.
1 change: 1 addition & 0 deletions website/dbt-versions.js
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,7 @@ exports.versions = [
},
{
version: "1.9",
customDisplay: "1.9 (Compatible)",
EOLDate: "2025-12-08",
},
{
Expand Down
63 changes: 59 additions & 4 deletions website/docs/docs/build/packages.md
Original file line number Diff line number Diff line change
Expand Up @@ -157,9 +157,57 @@ packages:

Where `name: 'dbt_utils'` specifies the subfolder of `dbt_packages` that's created for the package source code to be installed within.

### Private packages
## Private packages

#### SSH Key Method (Command Line only)
### Native private packages <Lifecycle status='beta'/>

dbt Cloud supports private packages from [supported](#prerequisites) Git repos leveraging an exisiting [configuration](/docs/cloud/git/git-configuration-in-dbt-cloud) in your environment. Previously, you had to configure a [token](#git-token-method) to retrieve packages from your private repos.

#### Prerequisites

To use native private packages, you must have one of the following Git providers configured in the **Integrations** section of your **Account settings**:
- [GitHub](/docs/cloud/git/connect-github)
- [Azure DevOps](/docs/cloud/git/connect-azure-devops)
- Support for GitLab is coming soon.


#### Configuration

Use the `private` key in your `packages.yml` or `dependencies.yml` to clone package repos using your existing dbt Cloud Git integration without having to provision an access token or create a dbt Cloud environment variable:

<File name="packages.yml">

```yaml
packages:
- private: dbt-labs/awesome_repo
- package: normal packages
[...]
```

</File>

You can pin private packages similar to regular dbt packages:

```yaml
packages:
- private: dbt-labs/awesome_repo
revision: "0.9.5" # Pin to a tag, branch, or complete 40-character commit hash
```

If you are using multiple Git integrations, disambiguate by adding the provider key:

```yaml
packages:
- private: dbt-labs/awesome_repo
provider: "github" # GitHub and Azure are currently supported. GitLab is coming soon.
```

With this method, you can retrieve private packages from an integrated Git provider without any additional steps to connect.

### SSH key method (command line only)
If you're using the Command Line, private packages can be cloned via SSH and an SSH key.

When you use SSH keys to authenticate to your git remote server, you don’t need to supply your username and password each time. Read more about SSH keys, how to generate them, and how to add them to your git provider here: [Github](https://docs.github.com/en/github/authenticating-to-github/connecting-to-github-with-ssh) and [GitLab](https://docs.gitlab.com/ee/user/ssh.html).
Expand All @@ -177,7 +225,14 @@ packages:
If you're using dbt Cloud, the SSH key method will not work, but you can use the [HTTPS Git Token Method](https://docs.getdbt.com/docs/build/packages#git-token-method).


#### Git token method
### Git token method

:::note

dbt Cloud has [native support](#native-private-packages) for Git hosted private packages with GitHub and Azure DevOps (GitLab coming soon). If you are using a supported [integrated Git environment](/docs/cloud/git/git-configuration-in-dbt-cloud), you no longer need to configure Git tokens to retrieve private packages.

:::

This method allows the user to clone via HTTPS by passing in a git token via an environment variable. Be careful of the expiration date of any token you use, as an expired token could cause a scheduled run to fail. Additionally, user tokens can create a challenge if the user ever loses access to a specific repo.


Expand Down Expand Up @@ -246,7 +301,7 @@ Read more about creating a Personal Access Token [here](https://confluence.atlas



#### Configure subdirectory for packaged projects
## Configure subdirectory for packaged projects

In general, dbt expects `dbt_project.yml` to be located as a top-level file in a package. If the packaged project is instead nested in a subdirectory—perhaps within a much larger mono repo—you can optionally specify the folder path as `subdirectory`. dbt will attempt a [sparse checkout](https://git-scm.com/docs/git-sparse-checkout) of just the files located within that subdirectory. Note that you must be using a recent version of `git` (`>=2.26.0`).

Expand Down
26 changes: 13 additions & 13 deletions website/docs/docs/dbt-versions/cloud-release-tracks.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,16 +14,18 @@ By moving your environments and jobs to release tracks you can get all the funct

## Which release tracks are available?

- **"Latest"** (available to all plans, formerly called "Versionless"): Provides a continuous release of the latest functionality in dbt Cloud. Includes early access to new features of the dbt framework before they're available in open source releases of dbt Core.
- <Lifecycle status="coming soon"/> **"Compatible"** (available to Team + Enterprise): Provides a monthly release aligned with the most recent open source versions of dbt Core and adapters, plus functionality exclusively available in dbt Cloud.
- <Lifecycle status="coming soon"/> **"Extended"** (available to Enterprise): Provides a delayed release of the previous month's "Compatible" release.
| Release track | Description | Plan availability | API value |
| ------------- | ----------- | ----------------- | --------- |
| **Latest** <br /> <Lifecycle status="GA"/> | Formerly called "Versionless", provides a continuous release of the latest functionality in dbt Cloud. Includes early access to new features of the dbt framework before they're available in open source releases of dbt Core. | All plans | `latest` (or `versionless`) |
| **Compatible** <Lifecycle status="preview"/> | Provides a monthly release aligned with the most recent open source versions of dbt Core and adapters, plus functionality exclusively available in dbt Cloud. | Team + Enterprise | `compatible` |
| **Extended** <Lifecycle status="preview"/> | The previous month's "Compatible" release. | Enterprise | `extended` |

The first "Compatible" release will be in December 2024, after the final release of dbt Core v1.9.0. For December 2024 only, the "Extended" release is the same as "Compatible." Starting in January 2025, "Extended" will be one month behind "Compatible."
The first "Compatible" release was on December 12, 2024, after the final release of dbt Core v1.9.0. For December 2024 only, the "Extended" release is the same as "Compatible." Starting in January 2025, "Extended" will be one month behind "Compatible."

To configure an environment in the [dbt Cloud Admin API](/docs/dbt-cloud-apis/admin-cloud-api) or [Terraform](https://registry.terraform.io/providers/dbt-labs/dbtcloud/latest) to use a release track, set `dbt_version` to the release track name:
- `latest` (formerly called `versionless`; the old name is still supported)
- `compatible` (available to Team + Enterprise)
- `extended` (available to Enterprise)
- `latest` (or `versionless`, the old name is still supported)
- `compatible`
- `extended`

## Which release track should I choose?

Expand Down Expand Up @@ -69,17 +71,15 @@ To learn more about how dbt Labs deploys stable dbt upgrades in a safe manner to

If you're running dbt version 1.6 or older, please know that your version of dbt Core has reached [end-of-life (EOL)](/docs/dbt-versions/core#eol-version-support) and is no longer supported. We strongly recommend that you update to a newer version as soon as reasonably possible.

dbt Labs has extended the critical support period of dbt Core v1.7 for dbt Cloud Enterprise customers to January 31, 2024. At that point, we will be asking all customers to select a Release Track for receiving ongoing updates to dbt in dbt Cloud.
dbt Labs has extended the critical support period of dbt Core v1.7 for dbt Cloud Enterprise customers to March 2025. At that point, we will be encouraging all customers to select a Release Track for ongoing updates in dbt Cloud.

<Expandable alt_header="I'm using an older version of dbt in dbt Cloud. What should I do? What happens if I do nothing?" >

If you're running dbt version v1.6 or older, please know that your version of dbt Core has reached [end-of-life (EOL)](/docs/dbt-versions/core#eol-version-support) and is no longer supported. We strongly recommend that you update to a newer version as soon as reasonably possible.

dbt Labs has extended the "Critical Support" period of dbt Core v1.7 for dbt Cloud Enterprise customers while we work through the migration with those customers to Release Tracks. In the meantime, this means that v1.7 will continue to be accessible in dbt Cloud for Enteprise customers, jobs and environments on v1.7 for those customers will not be automatically migrated to "Latest," and dbt Labs will continue to fix critical bugs and security issues.

dbt Cloud accounts on the Developer and Team plans will be migrated to the "Latest" release track after November 1, 2024. If you know that your project will not be compatible with the upgrade, for one of the reasons described here, or a different reason in your own testing, you should [contact dbt Cloud support](https://docs.getdbt.com/docs/dbt-support#dbt-cloud-support) to request an extension.

If your account has been migrated to the "Latest" release track, and you are seeing net-new failures in your scheduled dbt jobs, you should also [contact dbt Cloud support](https://docs.getdbt.com/docs/dbt-support#dbt-cloud-support) to request an extension.
Starting in October 2024, dbt Cloud accounts on the Developer and Team plans have been migrated to release tracks from older dbt Core versions. If your account was migrated to the "Latest" release track and you notice new failures in scheduled jobs, please [contact dbt Cloud support](https://docs.getdbt.com/docs/dbt-support#dbt-cloud-support) to report the problem or request an extension.

</Expandable>

Expand Down Expand Up @@ -134,8 +134,8 @@ In 2024, we've changed the way that new dbt functionality is made available for

Opting into a release cadence with automated upgrades is required for accessing any new functionality that we've released in 2024, and going forward.

We continue to release new minor versions of dbt Core (OSS), including v1.9 which will be available later this year. When we do, it will be a subset of the functionality that's already available to dbt Cloud customers, and always after the functionality has been available in dbt Cloud.
We continue to release new minor versions of dbt Core (OSS). We most recently released dbt Core v1.9 on December 9, 2024. These releases always include a subset of the functionality that's already available to dbt Cloud customers, and always after the functionality has been available in dbt Cloud.

</Expandable>

If you have comments or concerns, we’re happy to help. If you’re an existing dbt Cloud customer, you may reach out to your account team or [contact support](/docs/dbt-support#dbt-cloud-support).
If you have comments or concerns, we’re happy to help. If you’re an existing dbt Cloud customer, you may reach out to your account team or [contact support](/docs/dbt-support#dbt-cloud-support).
48 changes: 41 additions & 7 deletions website/docs/docs/dbt-versions/compatible-track-changelog.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,9 +4,9 @@ sidebar_label: "Compatible Track Changelog"
description: "The Compatible release track updates once per month, and it includes up-to-date open source versions as of the monthly release."
---

:::info Coming soon
:::info Preview

The "Compatible" and "Extended" release tracks will be available in Preview to eligible dbt Cloud accounts in December 2024.
The "Compatible" and "Extended" [release tracks](/docs/dbt-versions/cloud-release-tracks) are available in Preview. Access will be rolling out to dbt Cloud accounts on eligible plans during the week of December 16-20, 2024.

:::

Expand All @@ -20,8 +20,42 @@ For more information, see [release tracks](/docs/dbt-versions/cloud-release-trac

## December 2024

Planned release: December 11-13

This release will include functionality from `dbt-core==1.9.0` and the most recent versions of all adapters supported in dbt Cloud. After the Compatible release is cut, we will update with:
- exact versions of open source dbt packages
- changelog notes concerning functionality specific to dbt Cloud
Release date: December 12, 2024

This release includes functionality from the following versions of dbt Core OSS:
```
dbt-core==1.9.0
# shared interfaces
dbt-adapters==1.10.4
dbt-common==1.14.0
dbt-semantic-interfaces==0.7.4
# adapters
dbt-athena==1.9.0
dbt-bigquery==1.9.0
dbt-databricks==1.9.0
dbt-fabric==1.8.8
dbt-postgres==1.9.0
dbt-redshift==1.9.0
dbt-snowflake==1.9.0
dbt-spark==1.9.0
dbt-synapse==1.8.2
dbt-teradata==1.8.2
dbt-trino==1.8.5
```

Changelogs:
- [dbt-core 1.9.0](https://github.com/dbt-labs/dbt-core/blob/1.9.latest/CHANGELOG.md#dbt-core-190---december-09-2024)
- [dbt-adapters 1.10.4](https://github.com/dbt-labs/dbt-adapters/blob/main/CHANGELOG.md#dbt-adapters-1104---november-11-2024)
- [dbt-common 1.14.0](https://github.com/dbt-labs/dbt-common/blob/main/CHANGELOG.md)
- [dbt-bigquery 1.9.0](https://github.com/dbt-labs/dbt-bigquery/blob/1.9.latest/CHANGELOG.md#dbt-bigquery-190---december-09-2024)
- [dbt-databricks 1.9.0](https://github.com/databricks/dbt-databricks/blob/main/CHANGELOG.md#dbt-databricks-190-december-9-2024)
- [dbt-fabric 1.8.8](https://github.com/microsoft/dbt-fabric/blob/v1.8.latest/CHANGELOG.md)
- [dbt-postgres 1.9.0](https://github.com/dbt-labs/dbt-postgres/blob/main/CHANGELOG.md#dbt-postgres-190---december-09-2024)
- [dbt-redshift 1.9.0](https://github.com/dbt-labs/dbt-redshift/blob/1.9.latest/CHANGELOG.md#dbt-redshift-190---december-09-2024)
- [dbt-snowflake 1.9.0](https://github.com/dbt-labs/dbt-snowflake/blob/1.9.latest/CHANGELOG.md#dbt-snowflake-190---december-09-2024)
- [dbt-spark 1.9.0](https://github.com/dbt-labs/dbt-spark/blob/1.9.latest/CHANGELOG.md#dbt-spark-190---december-10-2024)
- [dbt-synapse 1.8.2](https://github.com/microsoft/dbt-synapse/blob/v1.8.latest/CHANGELOG.md)
- [dbt-teradata 1.8.2](https://github.com/Teradata/dbt-teradata/releases/tag/v1.8.2)
- [dbt-trino 1.8.5](https://github.com/starburstdata/dbt-trino/blob/master/CHANGELOG.md#dbt-trino-185---december-11-2024)
4 changes: 2 additions & 2 deletions website/docs/docs/deploy/about-ci.md
Original file line number Diff line number Diff line change
Expand Up @@ -19,9 +19,9 @@ Refer to the guide [Get started with continuous integration tests](/guides/set-u
icon="dbt-bit"/>

<Card
title="Advanced CI (preview)"
title="Advanced CI"
body="Compare the differences between what's in the production environment and the pull request before merging those changes, ensuring that you're always shipping trusted data products."
link="/docs/deploy/advanced-ci"
icon="dbt-bit"/>

</div><br />
</div><br />
15 changes: 11 additions & 4 deletions website/docs/docs/deploy/deploy-jobs.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@ You can use deploy jobs to build production data assets. Deploy jobs make it eas
- Job run details, including run timing, [model timing data](/docs/deploy/run-visibility#model-timing), and [artifacts](/docs/deploy/artifacts)
- Detailed run steps with logs and their run step statuses

You can create a deploy job and configure it to run on [scheduled days and times](#schedule-days) or enter a [custom cron schedule](#cron-schedule).
You can create a deploy job and configure it to run on [scheduled days and times](#schedule-days), enter a [custom cron schedule](#cron-schedule), or [trigger the job after another job completes](#trigger-on-job-completion).


## Prerequisites
Expand Down Expand Up @@ -115,11 +115,18 @@ Examples of cron job schedules:

### Trigger on job completion <Lifecycle status="team,enterprise" />

To _chain_ deploy jobs together, enable the **Run when another job finishes** option and specify the upstream (parent) job that, when it completes, will trigger your job. You can also use the [Create Job API](/dbt-cloud/api-v2#/operations/Create%20Job) to do this.
To _chain_ deploy jobs together:
1. In the **Triggers** section, enable the **Run when another job finishes** option.
2. Select the project that has the deploy job you want to run after completion.
3. Specify the upstream (parent) job that, when completed, will trigger your job.
- You can also use the [Create Job API](/dbt-cloud/api-v2#/operations/Create%20Job) to do this.
4. In the **Completes on** option, select the job run status(es) that will [enqueue](/docs/deploy/job-scheduler#scheduler-queue) the deploy job.

You can set up a configuration where an upstream job triggers multiple downstream (child) jobs and jobs in other projects. You must have proper [permissions](/docs/cloud/manage-access/enterprise-permissions#project-role-permissions) to the project and job to configure the trigger.
<Lightbox src="/img/docs/deploy/deploy-job-completion.jpg" width="100%" title="Example of Trigger on job completion on the Deploy job page"/>

For jobs that are triggered to run by another job, a link to the upstream job run is available from your [job's run details](/docs/deploy/run-visibility#job-run-details).
5. You can set up a configuration where an upstream job triggers multiple downstream (child) jobs and jobs in other projects. You must have proper [permissions](/docs/cloud/manage-access/enterprise-permissions#project-role-permissions) to the project and job to configure the trigger.

If another job triggers your job to run, you can find a link to the upstream job in the [run details section](/docs/deploy/run-visibility#job-run-details).

## Related docs

Expand Down
2 changes: 1 addition & 1 deletion website/docs/docs/deploy/deployment-overview.md
Original file line number Diff line number Diff line change
Expand Up @@ -33,7 +33,7 @@ Learn how to use dbt Cloud's features to help your team ship timely and quality

<Card
title="Deploy jobs"
body="Create and schedule jobs for the dbt Cloud scheduler to run."
body="Create and schedule jobs for the job scheduler to run. <br /><br />Runs on a schedule, by API, or after another job completes."
link="/docs/deploy/deploy-jobs"
icon="dbt-bit"/>

Expand Down
Loading

0 comments on commit 76fd57a

Please sign in to comment.