Skip to content

Commit

Permalink
Merge branch 'current' into wazi55/dbt-bigquery_python_model_build_se…
Browse files Browse the repository at this point in the history
…ction_revamp
  • Loading branch information
wazi55 authored Oct 18, 2023
2 parents 6a0add3 + 11d2093 commit 872a8df
Show file tree
Hide file tree
Showing 9 changed files with 95 additions and 17 deletions.
91 changes: 78 additions & 13 deletions website/docs/docs/cloud/billing.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,22 +11,73 @@ dbt Cloud offers a variety of [plans and pricing](https://www.getdbt.com/pricing

## How does dbt Cloud pricing work?

As a customer, you pay for the number of seats you have and the amount of usage consumed each month. Usage is based on the number of Successful Models Built, and seats are billed primarily on the amount of Developer licenses purchased. All billing computations are conducted in Coordinated Universal Time (UTC).
As a customer, you pay for the number of seats you have and the amount of usage consumed each month. Seats are billed primarily on the amount of Developer and Read licenses purchased. Usage is based on the number of [Successful Models Built](#what-counts-as-a-successful-model-built) and, if purchased and used, Semantic Layer Query Units subject to reasonable usage. All billing computations are conducted in Coordinated Universal Time (UTC).

### What counts as a seat license?

There are three types of possible seat licenses:

* **Developer** — for roles and permissions that require interaction with the dbt Cloud environment day-to-day.
* **Read-Only** — for access to view certain documents and reports.
* **IT** — for access to specific features related to account management (for example, configuring git integration).

### What counts as a Successful Model Built?

dbt Cloud considers a Successful Model Built as any model that is successfully built via a run through dbt Cloud’s orchestration functionality in a dbt Cloud deployment environment. Models are counted when built and run. This includes any jobs run via dbt Cloud's scheduler, CI builds (jobs triggered by pull requests), runs kicked off via the dbt Cloud API, and any successor dbt Cloud tools with similar functionality. This also includes models that are successfully built even when a run may fail to complete. For example, you may have a job that contains 100 models and on one of its runs, 51 models are successfully built and then the job fails. In this situation, only 51 models would be counted.
dbt Cloud considers a Successful Model Built as any <Term id="model">model</Term> that is successfully built via a run through dbt Cloud’s orchestration functionality in a dbt Cloud deployment environment. Models are counted when built and run. This includes any jobs run via dbt Cloud's scheduler, CI builds (jobs triggered by pull requests), runs kicked off via the dbt Cloud API, and any successor dbt Cloud tools with similar functionality. This also includes models that are successfully built even when a run may fail to complete. For example, you may have a job that contains 100 models and on one of its runs, 51 models are successfully built and then the job fails. In this situation, only 51 models would be counted.

Any models built in a dbt Cloud development environment (for example, via the IDE) do not count towards your usage. Tests, seeds, ephemeral models, and snapshots also do not count.

| What counts towards Successful Models Built | |
|---------------------------------------------|---------------------|
| View ||
| Table ||
| Incremental ||
| Ephemeral Models ||
| Tests ||
| Seeds ||
| Snapshots ||

### What counts as a seat license?
### What counts as a Query Unit?​

There are three types of possible seat licenses:
The dbt Semantic Layer, powered by MetricFlow, measures usage in distinct query units. Every successful request you make to render or run SQL to the Semantic Layer API counts as at least one query unit, even if no data is returned. If the query calculates or renders SQL for multiple metrics, each calculated metric will be counted as a query unit.
If a request to run a query is not executed successfully in the data platform or if a query results in an error without completion, it is not counted as a query unit. Requests for metadata from the Semantic Layer are also not counted as query units.

* **Developer** &mdash; for roles and permissions that require interaction with the dbt Cloud environment day-to-day.
* **Read-Only** &mdash; for access to view certain documents and reports.
* **IT** &mdash; for access to specific features related to account management (for example, configuring git integration).
Examples of query units include:

Querying one metric, grouping by one dimension → 1 query unit

```shell
dbt sl query --metrics revenue --group_by metric_time
```
Querying one metric, grouping by two dimensions → 1 query unit

```shell
dbt sl query --metrics revenue --group_by metric_time,user__country
```

Querying two metrics, grouping by two dimensions → 2 query units

```shell
dbt sl query --metrics revenue,gross_sales --group_by metric_time,user__country
```

Running an explain for one metric → 1 query unit

```shell
dbt sl query --metrics revenue --group_by metric_time --explain
```

Running an explain for two metrics → 2 query units

```shell
dbt sl query --metrics revenue,gross_sales --group_by metric_time --explain
```

Running a query for only dimensions such as dimension_values or a query with no metrics → 1 query unit

```shell
bt sl list dimension-values --dimension user__country
```

### Viewing usage in the product

Expand Down Expand Up @@ -61,7 +112,7 @@ All included successful models built numbers above reflect our most current pric

Team customers pay monthly via credit card for seats and usage, and accounts include 15,000 models monthly. Seats are charged upfront at the beginning of the month. If you add seats during the month, seats will be prorated and charged on the same day. Seats removed during the month will be reflected on the next invoice and are not eligible for refunds. You can change the credit card information and the number of seats from the billings section anytime. Accounts will receive one monthly invoice that includes the upfront charge for the seats and the usage charged in arrears from the previous month.

Usage is calculated and charged in arrears for the previous month. If you exceed 15,000 models in any month, you will be billed for additional usage on your next invoice. Additional use is billed at the rates on our [pricing page](https://www.getdbt.com/pricing).
Usage is calculated and charged in arrears for the previous month. If you exceed 15,000 models in any month, you will be billed for additional usage on your next invoice. Additional usage is billed at the rates on our [pricing page](https://www.getdbt.com/pricing).


Included models that are not consumed do not roll over to future months. You can estimate your bill with a simple formula:
Expand All @@ -70,15 +121,22 @@ Included models that are not consumed do not roll over to future months. You can

All included successful models built numbers above reflect our most current pricing and packaging. Based on your usage terms when you signed up for the Team Plan, the included model entitlements may be different from what’s reflected above.

:::note Legacy pricing plans

Customers who purchased the dbt Cloud Team plan before August 11, 2023, remain on a legacy pricing plan as long as their account is in good standing. The legacy pricing plan is based on seats and includes unlimited models subject to reasonable use. dbt Labs may institute use limits if reasonable use is exceeded. Additional features, upgrades, or updates may be subject to separate charges. Any changes to your current plan pricing will be communicated in advance according to our Terms of Use.
### Enterprise plan billing

As an Enterprise customer, you pay annually via invoice, monthly in arrears for additional usage (if applicable), and may benefit from negotiated usage rates. Please refer to your order form or contract for your specific pricing details, or [contact the account team](https://www.getdbt.com/contact-demo) with any questions.

### Legacy plans

Customers who purchased the dbt Cloud Team plan before August 11, 2023, remain on a legacy pricing plan as long as your account is in good standing. The legacy pricing plan is based on seats and includes unlimited models, subject to reasonable use.

:::note Legacy Semantic Layer

For customers using the legacy Semantic Layer with dbt_metrics package, this product will be deprecated in December 2023. Legacy users may choose to upgrade at any time to the revamped version, Semantic Layer powered by MetricFlow. The revamped version is available to most customers (see [prerequisites](/docs/use-dbt-semantic-layer/quickstart-sl#prerequisites)) for a limited time on a free trial basis, subject to reasonable use.

:::

### Enterprise plan billing
dbt Labs may institute use limits if reasonable use is exceeded. Additional features, upgrades, or updates may be subject to separate charges. Any changes to your current plan pricing will be communicated in advance according to our Terms of Use.

As an Enterprise customer, you pay annually via invoice, monthly in arrears for additional usage (if applicable), and may benefit from negotiated usage rates. Please refer to your order form or contract for your specific pricing details, or [contact the account team](https://www.getdbt.com/contact-demo) with any questions.

## Managing usage

Expand Down Expand Up @@ -193,3 +251,10 @@ _Yes. Your dbt Cloud account will be upgraded without impacting your existing pr

* How do I determine the right plan for me?
_The best option is to consult with our sales team. They'll help you figure out what is right for your needs. We also offer a free two-week trial on the Team plan._

* What are the Semantic Layer trial terms?
_Team and Enterprise customers can sign up for a free trial of the dbt Semantic Layer, powered by MetricFlow, for use of up to 1,000 query units per month. The trial will be available at least through January 2024. dbt Labs may extend the trial period in its sole discretion. During the trial period, we may reach out to discuss pricing options or ask for feedback. At the end of the trial, free access may be removed and a purchase may be required to continue use. dbt Labs reserves the right to change limits in a free trial or institute pricing when required or at any time in its sole discretion._

* What is the reasonable use limitation for the dbt Semantic Layer powered by MetricFlow during the trial?
_Each account will be limited to 1,000 Queried Metrics per month during the trial period and may be changed at the sole discretion of dbt Labs._

4 changes: 4 additions & 0 deletions website/docs/docs/cloud/cloud-cli-installation.md
Original file line number Diff line number Diff line change
Expand Up @@ -21,8 +21,12 @@ dbt commands are run against dbt Cloud's infrastructure and benefit from:
* Support for dbt Mesh ([cross-project `ref`](/docs/collaborate/govern/project-dependencies)),
* Significant platform improvements, to be released over the coming months.


## Prerequisites
The dbt Cloud CLI is available in all [deployment regions](/docs/cloud/about-cloud/regions-ip-addresses) and and for both multi-tenant and single-tenant accounts (Azure single-tenant not supported at this time).

You must be on dbt version 1.5 or higher. Refer to [dbt Cloud versions](/docs/dbt-versions/upgrade-core-in-cloud) to upgrade.

## Install dbt Cloud CLI

You can install the dbt Cloud CLI on the command line by using one of these methods:
Expand Down
2 changes: 1 addition & 1 deletion website/docs/docs/cloud/configure-cloud-cli.md
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@ import CloudCLIFlag from '/snippets/_cloud-cli-flag.md';
- You must have your [personal development credentials](/docs/dbt-cloud-environments#set-developer-credentials) set for that project. The dbt Cloud CLI will use these credentials, stored securely in dbt Cloud, to communicate with your data platform.
- You must [enroll](/docs/dbt-versions/experimental-features) in the dbt Cloud beta features.
- To enroll, navigate to your **Profile Settings** and enable the **Beta** flag under **Experimental Features**.

- You must be on dbt version 1.5 or higher. Refer to [dbt Cloud versions](/docs/dbt-versions/upgrade-core-in-cloud) to upgrade.

## Configure the dbt Cloud CLI

Expand Down
2 changes: 1 addition & 1 deletion website/docs/docs/cloud/dbt-cloud-ide/dbt-cloud-tips.md
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@ There are default keyboard shortcuts that can help make development more product
- Press Fn-F1 to view a full list of the editor shortcuts
- Command-O on macOS or Control-O on Windows to select a file to open
- Command-P/Command-Shift-P on macOS or Control-P/Control-Shift-P on Windows to see the command palette
- Hold Option-click-on-area on macOS or Hold-Alt-click-on-area on Windows to select multiple lines and perform a multi-edit. You can also press Command-E to perform this operation on the command line.
- Hold Option-click-on-area or press Shift-Option-Command on macOS or Hold-Alt-click-on-area on Windows to select multiple lines and perform a multi-edit. You can also press Command-E to perform this operation on the command line.
- Command-Enter on macOS or Control-Enter on Windows to Preview your code
- Command-Shift-Enter on macOS or Control-Shift-Enter on Windows to Compile
- Highlight a portion of code and use the above shortcuts to Preview or Compile code
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -23,6 +23,7 @@ It aims to bring the best of modeling and semantics to downstream applications b
<Lightbox src="/img/docs/dbt-cloud/semantic-layer/sl-architecture.jpg" width="80%" title="Use the universal dbt Semantic Layer to define and query metrics in integration tools."/>

The dbt Semantic Layer is available to [dbt Cloud Team or Enterprise](https://www.getdbt.com/) multi-tenant plans on dbt v1.6 or higher.
- Team and Enterprise customers can use 1,000 Queried Units per month for no additional cost on a limited trial basis, subject to reasonable use limitations. Refer to [Billing](/docs/cloud/billing#what-counts-as-a-query-unit) for more information.
- dbt Cloud Developer plans and dbt Core users can define metrics but won't be able to query them with integrated tools.


1 change: 0 additions & 1 deletion website/docs/quickstarts/bigquery-qs.md
Original file line number Diff line number Diff line change
Expand Up @@ -73,7 +73,6 @@ In order to let dbt connect to your warehouse, you'll need to generate a keyfile
1. Start the [GCP credentials wizard](https://console.cloud.google.com/apis/credentials/wizard). Make sure your new project is selected in the header. If you do not see your account or project, click your profile picture to the right and verify you are using the correct email account. For **Credential Type**:
- From the **Select an API** dropdown, choose **BigQuery API**
- Select **Application data** for the type of data you will be accessing
- Select **No, I’m not using them** and click **Next**.
- Click **Next** to create a new service account.
2. Create a service account for your new project from the [Service accounts page](https://console.cloud.google.com/projectselector2/iam-admin/serviceaccounts?supportedpurview=project). For more information, refer to [Create a service account](https://developers.google.com/workspace/guides/create-credentials#create_a_service_account) in the Google Cloud docs. As an example for this guide, you can:
- Type `dbt-user` as the **Service account name**
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -414,7 +414,7 @@ models:
columns:
- name: field
policy_tags:
- 'projects/<gcp-project>/locations/<location>/taxonomies/<organization>/policyTags/<tag>'
- 'projects/<gcp-project>/locations/<location>/taxonomies/<taxonomy>/policyTags/<tag>'
```

</File>
Expand Down
9 changes: 9 additions & 0 deletions website/docs/terms/model.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
---
id: model
title: Model
description: A model is an essential building block of the DAG
displayText: model
hoverSnippet: A model is an essential building block of the DAG
---

A model is an essential building block of the DAG that lives in a single file and contains logic that transforms data. This logic can be expressed as a SQL `select` statement or a Python dataframe operation. Models can be materialized in the warehouse in different ways &mdash; most of these materializations require models to be built in the warehouse.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.

0 comments on commit 872a8df

Please sign in to comment.