Skip to content

Commit

Permalink
Merge branch 'current' into wazi55/dbt-bigquery_python_model_build_se…
Browse files Browse the repository at this point in the history
…ction_revamp
  • Loading branch information
wazi55 authored Oct 12, 2023
2 parents f2a9a4b + 9308a06 commit ab391be
Show file tree
Hide file tree
Showing 68 changed files with 1,000 additions and 271 deletions.
2 changes: 1 addition & 1 deletion .github/ISSUE_TEMPLATE/a-improve-docs.yml
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ body:
- type: markdown
attributes:
value: |
* You can ask questions or submit ideas for the dbt docs in [Discussions](https://github.com/dbt-labs/docs.getdbt.com/discussions)
* You can ask questions or submit ideas for the dbt docs in [Issues](https://github.com/dbt-labs/docs.getdbt.com/issues/new/choose)
* Before you file an issue read the [Contributing guide](https://github.com/dbt-labs/docs.getdbt.com#contributing).
* Check to make sure someone hasn't already opened a similar [issue](https://github.com/dbt-labs/docs.getdbt.com/issues).
Expand Down
2 changes: 1 addition & 1 deletion .github/ISSUE_TEMPLATE/improve-the-site.yml
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ body:
- type: markdown
attributes:
value: |
* You can ask questions or submit ideas for the dbt docs in [Discussions](https://github.com/dbt-labs/docs.getdbt.com/discussions)
* You can ask questions or submit ideas for the dbt docs in [Issues](https://github.com/dbt-labs/docs.getdbt.com/issues/new/choose)
* Before you file an issue read the [Contributing guide](https://github.com/dbt-labs/docs.getdbt.com#contributing).
* Check to make sure someone hasn't already opened a similar [issue](https://github.com/dbt-labs/docs.getdbt.com/issues).
Expand Down
16 changes: 8 additions & 8 deletions .github/workflows/label.yml
Original file line number Diff line number Diff line change
Expand Up @@ -2,19 +2,18 @@ name: Add/Remove Labels

on:
pull_request_target:
types: [ opened ]
types: [opened]

jobs:
add_new_contributor_label:
if: github.event.action == 'opened'
permissions:
contents: read
pull-requests: write
runs-on: ubuntu-latest
steps:
- uses: actions/github-script@v6
with:
script: |
- name: Add new contributor label
uses: actions/github-script@v6
with:
github-token: ${{ secrets.DOCS_SECRET }}
script: |
const creator = context.payload.sender.login;
const opts = github.rest.issues.listForRepo.endpoint.merge({
...context.issue,
Expand All @@ -37,10 +36,11 @@ jobs:
}
if (!isAlreadyContributor) {
console.log('Adding label: new contributor');
await github.rest.issues.addLabels({
issue_number: context.issue.number,
owner: context.repo.owner,
repo: context.repo.repo,
labels: ['new contributor'],
});
}
}
4 changes: 2 additions & 2 deletions .github/workflows/labeler.yml
Original file line number Diff line number Diff line change
Expand Up @@ -5,8 +5,8 @@

name: "Pull Request Labeler"
on:
- pull_request_target

pull_request_target:
types: [opened]
jobs:
triage:
permissions:
Expand Down
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@ Creating an inclusive and equitable environment for our documents is more import
We welcome contributions from community members to this repo:
- **Fixes**: When you notice an error, you can use the `Edit this page` button at the bottom of each page to suggest a change.
- **New documentation**: If you contributed code in [dbt-core](https://github.com/dbt-labs/dbt-core), we encourage you to also write the docs here! Please reach out in the dbt community if you need help finding a place for these docs.
- **Major rewrites**: You can [file an issue](https://github.com/dbt-labs/docs.getdbt.com/issues/new?assignees=&labels=content%2Cimprovement&template=improve-docs.yml) or [start a discussion](https://github.com/dbt-labs/docs.getdbt.com/discussions) to propose ideas for a content area that requires attention.
- **Major rewrites**: You can [file an issue](https://github.com/dbt-labs/docs.getdbt.com/issues/new/choose) to propose ideas for a content area that requires attention.

You can use components documented in the [docusaurus library](https://v2.docusaurus.io/docs/markdown-features/).

Expand Down
4 changes: 2 additions & 2 deletions website/docs/community/resources/oss-expectations.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ title: "Expectations for OSS contributors"

Whether it's a dbt package, a plugin, `dbt-core`, or this very documentation site, contributing to the open source code that supports the dbt ecosystem is a great way to level yourself up as a developer, and to give back to the community. The goal of this page is to help you understand what to expect when contributing to dbt open source software (OSS). While we can only speak for our own experience as open source maintainers, many of these guidelines apply when contributing to other open source projects, too.

Have you seen things in other OSS projects that you quite like, and think we could learn from? [Open a discussion on the Developer Hub](https://github.com/dbt-labs/docs.getdbt.com/discussions/new), or start a conversation in the dbt Community Slack (for example: `#community-strategy`, `#dbt-core-development`, `#package-ecosystem`, `#adapter-ecosystem`). We always appreciate hearing from you!
Have you seen things in other OSS projects that you quite like, and think we could learn from? [Open a discussion on the dbt Community Forum](https://discourse.getdbt.com), or start a conversation in the dbt Community Slack (for example: `#community-strategy`, `#dbt-core-development`, `#package-ecosystem`, `#adapter-ecosystem`). We always appreciate hearing from you!

## Principles

Expand Down Expand Up @@ -51,7 +51,7 @@ An issue could be a bug you’ve identified while using the product or reading t

### Best practices for issues

- Issues are **not** for support / troubleshooting / debugging help. Please [open a discussion on the Developer Hub](https://github.com/dbt-labs/docs.getdbt.com/discussions/new), so other future users can find and read proposed solutions. If you need help formulating your question, you can post in the `#advice-dbt-help` channel in the [dbt Community Slack](https://www.getdbt.com/community/).
- Issues are **not** for support / troubleshooting / debugging help. Please [open a discussion on the dbt Community Forum](https://discourse.getdbt.com), so other future users can find and read proposed solutions. If you need help formulating your question, you can post in the `#advice-dbt-help` channel in the [dbt Community Slack](https://www.getdbt.com/community/).
- Always search existing issues first, to see if someone else had the same idea / found the same bug you did.
- Many repositories offer templates for creating issues, such as when reporting a bug or requesting a new feature. If available, please select the relevant template and fill it out to the best of your ability. This will help other people understand your issue and respond.

Expand Down
4 changes: 2 additions & 2 deletions website/docs/docs/build/groups.md
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,7 @@ This functionality is new in v1.5.

## About groups

A group is a collection of nodes within a dbt DAG. Groups are named, and every group has an `owner`. They enable intentional collaboration within and across teams by restricting [access to private](/reference/resource-properties/access) models.
A group is a collection of nodes within a dbt DAG. Groups are named, and every group has an `owner`. They enable intentional collaboration within and across teams by restricting [access to private](/reference/resource-configs/access) models.

Group members may include models, tests, seeds, snapshots, analyses, and metrics. (Not included: sources and exposures.) Each node may belong to only one group.

Expand Down Expand Up @@ -94,7 +94,7 @@ select ...

### Referencing a model in a group

By default, all models within a group have the `protected` [access modifier](/reference/resource-properties/access). This means they can be referenced by downstream resources in _any_ group in the same project, using the [`ref`](/reference/dbt-jinja-functions/ref) function. If a grouped model's `access` property is set to `private`, only resources within its group can reference it.
By default, all models within a group have the `protected` [access modifier](/reference/resource-configs/access). This means they can be referenced by downstream resources in _any_ group in the same project, using the [`ref`](/reference/dbt-jinja-functions/ref) function. If a grouped model's `access` property is set to `private`, only resources within its group can reference it.

<File name='models/schema.yml'>

Expand Down
3 changes: 1 addition & 2 deletions website/docs/docs/build/materializations.md
Original file line number Diff line number Diff line change
Expand Up @@ -140,8 +140,7 @@ required with incremental materializations
less configuration options available, see your database platform's docs for more details
* Materialized views may not be supported by every database platform
* **Advice:**
* Consider materialized views for use cases where incremental models are sufficient,
but you would like the data platform to manage the incremental logic and refresh.
* Consider materialized views for use cases where incremental models are sufficient, but you would like the data platform to manage the incremental logic and refresh.

## Python materializations

Expand Down
4 changes: 2 additions & 2 deletions website/docs/docs/build/metrics-overview.md
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@ The keys for metrics definitions are:
| `description` | Provide the description for your metric. | Optional |
| `type` | Define the type of metric, which can be `simple`, `ratio`, `cumulative`, or `derived`. | Required |
| `type_params` | Additional parameters used to configure metrics. `type_params` are different for each metric type. | Required |
| `configs` | Provide the specific configurations for your metric. | Optional |
| `config` | Provide the specific configurations for your metric. | Optional |
| `label` | The display name for your metric. This value will be shown in downstream tools. | Required |
| `filter` | You can optionally add a filter string to any metric type, applying filters to dimensions, entities, or time dimensions during metric computation. Consider it as your WHERE clause. | Optional |
| `meta` | Additional metadata you want to add to your metric. | Optional |
Expand All @@ -31,7 +31,7 @@ metrics:
type: the type of the metric ## Required
type_params: ## Required
- specific properties for the metric type
configs: here for `enabled` ## Optional
config: here for `enabled` ## Optional
label: The display name for your metric. This value will be shown in downstream tools. ## Required
filter: | ## Optional
{{ Dimension('entity__name') }} > 0 and {{ Dimension(' entity__another name') }} is not
Expand Down
42 changes: 34 additions & 8 deletions website/docs/docs/build/semantic-models.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,24 +8,28 @@ sidebar_label: Semantic models
tags: [Metrics, Semantic Layer]
---

Semantic models serve as the foundation for defining data in MetricFlow, which powers the dbt Semantic Layer. You can think of semantic models as nodes in your semantic graph, connected via entities as edges. MetricFlow takes semantic models defined in YAML configuration files as inputs and creates a semantic graph that can be used to query metrics.
Semantic models are the foundation for data definition in MetricFlow, which powers the dbt Semantic Layer:

Each semantic model corresponds to a dbt model in your DAG. Therefore you will have one YAML config for each semantic model in your dbt project. You can create multiple semantic models out of a single dbt model, as long as you give each semantic model a unique name.

You can configure semantic models in your dbt project directory in a `YAML` file. Depending on your project structure, you can nest semantic models under a `metrics:` folder or organize them under project sources.
- Think of semantic models as nodes connected by entities in a semantic graph.
- MetricFlow uses YAML configuration files to create this graph for querying metrics.
- Each semantic model corresponds to a dbt model in your DAG, requiring a unique YAML configuration for each semantic model.
- You can create multiple semantic models from a single dbt model, as long as you give each semantic model a unique name.
- Configure semantic models in a YAML file within your dbt project directory.
- Organize them under a `metrics:` folder or within project sources as needed.

Semantic models have 6 components and this page explains the definitions with some examples:

| Component | Description | Type |
| --------- | ----------- | ---- |
| [Name](#name) | Unique name for the semantic model | Required |
| [Name](#name) | Choose a unique name for the semantic model. Avoid using double underscores (__) in the name as they're not supported. | Required |
| [Description](#description) | Includes important details in the description | Optional |
| [Model](#model) | Specifies the dbt model for the semantic model using the `ref` function | Required |
| [Defaults](#defaults) | The defaults for the model, currently only `agg_time_dimension` is supported. | Required |
| [Entities](#entities) | Uses the columns from entities as join keys and indicate their type as primary, foreign, or unique keys with the `type` parameter | Required |
| [Primary Entity](#primary-entity) | If a primary entity exists, this component is Optional. If the semantic model has no primary entity, then this property is required. | Optional |
| [Dimensions](#dimensions) | Different ways to group or slice data for a metric, they can be `time` or `categorical` | Required |
| [Measures](#measures) | Aggregations applied to columns in your data model. They can be the final metric or used as building blocks for more complex metrics | Optional |
| Label | The display name for your semantic model `node`, `dimension`, `entity`, and/or `measures` | Optional |

## Semantic models components

Expand Down Expand Up @@ -105,9 +109,32 @@ semantic_models:
type: categorical
```
<VersionBlock firstVersion="1.7">
Semantic models support configs in either the schema file or at the project level.
Semantic model config in `models/semantic.yml`:
```yml
semantic_models:
- name: orders
config:
enabled: true | false
group: some_group
```

Semantic model config in `dbt_project.yml`:
```yml
semantic_models:
my_project_name:
+enabled: true | false
+group: some_group
```

</VersionBlock>

### Name

Define the name of the semantic model. You must define a unique name for the semantic model. The semantic graph will use this name to identify the model, and you can update it at any time.
Define the name of the semantic model. You must define a unique name for the semantic model. The semantic graph will use this name to identify the model, and you can update it at any time. Avoid using double underscores (__) in the name as they're not supported.

### Description

Expand Down Expand Up @@ -205,8 +232,7 @@ For semantic models with a measure, you must have a [primary time group](/docs/b
| `agg` | dbt supports the following aggregations: `sum`, `max`, `min`, `count_distinct`, and `sum_boolean`. | Required |
| `expr` | You can either reference an existing column in the table or use a SQL expression to create or derive a new one. | Optional |
| `non_additive_dimension` | Non-additive dimensions can be specified for measures that cannot be aggregated over certain dimensions, such as bank account balances, to avoid producing incorrect results. | Optional |
| `create_metric`* | You can create a metric directly from a measure with create_metric: True and specify its display name with create_metric_display_name. | Optional |
_*Coming soon_
| `create_metric` | You can create a metric directly from a measure with `create_metric: True` and specify its display name with create_metric_display_name. Default is false. | Optional |


import SetUpPages from '/snippets/_metrics-dependencies.md';
Expand Down
Loading

0 comments on commit ab391be

Please sign in to comment.