Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add visual editor docs #5603

Merged
merged 40 commits into from
Nov 12, 2024
Merged
Show file tree
Hide file tree
Changes from 16 commits
Commits
Show all changes
40 commits
Select commit Hold shift + click to select a range
786ba98
add visual editor content
mirnawong1 Jun 4, 2024
86fe892
add to sidebar
mirnawong1 Jun 4, 2024
a614077
add version control
mirnawong1 Jun 4, 2024
3237955
add edit img
mirnawong1 Jun 4, 2024
3b5d9e8
remove lowcode
mirnawong1 Jun 4, 2024
768f2ea
add limitation
mirnawong1 Jun 4, 2024
7e27eef
add limitation
mirnawong1 Jun 4, 2024
b1ed271
limitation
mirnawong1 Jun 4, 2024
a8f296a
Merge branch 'current' into mwong-add-visual-editor
mirnawong1 Jun 4, 2024
1aaefd9
Update use-visual-editor.md
mirnawong1 Jun 4, 2024
5bb39bd
Update website/docs/docs/cloud/visual-editor-interface.md
mirnawong1 Jun 4, 2024
6994f16
update img and sidebar
mirnawong1 Jun 4, 2024
5f69a73
Update website/docs/docs/cloud/use-visual-editor.md
mirnawong1 Jun 4, 2024
054f070
Update use-visual-editor.md
mirnawong1 Jun 4, 2024
9935646
Update visual-editor.md
mirnawong1 Jun 4, 2024
71c03a4
Update website/docs/docs/cloud/visual-editor-interface.md
mirnawong1 Jun 4, 2024
d5efafb
Update website/docs/docs/cloud/use-visual-editor.md
mirnawong1 Jun 4, 2024
16a9585
add introductory text below main header
mirnawong1 Jun 5, 2024
8f347a2
change word
mirnawong1 Jun 5, 2024
80b6837
Merge branch 'current' into mwong-add-visual-editor
mirnawong1 Jun 5, 2024
eca861e
fix sidebar
mirnawong1 Jun 5, 2024
913ded5
Merge branch 'current' into mwong-add-visual-editor
mirnawong1 Jun 5, 2024
5745f5f
update titles
mirnawong1 Jun 5, 2024
31a67eb
Merge branch 'current' into mwong-add-visual-editor
mirnawong1 Jun 5, 2024
a17d8d1
Merge branch 'current' into mwong-add-visual-editor
mirnawong1 Jun 7, 2024
b4a07d3
Update website/docs/docs/cloud/use-visual-editor.md
mirnawong1 Jun 7, 2024
b966e5e
Update website/docs/docs/cloud/use-visual-editor.md
mirnawong1 Jun 7, 2024
c5dfd8c
updates
mirnawong1 Jun 7, 2024
5c1a7c8
remove from sidebar
mirnawong1 Jun 7, 2024
301335c
Merge branch 'current' into mwong-add-visual-editor
mirnawong1 Jun 25, 2024
e61a855
Merge branch 'current' into mwong-add-visual-editor
mirnawong1 Jun 26, 2024
c413941
Merge branch 'current' into mwong-add-visual-editor
mirnawong1 Oct 17, 2024
37a1a9c
Update website/docs/docs/cloud/use-visual-editor.md
matthewshaver Nov 8, 2024
891c129
Apply suggestions from code review
matthewshaver Nov 8, 2024
5ab2a15
Apply suggestions from code review
matthewshaver Nov 8, 2024
9bc218e
Apply suggestions from code review
matthewshaver Nov 8, 2024
658025a
Merge branch 'current' into mwong-add-visual-editor
mirnawong1 Nov 11, 2024
4c6a448
Updated images
matthewshaver Nov 12, 2024
23d4994
Apply suggestions from code review
matthewshaver Nov 12, 2024
a7978ef
Merge branch 'current' into mwong-add-visual-editor
matthewshaver Nov 12, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 7 additions & 1 deletion website/docs/docs/cloud/about-cloud/about-dbt-cloud.md
Original file line number Diff line number Diff line change
Expand Up @@ -28,6 +28,12 @@ dbt Cloud's [flexible plans](https://www.getdbt.com/pricing/) and features make
link="/docs/cloud/dbt-cloud-ide/develop-in-the-cloud"
icon="dbt-bit"/>

<Card
title="Visual editor (beta)"
body="Develop with the visual editor, a seamless drag-and-drop experience that helps analysts quickly create and visualize dbt models in dbt Cloud."
link="/docs/cloud/visual-editor"
icon="dbt-bit"/>

<Card
title="Manage environments"
body="Set up and manage separate production and development environments in dbt Cloud to help engineers develop and test code more efficiently, without impacting users or data."
Expand Down Expand Up @@ -99,6 +105,6 @@ dbt Cloud's [flexible plans](https://www.getdbt.com/pricing/) and features make
## Related docs

- [dbt Cloud plans and pricing](https://www.getdbt.com/pricing/)
- [Quickstart guides](/guides)
- [Quickstart guides](/docs/get-started-dbt)
- [dbt Cloud IDE](/docs/cloud/dbt-cloud-ide/develop-in-the-cloud)

10 changes: 8 additions & 2 deletions website/docs/docs/cloud/about-develop-dbt.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,9 +9,9 @@ hide_table_of_contents: true

Develop dbt projects using dbt Cloud, which offers a fast and reliable way to work on your dbt project. It runs dbt Core in a hosted (single or multi-tenant) environment.

You can develop in your browser using an integrated development environment (IDE) or in a dbt Cloud-powered command line interface (CLI).
You can develop in your browser using an integrated development environment (IDE), a dbt Cloud-powered command line interface (CLI), or visual editor.

<div className="grid--2-col" >
<div className="grid--3-col" >

<Card
title="dbt Cloud CLI"
Expand All @@ -25,6 +25,12 @@ You can develop in your browser using an integrated development environment (IDE
link="/docs/cloud/dbt-cloud-ide/develop-in-the-cloud"
icon="dbt-bit"/>

<Card
title="Visual editor (beta)"
body="Develop with the visual editor, a seamless drag-and-drop experience that helps analysts quickly create and visualize dbt models in dbt Cloud."
link="/docs/cloud/visual-editor"
icon="dbt-bit"/>

</div><br />

To get started, you'll need a [dbt Cloud](https://www.getdbt.com/signup) account and a developer seat. For a more comprehensive guide about developing in dbt, refer to the [quickstart guides](/docs/get-started-dbt).
74 changes: 74 additions & 0 deletions website/docs/docs/cloud/use-visual-editor.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,74 @@
---
title: "Use the visual editor"
mirnawong1 marked this conversation as resolved.
Show resolved Hide resolved
id: use-visual-editor
sidebar_label: "Use the visual editor"
description: "Access and use the visual editor to create or edit dbt models through a visual, drag-and-drop experience inside of dbt Cloud."
---

# Use the visual editor <Lifecycle status='beta'/>

:::tip Beta feature
The visual editor provides users with a seamless and drag-and-drop experience inside of dbt Cloud. It's available in private beta for [dbt Cloud Enterprise accounts](https://www.getdbt.com/pricing). To join the private beta, [register your interest](https://docs.google.com/forms/d/e/1FAIpQLScPjRGyrtgfmdY919Pf3kgqI5E95xxPXz-8JoVruw-L9jVtxg/viewform) or reach out to your account team to begin this process.
mirnawong1 marked this conversation as resolved.
Show resolved Hide resolved
:::

nghi-ly marked this conversation as resolved.
Show resolved Hide resolved
## Prerequisites
- You have an active [dbt Cloud Enterprise](https://www.getdbt.com/pricing) account.
mirnawong1 marked this conversation as resolved.
Show resolved Hide resolved
- You have a [developer license](/docs/cloud/manage-access/seats-and-users) ??? WHO can access editor? dev license? read-only?
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

dev license required and dev credentials - yes?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

dev license
keep on latest

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can't create source model from this. need to have prod run that has sources created.

- You have a dbt Cloud project created already with dev credentials set up.
- Successful job run on Production or Staging [environment](/docs/dbt-cloud-environments)
- Have AI-powered features toggle enabled

## Access visual editor

Before accessing the editor, you should have a dbt Cloud project already set up. This includes a Git repository, data platform connection, environments, and developer credentials. If you don't have this set up, please contact your dbt Cloud Admin.

To access the visual editor:
- Type in the following URL, replacing the ACCOUNT_ID and ENVIRONMENT_ID with your own account and environment ID: `https://ACCESS_URL/visual-editor/ACCOUNT_ID/env/ENVIRONMENT_ID/`
matthewshaver marked this conversation as resolved.
Show resolved Hide resolved
- The environment ID must have had runs that generated catalogs in it.

- For example, if my region is North America multi-tenant, account ID is 10, environment ID with a generated catalog run is 100, my URL should be:

- `https://cloud.getdbt.com/visual-editor/10/env/100/`

<Lightbox src="/img/docs/dbt-cloud/visual-editor/visual-editor.jpg" width="80%" title="Visual editor interface that contains a node toolbar and canvas." />
matthewshaver marked this conversation as resolved.
Show resolved Hide resolved

## Create a model
To create a new dbt SQL model, click on **Create a new model** and perform the following steps:

1. Drag an operator from the operator toolbar and drop it onto the canvas.
2. Click on the operator to open its configuration panel:
- **Table**: Select the table and columns you want to use.
matthewshaver marked this conversation as resolved.
Show resolved Hide resolved
- **Join**: Define the join conditions and choose columns from both tables.
- **Select**: Pick the columns you need from the table.
matthewshaver marked this conversation as resolved.
Show resolved Hide resolved
- **Aggregate**: Specify the aggregation functions and the columns they apply to.
- **Formula**: Add the formula to create a new column. Use the built-AI code generator to help generate SQL code by clicking on the question mark (?) icon. Enter your prompt and wait to see the results.
- **Filter**: Set the conditions to filter data.
- **Order**: Select the columns to sort by and the sort order.
- **Limit**: Set the maximum number of rows you want to return.
3. View the **Output** and **SQL Code** tabs.
- Each operator has an Output tab that allows you to preview the data from that configured node.
- The Code tab displays the SQL code generated by the node's configuration. Use this to see the SQL for your visual model config.
3. Connect the operators by using the connector by dragging your cursor between the operator's "+" start point and linking it to the other operators you want to connect to. This should create a connector line.
- Doing this allows the data to flow from the source table through various transformations you configured, to the final output.
5. Keep building your dbt model and ensure you confirm the out through the **Output** tab.

<!--
### Configure nodes
- Built-in AI code generator

### View output
-->

## Edit an existing model
To edit an existing model, navigate to the visual editor and click on the **Edit a model** button on the upper right. This will allow you to select the model you'd like to edit.
matthewshaver marked this conversation as resolved.
Show resolved Hide resolved

<Lightbox src="/img/docs/dbt-cloud/visual-editor/edit-model.jpg" width="90%" title="Edit a model iin the visual editor using the 'Edit a model' button." />
matthewshaver marked this conversation as resolved.
Show resolved Hide resolved

## Version control

Testing and documenting your models is an important part of the development process.

Stay tuned—coming soon, the visual editor will allow you to version-control your dbt models, ensuring that you can track changes and revert to previous versions if needed.
mirnawong1 marked this conversation as resolved.
Show resolved Hide resolved

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

how to save created / modified models? how to ensure they're ran in prod jobs?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

design coming
can version control is coming soon

## Limitations
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

how to integrate with explorer and ide?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

no integrations yet, once we can save branch, they'll be able open the branch in ide.

Are there limitations here?
79 changes: 79 additions & 0 deletions website/docs/docs/cloud/visual-editor-interface.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,79 @@
---
title: "Visual editor interface"
id: visual-editor-interface
sidebar_label: "Visual editor interface"
description: "The visual editor interface contains a node toolbar, nodes, and canvas to help you create dbt models through a seamless drag-and-drop experience in dbt Cloud."

---

# Visual editor interface <Lifecycle status='beta'/>

:::tip Beta feature
The visual editor provides users with a seamless and visual, drag-and-drop experience inside dbt Cloud. It's available in private beta for [dbt Cloud Enterprise accounts](https://www.getdbt.com/pricing).

To join the private beta, [register your interest](https://docs.google.com/forms/d/e/1FAIpQLScPjRGyrtgfmdY919Pf3kgqI5E95xxPXz-8JoVruw-L9jVtxg/viewform) or reach out to your account team to begin this process.
:::

mirnawong1 marked this conversation as resolved.
Show resolved Hide resolved
The visual editor enables analysts to create or edit dbt SQL models through a visual, drag-and-drop experience in dbt Cloud. This lets organizations benefit from code-driven development &mdash; including better precision, easier debugging, and testing &mdash; while allowing contributors in a comfortable environment. The editor also includes built-in AI for custom code generation, allowing analysts to focus on building data models.

This page offers comprehensive definitions and terminology of user interface elements, allowing you to navigate the visual editor landscape with ease.

The visual editor interface is composed of:

- **Operator toolbar** &mdash; Located at the top of the interface, the toolbar displays all the nodes available. Use the toggle on the left of the toolbar to display or hide it.
- **Operators** &mdash; perform specific transformations or configurations (such as table, join, aggregate, filter, and so on). Use connectors to link the operators and build a complete data transformation pipeline.
matthewshaver marked this conversation as resolved.
Show resolved Hide resolved
- **Canvas** &mdash; The main whiteboard space below the node toolbar. The canvas allows you to create or modify models through a sleek drag-and-drop experience.

## Operators

The operator toolbar above the canvas contains the different transformation operators available to use. Use each operator to configure or perform specific tasks, like adding filters or joining tables by dragging an operator onto the canvas. You can connect operators using the connector line, which allows you to form a complete dbt model for your data transformation.
matthewshaver marked this conversation as resolved.
Show resolved Hide resolved

<Lightbox src="/img/docs/dbt-cloud/visual-editor/edit-model.jpg" width="90%" title="Use the operator toolbar to perform different transformation operations." />
matthewshaver marked this conversation as resolved.
Show resolved Hide resolved

Here the following operators are available:
- **Table**: This represents a data table. Use this to select the source table and the columns you want to include. There are no limits to the number of tables you're allowed to have in a session.
matthewshaver marked this conversation as resolved.
Show resolved Hide resolved
- **Join**: Join two tables and configure the join conditions by selecting which columns to include from each table. Requires two inputs. For example, you might want to join both tables using the 'ID' column found in both tables.
matthewshaver marked this conversation as resolved.
Show resolved Hide resolved
- **Select**: Use this to 'select' specific columns from a table.
- **Aggregate**: Allows you to perform aggregations like GROUP, SUM, AVG, COUNT, and so on.
- **Formula**: Create new columns using custom SQL formulas. Use a built-in AI code generator to generate SQL by clicking the ? icon. For example, you can use the formula node to only extract the email domain and ask the AI code generator to help you write the SQL for that code extraction.
- **Filter**: Filter data based on conditions you set.
- **Order**: Sort data by specific columns.
- **Limit**: Limits the number of rows returned back.

When you click on each operator, it opens a configuration panel. The configuration panel allows you to configure the operator, review the current table, preview changes to the table, view the SQL code for the node, and delete the operator.
matthewshaver marked this conversation as resolved.
Show resolved Hide resolved

<Lightbox src="/img/docs/dbt-cloud/visual-editor/visual-editor.jpg" width="90%" title="Visual editor interface that contains a node toolbar and canvas." />
matthewshaver marked this conversation as resolved.
Show resolved Hide resolved

If you have any feedback on additional operators that you might need, we'd love to hear it! Please contact your dbt Labs account team and share your thoughts.

## Canvas

The visual editor has a sleek drag-and-drop canvas interface that allows you to create or modify dbt SQL models. It's like a digital whiteboard space that allows analysts to deliver trustworthy data. Use the canvas to:

- Drag-and-drop operators to create and configure your model(s)
- Generate SQL code using the built-in AI generator
- Zoom in or out for better visualization
- Version-control your dbt models
- [Coming soon] Test and document your created models

<Lightbox src="/img/docs/dbt-cloud/visual-editor/operator.jpg" width="90%" title="The operator toolbar allows you to select different nodes to configure or perform specific tasks, like adding filters or joining tables." />
matthewshaver marked this conversation as resolved.
Show resolved Hide resolved

### Connector

Connectors allow you to connect your operators to create dbt models. Once you've added operators to the canvas:
- Hover over the "+" sign next to the operator and click.
- Drag your cursor between the operator's "+" start point to the other node you want to connect to. This should create a connector line.
- As an example, to create a join, connect one operator to the "L" (Left) and the other to the "R" (Right). The endpoints are located to the left of the operator so you can easily drag the connectors to the endpoint.

<Lightbox src="/img/docs/dbt-cloud/visual-editor/connector.jpg" width="100%" title="Click and drag your cursor to connect operators." />
matthewshaver marked this conversation as resolved.
Show resolved Hide resolved

## Configuration panel
Each operator has a configuration side panel that opens when you click on it. The configuration panel allows you to configure the operator, review the current table, preview changes to the table, view the SQL code for the operator, and delete the operator.
matthewshaver marked this conversation as resolved.
Show resolved Hide resolved

The configuration side panel has the following:
- Configure tab &mdash; This section allows you to configure the operator to your specified requirements.
- Input tab &mdash; This section allows you to view the data for the current source table. Not available for table operators.
matthewshaver marked this conversation as resolved.
Show resolved Hide resolved
- Output tab &mdash; This section allows you to preview the data for the modified source table.
matthewshaver marked this conversation as resolved.
Show resolved Hide resolved
- Code &mdash; This section allows you to view the underlying SQL code for the data transformation.

<Lightbox src="/img/docs/dbt-cloud/visual-editor/config-panel.jpg" width="90%" title="A sleek drag-and-drop canvas interface that allows you to create or modify dbt SQL models." />
matthewshaver marked this conversation as resolved.
Show resolved Hide resolved
35 changes: 35 additions & 0 deletions website/docs/docs/cloud/visual-editor.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,35 @@
---
title: "About the visual editor"
id: visual-editor
sidebar_label: "About the visual editor"
description: "The visual editor enables analysts to quickly create and visualize dbt models through a visual, drag-and-drop experience inside of dbt Cloud."
pagination_next: "docs/cloud/visual-editor-interface"
pagination_prev: null
---

# About the visual editor <Lifecycle status='beta'/>

:::tip Beta feature
The visual editor provides users with a seamless and visual, drag-and-drop experience inside dbt Cloud. It's available in private beta for [dbt Cloud Enterprise accounts](https://www.getdbt.com/pricing).

To join the private beta, [register your interest](https://docs.google.com/forms/d/e/1FAIpQLScPjRGyrtgfmdY919Pf3kgqI5E95xxPXz-8JoVruw-L9jVtxg/viewform) or reach out to your account team to begin this process.
:::
mirnawong1 marked this conversation as resolved.
Show resolved Hide resolved

The visual editor helps analysts quickly create and visualize dbt models through a visual, drag-and-drop experience inside of dbt Cloud. It allows organizations to enjoy the many benefits of code-driven development—such as increased precision, ease of debugging, and ease of validation—while retaining the flexibility to have different contributors develop wherever they are most comfortable.

The visual editor development experience allows users to also take advantage of built-in AI for custom code generation, making it a frictionless experience.

These models compile directly to SQL and are indistinguishable from other dbt models in your projects:
- Visual models are version-controlled in your backing Git provider.
- All models are accessible across projects in [dbt Mesh](/best-practices/how-we-mesh/mesh-1-intro).
- Models can be materialized into production through [dbt Cloud orchestration](/docs/deploy/deployments), or be built directly into a user's development schema.
- Integrate with [dbt Explorer](/docs/collaborate/explore-projects) and the [dbt Cloud IDE](/docs/cloud/dbt-cloud-ide/develop-in-the-cloud).

<Lightbox src="/img/docs/dbt-cloud/visual-editor/visual-editor.jpg" width="90%" title="Create or edit dbt models with the Visual editor, enabling everyone to develop with dbt through a drag-and-drop experience inside of dbt Cloud." />
matthewshaver marked this conversation as resolved.
Show resolved Hide resolved

## Feedback

Please note, always review AI-generated code and content as it may produce incorrect results. The visual editor features and/or functionality may be added or eliminated as part of the beta trial.

To give feedback, please reach out to your dbt Labs account team. We appreciate your feedback and suggestions as we improve the visual editor.

11 changes: 11 additions & 0 deletions website/sidebars.js
Original file line number Diff line number Diff line change
Expand Up @@ -266,6 +266,17 @@ const sidebarSettings = {
},
],
},
{
type: "category",
label: "Visual editor",
collapsed: true,
link: { type: "doc", id: "docs/cloud/visual-editor" },
items: [
"docs/cloud/visual-editor",
"docs/cloud/visual-editor-interface",
"docs/cloud/use-visual-editor",
],
},
],
},
{
Expand Down
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading