Skip to content

Commit

Permalink
Merge branch 'main' into DOC-8960
Browse files Browse the repository at this point in the history
  • Loading branch information
mdlinville authored Nov 4, 2024
2 parents 89c8b73 + b661800 commit b2c11b2
Show file tree
Hide file tree
Showing 13 changed files with 40 additions and 40 deletions.
2 changes: 1 addition & 1 deletion src/current/_includes/v24.3/cdc/core-csv.md
Original file line number Diff line number Diff line change
@@ -1,3 +1,3 @@
{{site.data.alerts.callout_info}}
To determine how wide the columns need to be, the default `table` display format in `cockroach sql` buffers the results it receives from the server before printing them to the console. When consuming core changefeed data using `cockroach sql`, it's important to use a display format like `csv` that does not buffer its results. To set the display format, use the [`--format=csv` flag]({% link {{ page.version.version }}/cockroach-sql.md %}#sql-flag-format) when starting the [built-in SQL client]({% link {{ page.version.version }}/cockroach-sql.md %}), or set the [`\set display_format=csv` option]({% link {{ page.version.version }}/cockroach-sql.md %}#client-side-options) once the SQL client is open.
To determine how wide the columns need to be, the default `table` display format in `cockroach sql` buffers the results it receives from the server before printing them to the console. When consuming basic changefeed data using `cockroach sql`, it's important to use a display format like `csv` that does not buffer its results. To set the display format, use the [`--format=csv` flag]({% link {{ page.version.version }}/cockroach-sql.md %}#sql-flag-format) when starting the [built-in SQL client]({% link {{ page.version.version }}/cockroach-sql.md %}), or set the [`\set display_format=csv` option]({% link {{ page.version.version }}/cockroach-sql.md %}#client-side-options) once the SQL client is open.
{{site.data.alerts.end}}
2 changes: 1 addition & 1 deletion src/current/_includes/v24.3/cdc/core-url.md
Original file line number Diff line number Diff line change
@@ -1,3 +1,3 @@
{{site.data.alerts.callout_info}}
Because core changefeeds return results differently than other SQL statements, they require a dedicated database connection with specific settings around result buffering. In normal operation, CockroachDB improves performance by buffering results server-side before returning them to a client; however, result buffering is automatically turned off for core changefeeds. Core changefeeds also have different cancellation behavior than other queries: they can only be canceled by closing the underlying connection or issuing a [`CANCEL QUERY`]({% link {{ page.version.version }}/cancel-query.md %}) statement on a separate connection. Combined, these attributes of changefeeds mean that applications should explicitly create dedicated connections to consume changefeed data, instead of using a connection pool as most client drivers do by default.
Because basic changefeeds return results differently than other SQL statements, they require a dedicated database connection with specific settings around result buffering. In normal operation, CockroachDB improves performance by buffering results server-side before returning them to a client; however, result buffering is automatically turned off for basic changefeeds. basic changefeeds also have different cancellation behavior than other queries: they can only be canceled by closing the underlying connection or issuing a [`CANCEL QUERY`]({% link {{ page.version.version }}/cancel-query.md %}) statement on a separate connection. Combined, these attributes of changefeeds mean that applications should explicitly create dedicated connections to consume changefeed data, instead of using a connection pool as most client drivers do by default.
{{site.data.alerts.end}}
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
In this example, you'll set up a core changefeed for a single-node cluster that emits Avro records. CockroachDB's Avro binary encoding convention uses the [Confluent Schema Registry](https://docs.confluent.io/current/schema-registry/docs/serializer-formatter.html) to store Avro schemas.
In this example, you'll set up a basic changefeed for a single-node cluster that emits Avro records. CockroachDB's Avro binary encoding convention uses the [Confluent Schema Registry](https://docs.confluent.io/current/schema-registry/docs/serializer-formatter.html) to store Avro schemas.

1. Use the [`cockroach start-single-node`]({% link {{ page.version.version }}/cockroach-start-single-node.md %}) command to start a single-node cluster:

Expand Down Expand Up @@ -53,7 +53,7 @@ In this example, you'll set up a core changefeed for a single-node cluster that
> INSERT INTO bar VALUES (0);
~~~

1. Start the core changefeed:
1. Start the basic changefeed:

{% include_cached copy-clipboard.html %}
~~~ sql
Expand All @@ -72,13 +72,13 @@ In this example, you'll set up a core changefeed for a single-node cluster that
$ cockroach sql --insecure -e "INSERT INTO bar VALUES (1)"
~~~

1. Back in the terminal where the core changefeed is streaming, the output will appear:
1. Back in the terminal where the basic changefeed is streaming, the output will appear:

~~~
bar,\000\000\000\000\001\002\002,\000\000\000\000\002\002\002\002
~~~

Note that records may take a couple of seconds to display in the core changefeed.
Note that records may take a couple of seconds to display in the basic changefeed.

1. To stop streaming the changefeed, enter **CTRL+C** into the terminal where the changefeed is running.

Expand Down
8 changes: 4 additions & 4 deletions src/current/_includes/v24.3/cdc/create-core-changefeed.md
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
In this example, you'll set up a core changefeed for a single-node cluster.
In this example, you'll set up a basic changefeed for a single-node cluster.

1. In a terminal window, start `cockroach`:

Expand Down Expand Up @@ -44,7 +44,7 @@ In this example, you'll set up a core changefeed for a single-node cluster.
> INSERT INTO foo VALUES (0);
~~~

1. Start the core changefeed:
1. Start the basic changefeed:

{% include_cached copy-clipboard.html %}
~~~ sql
Expand All @@ -62,13 +62,13 @@ In this example, you'll set up a core changefeed for a single-node cluster.
$ cockroach sql --insecure -e "INSERT INTO foo VALUES (1)"
~~~

1. Back in the terminal where the core changefeed is streaming, the following output has appeared:
1. Back in the terminal where the basic changefeed is streaming, the following output has appeared:

~~~
foo,[1],"{""after"": {""a"": 1}}"
~~~

Note that records may take a couple of seconds to display in the core changefeed.
Note that records may take a couple of seconds to display in the basic changefeed.

1. To stop streaming the changefeed, enter **CTRL+C** into the terminal where the changefeed is running.

Expand Down
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
{{site.data.alerts.callout_info}}
It is possible to [backup]({% link {{ page.version.version }}/backup.md %}), [restore]({% link {{ page.version.version }}/restore.md %}), [import]({% link {{ page.version.version }}/import-into.md %}), and run [core changefeeds](changefeed-for.html) in [CockroachDB {{ site.data.products.serverless }}](../cockroachcloud/quickstart.html) that have upgraded to v21.1 or later.
It is possible to [backup]({% link {{ page.version.version }}/backup.md %}), [restore]({% link {{ page.version.version }}/restore.md %}), [import]({% link {{ page.version.version }}/import-into.md %}), and run [basic changefeeds](changefeed-for.html) in [CockroachDB {{ site.data.products.serverless }}](../cockroachcloud/quickstart.html) that have upgraded to v21.1 or later.

[`userfile`]({% link {{ page.version.version }}/use-userfile-storage.md %}) storage is available in CockroachDB {{ site.data.products.serverless }} clusters for backups, restores, and imports.
{{site.data.alerts.end}}
4 changes: 2 additions & 2 deletions src/current/v24.3/change-data-capture-overview.md
Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,7 @@ CockroachDB has two implementations of changefeeds:
<table class="comparison-chart">
<tr>
<th></th>
<th>Core changefeeds</th>
<th>Basic changefeeds</th>
<th>Enterprise changefeeds</th>
</tr>

Expand Down Expand Up @@ -124,7 +124,7 @@ CockroachDB has two implementations of changefeeds:

To get started with changefeeds in CockroachDB, refer to:

- [Create and Configure Changefeeds]({% link {{ page.version.version }}/create-and-configure-changefeeds.md %}): Learn about the fundamentals of using SQL statements to create and manage Enterprise and Core changefeeds.
- [Create and Configure Changefeeds]({% link {{ page.version.version }}/create-and-configure-changefeeds.md %}): Learn about the fundamentals of using SQL statements to create and manage Enterprise and basic changefeeds.
- [Changefeed Sinks]({% link {{ page.version.version }}/changefeed-sinks.md %}): The downstream system to which the changefeed emits changes. Learn about the supported sinks and configuration capabilities.
- [Changefeed Messages]({% link {{ page.version.version }}/changefeed-messages.md %}): The change events that emit from the changefeed to your sink. Learn about how messages are ordered at your sink and the options to configure and format messages.
- [Changefeed Examples]({% link {{ page.version.version }}/changefeed-examples.md %}): Step-by-step examples for connecting to each changefeed sink.
Expand Down
12 changes: 6 additions & 6 deletions src/current/v24.3/changefeed-examples.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ toc: true
docs_area: stream_data
---

This page provides step-by-step examples for using Core and {{ site.data.products.enterprise }} changefeeds. Creating {{ site.data.products.enterprise }} changefeeds is available on CockroachDB {{ site.data.products.basic }}, {{ site.data.products.standard }}, {{ site.data.products.advanced }}, and with an [{{ site.data.products.enterprise }} license](enterprise-licensing.html) on CockroachDB {{ site.data.products.core }} clusters. Core changefeeds are available in all products.
This page provides step-by-step examples for using Core and {{ site.data.products.enterprise }} changefeeds. Creating {{ site.data.products.enterprise }} changefeeds is available on CockroachDB {{ site.data.products.basic }}, {{ site.data.products.standard }}, {{ site.data.products.advanced }}, and with an [{{ site.data.products.enterprise }} license](enterprise-licensing.html) on CockroachDB {{ site.data.products.core }} clusters. Basic changefeeds are available in all products.

For a summary of Core and {{ site.data.products.enterprise }} changefeed features, refer to the [Change Data Capture Overview]({% link {{ page.version.version }}/change-data-capture-overview.md %}) page.

Expand All @@ -26,7 +26,7 @@ Use the following filters to show usage examples for either **Enterprise** or **

<div class="filters clearfix">
<button class="filter-button" data-scope="enterprise">Enterprise Changefeeds</button>
<button class="filter-button" data-scope="core">Core Changefeeds</button>
<button class="filter-button" data-scope="core">Basic Changefeeds</button>
</div>

<section class="filter-content" markdown="1" data-scope="enterprise">
Expand Down Expand Up @@ -775,17 +775,17 @@ In this example, you'll set up a changefeed for a single-node cluster that is co

<section class="filter-content" markdown="1" data-scope="core">

Core changefeeds stream row-level changes to a client until the underlying SQL connection is closed.
Basic changefeeds stream row-level changes to a client until the underlying SQL connection is closed.

## Create a Core changefeed
## Create a basic changefeed

{% include {{ page.version.version }}/cdc/create-core-changefeed.md %}

## Create a Core changefeed using Avro
## Create a basic changefeed using Avro

{% include {{ page.version.version }}/cdc/create-core-changefeed-avro.md %}

For further information on Core changefeeds, see [`EXPERIMENTAL CHANGEFEED FOR`]({% link {{ page.version.version }}/changefeed-for.md %}).
For further information on basic changefeeds, see [`EXPERIMENTAL CHANGEFEED FOR`]({% link {{ page.version.version }}/changefeed-for.md %}).

</section>

Expand Down
18 changes: 9 additions & 9 deletions src/current/v24.3/changefeed-for.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ toc: true
docs_area: reference.sql
---

The `EXPERIMENTAL CHANGEFEED FOR` [statement]({% link {{ page.version.version }}/sql-statements.md %}) creates a new core changefeed, which streams row-level changes to the client indefinitely until the underlying connection is closed or the changefeed is canceled. A core changefeed can watch one table or multiple tables in a comma-separated list.
The `EXPERIMENTAL CHANGEFEED FOR` [statement]({% link {{ page.version.version }}/sql-statements.md %}) creates a new basic changefeed, which streams row-level changes to the client indefinitely until the underlying connection is closed or the changefeed is canceled. A basic changefeed can watch one table or multiple tables in a comma-separated list.

For more information, see [Change Data Capture Overview]({% link {{ page.version.version }}/change-data-capture-overview.md %}).

Expand All @@ -23,7 +23,7 @@ There is continued support for the [legacy privilege model](#legacy-privilege-mo

To create a changefeed with `EXPERIMENTAL CHANGEFEED FOR`, a user must have the `SELECT` privilege on the changefeed's source tables.

You can [grant]({% link {{ page.version.version }}/grant.md %}#grant-privileges-on-specific-tables-in-a-database) a user the `SELECT` privilege to allow them to create core changefeeds on a specific table:
You can [grant]({% link {{ page.version.version }}/grant.md %}#grant-privileges-on-specific-tables-in-a-database) a user the `SELECT` privilege to allow them to create basic changefeeds on a specific table:

{% include_cached copy-clipboard.html %}
~~~sql
Expand All @@ -36,9 +36,9 @@ Changefeeds can only be created by superusers, i.e., [members of the `admin` rol

## Considerations

- Because core changefeeds return results differently than other SQL statements, they require a dedicated database connection with specific settings around result buffering. In normal operation, CockroachDB improves performance by buffering results server-side before returning them to a client; however, result buffering is automatically turned off for core changefeeds. Core changefeeds also have different cancellation behavior than other queries: they can only be canceled by closing the underlying connection or issuing a [`CANCEL QUERY`]({% link {{ page.version.version }}/cancel-query.md %}) statement on a separate connection. Combined, these attributes of changefeeds mean that applications should explicitly create dedicated connections to consume changefeed data, instead of using a connection pool as most client drivers do by default.
- Because basic changefeeds return results differently than other SQL statements, they require a dedicated database connection with specific settings around result buffering. In normal operation, CockroachDB improves performance by buffering results server-side before returning them to a client; however, result buffering is automatically turned off for basic changefeeds. basic changefeeds also have different cancellation behavior than other queries: they can only be canceled by closing the underlying connection or issuing a [`CANCEL QUERY`]({% link {{ page.version.version }}/cancel-query.md %}) statement on a separate connection. Combined, these attributes of changefeeds mean that applications should explicitly create dedicated connections to consume changefeed data, instead of using a connection pool as most client drivers do by default.

This cancellation behavior (i.e., close the underlying connection to cancel the changefeed) also extends to client driver usage; in particular, when a client driver calls `Rows.Close()` after encountering errors for a stream of rows. The pgwire protocol requires that the rows be consumed before the connection is again usable, but in the case of a core changefeed, the rows are never consumed. It is therefore critical that you close the connection, otherwise the application will be blocked forever on `Rows.Close()`.
This cancellation behavior (i.e., close the underlying connection to cancel the changefeed) also extends to client driver usage; in particular, when a client driver calls `Rows.Close()` after encountering errors for a stream of rows. The pgwire protocol requires that the rows be consumed before the connection is again usable, but in the case of a basic changefeed, the rows are never consumed. It is therefore critical that you close the connection, otherwise the application will be blocked forever on `Rows.Close()`.

- In most cases, each version of a row will be emitted once. However, some infrequent conditions (e.g., node failures, network partitions) will cause them to be repeated. This gives our changefeeds an at-least-once delivery guarantee. For more information, see [Ordering Guarantees]({% link {{ page.version.version }}/changefeed-messages.md %}#ordering-and-delivery-guarantees).
- As of v22.1, changefeeds filter out [`VIRTUAL` computed columns]({% link {{ page.version.version }}/computed-columns.md %}) from events by default. This is a [backward-incompatible change]({% link releases/v22.1.md %}#v22-1-0-backward-incompatible-changes). To maintain the changefeed behavior in previous versions where [`NULL`]({% link {{ page.version.version }}/null-handling.md %}) values are emitted for virtual computed columns, see the [`virtual_columns`]({% link {{ page.version.version }}/changefeed-for.md %}#virtual-columns) option for more detail.
Expand Down Expand Up @@ -95,14 +95,14 @@ To start a changefeed:
EXPERIMENTAL CHANGEFEED FOR cdc_test;
~~~

In the terminal where the core changefeed is streaming, the output will appear:
In the terminal where the basic changefeed is streaming, the output will appear:

~~~
table,key,value
cdc_test,[0],"{""after"": {""a"": 0}}"
~~~

For step-by-step guidance on creating a Core changefeed, see the [Changefeed Examples]({% link {{ page.version.version }}/changefeed-examples.md %}) page.
For step-by-step guidance on creating a basic changefeed, see the [Changefeed Examples]({% link {{ page.version.version }}/changefeed-examples.md %}) page.

### Create a changefeed with Avro

Expand All @@ -113,14 +113,14 @@ To start a changefeed in Avro format:
EXPERIMENTAL CHANGEFEED FOR cdc_test WITH format = avro, confluent_schema_registry = 'http://localhost:8081';
~~~

In the terminal where the core changefeed is streaming, the output will appear:
In the terminal where the basic changefeed is streaming, the output will appear:

~~~
table,key,value
cdc_test,\000\000\000\000\001\002\000,\000\000\000\000\002\002\002\000
~~~

For step-by-step guidance on creating a Core changefeed with Avro, see the [Changefeed Examples]({% link {{ page.version.version }}/changefeed-examples.md %}) page.
For step-by-step guidance on creating a basic changefeed with Avro, see the [Changefeed Examples]({% link {{ page.version.version }}/changefeed-examples.md %}) page.

### Create a changefeed on a table with column families

Expand All @@ -138,7 +138,7 @@ To create a changefeed on a table and output changes for each column family, use
EXPERIMENTAL CHANGEFEED FOR TABLE cdc_test WITH split_column_families;
~~~

For step-by-step guidance creating a Core changefeed on a table with multiple column families, see the [Changefeed Examples]({% link {{ page.version.version }}/changefeed-examples.md %}) page.
For step-by-step guidance creating a basic changefeed on a table with multiple column families, see the [Changefeed Examples]({% link {{ page.version.version }}/changefeed-examples.md %}) page.

## See also

Expand Down
Loading

0 comments on commit b2c11b2

Please sign in to comment.