From f7a6b2580bab010e155504f8f537aca694144f42 Mon Sep 17 00:00:00 2001 From: Rich Loveland Date: Wed, 11 Dec 2024 15:37:24 -0500 Subject: [PATCH 01/36] Clarify 'free up disk space more quickly' FAQ item (#19087) * Clarify 'free up disk space more quickly' FAQ item Fixes DOC-11638 Co-authored-by: Adam Storm --- src/current/_includes/v23.1/storage/free-up-disk-space.md | 2 +- src/current/_includes/v23.2/storage/free-up-disk-space.md | 2 +- src/current/_includes/v24.1/storage/free-up-disk-space.md | 2 +- src/current/_includes/v24.2/storage/free-up-disk-space.md | 2 +- src/current/_includes/v24.3/storage/free-up-disk-space.md | 2 +- src/current/v23.1/operational-faqs.md | 8 ++++++-- src/current/v23.2/operational-faqs.md | 8 ++++++-- src/current/v24.1/operational-faqs.md | 8 ++++++-- src/current/v24.2/operational-faqs.md | 8 ++++++-- src/current/v24.3/operational-faqs.md | 8 ++++++-- 10 files changed, 35 insertions(+), 15 deletions(-) diff --git a/src/current/_includes/v23.1/storage/free-up-disk-space.md b/src/current/_includes/v23.1/storage/free-up-disk-space.md index c63b70b766e..e4a6b08a57a 100644 --- a/src/current/_includes/v23.1/storage/free-up-disk-space.md +++ b/src/current/_includes/v23.1/storage/free-up-disk-space.md @@ -1 +1 @@ -For instructions on how to free up disk space as quickly as possible after deleting data, see [How can I free up disk space quickly?]({% link {{ page.version.version }}/operational-faqs.md %}#how-can-i-free-up-disk-space-quickly) +For instructions on how to free up disk space as quickly as possible after dropping a table, see [How can I free up disk space that was used by a dropped table?]({% link {{ page.version.version }}/operational-faqs.md %}#how-can-i-free-up-disk-space-when-dropping-a-table) diff --git a/src/current/_includes/v23.2/storage/free-up-disk-space.md b/src/current/_includes/v23.2/storage/free-up-disk-space.md index c63b70b766e..e4a6b08a57a 100644 --- a/src/current/_includes/v23.2/storage/free-up-disk-space.md +++ b/src/current/_includes/v23.2/storage/free-up-disk-space.md @@ -1 +1 @@ -For instructions on how to free up disk space as quickly as possible after deleting data, see [How can I free up disk space quickly?]({% link {{ page.version.version }}/operational-faqs.md %}#how-can-i-free-up-disk-space-quickly) +For instructions on how to free up disk space as quickly as possible after dropping a table, see [How can I free up disk space that was used by a dropped table?]({% link {{ page.version.version }}/operational-faqs.md %}#how-can-i-free-up-disk-space-when-dropping-a-table) diff --git a/src/current/_includes/v24.1/storage/free-up-disk-space.md b/src/current/_includes/v24.1/storage/free-up-disk-space.md index c63b70b766e..e4a6b08a57a 100644 --- a/src/current/_includes/v24.1/storage/free-up-disk-space.md +++ b/src/current/_includes/v24.1/storage/free-up-disk-space.md @@ -1 +1 @@ -For instructions on how to free up disk space as quickly as possible after deleting data, see [How can I free up disk space quickly?]({% link {{ page.version.version }}/operational-faqs.md %}#how-can-i-free-up-disk-space-quickly) +For instructions on how to free up disk space as quickly as possible after dropping a table, see [How can I free up disk space that was used by a dropped table?]({% link {{ page.version.version }}/operational-faqs.md %}#how-can-i-free-up-disk-space-when-dropping-a-table) diff --git a/src/current/_includes/v24.2/storage/free-up-disk-space.md b/src/current/_includes/v24.2/storage/free-up-disk-space.md index c63b70b766e..e4a6b08a57a 100644 --- a/src/current/_includes/v24.2/storage/free-up-disk-space.md +++ b/src/current/_includes/v24.2/storage/free-up-disk-space.md @@ -1 +1 @@ -For instructions on how to free up disk space as quickly as possible after deleting data, see [How can I free up disk space quickly?]({% link {{ page.version.version }}/operational-faqs.md %}#how-can-i-free-up-disk-space-quickly) +For instructions on how to free up disk space as quickly as possible after dropping a table, see [How can I free up disk space that was used by a dropped table?]({% link {{ page.version.version }}/operational-faqs.md %}#how-can-i-free-up-disk-space-when-dropping-a-table) diff --git a/src/current/_includes/v24.3/storage/free-up-disk-space.md b/src/current/_includes/v24.3/storage/free-up-disk-space.md index c63b70b766e..e4a6b08a57a 100644 --- a/src/current/_includes/v24.3/storage/free-up-disk-space.md +++ b/src/current/_includes/v24.3/storage/free-up-disk-space.md @@ -1 +1 @@ -For instructions on how to free up disk space as quickly as possible after deleting data, see [How can I free up disk space quickly?]({% link {{ page.version.version }}/operational-faqs.md %}#how-can-i-free-up-disk-space-quickly) +For instructions on how to free up disk space as quickly as possible after dropping a table, see [How can I free up disk space that was used by a dropped table?]({% link {{ page.version.version }}/operational-faqs.md %}#how-can-i-free-up-disk-space-when-dropping-a-table) diff --git a/src/current/v23.1/operational-faqs.md b/src/current/v23.1/operational-faqs.md index b84fff484ca..56c0c9ff40b 100644 --- a/src/current/v23.1/operational-faqs.md +++ b/src/current/v23.1/operational-faqs.md @@ -81,9 +81,13 @@ When MVCC garbage is deleted by garbage collection, the data is still not yet ph {% include {{page.version.version}}/storage/free-up-disk-space.md %} -## How can I free up disk space quickly? +## How can I free up disk space when dropping a table? -If you've noticed that [your disk space is not freeing up quickly enough after deleting data](#why-is-my-disk-usage-not-decreasing-after-deleting-data), you can take the following steps to free up disk space more quickly. This example assumes a table `t`. +If you've noticed that [your disk space is not freeing up quickly enough after dropping a table](#why-is-my-disk-usage-not-decreasing-after-deleting-data), you can take the following steps to free up disk space more quickly the next time you drop a table. This example assumes a table `t` exists. + +{{site.data.alerts.callout_info}} +The procedure shown here only works if you get the range IDs from the table **before** running [`DROP TABLE`]({% link {{ page.version.version }}/drop-table.md %}). If you are in an emergency situation due to running out of disk, see [What happens when a node runs out of disk space?](#what-happens-when-a-node-runs-out-of-disk-space) +{{site.data.alerts.end}} 1. Lower the [`gc.ttlseconds` parameter]({% link {{ page.version.version }}/configure-replication-zones.md %}#gc-ttlseconds) to 10 minutes. diff --git a/src/current/v23.2/operational-faqs.md b/src/current/v23.2/operational-faqs.md index 6fa24aa8336..41bc09654b7 100644 --- a/src/current/v23.2/operational-faqs.md +++ b/src/current/v23.2/operational-faqs.md @@ -81,9 +81,13 @@ When MVCC garbage is deleted by garbage collection, the data is still not yet ph {% include {{page.version.version}}/storage/free-up-disk-space.md %} -## How can I free up disk space quickly? +## How can I free up disk space when dropping a table? -If you've noticed that [your disk space is not freeing up quickly enough after deleting data](#why-is-my-disk-usage-not-decreasing-after-deleting-data), you can take the following steps to free up disk space more quickly. This example assumes a table `t`. +If you've noticed that [your disk space is not freeing up quickly enough after dropping a table](#why-is-my-disk-usage-not-decreasing-after-deleting-data), you can take the following steps to free up disk space more quickly the next time you drop a table. This example assumes a table `t` exists. + +{{site.data.alerts.callout_info}} +The procedure shown here only works if you get the range IDs from the table **before** running [`DROP TABLE`]({% link {{ page.version.version }}/drop-table.md %}). If you are in an emergency situation due to running out of disk, see [What happens when a node runs out of disk space?](#what-happens-when-a-node-runs-out-of-disk-space) +{{site.data.alerts.end}} 1. Lower the [`gc.ttlseconds` parameter]({% link {{ page.version.version }}/configure-replication-zones.md %}#gc-ttlseconds) to 10 minutes. diff --git a/src/current/v24.1/operational-faqs.md b/src/current/v24.1/operational-faqs.md index d482b34322d..d132ffcded5 100644 --- a/src/current/v24.1/operational-faqs.md +++ b/src/current/v24.1/operational-faqs.md @@ -81,9 +81,13 @@ When MVCC garbage is deleted by garbage collection, the data is still not yet ph {% include {{page.version.version}}/storage/free-up-disk-space.md %} -## How can I free up disk space quickly? +## How can I free up disk space when dropping a table? -If you've noticed that [your disk space is not freeing up quickly enough after deleting data](#why-is-my-disk-usage-not-decreasing-after-deleting-data), you can take the following steps to free up disk space more quickly. This example assumes a table `t`. +If you've noticed that [your disk space is not freeing up quickly enough after dropping a table](#why-is-my-disk-usage-not-decreasing-after-deleting-data), you can take the following steps to free up disk space more quickly the next time you drop a table. This example assumes a table `t` exists. + +{{site.data.alerts.callout_info}} +The procedure shown here only works if you get the range IDs from the table **before** running [`DROP TABLE`]({% link {{ page.version.version }}/drop-table.md %}). If you are in an emergency situation due to running out of disk, see [What happens when a node runs out of disk space?](#what-happens-when-a-node-runs-out-of-disk-space) +{{site.data.alerts.end}} 1. Lower the [`gc.ttlseconds` parameter]({% link {{ page.version.version }}/configure-replication-zones.md %}#gc-ttlseconds) to 10 minutes. diff --git a/src/current/v24.2/operational-faqs.md b/src/current/v24.2/operational-faqs.md index 10c04d37472..7a073b9c0dc 100644 --- a/src/current/v24.2/operational-faqs.md +++ b/src/current/v24.2/operational-faqs.md @@ -81,9 +81,13 @@ When MVCC garbage is deleted by garbage collection, the data is still not yet ph {% include {{page.version.version}}/storage/free-up-disk-space.md %} -## How can I free up disk space quickly? +## How can I free up disk space when dropping a table? -If you've noticed that [your disk space is not freeing up quickly enough after deleting data](#why-is-my-disk-usage-not-decreasing-after-deleting-data), you can take the following steps to free up disk space more quickly. This example assumes a table `t`. +If you've noticed that [your disk space is not freeing up quickly enough after dropping a table](#why-is-my-disk-usage-not-decreasing-after-deleting-data), you can take the following steps to free up disk space more quickly the next time you drop a table. This example assumes a table `t` exists. + +{{site.data.alerts.callout_info}} +The procedure shown here only works if you get the range IDs from the table **before** running [`DROP TABLE`]({% link {{ page.version.version }}/drop-table.md %}). If you are in an emergency situation due to running out of disk, see [What happens when a node runs out of disk space?](#what-happens-when-a-node-runs-out-of-disk-space) +{{site.data.alerts.end}} 1. Lower the [`gc.ttlseconds` parameter]({% link {{ page.version.version }}/configure-replication-zones.md %}#gc-ttlseconds) to 10 minutes. diff --git a/src/current/v24.3/operational-faqs.md b/src/current/v24.3/operational-faqs.md index 10c04d37472..7a073b9c0dc 100644 --- a/src/current/v24.3/operational-faqs.md +++ b/src/current/v24.3/operational-faqs.md @@ -81,9 +81,13 @@ When MVCC garbage is deleted by garbage collection, the data is still not yet ph {% include {{page.version.version}}/storage/free-up-disk-space.md %} -## How can I free up disk space quickly? +## How can I free up disk space when dropping a table? -If you've noticed that [your disk space is not freeing up quickly enough after deleting data](#why-is-my-disk-usage-not-decreasing-after-deleting-data), you can take the following steps to free up disk space more quickly. This example assumes a table `t`. +If you've noticed that [your disk space is not freeing up quickly enough after dropping a table](#why-is-my-disk-usage-not-decreasing-after-deleting-data), you can take the following steps to free up disk space more quickly the next time you drop a table. This example assumes a table `t` exists. + +{{site.data.alerts.callout_info}} +The procedure shown here only works if you get the range IDs from the table **before** running [`DROP TABLE`]({% link {{ page.version.version }}/drop-table.md %}). If you are in an emergency situation due to running out of disk, see [What happens when a node runs out of disk space?](#what-happens-when-a-node-runs-out-of-disk-space) +{{site.data.alerts.end}} 1. Lower the [`gc.ttlseconds` parameter]({% link {{ page.version.version }}/configure-replication-zones.md %}#gc-ttlseconds) to 10 minutes. From 2a56ebd595bee10f06bdfb76dfe50924d51a1669 Mon Sep 17 00:00:00 2001 From: Kathryn May <44557882+kathancox@users.noreply.github.com> Date: Wed, 11 Dec 2024 15:56:45 -0500 Subject: [PATCH 02/36] Add the total ranges changefeed metric to lagging ranges section (#19196) --- src/current/_includes/releases/v23.2/v23.2.13.md | 3 ++- src/current/_includes/v23.1/cdc/lagging-ranges.md | 6 ++++-- src/current/_includes/v23.2/cdc/lagging-ranges.md | 6 ++++-- src/current/_includes/v24.1/cdc/lagging-ranges.md | 6 ++++-- src/current/_includes/v24.2/cdc/lagging-ranges.md | 6 ++++-- src/current/_includes/v24.3/cdc/lagging-ranges.md | 6 ++++-- src/current/v23.1/monitor-and-debug-changefeeds.md | 1 + src/current/v23.2/monitor-and-debug-changefeeds.md | 1 + src/current/v24.1/monitor-and-debug-changefeeds.md | 1 + src/current/v24.2/monitor-and-debug-changefeeds.md | 1 + src/current/v24.3/monitor-and-debug-changefeeds.md | 1 + 11 files changed, 27 insertions(+), 11 deletions(-) diff --git a/src/current/_includes/releases/v23.2/v23.2.13.md b/src/current/_includes/releases/v23.2/v23.2.13.md index 28ce188820c..26f4a561790 100644 --- a/src/current/_includes/releases/v23.2/v23.2.13.md +++ b/src/current/_includes/releases/v23.2/v23.2.13.md @@ -16,7 +16,7 @@ Release Date: October 17, 2024 [#130664][#130664] -- The new [metric]({% link v23.2/metrics.md %}) `changefeed.total_ranges` allows observation of the number of changes that are watched by a changefeed aggregator. It uses the same polling interval as `changefeed.lagging_ranges`, which is controlled by the changefeed option `lagging_ranges_polling_interval`. [#130984][#130984] +- The new [metric]({% link v23.2/metrics.md %}) `changefeed.total_ranges` allows observation of the number of ranges that are watched by a changefeed aggregator. It uses the same polling interval as `changefeed.lagging_ranges`, which is controlled by the changefeed option `lagging_ranges_polling_interval`. [#130984][#130984] - The following groups of [metrics]({% link v23.2/metrics.md %}) and [logs]({% link v23.2/logging.md %}) have been renamed to include the buffer they are associated with. The previous metrics are still maintained for backward compatibility. - `changefeed.buffer_entries.*` - `changefeed.buffer_entries_mem.*` @@ -81,6 +81,7 @@ Release Date: October 17, 2024 [#130664]: https://github.com/cockroachdb/cockroach/pull/130664 [#130790]: https://github.com/cockroachdb/cockroach/pull/130790 [#130919]: https://github.com/cockroachdb/cockroach/pull/130919 +[#130984]: https://github.com/cockroachdb/cockroach/pull/130984 [#130988]: https://github.com/cockroachdb/cockroach/pull/130988 [#131065]: https://github.com/cockroachdb/cockroach/pull/131065 [#131128]: https://github.com/cockroachdb/cockroach/pull/131128 diff --git a/src/current/_includes/v23.1/cdc/lagging-ranges.md b/src/current/_includes/v23.1/cdc/lagging-ranges.md index 339a5571443..e0061df7d4b 100644 --- a/src/current/_includes/v23.1/cdc/lagging-ranges.md +++ b/src/current/_includes/v23.1/cdc/lagging-ranges.md @@ -1,10 +1,12 @@ -{% include_cached new-in.html version="v23.1.12" %} Use the `changefeed.lagging_ranges` metric to track the number of ranges that are behind in a changefeed. This is calculated based on the [cluster settings]({% link {{ page.version.version }}/cluster-settings.md %}): +{% include_cached new-in.html version="v23.1.12" %} Use the `changefeed.lagging_ranges` metric to track the number of [ranges]({% link {{ page.version.version }}/architecture/overview.md %}#architecture-range) that are behind in a changefeed. This is calculated based on the [cluster settings]({% link {{ page.version.version }}/cluster-settings.md %}): - `changefeed.lagging_ranges_threshold` sets a duration from the present that determines the length of time a range is considered to be lagging behind, which will then track in the [`lagging_ranges`]({% link {{ page.version.version }}/monitor-and-debug-changefeeds.md %}#using-changefeed-metrics-labels) metric. Note that ranges undergoing an [initial scan]({% link {{ page.version.version }}/create-changefeed.md %}#initial-scan) for longer than the threshold duration are considered to be lagging. Starting a changefeed with an initial scan on a large table will likely increment the metric for each range in the table. As ranges complete the initial scan, the number of ranges lagging behind will decrease. - **Default:** `3m` - `changefeed.lagging_ranges_polling_interval` sets the interval rate for when lagging ranges are checked and the `lagging_ranges` metric is updated. Polling adds latency to the `lagging_ranges` metric being updated. For example, if a range falls behind by 3 minutes, the metric may not update until an additional minute afterward. - **Default:** `1m` +{% include_cached new-in.html version="v23.1.29" %} Use the `changefeed.total_ranges` metric to monitor the number of ranges that are watched by [aggregator processors]({% link {{ page.version.version }}/how-does-an-enterprise-changefeed-work.md %}) participating in the changefeed job. If you're experiencing lagging ranges, `changefeed.total_ranges` may indicate that the number of ranges watched by aggregator processors in the job is unbalanced. You may want to try [pausing]({% link {{ page.version.version }}/pause-job.md %}) the changefeed and then [resuming]({% link {{ page.version.version }}/resume-job.md %}) it, so that the changefeed replans the work in the cluster. `changefeed.total_ranges` shares the same polling interval as the `changefeed.lagging_ranges` metric, which is controlled by the `changefeed.lagging_ranges_polling_interval` cluster setting. + {{site.data.alerts.callout_success}} -You can use the [`metrics_label`]({% link {{ page.version.version }}/monitor-and-debug-changefeeds.md %}#using-changefeed-metrics-labels) option to track the `lagging_ranges` metric per changefeed. +You can use the [`metrics_label`]({% link {{ page.version.version }}/monitor-and-debug-changefeeds.md %}#using-changefeed-metrics-labels) option to track the `lagging_ranges` and `total_ranges` metric per changefeed. {{site.data.alerts.end}} \ No newline at end of file diff --git a/src/current/_includes/v23.2/cdc/lagging-ranges.md b/src/current/_includes/v23.2/cdc/lagging-ranges.md index b784a93cbfb..35d269bc706 100644 --- a/src/current/_includes/v23.2/cdc/lagging-ranges.md +++ b/src/current/_includes/v23.2/cdc/lagging-ranges.md @@ -1,10 +1,12 @@ -{% include_cached new-in.html version="v23.2" %} Use the `changefeed.lagging_ranges` metric to track the number of ranges that are behind in a changefeed. This is calculated based on the [changefeed options]({% link {{ page.version.version }}/create-changefeed.md %}#options): +{% include_cached new-in.html version="v23.2" %} Use the `changefeed.lagging_ranges` metric to track the number of [ranges]({% link {{ page.version.version }}/architecture/overview.md %}#architecture-range) that are behind in a changefeed. This is calculated based on the [changefeed options]({% link {{ page.version.version }}/create-changefeed.md %}#options): - `lagging_ranges_threshold` sets a duration from the present that determines the length of time a range is considered to be lagging behind, which will then track in the [`lagging_ranges`]({% link {{ page.version.version }}/monitor-and-debug-changefeeds.md %}#lagging-ranges-metric) metric. Note that ranges undergoing an [initial scan]({% link {{ page.version.version }}/create-changefeed.md %}#initial-scan) for longer than the threshold duration are considered to be lagging. Starting a changefeed with an initial scan on a large table will likely increment the metric for each range in the table. As ranges complete the initial scan, the number of ranges lagging behind will decrease. - **Default:** `3m` - `lagging_ranges_polling_interval` sets the interval rate for when lagging ranges are checked and the `lagging_ranges` metric is updated. Polling adds latency to the `lagging_ranges` metric being updated. For example, if a range falls behind by 3 minutes, the metric may not update until an additional minute afterward. - **Default:** `1m` +{% include_cached new-in.html version="v23.2.13" %} Use the `changefeed.total_ranges` metric to monitor the number of ranges that are watched by [aggregator processors]({% link {{ page.version.version }}/how-does-an-enterprise-changefeed-work.md %}) participating in the changefeed job. If you're experiencing lagging ranges, `changefeed.total_ranges` may indicate that the number of ranges watched by aggregator processors in the job is unbalanced. You may want to try [pausing]({% link {{ page.version.version }}/pause-job.md %}) the changefeed and then [resuming]({% link {{ page.version.version }}/resume-job.md %}) it, so that the changefeed replans the work in the cluster. `changefeed.total_ranges` shares the same polling interval as the `changefeed.lagging_ranges` metric, which is controlled by the `lagging_ranges_polling_interval` option. + {{site.data.alerts.callout_success}} -You can use the [`metrics_label`]({% link {{ page.version.version }}/monitor-and-debug-changefeeds.md %}#using-changefeed-metrics-labels) option to track the `lagging_ranges` metric per changefeed. +You can use the [`metrics_label`]({% link {{ page.version.version }}/monitor-and-debug-changefeeds.md %}#using-changefeed-metrics-labels) option to track the `lagging_ranges` and `total_ranges` metric per changefeed. {{site.data.alerts.end}} \ No newline at end of file diff --git a/src/current/_includes/v24.1/cdc/lagging-ranges.md b/src/current/_includes/v24.1/cdc/lagging-ranges.md index 8d0b5eb6c23..17a7e035916 100644 --- a/src/current/_includes/v24.1/cdc/lagging-ranges.md +++ b/src/current/_includes/v24.1/cdc/lagging-ranges.md @@ -1,10 +1,12 @@ -Use the `changefeed.lagging_ranges` metric to track the number of ranges that are behind in a changefeed. This is calculated based on the [changefeed options]({% link {{ page.version.version }}/create-changefeed.md %}#options): +Use the `changefeed.lagging_ranges` metric to track the number of [ranges]({% link {{ page.version.version }}/architecture/overview.md %}#range) that are behind in a changefeed. This is calculated based on the [changefeed options]({% link {{ page.version.version }}/create-changefeed.md %}#options): - `lagging_ranges_threshold` sets a duration from the present that determines the length of time a range is considered to be lagging behind, which will then track in the [`lagging_ranges`]({% link {{ page.version.version }}/monitor-and-debug-changefeeds.md %}#lagging-ranges-metric) metric. Note that ranges undergoing an [initial scan]({% link {{ page.version.version }}/create-changefeed.md %}#initial-scan) for longer than the threshold duration are considered to be lagging. Starting a changefeed with an initial scan on a large table will likely increment the metric for each range in the table. As ranges complete the initial scan, the number of ranges lagging behind will decrease. - **Default:** `3m` - `lagging_ranges_polling_interval` sets the interval rate for when lagging ranges are checked and the `lagging_ranges` metric is updated. Polling adds latency to the `lagging_ranges` metric being updated. For example, if a range falls behind by 3 minutes, the metric may not update until an additional minute afterward. - **Default:** `1m` +{% include_cached new-in.html version="v24.1.6" %} Use the `changefeed.total_ranges` metric to monitor the number of ranges that are watched by [aggregator processors]({% link {{ page.version.version }}/how-does-an-enterprise-changefeed-work.md %}) participating in the changefeed job. If you're experiencing lagging ranges, `changefeed.total_ranges` may indicate that the number of ranges watched by aggregator processors in the job is unbalanced. You may want to try [pausing]({% link {{ page.version.version }}/pause-job.md %}) the changefeed and then [resuming]({% link {{ page.version.version }}/resume-job.md %}) it, so that the changefeed replans the work in the cluster. `changefeed.total_ranges` shares the same polling interval as the `changefeed.lagging_ranges` metric, which is controlled by the `lagging_ranges_polling_interval` option. + {{site.data.alerts.callout_success}} -You can use the [`metrics_label`]({% link {{ page.version.version }}/monitor-and-debug-changefeeds.md %}#using-changefeed-metrics-labels) option to track the `lagging_ranges` metric per changefeed. +You can use the [`metrics_label`]({% link {{ page.version.version }}/monitor-and-debug-changefeeds.md %}#using-changefeed-metrics-labels) option to track the `lagging_ranges` and `total_ranges` metric per changefeed. {{site.data.alerts.end}} \ No newline at end of file diff --git a/src/current/_includes/v24.2/cdc/lagging-ranges.md b/src/current/_includes/v24.2/cdc/lagging-ranges.md index 8d0b5eb6c23..eb22275849a 100644 --- a/src/current/_includes/v24.2/cdc/lagging-ranges.md +++ b/src/current/_includes/v24.2/cdc/lagging-ranges.md @@ -1,10 +1,12 @@ -Use the `changefeed.lagging_ranges` metric to track the number of ranges that are behind in a changefeed. This is calculated based on the [changefeed options]({% link {{ page.version.version }}/create-changefeed.md %}#options): +Use the `changefeed.lagging_ranges` metric to track the number of [ranges]({% link {{ page.version.version }}/architecture/overview.md %}#architecture-range) that are behind in a changefeed. This is calculated based on the [changefeed options]({% link {{ page.version.version }}/create-changefeed.md %}#options): - `lagging_ranges_threshold` sets a duration from the present that determines the length of time a range is considered to be lagging behind, which will then track in the [`lagging_ranges`]({% link {{ page.version.version }}/monitor-and-debug-changefeeds.md %}#lagging-ranges-metric) metric. Note that ranges undergoing an [initial scan]({% link {{ page.version.version }}/create-changefeed.md %}#initial-scan) for longer than the threshold duration are considered to be lagging. Starting a changefeed with an initial scan on a large table will likely increment the metric for each range in the table. As ranges complete the initial scan, the number of ranges lagging behind will decrease. - **Default:** `3m` - `lagging_ranges_polling_interval` sets the interval rate for when lagging ranges are checked and the `lagging_ranges` metric is updated. Polling adds latency to the `lagging_ranges` metric being updated. For example, if a range falls behind by 3 minutes, the metric may not update until an additional minute afterward. - **Default:** `1m` +{% include_cached new-in.html version="v24.2.4" %} Use the `changefeed.total_ranges` metric to monitor the number of ranges that are watched by [aggregator processors]({% link {{ page.version.version }}/how-does-an-enterprise-changefeed-work.md %}) participating in the changefeed job. If you're experiencing lagging ranges, `changefeed.total_ranges` may indicate that the number of ranges watched by aggregator processors in the job is unbalanced. You may want to try [pausing]({% link {{ page.version.version }}/pause-job.md %}) the changefeed and then [resuming]({% link {{ page.version.version }}/resume-job.md %}) it, so that the changefeed replans the work in the cluster. `changefeed.total_ranges` shares the same polling interval as the `changefeed.lagging_ranges` metric, which is controlled by the `lagging_ranges_polling_interval` option. + {{site.data.alerts.callout_success}} -You can use the [`metrics_label`]({% link {{ page.version.version }}/monitor-and-debug-changefeeds.md %}#using-changefeed-metrics-labels) option to track the `lagging_ranges` metric per changefeed. +You can use the [`metrics_label`]({% link {{ page.version.version }}/monitor-and-debug-changefeeds.md %}#using-changefeed-metrics-labels) option to track the `lagging_ranges` and `total_ranges` metric per changefeed. {{site.data.alerts.end}} \ No newline at end of file diff --git a/src/current/_includes/v24.3/cdc/lagging-ranges.md b/src/current/_includes/v24.3/cdc/lagging-ranges.md index 8d0b5eb6c23..45180baa57f 100644 --- a/src/current/_includes/v24.3/cdc/lagging-ranges.md +++ b/src/current/_includes/v24.3/cdc/lagging-ranges.md @@ -1,10 +1,12 @@ -Use the `changefeed.lagging_ranges` metric to track the number of ranges that are behind in a changefeed. This is calculated based on the [changefeed options]({% link {{ page.version.version }}/create-changefeed.md %}#options): +Use the `changefeed.lagging_ranges` metric to track the number of [ranges]({% link {{ page.version.version }}/architecture/overview.md %}#range) that are behind in a changefeed. This is calculated based on the [changefeed options]({% link {{ page.version.version }}/create-changefeed.md %}#options): - `lagging_ranges_threshold` sets a duration from the present that determines the length of time a range is considered to be lagging behind, which will then track in the [`lagging_ranges`]({% link {{ page.version.version }}/monitor-and-debug-changefeeds.md %}#lagging-ranges-metric) metric. Note that ranges undergoing an [initial scan]({% link {{ page.version.version }}/create-changefeed.md %}#initial-scan) for longer than the threshold duration are considered to be lagging. Starting a changefeed with an initial scan on a large table will likely increment the metric for each range in the table. As ranges complete the initial scan, the number of ranges lagging behind will decrease. - **Default:** `3m` - `lagging_ranges_polling_interval` sets the interval rate for when lagging ranges are checked and the `lagging_ranges` metric is updated. Polling adds latency to the `lagging_ranges` metric being updated. For example, if a range falls behind by 3 minutes, the metric may not update until an additional minute afterward. - **Default:** `1m` +{% include_cached new-in.html version="v24.3" %} Use the `changefeed.total_ranges` metric to monitor the number of ranges that are watched by [aggregator processors]({% link {{ page.version.version }}/how-does-an-enterprise-changefeed-work.md %}) participating in the changefeed job. If you're experiencing lagging ranges, `changefeed.total_ranges` may indicate that the number of ranges watched by aggregator processors in the job is unbalanced. You may want to try [pausing]({% link {{ page.version.version }}/pause-job.md %}) the changefeed and then [resuming]({% link {{ page.version.version }}/resume-job.md %}) it, so that the changefeed replans the work in the cluster. `changefeed.total_ranges` shares the same polling interval as the `changefeed.lagging_ranges` metric, which is controlled by the `lagging_ranges_polling_interval` option. + {{site.data.alerts.callout_success}} -You can use the [`metrics_label`]({% link {{ page.version.version }}/monitor-and-debug-changefeeds.md %}#using-changefeed-metrics-labels) option to track the `lagging_ranges` metric per changefeed. +You can use the [`metrics_label`]({% link {{ page.version.version }}/monitor-and-debug-changefeeds.md %}#using-changefeed-metrics-labels) option to track the `lagging_ranges` and `total_ranges` metric per changefeed. {{site.data.alerts.end}} \ No newline at end of file diff --git a/src/current/v23.1/monitor-and-debug-changefeeds.md b/src/current/v23.1/monitor-and-debug-changefeeds.md index 8801fa611cd..fab5c562f75 100644 --- a/src/current/v23.1/monitor-and-debug-changefeeds.md +++ b/src/current/v23.1/monitor-and-debug-changefeeds.md @@ -156,6 +156,7 @@ changefeed_emitted_bytes{scope="vehicles"} 183557 `error_retries` | Total retryable errors encountered by changefeeds. | Errors `backfill_pending_ranges` | Number of [ranges]({% link {{ page.version.version }}/architecture/overview.md %}#architecture-range) in an ongoing backfill that are yet to be fully emitted. | Ranges `message_size_hist` | Distribution in the size of emitted messages. | Bytes +New in v23.1.29: `total_ranges` | Total number of ranges that are watched by [aggregator processors]({% link {{ page.version.version }}/how-does-an-enterprise-changefeed-work.md %}) participating in the changefeed job. `changefeed.total_ranges` shares the same polling interval as the [`changefeed.lagging_ranges`](#lagging-ranges-metric) metric, which is controlled by the `changefeed.lagging_ranges_polling_interval` [cluster setting]({% link {{ page.version.version }}/cluster-settings.md %}). For more details, refer to [Lagging ranges](#lagging-ranges). ### Monitoring and measuring changefeed latency diff --git a/src/current/v23.2/monitor-and-debug-changefeeds.md b/src/current/v23.2/monitor-and-debug-changefeeds.md index d01a020ec8d..0e3b515696e 100644 --- a/src/current/v23.2/monitor-and-debug-changefeeds.md +++ b/src/current/v23.2/monitor-and-debug-changefeeds.md @@ -156,6 +156,7 @@ changefeed_emitted_bytes{scope="vehicles"} 183557 `error_retries` | Total retryable errors encountered by changefeeds. | Errors `backfill_pending_ranges` | Number of [ranges]({% link {{ page.version.version }}/architecture/overview.md %}#architecture-range) in an ongoing backfill that are yet to be fully emitted. | Ranges `message_size_hist` | Distribution in the size of emitted messages. | Bytes +New in v23.2.13: `total_ranges` | Total number of ranges that are watched by [aggregator processors]({% link {{ page.version.version }}/how-does-an-enterprise-changefeed-work.md %}) participating in the changefeed job. `changefeed.total_ranges` shares the same polling interval as the [`changefeed.lagging_ranges`](#lagging-ranges-metric) metric, which is controlled by the `lagging_ranges_polling_interval` option. For more details, refer to [Lagging ranges](#lagging-ranges). ### Monitoring and measuring changefeed latency diff --git a/src/current/v24.1/monitor-and-debug-changefeeds.md b/src/current/v24.1/monitor-and-debug-changefeeds.md index 44e0de3aab3..caf985a5b9b 100644 --- a/src/current/v24.1/monitor-and-debug-changefeeds.md +++ b/src/current/v24.1/monitor-and-debug-changefeeds.md @@ -153,6 +153,7 @@ changefeed_emitted_bytes{scope="vehicles"} 183557 `changefeed.message_size_hist` | Distribution in the size of emitted messages. | Bytes | Histogram `changefeed.running` | Number of currently running changefeeds, including sinkless changefeeds. | Changefeeds | Gauge `changefeed.sink_batch_hist_nanos` | Time messages spend batched in the sink buffer before being flushed and acknowledged. | Nanoseconds | Histogram +New in v24.1.6: `changefeed.total_ranges` | Total number of ranges that are watched by [aggregator processors]({% link {{ page.version.version }}/how-does-an-enterprise-changefeed-work.md %}) participating in the changefeed job. `changefeed.total_ranges` shares the same polling interval as the [`changefeed.lagging_ranges`](#lagging-ranges-metric) metric, which is controlled by the `lagging_ranges_polling_interval` option. For more details, refer to [Lagging ranges](#lagging-ranges). ### Monitoring and measuring changefeed latency diff --git a/src/current/v24.2/monitor-and-debug-changefeeds.md b/src/current/v24.2/monitor-and-debug-changefeeds.md index eb17ffe4f72..64d58d7174a 100644 --- a/src/current/v24.2/monitor-and-debug-changefeeds.md +++ b/src/current/v24.2/monitor-and-debug-changefeeds.md @@ -153,6 +153,7 @@ changefeed_emitted_bytes{scope="vehicles"} 183557 `changefeed.message_size_hist` | Distribution in the size of emitted messages. | Bytes | Histogram `changefeed.running` | Number of currently running changefeeds, including sinkless changefeeds. | Changefeeds | Gauge `changefeed.sink_batch_hist_nanos` | Time messages spend batched in the sink buffer before being flushed and acknowledged. | Nanoseconds | Histogram +New in v24.2.4: `changefeed.total_ranges` | Total number of ranges that are watched by [aggregator processors]({% link {{ page.version.version }}/how-does-an-enterprise-changefeed-work.md %}) participating in the changefeed job. `changefeed.total_ranges` shares the same polling interval as the [`changefeed.lagging_ranges`](#lagging-ranges-metric) metric, which is controlled by the `lagging_ranges_polling_interval` option. For more details, refer to [Lagging ranges](#lagging-ranges). ### Monitoring and measuring changefeed latency diff --git a/src/current/v24.3/monitor-and-debug-changefeeds.md b/src/current/v24.3/monitor-and-debug-changefeeds.md index bcdfd17293e..8ee4a0f5aa8 100644 --- a/src/current/v24.3/monitor-and-debug-changefeeds.md +++ b/src/current/v24.3/monitor-and-debug-changefeeds.md @@ -153,6 +153,7 @@ changefeed_emitted_bytes{scope="vehicles"} 183557 `changefeed.message_size_hist` | Distribution in the size of emitted messages. | Bytes | Histogram `changefeed.running` | Number of currently running changefeeds, including sinkless changefeeds. | Changefeeds | Gauge `changefeed.sink_batch_hist_nanos` | Time messages spend batched in the sink buffer before being flushed and acknowledged. | Nanoseconds | Histogram +New in v24.3: `changefeed.total_ranges` | Total number of ranges that are watched by [aggregator processors]({% link {{ page.version.version }}/how-does-an-enterprise-changefeed-work.md %}) participating in the changefeed job. `changefeed.total_ranges` shares the same polling interval as the [`changefeed.lagging_ranges`](#lagging-ranges-metric) metric, which is controlled by the `lagging_ranges_polling_interval` option. For more details, refer to [Lagging ranges](#lagging-ranges). ### Monitoring and measuring changefeed latency From 7c9b252fdb23211d716cf991a8334b049c457ee7 Mon Sep 17 00:00:00 2001 From: Kathryn May <44557882+kathancox@users.noreply.github.com> Date: Thu, 12 Dec 2024 10:14:56 -0500 Subject: [PATCH 03/36] Correct link to cdc KLs on get started page (#19220) --- src/current/v23.2/create-and-configure-changefeeds.md | 2 +- src/current/v24.1/create-and-configure-changefeeds.md | 2 +- src/current/v24.2/create-and-configure-changefeeds.md | 2 +- src/current/v24.3/create-and-configure-changefeeds.md | 2 +- 4 files changed, 4 insertions(+), 4 deletions(-) diff --git a/src/current/v23.2/create-and-configure-changefeeds.md b/src/current/v23.2/create-and-configure-changefeeds.md index 2cb7b8a6120..53fab8d09f8 100644 --- a/src/current/v23.2/create-and-configure-changefeeds.md +++ b/src/current/v23.2/create-and-configure-changefeeds.md @@ -68,7 +68,7 @@ To maintain a high number of changefeeds in your cluster: ### Considerations - If you require [`resolved`]({% link {{ page.version.version }}/create-changefeed.md %}#resolved-option) message frequency under `30s`, then you **must** set the [`min_checkpoint_frequency`]({% link {{ page.version.version }}/create-changefeed.md %}#min-checkpoint-frequency) option to at least the desired `resolved` frequency. -- Many DDL queries (including [`TRUNCATE`]({% link {{ page.version.version }}/truncate.md %}), [`DROP TABLE`]({% link {{ page.version.version }}/drop-table.md %}), and queries that add a column family) will cause errors on a changefeed watching the affected tables. You will need to [start a new changefeed]({% link {{ page.version.version }}/create-changefeed.md %}#start-a-new-changefeed-where-another-ended). If a table is truncated that a changefeed with `on_error='pause'` is watching, you will also need to start a new changefeed. See change data capture [Known Limitations]({% link {{ page.version.version }}/change-data-capture-overview.md %}) for more detail. +- Many DDL queries (including [`TRUNCATE`]({% link {{ page.version.version }}/truncate.md %}), [`DROP TABLE`]({% link {{ page.version.version }}/drop-table.md %}), and queries that add a column family) will cause errors on a changefeed watching the affected tables. You will need to [start a new changefeed]({% link {{ page.version.version }}/create-changefeed.md %}#start-a-new-changefeed-where-another-ended). If a table is truncated that a changefeed with `on_error='pause'` is watching, you will also need to start a new changefeed. Refer to the change data capture [Known Limitations](#known-limitations) for more detail. - Partial or intermittent sink unavailability may impact changefeed stability. If a sink is unavailable, messages can't send, which means that a changefeed's high-water mark timestamp is at risk of falling behind the cluster's [garbage collection window]({% link {{ page.version.version }}/configure-replication-zones.md %}#replication-zone-variables). Throughput and latency can be affected once the sink is available again. However, [ordering guarantees]({% link {{ page.version.version }}/changefeed-messages.md %}#ordering-and-delivery-guarantees) will still hold for as long as a changefeed [remains active]({% link {{ page.version.version }}/monitor-and-debug-changefeeds.md %}#monitor-a-changefeed). - When an [`IMPORT INTO`]({% link {{ page.version.version }}/import-into.md %}) statement is run, any current changefeed jobs targeting that table will fail. - After you [restore from a full-cluster backup]({% link {{ page.version.version }}/restore.md %}#full-cluster), changefeed jobs will **not** resume on the new cluster. It is necessary to manually create the changefeeds following the full-cluster restore. diff --git a/src/current/v24.1/create-and-configure-changefeeds.md b/src/current/v24.1/create-and-configure-changefeeds.md index ac4e4f08067..b56237149a4 100644 --- a/src/current/v24.1/create-and-configure-changefeeds.md +++ b/src/current/v24.1/create-and-configure-changefeeds.md @@ -68,7 +68,7 @@ To maintain a high number of changefeeds in your cluster: ### Considerations - If you require [`resolved`]({% link {{ page.version.version }}/create-changefeed.md %}#resolved) message frequency under `30s`, then you **must** set the [`min_checkpoint_frequency`]({% link {{ page.version.version }}/create-changefeed.md %}#min-checkpoint-frequency) option to at least the desired `resolved` frequency. -- Many DDL queries (including [`TRUNCATE`]({% link {{ page.version.version }}/truncate.md %}), [`DROP TABLE`]({% link {{ page.version.version }}/drop-table.md %}), and queries that add a column family) will cause errors on a changefeed watching the affected tables. You will need to [start a new changefeed]({% link {{ page.version.version }}/create-changefeed.md %}#start-a-new-changefeed-where-another-ended). If a table is truncated that a changefeed with `on_error='pause'` is watching, you will also need to start a new changefeed. See change data capture [Known Limitations]({% link {{ page.version.version }}/change-data-capture-overview.md %}) for more detail. +- Many DDL queries (including [`TRUNCATE`]({% link {{ page.version.version }}/truncate.md %}), [`DROP TABLE`]({% link {{ page.version.version }}/drop-table.md %}), and queries that add a column family) will cause errors on a changefeed watching the affected tables. You will need to [start a new changefeed]({% link {{ page.version.version }}/create-changefeed.md %}#start-a-new-changefeed-where-another-ended). If a table is truncated that a changefeed with `on_error='pause'` is watching, you will also need to start a new changefeed. Refer to the change data capture [Known Limitations](#known-limitations) for more detail. - Partial or intermittent sink unavailability may impact changefeed stability. If a sink is unavailable, messages can't send, which means that a changefeed's high-water mark timestamp is at risk of falling behind the cluster's [garbage collection window]({% link {{ page.version.version }}/configure-replication-zones.md %}#replication-zone-variables). Throughput and latency can be affected once the sink is available again. However, [ordering guarantees]({% link {{ page.version.version }}/changefeed-messages.md %}#ordering-and-delivery-guarantees) will still hold for as long as a changefeed [remains active]({% link {{ page.version.version }}/monitor-and-debug-changefeeds.md %}#monitor-a-changefeed). - When an [`IMPORT INTO`]({% link {{ page.version.version }}/import-into.md %}) statement is run, any current changefeed jobs targeting that table will fail. - After you [restore from a full-cluster backup]({% link {{ page.version.version }}/restore.md %}#full-cluster), changefeed jobs will **not** resume on the new cluster. It is necessary to manually create the changefeeds following the full-cluster restore. diff --git a/src/current/v24.2/create-and-configure-changefeeds.md b/src/current/v24.2/create-and-configure-changefeeds.md index 201c023e128..69c3a1b8e47 100644 --- a/src/current/v24.2/create-and-configure-changefeeds.md +++ b/src/current/v24.2/create-and-configure-changefeeds.md @@ -68,7 +68,7 @@ To maintain a high number of changefeeds in your cluster: ### Considerations - If you require [`resolved`]({% link {{ page.version.version }}/create-changefeed.md %}#resolved) message frequency under `30s`, then you **must** set the [`min_checkpoint_frequency`]({% link {{ page.version.version }}/create-changefeed.md %}#min-checkpoint-frequency) option to at least the desired `resolved` frequency. -- Many DDL queries (including [`TRUNCATE`]({% link {{ page.version.version }}/truncate.md %}), [`DROP TABLE`]({% link {{ page.version.version }}/drop-table.md %}), and queries that add a column family) will cause errors on a changefeed watching the affected tables. You will need to [start a new changefeed]({% link {{ page.version.version }}/create-changefeed.md %}#start-a-new-changefeed-where-another-ended). If a table is truncated that a changefeed with `on_error='pause'` is watching, you will also need to start a new changefeed. See change data capture [Known Limitations]({% link {{ page.version.version }}/change-data-capture-overview.md %}) for more detail. +- Many DDL queries (including [`TRUNCATE`]({% link {{ page.version.version }}/truncate.md %}), [`DROP TABLE`]({% link {{ page.version.version }}/drop-table.md %}), and queries that add a column family) will cause errors on a changefeed watching the affected tables. You will need to [start a new changefeed]({% link {{ page.version.version }}/create-changefeed.md %}#start-a-new-changefeed-where-another-ended). If a table is truncated that a changefeed with `on_error='pause'` is watching, you will also need to start a new changefeed. Refer to the change data capture [Known Limitations](#known-limitations) for more detail. - Partial or intermittent sink unavailability may impact changefeed stability. If a sink is unavailable, messages can't send, which means that a changefeed's high-water mark timestamp is at risk of falling behind the cluster's [garbage collection window]({% link {{ page.version.version }}/configure-replication-zones.md %}#replication-zone-variables). Throughput and latency can be affected once the sink is available again. However, [ordering guarantees]({% link {{ page.version.version }}/changefeed-messages.md %}#ordering-and-delivery-guarantees) will still hold for as long as a changefeed [remains active]({% link {{ page.version.version }}/monitor-and-debug-changefeeds.md %}#monitor-a-changefeed). - When an [`IMPORT INTO`]({% link {{ page.version.version }}/import-into.md %}) statement is run, any current changefeed jobs targeting that table will fail. - After you [restore from a full-cluster backup]({% link {{ page.version.version }}/restore.md %}#full-cluster), changefeed jobs will **not** resume on the new cluster. It is necessary to manually create the changefeeds following the full-cluster restore. diff --git a/src/current/v24.3/create-and-configure-changefeeds.md b/src/current/v24.3/create-and-configure-changefeeds.md index 494c1653c58..112f2a32410 100644 --- a/src/current/v24.3/create-and-configure-changefeeds.md +++ b/src/current/v24.3/create-and-configure-changefeeds.md @@ -46,7 +46,7 @@ For further detail on performance-related configuration, refer to the [Advanced ### Considerations - If you require [`resolved`]({% link {{ page.version.version }}/create-changefeed.md %}#resolved) message frequency under `30s`, then you **must** set the [`min_checkpoint_frequency`]({% link {{ page.version.version }}/create-changefeed.md %}#min-checkpoint-frequency) option to at least the desired `resolved` frequency. -- Many DDL queries (including [`TRUNCATE`]({% link {{ page.version.version }}/truncate.md %}), [`DROP TABLE`]({% link {{ page.version.version }}/drop-table.md %}), and queries that add a column family) will cause errors on a changefeed watching the affected tables. You will need to [start a new changefeed]({% link {{ page.version.version }}/create-changefeed.md %}#start-a-new-changefeed-where-another-ended). If a table is truncated that a changefeed with `on_error='pause'` is watching, you will also need to start a new changefeed. See change data capture [Known Limitations]({% link {{ page.version.version }}/change-data-capture-overview.md %}) for more detail. +- Many DDL queries (including [`TRUNCATE`]({% link {{ page.version.version }}/truncate.md %}), [`DROP TABLE`]({% link {{ page.version.version }}/drop-table.md %}), and queries that add a column family) will cause errors on a changefeed watching the affected tables. You will need to [start a new changefeed]({% link {{ page.version.version }}/create-changefeed.md %}#start-a-new-changefeed-where-another-ended). If a table is truncated that a changefeed with `on_error='pause'` is watching, you will also need to start a new changefeed. Refer to the change data capture [Known Limitations](#known-limitations) for more detail. - Partial or intermittent sink unavailability may impact changefeed stability. If a sink is unavailable, messages can't send, which means that a changefeed's high-water mark timestamp is at risk of falling behind the cluster's [garbage collection window]({% link {{ page.version.version }}/configure-replication-zones.md %}#replication-zone-variables). Throughput and latency can be affected once the sink is available again. However, [ordering guarantees]({% link {{ page.version.version }}/changefeed-messages.md %}#ordering-and-delivery-guarantees) will still hold for as long as a changefeed [remains active]({% link {{ page.version.version }}/monitor-and-debug-changefeeds.md %}#monitor-a-changefeed). - When an [`IMPORT INTO`]({% link {{ page.version.version }}/import-into.md %}) statement is run, any current changefeed jobs targeting that table will fail. - After you [restore from a full-cluster backup]({% link {{ page.version.version }}/restore.md %}#full-cluster), changefeed jobs will **not** resume on the new cluster. It is necessary to manually create the changefeeds following the full-cluster restore. From aa841414f0a018f2a30a1011ed6e25acd56f2ff7 Mon Sep 17 00:00:00 2001 From: Mike Lewis <76072290+mikeCRL@users.noreply.github.com> Date: Thu, 12 Dec 2024 10:53:29 -0500 Subject: [PATCH 04/36] Add Google Cloud Marketplace billing (#19219) --- .../cockroachcloud/billing-management.md | 60 ++++++++++++++----- 1 file changed, 44 insertions(+), 16 deletions(-) diff --git a/src/current/cockroachcloud/billing-management.md b/src/current/cockroachcloud/billing-management.md index f0343c2e413..3507ce4d8df 100644 --- a/src/current/cockroachcloud/billing-management.md +++ b/src/current/cockroachcloud/billing-management.md @@ -11,11 +11,12 @@ The **Billing** page contains an overview of your charges and the payment detail Users with the [Billing Coordinator]({% link cockroachcloud/authorization.md %}#billing-coordinator) role can manage billing for the organization. -You can pay for CockroachDB {{ site.data.products.cloud }} by using a credit card, or you can set up billing through [AWS Marketplace](https://aws.amazon.com/marketplace), along with other AWS charges. +You can pay for CockroachDB {{ site.data.products.cloud }} by using a credit card, or you can set up billing through [AWS Marketplace](https://aws.amazon.com/marketplace) or [Google Cloud Marketplace](https://cloud.google.com/marketplace).
- + +
@@ -27,7 +28,7 @@ You can pay for CockroachDB {{ site.data.products.cloud }} by using a credit car 1. In the **Edit payment method** dialog, enter the credit or debit card details. 1. Click **Save card**. 1. Click **Add a billing email** in the **Billing contact info** section. -1. In the **Edit billing email*** the email address at which you want to get invoices for the organization. +1. In **Edit billing email** enter the email address at which you want to receive invoices for the organization. 1. Click **Submit**. 1. Click **Add a billing address** in the **Billing contact info** section. 1. Enter the address associated with your payment method. This address appears on your monthly invoice and should be the legal address of your home or business. @@ -37,7 +38,7 @@ You can pay for CockroachDB {{ site.data.products.cloud }} by using a credit car 1. On the **Billing** page, select the **Payment details** tab. 1. Click the pencil icon for the **Email** field under **Billing contact info**. -1. Enter the new email address at which you want get invoices for the organization. +1. Enter the new email address at which you want to receive invoices for the organization. 1. Click **Submit**. ## Change the payment method @@ -54,17 +55,9 @@ You can pay for CockroachDB {{ site.data.products.cloud }} by using a credit car We keep a card on file after the associated organization is deleted so we can process pending charges. You can [contact Support](https://support.cockroachlabs.com) to remove your card once your charges are settled. -## Check trial code details - -If you used a CockroachDB {{ site.data.products.dedicated }} trial code while [creating a cluster]({% link cockroachcloud/create-an-advanced-cluster.md %}#step-7-enter-billing-details), you can check the trial expiration details on the **Overview** tab of the **Billing** page. - -{{site.data.alerts.callout_info}} -Your credit card will be charged after the trial ends. -{{site.data.alerts.end}} -
-
+
## Subscribe through AWS Marketplace @@ -76,7 +69,7 @@ To subscribe to CockroachDB {{ site.data.products.cloud }} through the AWS Marke 1. If you have access to multiple CockroachDB {{ site.data.products.cloud }} organizations, select an organization to update its billing configuration. {{site.data.alerts.callout_info}} - Organizations that are configured for invoice billing or are already linked to AWS Marketplace are not listed. + Organizations that are configured for invoice billing or are already linked to a cloud marketplace are not listed. {{site.data.alerts.end}} @@ -86,7 +79,7 @@ To subscribe to CockroachDB {{ site.data.products.cloud }} through the AWS Marke ## Unsubscribe from AWS Marketplace {{ site.data.alerts.callout_danger }} -After you unsubscribe your CockroachDB {{ site.data.products.cloud }} from AWS Marketplace billing, clusters will be deleted and applications may be disrupted unless you [add a credit card](#add-a-credit-card) within 6 hours after you unsubscribe from AWS Marketplace billing. +After you unsubscribe your CockroachDB {{ site.data.products.cloud }} from AWS Marketplace billing, clusters will be deleted and applications may be disrupted unless you add an alternate payment method within 6 hours after you unsubscribe from AWS Marketplace billing. {{ site.data.alerts.end }} @@ -99,9 +92,44 @@ To unsubscribe from CockroachDB {{ site.data.products.cloud }}:
+
+ +## Subscribe through Google Cloud Marketplace + +To subscribe to CockroachDB Cloud through the Google Cloud Marketplace: + +1. From the [Google Cloud Marketplace page for CockroachDB (pay-as-you-go)](https://console.cloud.google.com/marketplace/product/cockroachlabs/cockroachdb-pay-as-you-go), click **Subscribe** to open the **Order Summary** page +2. **Select a billing account** and agree to **Additional Terms**, then click **Subscribe**. +3. Click **Go to Product Page**. You will be redirected to the [Google Cloud Marketplace page for CockroachDB (pay-as-you-go)](https://console.cloud.google.com/marketplace/product/cockroachlabs/cockroachdb-pay-as-you-go). +4. From the product page, click **Manage on Provider**. You will be redirected to the CockroachDB Cloud console. +5. [Register]({% link cockroachcloud/create-an-account.md %}#register-a-new-account) a new CockroachDB Cloud account or sign in to your existing account. +6. If you have access to multiple CockroachDB Cloud organizations, select an organization to update its billing configuration. + {{site.data.alerts.callout_info}} + Organizations that are configured for invoice billing or are already linked to a cloud marketplace are not listed. + {{site.data.alerts.end}} +7. Click **Subscribe to Google Cloud Marketplace**. + {{site.data.alerts.callout_info}} + If your Google Account was previously subscribed to CockroachDB (pay-as-you-go) through the Google Cloud Marketplace and you are unable to re-subscribe, please contact our [support team](https://support.cockroachlabs.com) for assistance. + {{site.data.alerts.end}} + +## Unsubscribe from Google Cloud Marketplace + +{{ site.data.alerts.callout_danger }} +After you unsubscribe your CockroachDB {{ site.data.products.cloud }} from Google Cloud Marketplace billing, clusters will be deleted and applications may be disrupted unless you add an alternate payment method within 6 hours after you unsubscribe from AWS Marketplace billing. +{{ site.data.alerts.end }} + +To unsubscribe from CockroachDB Cloud: + +1. On the [Google Cloud Marketplace orders](https://console.cloud.google.com/marketplace/orders) page, **Select a billing account** from the dropdown. +2. In the list of subscriptions, find **CockroachDB (pay-as-you-go)**. +3. Click **Cancel Order** from the actions menu. +4. Follow the instructions in the Cancel Order dialog, then click **Cancel Order** to unsubscribe your organization from billing through Google Cloud Marketplace. + +
+ ## View Credits balance -If your organization has an annual contract with CockroachDB {{ site.data.products.cloud }}, the **Overview** tab of the **Billing** page will display the amount of available CockroachDB {{ site.data.products.cloud }} Credits you have remaining and the average number of Credits your organization consumes per day. At the end of each billing period, invoice payments will be deducted from your organization's Credits. +If your organization has an annual contract with CockroachDB {{ site.data.products.cloud }}, the **Overview** tab of the **Billing** page displays the amount of available CockroachDB {{ site.data.products.cloud }} Credits you have remaining and the average number of Credits your organization consumes per day. At the end of each billing period, invoice payments are deducted from your organization's Credits. Under the **Credits** section, you can see more information about each of your organization's contracts. Contracts are listed in the order in which they will be used. From 9ed50688a77220c26060d3fcecb8f834930d40c0 Mon Sep 17 00:00:00 2001 From: Mike Lewis <76072290+mikeCRL@users.noreply.github.com> Date: Thu, 12 Dec 2024 11:03:37 -0500 Subject: [PATCH 05/36] Update list of major releases in Cloud policy to add v24.3 (#19222) --- src/current/cockroachcloud/upgrade-policy.md | 1 + 1 file changed, 1 insertion(+) diff --git a/src/current/cockroachcloud/upgrade-policy.md b/src/current/cockroachcloud/upgrade-policy.md index 5189e217a29..7f479648d5f 100644 --- a/src/current/cockroachcloud/upgrade-policy.md +++ b/src/current/cockroachcloud/upgrade-policy.md @@ -31,6 +31,7 @@ Version | Release Type | Support period | Release date | EOS date v23.2 | Regular | 12 months | 2024-02-05 | 2025-02-05 v24.1 | Regular | 12 months | 2024-05-20 | 2025-05-20 v24.2 | Innovation | 6 months | 2024-08-12 | 2025-02-12 +v24.3 | Regular | 12 months | 2024-11-18 | 2025-11-18 For expected future versions, refer to [Upcoming releases]({% link releases/index.md %}#upcoming-releases). From ca0d4bf86076bf511d7fe78ce19968b60511e9fe Mon Sep 17 00:00:00 2001 From: Florence Morris Date: Thu, 12 Dec 2024 11:24:33 -0500 Subject: [PATCH 06/36] DOC-11714 Available Metrics audit (#19148) (1) Updated metric-names.md to use information from _data/metrics files. (2) Added _data/metrics subdirectory with (a) metrics-list.csv generated using v24.3.0-beta.3 ./cockroach gen metric-list --format=csv > metrics-list.csv, (b) available-metrics-in-metrics-list.csv downloaded from Metrics v24.3.0-beta.3 google sheet and manually added 24.3 metrics and expanded job metrics, (c) available-metrics-not-in-metrics-list.csv manually edited from certificates_metrics.go. --- .../available-metrics-in-metrics-list.csv | 477 ++++++++++++++++++ .../available-metrics-not-in-metrics-list.csv | 19 + src/current/_data/metrics/metrics-list.csv | 224 +++++++- src/current/_includes/v24.3/metric-names.md | 360 ++----------- 4 files changed, 727 insertions(+), 353 deletions(-) create mode 100644 src/current/_data/metrics/available-metrics-in-metrics-list.csv create mode 100644 src/current/_data/metrics/available-metrics-not-in-metrics-list.csv diff --git a/src/current/_data/metrics/available-metrics-in-metrics-list.csv b/src/current/_data/metrics/available-metrics-in-metrics-list.csv new file mode 100644 index 00000000000..ce8aa57543a --- /dev/null +++ b/src/current/_data/metrics/available-metrics-in-metrics-list.csv @@ -0,0 +1,477 @@ +metric_id +addsstable.applications +addsstable.copies +addsstable.proposals +admission.io.overload +capacity +capacity.available +capacity.reserved +capacity.used +exec.error +exec.latency +exec.success +gcbytesage +gossip.bytes.received +gossip.bytes.sent +gossip.connections.incoming +gossip.connections.outgoing +gossip.connections.refused +gossip.infos.received +gossip.infos.sent +intentage +intentbytes +intentcount +keybytes +keycount +leases.epoch +leases.error +leases.expiration +leases.success +leases.transfers.error +leases.transfers.success +livebytes +livecount +liveness.epochincrements +liveness.heartbeatfailures +liveness.heartbeatlatency +liveness.heartbeatsuccesses +liveness.livenodes +node-id +queue.consistency.pending +queue.consistency.process.failure +queue.consistency.process.success +queue.consistency.processingnanos +queue.gc.info.abortspanconsidered +queue.gc.info.abortspangcnum +queue.gc.info.abortspanscanned +queue.gc.info.clearrangefailed +queue.gc.info.clearrangesuccess +queue.gc.info.intentsconsidered +queue.gc.info.intenttxns +queue.gc.info.numkeysaffected +queue.gc.info.pushtxn +queue.gc.info.resolvesuccess +queue.gc.info.resolvetotal +queue.gc.info.transactionspangcaborted +queue.gc.info.transactionspangccommitted +queue.gc.info.transactionspangcpending +queue.gc.info.transactionspanscanned +queue.gc.pending +queue.gc.process.failure +queue.gc.process.success +queue.gc.processingnanos +queue.raftlog.pending +queue.raftlog.process.failure +queue.raftlog.process.success +queue.raftlog.processingnanos +queue.raftsnapshot.pending +queue.raftsnapshot.process.failure +queue.raftsnapshot.process.success +queue.raftsnapshot.processingnanos +queue.replicagc.pending +queue.replicagc.process.failure +queue.replicagc.process.success +queue.replicagc.processingnanos +queue.replicagc.removereplica +queue.replicate.addreplica +queue.replicate.addreplica.error +queue.replicate.addreplica.success +queue.replicate.pending +queue.replicate.process.failure +queue.replicate.process.success +queue.replicate.processingnanos +queue.replicate.purgatory +queue.replicate.rebalancereplica +queue.replicate.removedeadreplica +queue.replicate.removedeadreplica.error +queue.replicate.removedeadreplica.success +queue.replicate.removedecommissioningreplica.error +queue.replicate.removedecommissioningreplica.success +queue.replicate.removereplica +queue.replicate.removereplica.error +queue.replicate.removereplica.success +queue.replicate.replacedeadreplica.error +queue.replicate.replacedeadreplica.success +queue.replicate.replacedecommissioningreplica.error +queue.replicate.replacedecommissioningreplica.success +queue.replicate.transferlease +queue.split.pending +queue.split.process.failure +queue.split.process.success +queue.split.processingnanos +queue.tsmaintenance.pending +queue.tsmaintenance.process.failure +queue.tsmaintenance.process.success +queue.tsmaintenance.processingnanos +raft.commandsapplied +raft.heartbeats.pending +raft.process.commandcommit.latency +raft.process.logcommit.latency +raft.process.tickingnanos +raft.process.workingnanos +raft.rcvd.app +raft.rcvd.appresp +raft.rcvd.dropped +raft.rcvd.heartbeat +raft.rcvd.heartbeatresp +raft.rcvd.prevote +raft.rcvd.prevoteresp +raft.rcvd.prop +raft.rcvd.snap +raft.rcvd.timeoutnow +raft.rcvd.transferleader +raft.rcvd.vote +raft.rcvd.voteresp +raft.ticks +raftlog.behind +raftlog.truncated +range.adds +range.merges +range.raftleadertransfers +range.removes +range.snapshots.generated +range.snapshots.rcvd-bytes +range.snapshots.rebalancing.rcvd-bytes +range.snapshots.rebalancing.sent-bytes +range.snapshots.recovery.rcvd-bytes +range.snapshots.recovery.sent-bytes +range.snapshots.recv-in-progress +range.snapshots.recv-queue +range.snapshots.recv-total-in-progress +range.snapshots.send-in-progress +range.snapshots.send-queue +range.snapshots.send-total-in-progress +range.snapshots.sent-bytes +range.snapshots.unknown.rcvd-bytes +range.snapshots.unknown.sent-bytes +range.splits +rangekeybytes +rangekeycount +ranges +ranges.overreplicated +ranges.unavailable +ranges.underreplicated +rangevalbytes +rangevalcount +rebalancing.queriespersecond +rebalancing.readbytespersecond +rebalancing.readspersecond +rebalancing.requestspersecond +rebalancing.writebytespersecond +rebalancing.writespersecond +replicas +replicas.leaders +replicas.leaders_invalid_lease +replicas.leaders_not_leaseholders +replicas.leaseholders +replicas.quiescent +replicas.reserved +requests.backpressure.split +requests.slow.lease +requests.slow.raft +rocksdb.block.cache.hits +rocksdb.block.cache.misses +rocksdb.block.cache.usage +rocksdb.bloom.filter.prefix.checked +rocksdb.bloom.filter.prefix.useful +rocksdb.compactions +rocksdb.flushes +rocksdb.memtable.total-size +rocksdb.num-sstables +rocksdb.read-amplification +rocksdb.table-readers-mem-estimate +storage.keys.range-key-set.count +storage.l0-level-score +storage.l0-level-size +storage.l0-num-files +storage.l0-sublevels +storage.l1-level-score +storage.l1-level-size +storage.l2-level-score +storage.l2-level-size +storage.l3-level-score +storage.l3-level-size +storage.l4-level-score +storage.l4-level-size +storage.l5-level-score +storage.l5-level-size +storage.l6-level-score +storage.l6-level-size +storage.marked-for-compaction-files +storage.write-stalls +sysbytes +syscount +tenant.consumption.cross_region_network_ru +tenant.consumption.external_io_egress_bytes +tenant.consumption.pgwire_egress_bytes +tenant.consumption.read_batches +tenant.consumption.read_bytes +tenant.consumption.read_requests +tenant.consumption.request_units +tenant.consumption.sql_pods_cpu_seconds +tenant.consumption.write_batches +tenant.consumption.write_bytes +tenant.consumption.write_requests +timeseries.write.bytes +timeseries.write.errors +timeseries.write.samples +totalbytes +txnwaitqueue.deadlocks_total +valbytes +valcount +changefeed.aggregator_progress +changefeed.backfill_count +changefeed.backfill_pending_ranges +changefeed.checkpoint_progress +changefeed.commit_latency +changefeed.emitted_bytes +changefeed.emitted_messages +changefeed.error_retries +changefeed.failures +changefeed.lagging_ranges +changefeed.max_behind_nanos +changefeed.message_size_hist +changefeed.running +clock-offset.meannanos +clock-offset.stddevnanos +cluster.preserve-downgrade-option.last-updated +distsender.batches +distsender.batches.partial +distsender.errors.notleaseholder +distsender.rpc.sent +distsender.rpc.sent.local +distsender.rpc.sent.nextreplicaerror +jobs.auto_create_stats.currently_paused +jobs.auto_create_stats.currently_running +jobs.auto_create_stats.resume_failed +jobs.backup.currently_paused +jobs.backup.currently_running +jobs.changefeed.currently_paused +jobs.changefeed.expired_pts_records +jobs.changefeed.protected_age_sec +jobs.changefeed.resume_retry_error +jobs.create_stats.currently_running +jobs.row_level_ttl.currently_paused +jobs.row_level_ttl.currently_running +jobs.row_level_ttl.delete_duration +jobs.row_level_ttl.num_active_spans +jobs.row_level_ttl.resume_completed +jobs.row_level_ttl.resume_failed +jobs.row_level_ttl.rows_deleted +jobs.row_level_ttl.rows_selected +jobs.row_level_ttl.select_duration +jobs.row_level_ttl.span_total_duration +jobs.row_level_ttl.total_expired_rows +jobs.row_level_ttl.total_rows +physical_replication.logical_bytes +physical_replication.replicated_time_seconds +requests.slow.distsender +round-trip-latency +rpc.connection.avg_round_trip_latency +rpc.connection.failures +rpc.connection.healthy +rpc.connection.healthy_nanos +rpc.connection.heartbeats +rpc.connection.unhealthy +rpc.connection.unhealthy_nanos +schedules.BACKUP.failed +schedules.BACKUP.last-completed-time +schedules.BACKUP.protected_age_sec +schedules.BACKUP.protected_record_count +schedules.BACKUP.started +schedules.BACKUP.succeeded +schedules.scheduled-row-level-ttl-executor.failed +sql.bytesin +sql.bytesout +sql.conn.latency +sql.conns +sql.ddl.count +sql.delete.count +sql.distsql.contended_queries.count +sql.distsql.exec.latency +sql.distsql.flows.active +sql.distsql.flows.total +sql.distsql.queries.active +sql.distsql.queries.total +sql.distsql.select.count +sql.distsql.service.latency +sql.exec.latency +sql.failure.count +sql.full.scan.count +sql.guardrails.max_row_size_err.count +sql.guardrails.max_row_size_log.count +sql.insert.count +sql.mem.distsql.current +sql.mem.distsql.max +sql.mem.internal.session.current +sql.mem.internal.session.max +sql.mem.internal.txn.current +sql.mem.internal.txn.max +sql.mem.root.current +sql.mem.root.max +sql.misc.count +sql.new_conns +sql.pgwire_cancel.ignored +sql.pgwire_cancel.successful +sql.pgwire_cancel.total +sql.query.count +sql.select.count +sql.service.latency +sql.statements.active +sql.txn.abort.count +sql.txn.begin.count +sql.txn.commit.count +sql.txn.contended.count +sql.txn.latency +sql.txn.rollback.count +sql.txns.open +sql.update.count +tenant.sql_usage.cross_region_network_ru +tenant.sql_usage.estimated_cpu_seconds +tenant.sql_usage.external_io_egress_bytes +tenant.sql_usage.external_io_ingress_bytes +tenant.sql_usage.kv_request_units +tenant.sql_usage.pgwire_egress_bytes +tenant.sql_usage.provisioned_vcpus +tenant.sql_usage.read_batches +tenant.sql_usage.read_bytes +tenant.sql_usage.read_requests +tenant.sql_usage.request_units +tenant.sql_usage.sql_pods_cpu_seconds +tenant.sql_usage.write_batches +tenant.sql_usage.write_bytes +tenant.sql_usage.write_requests +txn.aborts +txn.commits +txn.commits1PC +txn.durations +txn.restarts +txn.restarts.asyncwritefailure +txn.restarts.readwithinuncertainty +txn.restarts.serializable +txn.restarts.txnaborted +txn.restarts.txnpush +txn.restarts.unknown +txn.restarts.writetooold +build.timestamp +sys.cgo.allocbytes +sys.cgo.totalbytes +sys.cgocalls +sys.cpu.combined.percent-normalized +sys.cpu.host.combined.percent-normalized +sys.cpu.sys.ns +sys.cpu.sys.percent +sys.cpu.user.ns +sys.cpu.user.percent +sys.fd.open +sys.fd.softlimit +sys.gc.count +sys.gc.pause.ns +sys.gc.pause.percent +sys.go.allocbytes +sys.go.totalbytes +sys.goroutines +sys.host.disk.iopsinprogress +sys.host.disk.read.bytes +sys.host.disk.read.count +sys.host.disk.write.bytes +sys.host.disk.write.count +sys.host.net.recv.bytes +sys.host.net.send.bytes +sys.rss +sys.runnable.goroutines.per.cpu +sys.totalmem +sys.uptime +jobs.auto_config_env_runner.currently_paused +jobs.auto_config_env_runner.protected_age_sec +jobs.auto_config_env_runner.protected_record_count +jobs.auto_config_runner.currently_paused +jobs.auto_config_runner.protected_age_sec +jobs.auto_config_runner.protected_record_count +jobs.auto_config_task.currently_paused +jobs.auto_config_task.protected_age_sec +jobs.auto_config_task.protected_record_count +jobs.auto_create_partial_stats.currently_paused +jobs.auto_create_partial_stats.protected_age_sec +jobs.auto_create_partial_stats.protected_record_count +jobs.auto_create_stats.protected_age_sec +jobs.auto_create_stats.protected_record_count +jobs.auto_schema_telemetry.currently_paused +jobs.auto_schema_telemetry.protected_age_sec +jobs.auto_schema_telemetry.protected_record_count +jobs.auto_span_config_reconciliation.currently_paused +jobs.auto_span_config_reconciliation.protected_age_sec +jobs.auto_span_config_reconciliation.protected_record_count +jobs.auto_sql_stats_compaction.currently_paused +jobs.auto_sql_stats_compaction.protected_age_sec +jobs.auto_sql_stats_compaction.protected_record_count +jobs.auto_update_sql_activity.currently_paused +jobs.auto_update_sql_activity.protected_age_sec +jobs.auto_update_sql_activity.protected_record_count +jobs.backup.protected_age_sec +jobs.backup.protected_record_count +jobs.changefeed.protected_record_count +jobs.create_stats.currently_paused +jobs.create_stats.protected_age_sec +jobs.create_stats.protected_record_count +jobs.history_retention.currently_paused +jobs.history_retention.protected_age_sec +jobs.history_retention.protected_record_count +jobs.import.currently_paused +jobs.import.protected_age_sec +jobs.import.protected_record_count +jobs.import_rollback.currently_paused +jobs.import_rollback.protected_age_sec +jobs.import_rollback.protected_record_count +jobs.key_visualizer.currently_paused +jobs.key_visualizer.protected_age_sec +jobs.key_visualizer.protected_record_count +jobs.logical_replication.currently_paused +jobs.logical_replication.protected_age_sec +jobs.logical_replication.protected_record_count +jobs.migration.currently_paused +jobs.migration.protected_age_sec +jobs.migration.protected_record_count +jobs.mvcc_statistics_update.currently_paused +jobs.mvcc_statistics_update.protected_age_sec +jobs.mvcc_statistics_update.protected_record_count +jobs.new_schema_change.currently_paused +jobs.new_schema_change.protected_age_sec +jobs.new_schema_change.protected_record_count +jobs.poll_jobs_stats.currently_paused +jobs.poll_jobs_stats.protected_age_sec +jobs.poll_jobs_stats.protected_record_count +jobs.replication_stream_ingestion.currently_paused +jobs.replication_stream_ingestion.protected_age_sec +jobs.replication_stream_ingestion.protected_record_count +jobs.replication_stream_producer.currently_paused +jobs.replication_stream_producer.protected_age_sec +jobs.replication_stream_producer.protected_record_count +jobs.restore.currently_paused +jobs.restore.protected_age_sec +jobs.restore.protected_record_count +jobs.row_level_ttl.protected_age_sec +jobs.row_level_ttl.protected_record_count +jobs.schema_change.currently_paused +jobs.schema_change.protected_age_sec +jobs.schema_change.protected_record_count +jobs.schema_change_gc.currently_paused +jobs.schema_change_gc.protected_age_sec +jobs.schema_change_gc.protected_record_count +jobs.standby_read_ts_poller.currently_paused +jobs.standby_read_ts_poller.protected_age_sec +jobs.standby_read_ts_poller.protected_record_count +jobs.typedesc_schema_change.currently_paused +jobs.typedesc_schema_change.protected_age_sec +jobs.typedesc_schema_change.protected_record_count +jobs.update_table_metadata_cache.currently_paused +jobs.update_table_metadata_cache.protected_age_sec +jobs.update_table_metadata_cache.protected_record_count +sql.crud_query.count +sql.crud_query.started.count +auth.cert.conn.latency +auth.gss.conn.latency +auth.jwt.conn.latency +auth.ldap.conn.latency +auth.password.conn.latency +auth.scram.conn.latency \ No newline at end of file diff --git a/src/current/_data/metrics/available-metrics-not-in-metrics-list.csv b/src/current/_data/metrics/available-metrics-not-in-metrics-list.csv new file mode 100644 index 00000000000..1cd86aace0a --- /dev/null +++ b/src/current/_data/metrics/available-metrics-not-in-metrics-list.csv @@ -0,0 +1,19 @@ +metric_id,description,y-axis label,type,unit +"security.certificate.expiration.ca","Expiration for the CA certificate. 0 means no certificate or error.","Certificate Expiration",GAUGE,TIMESTAMP_SEC +"security.certificate.expiration.client-ca","Expiration for the client CA certificate. 0 means no certificate or error.","Certificate Expiration",GAUGE,TIMESTAMP_SEC +"security.certificate.expiration.client","Minimum expiration for client certificates, labeled by SQL user. 0 means no certificate or error.","Certificate Expiration",GAUGE,TIMESTAMP_SEC +"security.certificate.expiration.ui-ca","Expiration for the UI CA certificate. 0 means no certificate or error.","Certificate Expiration",GAUGE,TIMESTAMP_SEC +"security.certificate.expiration.node","Expiration for the node certificate. 0 means no certificate or error.","Certificate Expiration",GAUGE,TIMESTAMP_SEC +"security.certificate.expiration.node-client","Expiration for the node's client certificate. 0 means no certificate or error.","Certificate Expiration",GAUGE,TIMESTAMP_SEC +"security.certificate.expiration.ui","Expiration for the UI certificate. 0 means no certificate or error.","Certificate Expiration",GAUGE,TIMESTAMP_SEC +"security.certificate.expiration.ca-client-tenant","Expiration for the Tenant Client CA certificate. 0 means no certificate or error.","Certificate Expiration",GAUGE,TIMESTAMP_SEC +"security.certificate.expiration.client-tenant","Expiration for the Tenant Client certificate. 0 means no certificate or error.","Certificate Expiration",GAUGE,TIMESTAMP_SEC +"security.certificate.ttl.ca","Seconds till expiration for the CA certificate. 0 means expired, no certificate or error.","Certificate TTL",GAUGE,TIMESTAMP_SEC +"security.certificate.ttl.client-ca","Seconds till expiration for the client CA certificate. 0 means expired, no certificate or error.","Certificate TTL",GAUGE,TIMESTAMP_SEC +"security.certificate.ttl.client","Seconds till expiration for the client certificates, labeled by SQL user. 0 means expired, no certificate or error.","Certificate TTL",GAUGE,TIMESTAMP_SEC +"security.certificate.ttl.ui-ca","Seconds till expiration for the UI CA certificate. 0 means expired, no certificate or error.","Certificate TTL",GAUGE,TIMESTAMP_SEC +"security.certificate.ttl.node","Seconds till expiration for the node certificate. 0 means expired, no certificate or error.","Certificate TTL",GAUGE,TIMESTAMP_SEC +"security.certificate.ttl.node-client","Seconds till expiration for the node's client certificate. 0 means expired, no certificate or error.","Certificate TTL",GAUGE,TIMESTAMP_SEC +"security.certificate.ttl.ui","Seconds till expiration for the UI certificate. 0 means expired, no certificate or error.","Certificate TTL",GAUGE,TIMESTAMP_SEC +"security.certificate.ttl.ca-client-tenant","Seconds till expiration for the Tenant Client CA certificate. 0 means expired, no certificate or error.","Certificate TTL",GAUGE,TIMESTAMP_SEC +"security.certificate.ttl.client-tenant","Seconds till expiration for the Tenant Client certificate. 0 means expired, no certificate or error.","Certificate TTL",GAUGE,TIMESTAMP_SEC \ No newline at end of file diff --git a/src/current/_data/metrics/metrics-list.csv b/src/current/_data/metrics/metrics-list.csv index 32a5cb72e41..6b195f89ac2 100644 --- a/src/current/_data/metrics/metrics-list.csv +++ b/src/current/_data/metrics/metrics-list.csv @@ -16,8 +16,8 @@ STORAGE,admission.admitted.elastic-cpu,Number of requests admitted,Requests,COUN STORAGE,admission.admitted.elastic-cpu.bulk-normal-pri,Number of requests admitted,Requests,COUNTER,COUNT,AVG,NON_NEGATIVE_DERIVATIVE STORAGE,admission.admitted.elastic-cpu.normal-pri,Number of requests admitted,Requests,COUNTER,COUNT,AVG,NON_NEGATIVE_DERIVATIVE STORAGE,admission.admitted.elastic-stores,Number of requests admitted,Requests,COUNTER,COUNT,AVG,NON_NEGATIVE_DERIVATIVE +STORAGE,admission.admitted.elastic-stores.bulk-low-pri,Number of requests admitted,Requests,COUNTER,COUNT,AVG,NON_NEGATIVE_DERIVATIVE STORAGE,admission.admitted.elastic-stores.bulk-normal-pri,Number of requests admitted,Requests,COUNTER,COUNT,AVG,NON_NEGATIVE_DERIVATIVE -STORAGE,admission.admitted.elastic-stores.ttl-low-pri,Number of requests admitted,Requests,COUNTER,COUNT,AVG,NON_NEGATIVE_DERIVATIVE STORAGE,admission.admitted.kv,Number of requests admitted,Requests,COUNTER,COUNT,AVG,NON_NEGATIVE_DERIVATIVE STORAGE,admission.admitted.kv-stores,Number of requests admitted,Requests,COUNTER,COUNT,AVG,NON_NEGATIVE_DERIVATIVE STORAGE,admission.admitted.kv-stores.high-pri,Number of requests admitted,Requests,COUNTER,COUNT,AVG,NON_NEGATIVE_DERIVATIVE @@ -51,8 +51,8 @@ STORAGE,admission.errored.elastic-cpu,Number of requests not admitted due to err STORAGE,admission.errored.elastic-cpu.bulk-normal-pri,Number of requests not admitted due to error,Requests,COUNTER,COUNT,AVG,NON_NEGATIVE_DERIVATIVE STORAGE,admission.errored.elastic-cpu.normal-pri,Number of requests not admitted due to error,Requests,COUNTER,COUNT,AVG,NON_NEGATIVE_DERIVATIVE STORAGE,admission.errored.elastic-stores,Number of requests not admitted due to error,Requests,COUNTER,COUNT,AVG,NON_NEGATIVE_DERIVATIVE +STORAGE,admission.errored.elastic-stores.bulk-low-pri,Number of requests not admitted due to error,Requests,COUNTER,COUNT,AVG,NON_NEGATIVE_DERIVATIVE STORAGE,admission.errored.elastic-stores.bulk-normal-pri,Number of requests not admitted due to error,Requests,COUNTER,COUNT,AVG,NON_NEGATIVE_DERIVATIVE -STORAGE,admission.errored.elastic-stores.ttl-low-pri,Number of requests not admitted due to error,Requests,COUNTER,COUNT,AVG,NON_NEGATIVE_DERIVATIVE STORAGE,admission.errored.kv,Number of requests not admitted due to error,Requests,COUNTER,COUNT,AVG,NON_NEGATIVE_DERIVATIVE STORAGE,admission.errored.kv-stores,Number of requests not admitted due to error,Requests,COUNTER,COUNT,AVG,NON_NEGATIVE_DERIVATIVE STORAGE,admission.errored.kv-stores.high-pri,Number of requests not admitted due to error,Requests,COUNTER,COUNT,AVG,NON_NEGATIVE_DERIVATIVE @@ -106,8 +106,8 @@ STORAGE,admission.requested.elastic-cpu,Number of requests,Requests,COUNTER,COUN STORAGE,admission.requested.elastic-cpu.bulk-normal-pri,Number of requests,Requests,COUNTER,COUNT,AVG,NON_NEGATIVE_DERIVATIVE STORAGE,admission.requested.elastic-cpu.normal-pri,Number of requests,Requests,COUNTER,COUNT,AVG,NON_NEGATIVE_DERIVATIVE STORAGE,admission.requested.elastic-stores,Number of requests,Requests,COUNTER,COUNT,AVG,NON_NEGATIVE_DERIVATIVE +STORAGE,admission.requested.elastic-stores.bulk-low-pri,Number of requests,Requests,COUNTER,COUNT,AVG,NON_NEGATIVE_DERIVATIVE STORAGE,admission.requested.elastic-stores.bulk-normal-pri,Number of requests,Requests,COUNTER,COUNT,AVG,NON_NEGATIVE_DERIVATIVE -STORAGE,admission.requested.elastic-stores.ttl-low-pri,Number of requests,Requests,COUNTER,COUNT,AVG,NON_NEGATIVE_DERIVATIVE STORAGE,admission.requested.kv,Number of requests,Requests,COUNTER,COUNT,AVG,NON_NEGATIVE_DERIVATIVE STORAGE,admission.requested.kv-stores,Number of requests,Requests,COUNTER,COUNT,AVG,NON_NEGATIVE_DERIVATIVE STORAGE,admission.requested.kv-stores.high-pri,Number of requests,Requests,COUNTER,COUNT,AVG,NON_NEGATIVE_DERIVATIVE @@ -133,8 +133,8 @@ STORAGE,admission.wait_durations.elastic-cpu,Wait time durations for requests th STORAGE,admission.wait_durations.elastic-cpu.bulk-normal-pri,Wait time durations for requests that waited,Wait time Duration,HISTOGRAM,NANOSECONDS,AVG,NONE STORAGE,admission.wait_durations.elastic-cpu.normal-pri,Wait time durations for requests that waited,Wait time Duration,HISTOGRAM,NANOSECONDS,AVG,NONE STORAGE,admission.wait_durations.elastic-stores,Wait time durations for requests that waited,Wait time Duration,HISTOGRAM,NANOSECONDS,AVG,NONE +STORAGE,admission.wait_durations.elastic-stores.bulk-low-pri,Wait time durations for requests that waited,Wait time Duration,HISTOGRAM,NANOSECONDS,AVG,NONE STORAGE,admission.wait_durations.elastic-stores.bulk-normal-pri,Wait time durations for requests that waited,Wait time Duration,HISTOGRAM,NANOSECONDS,AVG,NONE -STORAGE,admission.wait_durations.elastic-stores.ttl-low-pri,Wait time durations for requests that waited,Wait time Duration,HISTOGRAM,NANOSECONDS,AVG,NONE STORAGE,admission.wait_durations.kv,Wait time durations for requests that waited,Wait time Duration,HISTOGRAM,NANOSECONDS,AVG,NONE STORAGE,admission.wait_durations.kv-stores,Wait time durations for requests that waited,Wait time Duration,HISTOGRAM,NANOSECONDS,AVG,NONE STORAGE,admission.wait_durations.kv-stores.high-pri,Wait time durations for requests that waited,Wait time Duration,HISTOGRAM,NANOSECONDS,AVG,NONE @@ -143,6 +143,7 @@ STORAGE,admission.wait_durations.kv-stores.normal-pri,Wait time durations for re STORAGE,admission.wait_durations.kv.high-pri,Wait time durations for requests that waited,Wait time Duration,HISTOGRAM,NANOSECONDS,AVG,NONE STORAGE,admission.wait_durations.kv.locking-normal-pri,Wait time durations for requests that waited,Wait time Duration,HISTOGRAM,NANOSECONDS,AVG,NONE STORAGE,admission.wait_durations.kv.normal-pri,Wait time durations for requests that waited,Wait time Duration,HISTOGRAM,NANOSECONDS,AVG,NONE +STORAGE,admission.wait_durations.snapshot_ingest,Wait time for snapshot ingest requests that waited,Wait time Duration,HISTOGRAM,NANOSECONDS,AVG,NONE STORAGE,admission.wait_durations.sql-kv-response,Wait time durations for requests that waited,Wait time Duration,HISTOGRAM,NANOSECONDS,AVG,NONE STORAGE,admission.wait_durations.sql-kv-response.locking-normal-pri,Wait time durations for requests that waited,Wait time Duration,HISTOGRAM,NANOSECONDS,AVG,NONE STORAGE,admission.wait_durations.sql-kv-response.normal-pri,Wait time durations for requests that waited,Wait time Duration,HISTOGRAM,NANOSECONDS,AVG,NONE @@ -159,8 +160,8 @@ STORAGE,admission.wait_queue_length.elastic-cpu,Length of wait queue,Requests,GA STORAGE,admission.wait_queue_length.elastic-cpu.bulk-normal-pri,Length of wait queue,Requests,GAUGE,COUNT,AVG,NONE STORAGE,admission.wait_queue_length.elastic-cpu.normal-pri,Length of wait queue,Requests,GAUGE,COUNT,AVG,NONE STORAGE,admission.wait_queue_length.elastic-stores,Length of wait queue,Requests,GAUGE,COUNT,AVG,NONE +STORAGE,admission.wait_queue_length.elastic-stores.bulk-low-pri,Length of wait queue,Requests,GAUGE,COUNT,AVG,NONE STORAGE,admission.wait_queue_length.elastic-stores.bulk-normal-pri,Length of wait queue,Requests,GAUGE,COUNT,AVG,NONE -STORAGE,admission.wait_queue_length.elastic-stores.ttl-low-pri,Length of wait queue,Requests,GAUGE,COUNT,AVG,NONE STORAGE,admission.wait_queue_length.kv,Length of wait queue,Requests,GAUGE,COUNT,AVG,NONE STORAGE,admission.wait_queue_length.kv-stores,Length of wait queue,Requests,GAUGE,COUNT,AVG,NONE STORAGE,admission.wait_queue_length.kv-stores.high-pri,Length of wait queue,Requests,GAUGE,COUNT,AVG,NONE @@ -283,6 +284,8 @@ STORAGE,kv.prober.write.quarantine.oldest_duration,The duration that the oldest STORAGE,kv.rangefeed.budget_allocation_blocked,Number of times RangeFeed waited for budget availability,Events,COUNTER,COUNT,AVG,NON_NEGATIVE_DERIVATIVE STORAGE,kv.rangefeed.budget_allocation_failed,Number of times RangeFeed failed because memory budget was exceeded,Events,COUNTER,COUNT,AVG,NON_NEGATIVE_DERIVATIVE STORAGE,kv.rangefeed.catchup_scan_nanos,Time spent in RangeFeed catchup scan,Nanoseconds,COUNTER,NANOSECONDS,AVG,NON_NEGATIVE_DERIVATIVE +STORAGE,kv.rangefeed.closed_timestamp.slow_ranges,Number of ranges that have a closed timestamp lagging by more than 5x target lag. Periodically re-calculated,Ranges,GAUGE,COUNT,AVG,NONE +STORAGE,kv.rangefeed.closed_timestamp_max_behind_nanos,Largest latency between realtime and replica max closed timestamp for replicas that have active rangeeds on them,Nanoseconds,GAUGE,NANOSECONDS,AVG,NONE STORAGE,kv.rangefeed.mem_shared,Memory usage by rangefeeds,Memory,GAUGE,BYTES,AVG,NONE STORAGE,kv.rangefeed.mem_system,Memory usage by rangefeeds on system ranges,Memory,GAUGE,BYTES,AVG,NONE STORAGE,kv.rangefeed.processors_goroutine,Number of active RangeFeed processors using goroutines,Processors,GAUGE,COUNT,AVG,NONE @@ -358,9 +361,57 @@ STORAGE,kvadmission.flow_token_dispatch.pending_nodes,Number of nodes pending fl STORAGE,kvadmission.flow_token_dispatch.pending_regular,Number of pending regular flow token dispatches,Dispatches,GAUGE,COUNT,AVG,NONE STORAGE,kvadmission.flow_token_dispatch.remote_elastic,Number of remote elastic flow token dispatches,Dispatches,COUNTER,COUNT,AVG,NON_NEGATIVE_DERIVATIVE STORAGE,kvadmission.flow_token_dispatch.remote_regular,Number of remote regular flow token dispatches,Dispatches,COUNTER,COUNT,AVG,NON_NEGATIVE_DERIVATIVE +STORAGE,kvflowcontrol.eval_wait.elastic.duration,Latency histogram for time elastic requests spent waiting for flow tokens to evaluate,Nanoseconds,HISTOGRAM,NANOSECONDS,AVG,NONE +STORAGE,kvflowcontrol.eval_wait.elastic.requests.admitted,Number of elastic requests admitted by the flow controller,Requests,COUNTER,COUNT,AVG,NON_NEGATIVE_DERIVATIVE +STORAGE,kvflowcontrol.eval_wait.elastic.requests.bypassed,Number of waiting elastic requests that bypassed the flow controller due the evaluating replica not being the leader,Requests,COUNTER,COUNT,AVG,NON_NEGATIVE_DERIVATIVE +STORAGE,kvflowcontrol.eval_wait.elastic.requests.errored,Number of elastic requests that errored out while waiting for flow tokens,Requests,COUNTER,COUNT,AVG,NON_NEGATIVE_DERIVATIVE +STORAGE,kvflowcontrol.eval_wait.elastic.requests.waiting,Number of elastic requests waiting for flow tokens,Requests,GAUGE,COUNT,AVG,NONE +STORAGE,kvflowcontrol.eval_wait.regular.duration,Latency histogram for time regular requests spent waiting for flow tokens to evaluate,Nanoseconds,HISTOGRAM,NANOSECONDS,AVG,NONE +STORAGE,kvflowcontrol.eval_wait.regular.requests.admitted,Number of regular requests admitted by the flow controller,Requests,COUNTER,COUNT,AVG,NON_NEGATIVE_DERIVATIVE +STORAGE,kvflowcontrol.eval_wait.regular.requests.bypassed,Number of waiting regular requests that bypassed the flow controller due the evaluating replica not being the leader,Requests,COUNTER,COUNT,AVG,NON_NEGATIVE_DERIVATIVE +STORAGE,kvflowcontrol.eval_wait.regular.requests.errored,Number of regular requests that errored out while waiting for flow tokens,Requests,COUNTER,COUNT,AVG,NON_NEGATIVE_DERIVATIVE +STORAGE,kvflowcontrol.eval_wait.regular.requests.waiting,Number of regular requests waiting for flow tokens,Requests,GAUGE,COUNT,AVG,NONE +STORAGE,kvflowcontrol.range_controller.count,"Gauge of range flow controllers currently open, this should align with the number of leaders",Count,GAUGE,COUNT,AVG,NONE +STORAGE,kvflowcontrol.send_queue.bytes,"Byte size of all raft entries queued for sending to followers, waiting on available elastic send tokens",Bytes,GAUGE,BYTES,AVG,NONE +STORAGE,kvflowcontrol.send_queue.count,"Count of all raft entries queued for sending to followers, waiting on available elastic send tokens",Bytes,GAUGE,COUNT,AVG,NONE +STORAGE,kvflowcontrol.send_queue.prevent.count,Counter of replication streams that were prevented from forming a send queue,Preventions,COUNTER,COUNT,AVG,NON_NEGATIVE_DERIVATIVE +STORAGE,kvflowcontrol.send_queue.scheduled.deducted_bytes,Gauge of elastic send token bytes already deducted by replication streams waiting on the scheduler,Bytes,GAUGE,BYTES,AVG,NONE +STORAGE,kvflowcontrol.send_queue.scheduled.force_flush,Gauge of replication streams scheduled to force flush their send queue,Scheduled force flushes,GAUGE,COUNT,AVG,NONE +STORAGE,kvflowcontrol.streams.eval.elastic.blocked_count,Number of eval replication streams with no flow tokens available for elastic requests,Count,GAUGE,COUNT,AVG,NONE +STORAGE,kvflowcontrol.streams.eval.elastic.total_count,Total number of eval replication streams for elastic requests,Count,GAUGE,COUNT,AVG,NONE +STORAGE,kvflowcontrol.streams.eval.regular.blocked_count,Number of eval replication streams with no flow tokens available for regular requests,Count,GAUGE,COUNT,AVG,NONE +STORAGE,kvflowcontrol.streams.eval.regular.total_count,Total number of eval replication streams for regular requests,Count,GAUGE,COUNT,AVG,NONE +STORAGE,kvflowcontrol.streams.send.elastic.blocked_count,Number of send replication streams with no flow tokens available for elastic requests,Count,GAUGE,COUNT,AVG,NONE +STORAGE,kvflowcontrol.streams.send.elastic.total_count,Total number of send replication streams for elastic requests,Count,GAUGE,COUNT,AVG,NONE +STORAGE,kvflowcontrol.streams.send.regular.blocked_count,Number of send replication streams with no flow tokens available for regular requests,Count,GAUGE,COUNT,AVG,NONE +STORAGE,kvflowcontrol.streams.send.regular.total_count,Total number of send replication streams for regular requests,Count,GAUGE,COUNT,AVG,NONE +STORAGE,kvflowcontrol.tokens.eval.elastic.available,"Flow eval tokens available for elastic requests, across all replication streams",Bytes,GAUGE,BYTES,AVG,NONE +STORAGE,kvflowcontrol.tokens.eval.elastic.deducted,"Flow eval tokens deducted by elastic requests, across all replication streams",Bytes,COUNTER,BYTES,AVG,NON_NEGATIVE_DERIVATIVE +STORAGE,kvflowcontrol.tokens.eval.elastic.returned,"Flow eval tokens returned by elastic requests, across all replication streams",Bytes,COUNTER,BYTES,AVG,NON_NEGATIVE_DERIVATIVE +STORAGE,kvflowcontrol.tokens.eval.elastic.returned.disconnect,"Flow eval tokens returned early by elastic due disconnects, across all replication stream, this is a subset of returned tokens",Bytes,COUNTER,BYTES,AVG,NON_NEGATIVE_DERIVATIVE +STORAGE,kvflowcontrol.tokens.eval.elastic.unaccounted,"Flow eval tokens returned by elastic requests that were unaccounted for, across all replication streams",Bytes,COUNTER,BYTES,AVG,NON_NEGATIVE_DERIVATIVE +STORAGE,kvflowcontrol.tokens.eval.regular.available,"Flow eval tokens available for regular requests, across all replication streams",Bytes,GAUGE,BYTES,AVG,NONE +STORAGE,kvflowcontrol.tokens.eval.regular.deducted,"Flow eval tokens deducted by regular requests, across all replication streams",Bytes,COUNTER,BYTES,AVG,NON_NEGATIVE_DERIVATIVE +STORAGE,kvflowcontrol.tokens.eval.regular.returned,"Flow eval tokens returned by regular requests, across all replication streams",Bytes,COUNTER,BYTES,AVG,NON_NEGATIVE_DERIVATIVE +STORAGE,kvflowcontrol.tokens.eval.regular.returned.disconnect,"Flow eval tokens returned early by regular due disconnects, across all replication stream, this is a subset of returned tokens",Bytes,COUNTER,BYTES,AVG,NON_NEGATIVE_DERIVATIVE +STORAGE,kvflowcontrol.tokens.eval.regular.unaccounted,"Flow eval tokens returned by regular requests that were unaccounted for, across all replication streams",Bytes,COUNTER,BYTES,AVG,NON_NEGATIVE_DERIVATIVE +STORAGE,kvflowcontrol.tokens.send.elastic.available,"Flow send tokens available for elastic requests, across all replication streams",Bytes,GAUGE,BYTES,AVG,NONE +STORAGE,kvflowcontrol.tokens.send.elastic.deducted,"Flow send tokens deducted by elastic requests, across all replication streams",Bytes,COUNTER,BYTES,AVG,NON_NEGATIVE_DERIVATIVE +STORAGE,kvflowcontrol.tokens.send.elastic.deducted.force_flush_send_queue,"Flow send tokens deducted by elastic requests, across all replication streams due to force flushing the stream's send queue",Bytes,COUNTER,BYTES,AVG,NON_NEGATIVE_DERIVATIVE +STORAGE,kvflowcontrol.tokens.send.elastic.deducted.prevent_send_queue,"Flow send tokens deducted by elastic requests, across all replication streams to prevent forming a send queue",Bytes,COUNTER,BYTES,AVG,NON_NEGATIVE_DERIVATIVE +STORAGE,kvflowcontrol.tokens.send.elastic.returned,"Flow send tokens returned by elastic requests, across all replication streams",Bytes,COUNTER,BYTES,AVG,NON_NEGATIVE_DERIVATIVE +STORAGE,kvflowcontrol.tokens.send.elastic.returned.disconnect,"Flow send tokens returned early by elastic due disconnects, across all replication stream, this is a subset of returned tokens",Bytes,COUNTER,BYTES,AVG,NON_NEGATIVE_DERIVATIVE +STORAGE,kvflowcontrol.tokens.send.elastic.unaccounted,"Flow send tokens returned by elastic requests that were unaccounted for, across all replication streams",Bytes,COUNTER,BYTES,AVG,NON_NEGATIVE_DERIVATIVE +STORAGE,kvflowcontrol.tokens.send.regular.available,"Flow send tokens available for regular requests, across all replication streams",Bytes,GAUGE,BYTES,AVG,NONE +STORAGE,kvflowcontrol.tokens.send.regular.deducted,"Flow send tokens deducted by regular requests, across all replication streams",Bytes,COUNTER,BYTES,AVG,NON_NEGATIVE_DERIVATIVE +STORAGE,kvflowcontrol.tokens.send.regular.deducted.prevent_send_queue,"Flow send tokens deducted by regular requests, across all replication streams to prevent forming a send queue",Bytes,COUNTER,BYTES,AVG,NON_NEGATIVE_DERIVATIVE +STORAGE,kvflowcontrol.tokens.send.regular.returned,"Flow send tokens returned by regular requests, across all replication streams",Bytes,COUNTER,BYTES,AVG,NON_NEGATIVE_DERIVATIVE +STORAGE,kvflowcontrol.tokens.send.regular.returned.disconnect,"Flow send tokens returned early by regular due disconnects, across all replication stream, this is a subset of returned tokens",Bytes,COUNTER,BYTES,AVG,NON_NEGATIVE_DERIVATIVE +STORAGE,kvflowcontrol.tokens.send.regular.unaccounted,"Flow send tokens returned by regular requests that were unaccounted for, across all replication streams",Bytes,COUNTER,BYTES,AVG,NON_NEGATIVE_DERIVATIVE STORAGE,leases.epoch,Number of replica leaseholders using epoch-based leases,Replicas,GAUGE,COUNT,AVG,NONE STORAGE,leases.error,Number of failed lease requests,Lease Requests,COUNTER,COUNT,AVG,NON_NEGATIVE_DERIVATIVE STORAGE,leases.expiration,Number of replica leaseholders using expiration-based leases,Replicas,GAUGE,COUNT,AVG,NONE +STORAGE,leases.leader,Number of replica leaseholders using leader leases,Replicas,GAUGE,COUNT,AVG,NONE STORAGE,leases.liveness,Number of replica leaseholders for the liveness range(s),Replicas,GAUGE,COUNT,AVG,NONE STORAGE,leases.preferences.less-preferred,Number of replica leaseholders which satisfy a lease preference which is not the most preferred,Replicas,GAUGE,COUNT,AVG,NONE STORAGE,leases.preferences.violating,Number of replica leaseholders which violate lease preferences,Replicas,GAUGE,COUNT,AVG,NONE @@ -581,8 +632,11 @@ STORAGE,raft.rcvd.cross_zone.bytes,"Number of bytes received by this store for c regions. To ensure accurate monitoring of transmitted data, it is important to set up a consistent locality configuration across nodes. Note that this does not include raft snapshot received.",Bytes,COUNTER,BYTES,AVG,NON_NEGATIVE_DERIVATIVE +STORAGE,raft.rcvd.defortifyleader,Number of MsgDeFortifyLeader messages received by this store,Messages,COUNTER,COUNT,AVG,NON_NEGATIVE_DERIVATIVE STORAGE,raft.rcvd.dropped,Number of incoming Raft messages dropped (due to queue length or size),Messages,COUNTER,COUNT,AVG,NON_NEGATIVE_DERIVATIVE STORAGE,raft.rcvd.dropped_bytes,Bytes of dropped incoming Raft messages,Bytes,COUNTER,BYTES,AVG,NON_NEGATIVE_DERIVATIVE +STORAGE,raft.rcvd.fortifyleader,Number of MsgFortifyLeader messages received by this store,Messages,COUNTER,COUNT,AVG,NON_NEGATIVE_DERIVATIVE +STORAGE,raft.rcvd.fortifyleaderresp,Number of MsgFortifyLeaderResp messages received by this store,Messages,COUNTER,COUNT,AVG,NON_NEGATIVE_DERIVATIVE STORAGE,raft.rcvd.heartbeat,"Number of (coalesced, if enabled) MsgHeartbeat messages received by this store",Messages,COUNTER,COUNT,AVG,NON_NEGATIVE_DERIVATIVE STORAGE,raft.rcvd.heartbeatresp,"Number of (coalesced, if enabled) MsgHeartbeatResp messages received by this store",Messages,COUNTER,COUNT,AVG,NON_NEGATIVE_DERIVATIVE STORAGE,raft.rcvd.prevote,Number of MsgPreVote messages received by this store,Messages,COUNTER,COUNT,AVG,NON_NEGATIVE_DERIVATIVE @@ -768,6 +822,7 @@ STORAGE,range.splits,Number of range splits,Range Ops,COUNTER,COUNT,AVG,NON_NEGA STORAGE,rangekeybytes,Number of bytes taken up by range keys (e.g. MVCC range tombstones),Storage,GAUGE,BYTES,AVG,NONE STORAGE,rangekeycount,Count of all range keys (e.g. MVCC range tombstones),Keys,GAUGE,COUNT,AVG,NONE STORAGE,ranges,Number of ranges,Ranges,GAUGE,COUNT,AVG,NONE +STORAGE,ranges.decommissioning,Number of ranges with at lease one replica on a decommissioning node,Ranges,GAUGE,COUNT,AVG,NONE STORAGE,ranges.overreplicated,Number of ranges with more live replicas than the replication target,Ranges,GAUGE,COUNT,AVG,NONE STORAGE,ranges.unavailable,Number of ranges with fewer live replicas than needed for quorum,Ranges,GAUGE,COUNT,AVG,NONE STORAGE,ranges.underreplicated,Number of ranges with fewer live replicas than the replication target,Ranges,GAUGE,COUNT,AVG,NONE @@ -911,9 +966,6 @@ STORAGE,storage.batch-commit.wal-queue-wait.duration,"Cumulative time spent wait STORAGE,storage.batch-commit.wal-rotation.duration,"Cumulative time spent waiting for WAL rotation, for batch commit. See storage.AggregatedBatchCommitStats for details.",Nanoseconds,COUNTER,NANOSECONDS,AVG,NON_NEGATIVE_DERIVATIVE STORAGE,storage.block-load.active,The number of sstable block loads currently in progress,Block loads,GAUGE,COUNT,AVG,NONE STORAGE,storage.block-load.queued,The cumulative number of SSTable block loads that were delayed because too many loads were active (see also: `storage.block_load.node_max_active`),Block loads,COUNTER,COUNT,AVG,NON_NEGATIVE_DERIVATIVE -STORAGE,storage.category-pebble-manifest.bytes-written,Bytes written to disk,Bytes,GAUGE,BYTES,AVG,NONE -STORAGE,storage.category-pebble-wal.bytes-written,Bytes written to disk,Bytes,GAUGE,BYTES,AVG,NONE -STORAGE,storage.category-unspecified.bytes-written,Bytes written to disk,Bytes,GAUGE,BYTES,AVG,NONE STORAGE,storage.checkpoints,"The number of checkpoint directories found in storage. This is the number of directories found in the auxiliary/checkpoints directory. @@ -970,6 +1022,24 @@ STORAGE,storage.ingest.count,Number of successful ingestions performed,Events,GA STORAGE,storage.iterator.block-load.bytes,Bytes loaded by storage engine iterators (possibly cached). See storage.AggregatedIteratorStats for details.,Bytes,COUNTER,BYTES,AVG,NON_NEGATIVE_DERIVATIVE STORAGE,storage.iterator.block-load.cached-bytes,Bytes loaded by storage engine iterators from the block cache. See storage.AggregatedIteratorStats for details.,Bytes,COUNTER,BYTES,AVG,NON_NEGATIVE_DERIVATIVE STORAGE,storage.iterator.block-load.read-duration,Cumulative time storage engine iterators spent loading blocks from durable storage. See storage.AggregatedIteratorStats for details.,Nanoseconds,COUNTER,NANOSECONDS,AVG,NON_NEGATIVE_DERIVATIVE +STORAGE,storage.iterator.category-batch-eval.block-load.bytes,Bytes loaded by storage sstable iterators (possibly cached).,Bytes,COUNTER,BYTES,AVG,NON_NEGATIVE_DERIVATIVE +STORAGE,storage.iterator.category-batch-eval.block-load.cached-bytes,Bytes loaded by storage sstable iterators from the block cache,Bytes,COUNTER,BYTES,AVG,NON_NEGATIVE_DERIVATIVE +STORAGE,storage.iterator.category-batch-eval.block-load.latency-sum,"Cumulative latency for loading bytes not in the block cache, by storage sstable iterators",Latency,COUNTER,NANOSECONDS,AVG,NON_NEGATIVE_DERIVATIVE +STORAGE,storage.iterator.category-crdb-unknown.block-load.bytes,Bytes loaded by storage sstable iterators (possibly cached).,Bytes,COUNTER,BYTES,AVG,NON_NEGATIVE_DERIVATIVE +STORAGE,storage.iterator.category-crdb-unknown.block-load.cached-bytes,Bytes loaded by storage sstable iterators from the block cache,Bytes,COUNTER,BYTES,AVG,NON_NEGATIVE_DERIVATIVE +STORAGE,storage.iterator.category-crdb-unknown.block-load.latency-sum,"Cumulative latency for loading bytes not in the block cache, by storage sstable iterators",Latency,COUNTER,NANOSECONDS,AVG,NON_NEGATIVE_DERIVATIVE +STORAGE,storage.iterator.category-mvcc-gc.block-load.bytes,Bytes loaded by storage sstable iterators (possibly cached).,Bytes,COUNTER,BYTES,AVG,NON_NEGATIVE_DERIVATIVE +STORAGE,storage.iterator.category-mvcc-gc.block-load.cached-bytes,Bytes loaded by storage sstable iterators from the block cache,Bytes,COUNTER,BYTES,AVG,NON_NEGATIVE_DERIVATIVE +STORAGE,storage.iterator.category-mvcc-gc.block-load.latency-sum,"Cumulative latency for loading bytes not in the block cache, by storage sstable iterators",Latency,COUNTER,NANOSECONDS,AVG,NON_NEGATIVE_DERIVATIVE +STORAGE,storage.iterator.category-rangefeed.block-load.bytes,Bytes loaded by storage sstable iterators (possibly cached).,Bytes,COUNTER,BYTES,AVG,NON_NEGATIVE_DERIVATIVE +STORAGE,storage.iterator.category-rangefeed.block-load.cached-bytes,Bytes loaded by storage sstable iterators from the block cache,Bytes,COUNTER,BYTES,AVG,NON_NEGATIVE_DERIVATIVE +STORAGE,storage.iterator.category-rangefeed.block-load.latency-sum,"Cumulative latency for loading bytes not in the block cache, by storage sstable iterators",Latency,COUNTER,NANOSECONDS,AVG,NON_NEGATIVE_DERIVATIVE +STORAGE,storage.iterator.category-replication.block-load.bytes,Bytes loaded by storage sstable iterators (possibly cached).,Bytes,COUNTER,BYTES,AVG,NON_NEGATIVE_DERIVATIVE +STORAGE,storage.iterator.category-replication.block-load.cached-bytes,Bytes loaded by storage sstable iterators from the block cache,Bytes,COUNTER,BYTES,AVG,NON_NEGATIVE_DERIVATIVE +STORAGE,storage.iterator.category-replication.block-load.latency-sum,"Cumulative latency for loading bytes not in the block cache, by storage sstable iterators",Latency,COUNTER,NANOSECONDS,AVG,NON_NEGATIVE_DERIVATIVE +STORAGE,storage.iterator.category-scan-regular.block-load.bytes,Bytes loaded by storage sstable iterators (possibly cached).,Bytes,COUNTER,BYTES,AVG,NON_NEGATIVE_DERIVATIVE +STORAGE,storage.iterator.category-scan-regular.block-load.cached-bytes,Bytes loaded by storage sstable iterators from the block cache,Bytes,COUNTER,BYTES,AVG,NON_NEGATIVE_DERIVATIVE +STORAGE,storage.iterator.category-scan-regular.block-load.latency-sum,"Cumulative latency for loading bytes not in the block cache, by storage sstable iterators",Latency,COUNTER,NANOSECONDS,AVG,NON_NEGATIVE_DERIVATIVE STORAGE,storage.iterator.external.seeks,Cumulative count of seeks performed on storage engine iterators. See storage.AggregatedIteratorStats for details.,Iterator Ops,COUNTER,COUNT,AVG,NON_NEGATIVE_DERIVATIVE STORAGE,storage.iterator.external.steps,Cumulative count of steps performed on storage engine iterators. See storage.AggregatedIteratorStats for details.,Iterator Ops,COUNTER,COUNT,AVG,NON_NEGATIVE_DERIVATIVE STORAGE,storage.iterator.internal.seeks,"Cumulative count of seeks performed internally within storage engine iterators. @@ -1040,8 +1110,30 @@ STORAGE,storage.wal.failover.secondary.duration,Cumulative time spent writing to STORAGE,storage.wal.failover.switch.count,Count of the number of times WAL writing has switched from primary to secondary and vice versa.,Events,COUNTER,COUNT,AVG,NON_NEGATIVE_DERIVATIVE STORAGE,storage.wal.failover.write_and_sync.latency,The observed latency for writing and syncing to the write ahead log. Only populated when WAL failover is configured,Nanoseconds,HISTOGRAM,NANOSECONDS,AVG,NONE STORAGE,storage.wal.fsync.latency,The write ahead log fsync latency,Fsync Latency,HISTOGRAM,NANOSECONDS,AVG,NONE +STORAGE,storage.write-amplification,"Running measure of write-amplification. + +Write amplification is measured as the ratio of bytes written to disk relative to the logical +bytes present in sstables, over the life of a store. This metric is a running average +of the write amplification as tracked by Pebble.",Ratio of bytes written to logical bytes,GAUGE,COUNT,AVG,NONE STORAGE,storage.write-stall-nanos,Total write stall duration in nanos,Nanoseconds,GAUGE,NANOSECONDS,AVG,NONE STORAGE,storage.write-stalls,Number of instances of intentional write stalls to backpressure incoming writes,Events,GAUGE,COUNT,AVG,NONE +STORAGE,storeliveness.heartbeat.failures,Number of Store Liveness heartbeats that failed to be sent out by the Store Liveness Support Manager,Heartbeats,COUNTER,COUNT,AVG,NON_NEGATIVE_DERIVATIVE +STORAGE,storeliveness.heartbeat.successes,Number of Store Liveness heartbeats sent out by the Store Liveness Support Manager,Heartbeats,COUNTER,COUNT,AVG,NON_NEGATIVE_DERIVATIVE +STORAGE,storeliveness.message_handle.failures,Number of incoming Store Liveness messages that failed to be handled by the Store Liveness Support Manager,Messages,COUNTER,COUNT,AVG,NON_NEGATIVE_DERIVATIVE +STORAGE,storeliveness.message_handle.successes,Number of incoming Store Liveness messages handled by the Store Liveness Support Manager,Messages,COUNTER,COUNT,AVG,NON_NEGATIVE_DERIVATIVE +STORAGE,storeliveness.support_for.stores,Number of stores that the Store Liveness Support Manager has ever provided support for,Stores,GAUGE,COUNT,AVG,NONE +STORAGE,storeliveness.support_from.stores,Number of stores that the Store Liveness Support Manager is requesting support from by sending heartbeats,Stores,GAUGE,COUNT,AVG,NONE +STORAGE,storeliveness.support_withdraw.failures,Number of times the Store Liveness Support Manager has encountered an error while withdrawing support for another store,Support Withdrawals,COUNTER,COUNT,AVG,NON_NEGATIVE_DERIVATIVE +STORAGE,storeliveness.support_withdraw.successes,Number of times the Store Liveness Support Manager has successfully withdrawn support for another store,Support Withdrawals,COUNTER,COUNT,AVG,NON_NEGATIVE_DERIVATIVE +STORAGE,storeliveness.transport.receive-queue-bytes,Total byte size of pending incoming messages from Store Liveness Transport,Bytes,GAUGE,BYTES,AVG,NONE +STORAGE,storeliveness.transport.receive-queue-size,Number of pending incoming messages from the Store Liveness Transport,Messages,GAUGE,COUNT,AVG,NONE +STORAGE,storeliveness.transport.receive_dropped,Number of Store Liveness messages dropped by the Store Liveness Transport on the receiver side,Messages,COUNTER,COUNT,AVG,NON_NEGATIVE_DERIVATIVE +STORAGE,storeliveness.transport.received,Number of Store Liveness messages received by the Store Liveness Transport,Messages,COUNTER,COUNT,AVG,NON_NEGATIVE_DERIVATIVE +STORAGE,storeliveness.transport.send-queue-bytes,Total byte size of pending outgoing messages in all Store Liveness Transport per-store send queues,Bytes,GAUGE,BYTES,AVG,NONE +STORAGE,storeliveness.transport.send-queue-idle,Number of Store Liveness Transport per-store send queues that have become idle due to no recently-sent messages,Messages,COUNTER,COUNT,AVG,NON_NEGATIVE_DERIVATIVE +STORAGE,storeliveness.transport.send-queue-size,Number of pending outgoing messages in all Store Liveness Transport per-store send queues,Messages,GAUGE,COUNT,AVG,NONE +STORAGE,storeliveness.transport.send_dropped,Number of Store Liveness messages dropped by the Store Liveness Transport on the sender side,Messages,COUNTER,COUNT,AVG,NON_NEGATIVE_DERIVATIVE +STORAGE,storeliveness.transport.sent,Number of Store Liveness messages sent by the Store Liveness Transport,Messages,COUNTER,COUNT,AVG,NON_NEGATIVE_DERIVATIVE STORAGE,sysbytes,Number of bytes in system KV pairs,Storage,GAUGE,BYTES,AVG,NONE STORAGE,syscount,Count of system KV pairs,Keys,GAUGE,COUNT,AVG,NONE STORAGE,tenant.consumption.cross_region_network_ru,Total number of RUs charged for cross-region network traffic,Request Units,COUNTER,COUNT,AVG,NON_NEGATIVE_DERIVATIVE @@ -1087,6 +1179,12 @@ STORAGE,txnwaitqueue.query.wait_time,Histogram of durations spent in queue by qu STORAGE,txnwaitqueue.query.waiting,Number of transaction status queries waiting for an updated transaction record,Waiting Queries,GAUGE,COUNT,AVG,NONE STORAGE,valbytes,Number of bytes taken up by values,Storage,GAUGE,BYTES,AVG,NONE STORAGE,valcount,Count of all values,MVCC Values,GAUGE,COUNT,AVG,NONE +APPLICATION,auth.cert.conn.latency,Latency to establish and authenticate a SQL connection using certificate,Nanoseconds,HISTOGRAM,NANOSECONDS,AVG,NONE +APPLICATION,auth.gss.conn.latency,Latency to establish and authenticate a SQL connection using GSS,Nanoseconds,HISTOGRAM,NANOSECONDS,AVG,NONE +APPLICATION,auth.jwt.conn.latency,Latency to establish and authenticate a SQL connection using JWT Token,Nanoseconds,HISTOGRAM,NANOSECONDS,AVG,NONE +APPLICATION,auth.ldap.conn.latency,Latency to establish and authenticate a SQL connection using LDAP,Nanoseconds,HISTOGRAM,NANOSECONDS,AVG,NONE +APPLICATION,auth.password.conn.latency,Latency to establish and authenticate a SQL connection using password,Nanoseconds,HISTOGRAM,NANOSECONDS,AVG,NONE +APPLICATION,auth.scram.conn.latency,Latency to establish and authenticate a SQL connection using SCRAM,Nanoseconds,HISTOGRAM,NANOSECONDS,AVG,NONE APPLICATION,backup.last-failed-time.kms-inaccessible,The unix timestamp of the most recent failure of backup due to errKMSInaccessible by a backup specified as maintaining this metric,Jobs,GAUGE,TIMESTAMP_SEC,AVG,NONE APPLICATION,changefeed.admit_latency,"Event admission latency: a difference between event MVCC timestamp and the time it was admitted into changefeed pipeline; Note: this metric includes the time spent waiting until event can be processed due to backpressure or time spent resolving schema descriptors. Also note, this metric excludes latency during backfill",Nanoseconds,HISTOGRAM,NANOSECONDS,AVG,NONE APPLICATION,changefeed.aggregator_progress,The earliest timestamp up to which any aggregator is guaranteed to have emitted all values for,Unix Timestamp Nanoseconds,GAUGE,TIMESTAMP_NS,AVG,NONE @@ -1094,15 +1192,35 @@ APPLICATION,changefeed.backfill_count,Number of changefeeds currently executing APPLICATION,changefeed.backfill_pending_ranges,Number of ranges in an ongoing backfill that are yet to be fully emitted,Count,GAUGE,COUNT,AVG,NONE APPLICATION,changefeed.batch_reduction_count,Number of times a changefeed aggregator node attempted to reduce the size of message batches it emitted to the sink,Batch Size Reductions,GAUGE,COUNT,AVG,NONE APPLICATION,changefeed.buffer_entries.allocated_mem,Current quota pool memory allocation,Bytes,GAUGE,BYTES,AVG,NONE +APPLICATION,changefeed.buffer_entries.allocated_mem.aggregator,Current quota pool memory allocation - between the kvfeed and the sink,Bytes,GAUGE,BYTES,AVG,NONE +APPLICATION,changefeed.buffer_entries.allocated_mem.rangefeed,Current quota pool memory allocation - between the rangefeed and the kvfeed,Bytes,GAUGE,BYTES,AVG,NONE APPLICATION,changefeed.buffer_entries.flush,Number of flush elements added to the buffer,Events,COUNTER,COUNT,AVG,NON_NEGATIVE_DERIVATIVE +APPLICATION,changefeed.buffer_entries.flush.aggregator,Number of flush elements added to the buffer - between the kvfeed and the sink,Events,COUNTER,COUNT,AVG,NON_NEGATIVE_DERIVATIVE +APPLICATION,changefeed.buffer_entries.flush.rangefeed,Number of flush elements added to the buffer - between the rangefeed and the kvfeed,Events,COUNTER,COUNT,AVG,NON_NEGATIVE_DERIVATIVE APPLICATION,changefeed.buffer_entries.in,Total entries entering the buffer between raft and changefeed sinks,Entries,COUNTER,COUNT,AVG,NON_NEGATIVE_DERIVATIVE +APPLICATION,changefeed.buffer_entries.in.aggregator,Total entries entering the buffer between raft and changefeed sinks - between the kvfeed and the sink,Entries,COUNTER,COUNT,AVG,NON_NEGATIVE_DERIVATIVE +APPLICATION,changefeed.buffer_entries.in.rangefeed,Total entries entering the buffer between raft and changefeed sinks - between the rangefeed and the kvfeed,Entries,COUNTER,COUNT,AVG,NON_NEGATIVE_DERIVATIVE APPLICATION,changefeed.buffer_entries.kv,Number of kv elements added to the buffer,Events,COUNTER,COUNT,AVG,NON_NEGATIVE_DERIVATIVE +APPLICATION,changefeed.buffer_entries.kv.aggregator,Number of kv elements added to the buffer - between the kvfeed and the sink,Events,COUNTER,COUNT,AVG,NON_NEGATIVE_DERIVATIVE +APPLICATION,changefeed.buffer_entries.kv.rangefeed,Number of kv elements added to the buffer - between the rangefeed and the kvfeed,Events,COUNTER,COUNT,AVG,NON_NEGATIVE_DERIVATIVE APPLICATION,changefeed.buffer_entries.out,Total entries leaving the buffer between raft and changefeed sinks,Entries,COUNTER,COUNT,AVG,NON_NEGATIVE_DERIVATIVE +APPLICATION,changefeed.buffer_entries.out.aggregator,Total entries leaving the buffer between raft and changefeed sinks - between the kvfeed and the sink,Entries,COUNTER,COUNT,AVG,NON_NEGATIVE_DERIVATIVE +APPLICATION,changefeed.buffer_entries.out.rangefeed,Total entries leaving the buffer between raft and changefeed sinks - between the rangefeed and the kvfeed,Entries,COUNTER,COUNT,AVG,NON_NEGATIVE_DERIVATIVE APPLICATION,changefeed.buffer_entries.released,"Total entries processed, emitted and acknowledged by the sinks",Entries,COUNTER,COUNT,AVG,NON_NEGATIVE_DERIVATIVE +APPLICATION,changefeed.buffer_entries.released.aggregator,"Total entries processed, emitted and acknowledged by the sinks - between the kvfeed and the sink",Entries,COUNTER,COUNT,AVG,NON_NEGATIVE_DERIVATIVE +APPLICATION,changefeed.buffer_entries.released.rangefeed,"Total entries processed, emitted and acknowledged by the sinks - between the rangefeed and the kvfeed",Entries,COUNTER,COUNT,AVG,NON_NEGATIVE_DERIVATIVE APPLICATION,changefeed.buffer_entries.resolved,Number of resolved elements added to the buffer,Events,COUNTER,COUNT,AVG,NON_NEGATIVE_DERIVATIVE +APPLICATION,changefeed.buffer_entries.resolved.aggregator,Number of resolved elements added to the buffer - between the kvfeed and the sink,Events,COUNTER,COUNT,AVG,NON_NEGATIVE_DERIVATIVE +APPLICATION,changefeed.buffer_entries.resolved.rangefeed,Number of resolved elements added to the buffer - between the rangefeed and the kvfeed,Events,COUNTER,COUNT,AVG,NON_NEGATIVE_DERIVATIVE APPLICATION,changefeed.buffer_entries_mem.acquired,Total amount of memory acquired for entries as they enter the system,Entries,COUNTER,COUNT,AVG,NON_NEGATIVE_DERIVATIVE +APPLICATION,changefeed.buffer_entries_mem.acquired.aggregator,Total amount of memory acquired for entries as they enter the system - between the kvfeed and the sink,Entries,COUNTER,COUNT,AVG,NON_NEGATIVE_DERIVATIVE +APPLICATION,changefeed.buffer_entries_mem.acquired.rangefeed,Total amount of memory acquired for entries as they enter the system - between the rangefeed and the kvfeed,Entries,COUNTER,COUNT,AVG,NON_NEGATIVE_DERIVATIVE APPLICATION,changefeed.buffer_entries_mem.released,Total amount of memory released by the entries after they have been emitted,Entries,COUNTER,COUNT,AVG,NON_NEGATIVE_DERIVATIVE +APPLICATION,changefeed.buffer_entries_mem.released.aggregator,Total amount of memory released by the entries after they have been emitted - between the kvfeed and the sink,Entries,COUNTER,COUNT,AVG,NON_NEGATIVE_DERIVATIVE +APPLICATION,changefeed.buffer_entries_mem.released.rangefeed,Total amount of memory released by the entries after they have been emitted - between the rangefeed and the kvfeed,Entries,COUNTER,COUNT,AVG,NON_NEGATIVE_DERIVATIVE APPLICATION,changefeed.buffer_pushback_nanos,Total time spent waiting while the buffer was full,Nanoseconds,COUNTER,NANOSECONDS,AVG,NON_NEGATIVE_DERIVATIVE +APPLICATION,changefeed.buffer_pushback_nanos.aggregator,Total time spent waiting while the buffer was full - between the kvfeed and the sink,Nanoseconds,COUNTER,NANOSECONDS,AVG,NON_NEGATIVE_DERIVATIVE +APPLICATION,changefeed.buffer_pushback_nanos.rangefeed,Total time spent waiting while the buffer was full - between the rangefeed and the kvfeed,Nanoseconds,COUNTER,NANOSECONDS,AVG,NON_NEGATIVE_DERIVATIVE APPLICATION,changefeed.bytes.messages_pushback_nanos,Total time spent throttled for bytes quota,Nanoseconds,COUNTER,NANOSECONDS,AVG,NON_NEGATIVE_DERIVATIVE APPLICATION,changefeed.checkpoint_hist_nanos,Time spent checkpointing changefeed progress,Changefeeds,HISTOGRAM,NANOSECONDS,AVG,NONE APPLICATION,changefeed.checkpoint_progress,The earliest timestamp of any changefeed's persisted checkpoint (values prior to this timestamp will never need to be re-emitted),Unix Timestamp Nanoseconds,GAUGE,TIMESTAMP_NS,AVG,NONE @@ -1126,6 +1244,8 @@ APPLICATION,changefeed.lagging_ranges,The number of ranges considered to be lagg APPLICATION,changefeed.max_behind_nanos,(Deprecated in favor of checkpoint_progress) The most any changefeed's persisted checkpoint is behind the present,Nanoseconds,GAUGE,NANOSECONDS,AVG,NONE APPLICATION,changefeed.message_size_hist,Message size histogram,Bytes,HISTOGRAM,BYTES,AVG,NONE APPLICATION,changefeed.messages.messages_pushback_nanos,Total time spent throttled for messages quota,Nanoseconds,COUNTER,NANOSECONDS,AVG,NON_NEGATIVE_DERIVATIVE +APPLICATION,changefeed.network.bytes_in,The number of bytes received from the network by changefeeds,Bytes,COUNTER,COUNT,AVG,NON_NEGATIVE_DERIVATIVE +APPLICATION,changefeed.network.bytes_out,The number of bytes sent over the network by changefeeds,Bytes,COUNTER,COUNT,AVG,NON_NEGATIVE_DERIVATIVE APPLICATION,changefeed.nprocs_consume_event_nanos,Total time spent waiting to add an event to the parallel consumer,Nanoseconds,HISTOGRAM,NANOSECONDS,AVG,NONE APPLICATION,changefeed.nprocs_flush_nanos,Total time spent idle waiting for the parallel consumer to flush,Nanoseconds,HISTOGRAM,NANOSECONDS,AVG,NONE APPLICATION,changefeed.nprocs_in_flight_count,Number of buffered events in the parallel consumer,Count of Events,GAUGE,COUNT,AVG,NONE @@ -1140,8 +1260,18 @@ APPLICATION,changefeed.schema_registry.retry_count,Number of retries encountered APPLICATION,changefeed.schemafeed.table_history_scans,The number of table history scans during polling,Counts,COUNTER,COUNT,AVG,NON_NEGATIVE_DERIVATIVE APPLICATION,changefeed.schemafeed.table_metadata_nanos,Time blocked while verifying table metadata histories,Nanoseconds,COUNTER,NANOSECONDS,AVG,NON_NEGATIVE_DERIVATIVE APPLICATION,changefeed.sink_batch_hist_nanos,Time spent batched in the sink buffer before being flushed and acknowledged,Changefeeds,HISTOGRAM,NANOSECONDS,AVG,NONE +APPLICATION,changefeed.sink_errors,Number of changefeed errors caused by the sink,Count,COUNTER,COUNT,AVG,NON_NEGATIVE_DERIVATIVE APPLICATION,changefeed.sink_io_inflight,The number of keys currently inflight as IO requests being sent to the sink,Messages,GAUGE,COUNT,AVG,NONE APPLICATION,changefeed.size_based_flushes,Total size based flushes across all feeds,Flushes,COUNTER,COUNT,AVG,NON_NEGATIVE_DERIVATIVE +APPLICATION,changefeed.stage.checkpoint_job_progress.latency,Latency of the changefeed stage: checkpointing job progress,Latency,HISTOGRAM,NANOSECONDS,AVG,NONE +APPLICATION,changefeed.stage.downstream_client_send.latency,Latency of the changefeed stage: flushing messages from the sink's client to its downstream. This includes sends that failed for most but not all sinks.,Latency,HISTOGRAM,NANOSECONDS,AVG,NONE +APPLICATION,changefeed.stage.emit_row.latency,Latency of the changefeed stage: emitting row to sink,Latency,HISTOGRAM,NANOSECONDS,AVG,NONE +APPLICATION,changefeed.stage.encode.latency,Latency of the changefeed stage: encoding data,Latency,HISTOGRAM,NANOSECONDS,AVG,NONE +APPLICATION,changefeed.stage.kv_feed_buffer.latency,Latency of the changefeed stage: waiting to buffer kv events,Latency,HISTOGRAM,NANOSECONDS,AVG,NONE +APPLICATION,changefeed.stage.kv_feed_wait_for_table_event.latency,Latency of the changefeed stage: waiting for a table schema event to join to the kv event,Latency,HISTOGRAM,NANOSECONDS,AVG,NONE +APPLICATION,changefeed.stage.rangefeed_buffer_checkpoint.latency,Latency of the changefeed stage: buffering rangefeed checkpoint events,Latency,HISTOGRAM,NANOSECONDS,AVG,NONE +APPLICATION,changefeed.stage.rangefeed_buffer_value.latency,Latency of the changefeed stage: buffering rangefeed value events,Latency,HISTOGRAM,NANOSECONDS,AVG,NONE +APPLICATION,changefeed.total_ranges,The total number of ranges being watched by changefeed aggregators,Ranges,GAUGE,COUNT,AVG,NONE APPLICATION,changefeed.usage.error_count,Count of errors encountered while generating usage metrics for changefeeds,Errors,COUNTER,COUNT,AVG,NON_NEGATIVE_DERIVATIVE APPLICATION,changefeed.usage.query_duration,Time taken by the queries used to generate usage metrics for changefeeds,Nanoseconds,HISTOGRAM,NANOSECONDS,AVG,NONE APPLICATION,changefeed.usage.table_bytes,Aggregated number of bytes of data per table watched by changefeeds,Storage,GAUGE,BYTES,AVG,NONE @@ -1803,6 +1933,18 @@ APPLICATION,jobs.auto_config_task.protected_record_count,Number of protected tim APPLICATION,jobs.auto_config_task.resume_completed,Number of auto_config_task jobs which successfully resumed to completion,jobs,COUNTER,COUNT,AVG,NON_NEGATIVE_DERIVATIVE APPLICATION,jobs.auto_config_task.resume_failed,Number of auto_config_task jobs which failed with a non-retriable error,jobs,COUNTER,COUNT,AVG,NON_NEGATIVE_DERIVATIVE APPLICATION,jobs.auto_config_task.resume_retry_error,Number of auto_config_task jobs which failed with a retriable error,jobs,COUNTER,COUNT,AVG,NON_NEGATIVE_DERIVATIVE +APPLICATION,jobs.auto_create_partial_stats.currently_idle,Number of auto_create_partial_stats jobs currently considered Idle and can be freely shut down,jobs,GAUGE,COUNT,AVG,NONE +APPLICATION,jobs.auto_create_partial_stats.currently_paused,Number of auto_create_partial_stats jobs currently considered Paused,jobs,GAUGE,COUNT,AVG,NONE +APPLICATION,jobs.auto_create_partial_stats.currently_running,Number of auto_create_partial_stats jobs currently running in Resume or OnFailOrCancel state,jobs,GAUGE,COUNT,AVG,NONE +APPLICATION,jobs.auto_create_partial_stats.expired_pts_records,Number of expired protected timestamp records owned by auto_create_partial_stats jobs,records,COUNTER,COUNT,AVG,NON_NEGATIVE_DERIVATIVE +APPLICATION,jobs.auto_create_partial_stats.fail_or_cancel_completed,Number of auto_create_partial_stats jobs which successfully completed their failure or cancelation process,jobs,COUNTER,COUNT,AVG,NON_NEGATIVE_DERIVATIVE +APPLICATION,jobs.auto_create_partial_stats.fail_or_cancel_failed,Number of auto_create_partial_stats jobs which failed with a non-retriable error on their failure or cancelation process,jobs,COUNTER,COUNT,AVG,NON_NEGATIVE_DERIVATIVE +APPLICATION,jobs.auto_create_partial_stats.fail_or_cancel_retry_error,Number of auto_create_partial_stats jobs which failed with a retriable error on their failure or cancelation process,jobs,COUNTER,COUNT,AVG,NON_NEGATIVE_DERIVATIVE +APPLICATION,jobs.auto_create_partial_stats.protected_age_sec,The age of the oldest PTS record protected by auto_create_partial_stats jobs,seconds,GAUGE,SECONDS,AVG,NONE +APPLICATION,jobs.auto_create_partial_stats.protected_record_count,Number of protected timestamp records held by auto_create_partial_stats jobs,records,GAUGE,COUNT,AVG,NONE +APPLICATION,jobs.auto_create_partial_stats.resume_completed,Number of auto_create_partial_stats jobs which successfully resumed to completion,jobs,COUNTER,COUNT,AVG,NON_NEGATIVE_DERIVATIVE +APPLICATION,jobs.auto_create_partial_stats.resume_failed,Number of auto_create_partial_stats jobs which failed with a non-retriable error,jobs,COUNTER,COUNT,AVG,NON_NEGATIVE_DERIVATIVE +APPLICATION,jobs.auto_create_partial_stats.resume_retry_error,Number of auto_create_partial_stats jobs which failed with a retriable error,jobs,COUNTER,COUNT,AVG,NON_NEGATIVE_DERIVATIVE APPLICATION,jobs.auto_create_stats.currently_idle,Number of auto_create_stats jobs currently considered Idle and can be freely shut down,jobs,GAUGE,COUNT,AVG,NONE APPLICATION,jobs.auto_create_stats.currently_paused,Number of auto_create_stats jobs currently considered Paused,jobs,GAUGE,COUNT,AVG,NONE APPLICATION,jobs.auto_create_stats.currently_running,Number of auto_create_stats jobs currently running in Resume or OnFailOrCancel state,jobs,GAUGE,COUNT,AVG,NONE @@ -2091,6 +2233,18 @@ APPLICATION,jobs.schema_change_gc.protected_record_count,Number of protected tim APPLICATION,jobs.schema_change_gc.resume_completed,Number of schema_change_gc jobs which successfully resumed to completion,jobs,COUNTER,COUNT,AVG,NON_NEGATIVE_DERIVATIVE APPLICATION,jobs.schema_change_gc.resume_failed,Number of schema_change_gc jobs which failed with a non-retriable error,jobs,COUNTER,COUNT,AVG,NON_NEGATIVE_DERIVATIVE APPLICATION,jobs.schema_change_gc.resume_retry_error,Number of schema_change_gc jobs which failed with a retriable error,jobs,COUNTER,COUNT,AVG,NON_NEGATIVE_DERIVATIVE +APPLICATION,jobs.standby_read_ts_poller.currently_idle,Number of standby_read_ts_poller jobs currently considered Idle and can be freely shut down,jobs,GAUGE,COUNT,AVG,NONE +APPLICATION,jobs.standby_read_ts_poller.currently_paused,Number of standby_read_ts_poller jobs currently considered Paused,jobs,GAUGE,COUNT,AVG,NONE +APPLICATION,jobs.standby_read_ts_poller.currently_running,Number of standby_read_ts_poller jobs currently running in Resume or OnFailOrCancel state,jobs,GAUGE,COUNT,AVG,NONE +APPLICATION,jobs.standby_read_ts_poller.expired_pts_records,Number of expired protected timestamp records owned by standby_read_ts_poller jobs,records,COUNTER,COUNT,AVG,NON_NEGATIVE_DERIVATIVE +APPLICATION,jobs.standby_read_ts_poller.fail_or_cancel_completed,Number of standby_read_ts_poller jobs which successfully completed their failure or cancelation process,jobs,COUNTER,COUNT,AVG,NON_NEGATIVE_DERIVATIVE +APPLICATION,jobs.standby_read_ts_poller.fail_or_cancel_failed,Number of standby_read_ts_poller jobs which failed with a non-retriable error on their failure or cancelation process,jobs,COUNTER,COUNT,AVG,NON_NEGATIVE_DERIVATIVE +APPLICATION,jobs.standby_read_ts_poller.fail_or_cancel_retry_error,Number of standby_read_ts_poller jobs which failed with a retriable error on their failure or cancelation process,jobs,COUNTER,COUNT,AVG,NON_NEGATIVE_DERIVATIVE +APPLICATION,jobs.standby_read_ts_poller.protected_age_sec,The age of the oldest PTS record protected by standby_read_ts_poller jobs,seconds,GAUGE,SECONDS,AVG,NONE +APPLICATION,jobs.standby_read_ts_poller.protected_record_count,Number of protected timestamp records held by standby_read_ts_poller jobs,records,GAUGE,COUNT,AVG,NONE +APPLICATION,jobs.standby_read_ts_poller.resume_completed,Number of standby_read_ts_poller jobs which successfully resumed to completion,jobs,COUNTER,COUNT,AVG,NON_NEGATIVE_DERIVATIVE +APPLICATION,jobs.standby_read_ts_poller.resume_failed,Number of standby_read_ts_poller jobs which failed with a non-retriable error,jobs,COUNTER,COUNT,AVG,NON_NEGATIVE_DERIVATIVE +APPLICATION,jobs.standby_read_ts_poller.resume_retry_error,Number of standby_read_ts_poller jobs which failed with a retriable error,jobs,COUNTER,COUNT,AVG,NON_NEGATIVE_DERIVATIVE APPLICATION,jobs.typedesc_schema_change.currently_idle,Number of typedesc_schema_change jobs currently considered Idle and can be freely shut down,jobs,GAUGE,COUNT,AVG,NONE APPLICATION,jobs.typedesc_schema_change.currently_paused,Number of typedesc_schema_change jobs currently considered Paused,jobs,GAUGE,COUNT,AVG,NONE APPLICATION,jobs.typedesc_schema_change.currently_running,Number of typedesc_schema_change jobs currently running in Resume or OnFailOrCancel state,jobs,GAUGE,COUNT,AVG,NONE @@ -2103,6 +2257,18 @@ APPLICATION,jobs.typedesc_schema_change.protected_record_count,Number of protect APPLICATION,jobs.typedesc_schema_change.resume_completed,Number of typedesc_schema_change jobs which successfully resumed to completion,jobs,COUNTER,COUNT,AVG,NON_NEGATIVE_DERIVATIVE APPLICATION,jobs.typedesc_schema_change.resume_failed,Number of typedesc_schema_change jobs which failed with a non-retriable error,jobs,COUNTER,COUNT,AVG,NON_NEGATIVE_DERIVATIVE APPLICATION,jobs.typedesc_schema_change.resume_retry_error,Number of typedesc_schema_change jobs which failed with a retriable error,jobs,COUNTER,COUNT,AVG,NON_NEGATIVE_DERIVATIVE +APPLICATION,jobs.update_table_metadata_cache.currently_idle,Number of update_table_metadata_cache jobs currently considered Idle and can be freely shut down,jobs,GAUGE,COUNT,AVG,NONE +APPLICATION,jobs.update_table_metadata_cache.currently_paused,Number of update_table_metadata_cache jobs currently considered Paused,jobs,GAUGE,COUNT,AVG,NONE +APPLICATION,jobs.update_table_metadata_cache.currently_running,Number of update_table_metadata_cache jobs currently running in Resume or OnFailOrCancel state,jobs,GAUGE,COUNT,AVG,NONE +APPLICATION,jobs.update_table_metadata_cache.expired_pts_records,Number of expired protected timestamp records owned by update_table_metadata_cache jobs,records,COUNTER,COUNT,AVG,NON_NEGATIVE_DERIVATIVE +APPLICATION,jobs.update_table_metadata_cache.fail_or_cancel_completed,Number of update_table_metadata_cache jobs which successfully completed their failure or cancelation process,jobs,COUNTER,COUNT,AVG,NON_NEGATIVE_DERIVATIVE +APPLICATION,jobs.update_table_metadata_cache.fail_or_cancel_failed,Number of update_table_metadata_cache jobs which failed with a non-retriable error on their failure or cancelation process,jobs,COUNTER,COUNT,AVG,NON_NEGATIVE_DERIVATIVE +APPLICATION,jobs.update_table_metadata_cache.fail_or_cancel_retry_error,Number of update_table_metadata_cache jobs which failed with a retriable error on their failure or cancelation process,jobs,COUNTER,COUNT,AVG,NON_NEGATIVE_DERIVATIVE +APPLICATION,jobs.update_table_metadata_cache.protected_age_sec,The age of the oldest PTS record protected by update_table_metadata_cache jobs,seconds,GAUGE,SECONDS,AVG,NONE +APPLICATION,jobs.update_table_metadata_cache.protected_record_count,Number of protected timestamp records held by update_table_metadata_cache jobs,records,GAUGE,COUNT,AVG,NONE +APPLICATION,jobs.update_table_metadata_cache.resume_completed,Number of update_table_metadata_cache jobs which successfully resumed to completion,jobs,COUNTER,COUNT,AVG,NON_NEGATIVE_DERIVATIVE +APPLICATION,jobs.update_table_metadata_cache.resume_failed,Number of update_table_metadata_cache jobs which failed with a non-retriable error,jobs,COUNTER,COUNT,AVG,NON_NEGATIVE_DERIVATIVE +APPLICATION,jobs.update_table_metadata_cache.resume_retry_error,Number of update_table_metadata_cache jobs which failed with a retriable error,jobs,COUNTER,COUNT,AVG,NON_NEGATIVE_DERIVATIVE APPLICATION,kv.protectedts.reconciliation.errors,number of errors encountered during reconciliation runs on this node,Count,COUNTER,COUNT,AVG,NON_NEGATIVE_DERIVATIVE APPLICATION,kv.protectedts.reconciliation.num_runs,number of successful reconciliation runs on this node,Count,COUNTER,COUNT,AVG,NON_NEGATIVE_DERIVATIVE APPLICATION,kv.protectedts.reconciliation.records_processed,number of records processed without error during reconciliation on this node,Count,COUNTER,COUNT,AVG,NON_NEGATIVE_DERIVATIVE @@ -2112,37 +2278,38 @@ APPLICATION,logical_replication.checkpoint_events_ingested,Checkpoint events ing APPLICATION,logical_replication.commit_latency,"Event commit latency: a difference between event MVCC timestamp and the time it was flushed into disk. If we batch events, then the difference between the oldest event in the batch and flush is recorded",Nanoseconds,HISTOGRAM,NANOSECONDS,AVG,NONE APPLICATION,logical_replication.events_dlqed,Row update events sent to DLQ,Failures,COUNTER,COUNT,AVG,NON_NEGATIVE_DERIVATIVE APPLICATION,logical_replication.events_dlqed_age,Row update events sent to DLQ due to reaching the maximum time allowed in the retry queue,Failures,COUNTER,COUNT,AVG,NON_NEGATIVE_DERIVATIVE +APPLICATION,logical_replication.events_dlqed_by_label,Row update events sent to DLQ by label,Failures,COUNTER,COUNT,AVG,NON_NEGATIVE_DERIVATIVE APPLICATION,logical_replication.events_dlqed_errtype,Row update events sent to DLQ due to an error not considered retryable,Failures,COUNTER,COUNT,AVG,NON_NEGATIVE_DERIVATIVE APPLICATION,logical_replication.events_dlqed_space,Row update events sent to DLQ due to capacity of the retry queue,Failures,COUNTER,COUNT,AVG,NON_NEGATIVE_DERIVATIVE APPLICATION,logical_replication.events_ingested,Events ingested by all replication jobs,Events,COUNTER,COUNT,AVG,NON_NEGATIVE_DERIVATIVE +APPLICATION,logical_replication.events_ingested_by_label,Events ingested by all replication jobs by label,Events,COUNTER,COUNT,AVG,NON_NEGATIVE_DERIVATIVE APPLICATION,logical_replication.events_initial_failure,Failed attempts to apply an incoming row update,Failures,COUNTER,COUNT,AVG,NON_NEGATIVE_DERIVATIVE APPLICATION,logical_replication.events_initial_success,Successful applications of an incoming row update,Failures,COUNTER,COUNT,AVG,NON_NEGATIVE_DERIVATIVE APPLICATION,logical_replication.events_retry_failure,Failed re-attempts to apply a row update,Failures,COUNTER,COUNT,AVG,NON_NEGATIVE_DERIVATIVE APPLICATION,logical_replication.events_retry_success,Row update events applied after one or more retries,Failures,COUNTER,COUNT,AVG,NON_NEGATIVE_DERIVATIVE -APPLICATION,logical_replication.flush_bytes,Number of bytes in a given flush,Logical bytes,HISTOGRAM,BYTES,AVG,NONE -APPLICATION,logical_replication.flush_hist_nanos,Time spent flushing messages across all replication streams,Nanoseconds,HISTOGRAM,NANOSECONDS,AVG,NONE -APPLICATION,logical_replication.flush_row_count,Number of rows in a given flush,Rows,HISTOGRAM,COUNT,AVG,NONE +APPLICATION,logical_replication.kv.update_too_old,Total number of updates that were not applied because they were too old,Events,COUNTER,COUNT,AVG,NON_NEGATIVE_DERIVATIVE +APPLICATION,logical_replication.kv.value_refreshes,Total number of batches that refreshed the previous value,Events,COUNTER,COUNT,AVG,NON_NEGATIVE_DERIVATIVE APPLICATION,logical_replication.logical_bytes,Logical bytes (sum of keys + values) received by all replication jobs,Bytes,COUNTER,BYTES,AVG,NON_NEGATIVE_DERIVATIVE -APPLICATION,logical_replication.optimistic_insert_conflict_count,Total number of times the optimistic insert encountered a conflict,Events,COUNTER,COUNT,AVG,NON_NEGATIVE_DERIVATIVE APPLICATION,logical_replication.replan_count,Total number of dist sql replanning events,Events,COUNTER,COUNT,AVG,NON_NEGATIVE_DERIVATIVE +APPLICATION,logical_replication.replicated_time_by_label,Replicated time of the logical replication stream by label,Seconds,COUNTER,SECONDS,AVG,NON_NEGATIVE_DERIVATIVE APPLICATION,logical_replication.replicated_time_seconds,The replicated time of the logical replication stream in seconds since the unix epoch.,Seconds,GAUGE,SECONDS,AVG,NONE APPLICATION,logical_replication.retry_queue_bytes,The replicated time of the logical replication stream in seconds since the unix epoch.,Bytes,GAUGE,BYTES,AVG,NONE APPLICATION,logical_replication.retry_queue_events,The replicated time of the logical replication stream in seconds since the unix epoch.,Events,GAUGE,COUNT,AVG,NONE +APPLICATION,obs.tablemetadata.update_job.duration,Time spent running the update table metadata job.,Duration,HISTOGRAM,NANOSECONDS,AVG,NONE +APPLICATION,obs.tablemetadata.update_job.errors,The total number of errors that have been emitted from the update table metadata job.,Errors,COUNTER,COUNT,AVG,NON_NEGATIVE_DERIVATIVE +APPLICATION,obs.tablemetadata.update_job.runs,The total number of runs of the update table metadata job.,Executions,COUNTER,COUNT,AVG,NON_NEGATIVE_DERIVATIVE +APPLICATION,obs.tablemetadata.update_job.table_updates,The total number of rows that have been updated in system.table_metadata,Rows Updated,COUNTER,COUNT,AVG,NON_NEGATIVE_DERIVATIVE APPLICATION,physical_replication.admit_latency,Event admission latency: a difference between event MVCC timestamp and the time it was admitted into ingestion processor,Nanoseconds,HISTOGRAM,NANOSECONDS,AVG,NONE APPLICATION,physical_replication.commit_latency,"Event commit latency: a difference between event MVCC timestamp and the time it was flushed into disk. If we batch events, then the difference between the oldest event in the batch and flush is recorded",Nanoseconds,HISTOGRAM,NANOSECONDS,AVG,NONE -APPLICATION,physical_replication.cutover_progress,The number of ranges left to revert in order to complete an inflight cutover,Ranges,GAUGE,COUNT,AVG,NONE APPLICATION,physical_replication.distsql_replan_count,Total number of dist sql replanning events,Events,COUNTER,COUNT,AVG,NON_NEGATIVE_DERIVATIVE -APPLICATION,physical_replication.earliest_data_checkpoint_span,The earliest timestamp of the last checkpoint forwarded by an ingestion data processor,Timestamp,GAUGE,TIMESTAMP_NS,AVG,NONE APPLICATION,physical_replication.events_ingested,Events ingested by all replication jobs,Events,COUNTER,COUNT,AVG,NON_NEGATIVE_DERIVATIVE +APPLICATION,physical_replication.failover_progress,The number of ranges left to revert in order to complete an inflight cutover,Ranges,GAUGE,COUNT,AVG,NONE APPLICATION,physical_replication.flush_hist_nanos,Time spent flushing messages across all replication streams,Nanoseconds,HISTOGRAM,NANOSECONDS,AVG,NONE APPLICATION,physical_replication.flushes,Total flushes across all replication jobs,Flushes,COUNTER,COUNT,AVG,NON_NEGATIVE_DERIVATIVE -APPLICATION,physical_replication.job_progress_updates,Total number of updates to the ingestion job progress,Job Updates,COUNTER,COUNT,AVG,NON_NEGATIVE_DERIVATIVE -APPLICATION,physical_replication.latest_data_checkpoint_span,The latest timestamp of the last checkpoint forwarded by an ingestion data processor,Timestamp,GAUGE,TIMESTAMP_NS,AVG,NONE APPLICATION,physical_replication.logical_bytes,Logical bytes (sum of keys + values) ingested by all replication jobs,Bytes,COUNTER,BYTES,AVG,NON_NEGATIVE_DERIVATIVE APPLICATION,physical_replication.replicated_time_seconds,The replicated time of the physical replication stream in seconds since the unix epoch.,Seconds,GAUGE,SECONDS,AVG,NONE APPLICATION,physical_replication.resolved_events_ingested,Resolved events ingested by all replication jobs,Events,COUNTER,COUNT,AVG,NON_NEGATIVE_DERIVATIVE APPLICATION,physical_replication.running,Number of currently running replication streams,Replication Streams,GAUGE,COUNT,AVG,NONE -APPLICATION,physical_replication.sst_bytes,SST bytes (compressed) sent to KV by all replication jobs,Bytes,COUNTER,BYTES,AVG,NON_NEGATIVE_DERIVATIVE APPLICATION,requests.slow.distsender,"Number of range-bound RPCs currently stuck or retrying for a long time. Note that this is not a good signal for KV health. The remote side of the @@ -2159,6 +2326,8 @@ metrics such as packet loss, retransmits, etc, to conclusively diagnose network issues. Heartbeats are not very frequent (~seconds), so they may not capture rare or short-lived degradations. ",Round-trip time,HISTOGRAM,NANOSECONDS,AVG,NONE +APPLICATION,rpc.client.bytes.egress,Counter of TCP bytes sent via gRPC on connections we initiated.,Bytes,COUNTER,BYTES,AVG,NON_NEGATIVE_DERIVATIVE +APPLICATION,rpc.client.bytes.ingress,Counter of TCP bytes received via gRPC on connections we initiated.,Bytes,COUNTER,BYTES,AVG,NON_NEGATIVE_DERIVATIVE APPLICATION,rpc.connection.avg_round_trip_latency,"Sum of exponentially weighted moving average of round-trip latencies, as measured through a gRPC RPC. Dividing this Gauge by rpc.connection.healthy gives an approximation of average @@ -2170,6 +2339,12 @@ these provide per-peer moving averages. This metric does not track failed connection. A failed connection's contribution is reset to zero. ",Latency,GAUGE,NANOSECONDS,AVG,NONE +APPLICATION,rpc.connection.connected,"Counter of TCP level connected connections. + +This metric is the number of gRPC connections from the TCP level. Unlike rpc.connection.healthy +this metric does not take into account whether the application has been able to heartbeat +over this connection. +",Connections,GAUGE,COUNT,AVG,NONE APPLICATION,rpc.connection.failures,"Counter of failed connections. This includes both the event in which a healthy connection terminates as well as @@ -2215,6 +2390,7 @@ APPLICATION,schedules.scheduled-schema-telemetry-executor.succeeded,Number of sc APPLICATION,schedules.scheduled-sql-stats-compaction-executor.failed,Number of scheduled-sql-stats-compaction-executor jobs failed,Jobs,COUNTER,COUNT,AVG,NON_NEGATIVE_DERIVATIVE APPLICATION,schedules.scheduled-sql-stats-compaction-executor.started,Number of scheduled-sql-stats-compaction-executor jobs started,Jobs,COUNTER,COUNT,AVG,NON_NEGATIVE_DERIVATIVE APPLICATION,schedules.scheduled-sql-stats-compaction-executor.succeeded,Number of scheduled-sql-stats-compaction-executor jobs succeeded,Jobs,COUNTER,COUNT,AVG,NON_NEGATIVE_DERIVATIVE +APPLICATION,server.http.request.duration.nanos,Duration of an HTTP request in nanoseconds.,Duration,HISTOGRAM,NANOSECONDS,AVG,NONE APPLICATION,sql.bytesin,Number of SQL bytes received,SQL Bytes,COUNTER,BYTES,AVG,NON_NEGATIVE_DERIVATIVE APPLICATION,sql.bytesout,Number of SQL bytes sent,SQL Bytes,COUNTER,BYTES,AVG,NON_NEGATIVE_DERIVATIVE APPLICATION,sql.conn.failures,Number of SQL connection failures,Connections,COUNTER,COUNT,AVG,NON_NEGATIVE_DERIVATIVE @@ -2234,6 +2410,10 @@ APPLICATION,sql.copy.nonatomic.started.count,Number of non-atomic COPY SQL state APPLICATION,sql.copy.nonatomic.started.count.internal,Number of non-atomic COPY SQL statements started (internal queries),SQL Internal Statements,COUNTER,COUNT,AVG,NON_NEGATIVE_DERIVATIVE APPLICATION,sql.copy.started.count,Number of COPY SQL statements started,SQL Statements,COUNTER,COUNT,AVG,NON_NEGATIVE_DERIVATIVE APPLICATION,sql.copy.started.count.internal,Number of COPY SQL statements started (internal queries),SQL Internal Statements,COUNTER,COUNT,AVG,NON_NEGATIVE_DERIVATIVE +APPLICATION,sql.crud_query.count,"Number of SQL SELECT, INSERT, UPDATE, DELETE statements successfully executed",SQL Statements,COUNTER,COUNT,AVG,NON_NEGATIVE_DERIVATIVE +APPLICATION,sql.crud_query.count.internal,"Number of SQL SELECT, INSERT, UPDATE, DELETE statements successfully executed (internal queries)",SQL Internal Statements,COUNTER,COUNT,AVG,NON_NEGATIVE_DERIVATIVE +APPLICATION,sql.crud_query.started.count,"Number of SQL SELECT, INSERT, UPDATE, DELETE statements started",SQL Statements,COUNTER,COUNT,AVG,NON_NEGATIVE_DERIVATIVE +APPLICATION,sql.crud_query.started.count.internal,"Number of SQL SELECT, INSERT, UPDATE, DELETE statements started (internal queries)",SQL Internal Statements,COUNTER,COUNT,AVG,NON_NEGATIVE_DERIVATIVE APPLICATION,sql.ddl.count,Number of SQL DDL statements successfully executed,SQL Statements,COUNTER,COUNT,AVG,NON_NEGATIVE_DERIVATIVE APPLICATION,sql.ddl.count.internal,Number of SQL DDL statements successfully executed (internal queries),SQL Internal Statements,COUNTER,COUNT,AVG,NON_NEGATIVE_DERIVATIVE APPLICATION,sql.ddl.started.count,Number of SQL DDL statements started,SQL Statements,COUNTER,COUNT,AVG,NON_NEGATIVE_DERIVATIVE @@ -2349,10 +2529,10 @@ APPLICATION,sql.pre_serve.conn.failures,Number of SQL connection failures prior APPLICATION,sql.pre_serve.mem.cur,Current memory usage for SQL connections prior to routing the connection to the target SQL server,Memory,GAUGE,BYTES,AVG,NONE APPLICATION,sql.pre_serve.mem.max,Memory usage for SQL connections prior to routing the connection to the target SQL server,Memory,HISTOGRAM,BYTES,AVG,NONE APPLICATION,sql.pre_serve.new_conns,Number of SQL connections created prior to routing the connection to the target SQL server,Connections,COUNTER,COUNT,AVG,NON_NEGATIVE_DERIVATIVE -APPLICATION,sql.query.count,Number of SQL queries executed,SQL Statements,COUNTER,COUNT,AVG,NON_NEGATIVE_DERIVATIVE -APPLICATION,sql.query.count.internal,Number of SQL queries executed (internal queries),SQL Internal Statements,COUNTER,COUNT,AVG,NON_NEGATIVE_DERIVATIVE -APPLICATION,sql.query.started.count,Number of SQL queries started,SQL Statements,COUNTER,COUNT,AVG,NON_NEGATIVE_DERIVATIVE -APPLICATION,sql.query.started.count.internal,Number of SQL queries started (internal queries),SQL Internal Statements,COUNTER,COUNT,AVG,NON_NEGATIVE_DERIVATIVE +APPLICATION,sql.query.count,"Number of SQL operations started including queries, and transaction control statements",SQL Statements,COUNTER,COUNT,AVG,NON_NEGATIVE_DERIVATIVE +APPLICATION,sql.query.count.internal,"Number of SQL operations started including queries, and transaction control statements (internal queries)",SQL Internal Statements,COUNTER,COUNT,AVG,NON_NEGATIVE_DERIVATIVE +APPLICATION,sql.query.started.count,"Number of SQL operations started including queries, and transaction control statements",SQL Statements,COUNTER,COUNT,AVG,NON_NEGATIVE_DERIVATIVE +APPLICATION,sql.query.started.count.internal,"Number of SQL operations started including queries, and transaction control statements (internal queries)",SQL Internal Statements,COUNTER,COUNT,AVG,NON_NEGATIVE_DERIVATIVE APPLICATION,sql.restart_savepoint.count,Number of `SAVEPOINT cockroach_restart` statements successfully executed,SQL Statements,COUNTER,COUNT,AVG,NON_NEGATIVE_DERIVATIVE APPLICATION,sql.restart_savepoint.count.internal,Number of `SAVEPOINT cockroach_restart` statements successfully executed (internal queries),SQL Internal Statements,COUNTER,COUNT,AVG,NON_NEGATIVE_DERIVATIVE APPLICATION,sql.restart_savepoint.release.count,Number of `RELEASE SAVEPOINT cockroach_restart` statements successfully executed,SQL Statements,COUNTER,COUNT,AVG,NON_NEGATIVE_DERIVATIVE @@ -2450,6 +2630,7 @@ APPLICATION,tenant.sql_usage.external_io_egress_bytes,Total number of bytes writ APPLICATION,tenant.sql_usage.external_io_ingress_bytes,Total number of bytes read from external services such as cloud storage providers,Bytes,COUNTER,COUNT,AVG,NON_NEGATIVE_DERIVATIVE APPLICATION,tenant.sql_usage.kv_request_units,RU consumption attributable to KV,Request Units,COUNTER,COUNT,AVG,NON_NEGATIVE_DERIVATIVE APPLICATION,tenant.sql_usage.pgwire_egress_bytes,Total number of bytes transferred from a SQL pod to the client,Bytes,COUNTER,COUNT,AVG,NON_NEGATIVE_DERIVATIVE +APPLICATION,tenant.sql_usage.provisioned_vcpus,Number of vcpus available to the virtual cluster,Count,GAUGE,COUNT,AVG,NONE APPLICATION,tenant.sql_usage.read_batches,Total number of KV read batches,Requests,COUNTER,COUNT,AVG,NON_NEGATIVE_DERIVATIVE APPLICATION,tenant.sql_usage.read_bytes,Total number of bytes read from KV,Bytes,COUNTER,COUNT,AVG,NON_NEGATIVE_DERIVATIVE APPLICATION,tenant.sql_usage.read_requests,Total number of KV read requests,Requests,COUNTER,COUNT,AVG,NON_NEGATIVE_DERIVATIVE @@ -2485,7 +2666,6 @@ APPLICATION,txn.restarts.txnaborted,Number of restarts due to an abort by a conc APPLICATION,txn.restarts.txnpush,Number of restarts due to a transaction push failure,Restarted Transactions,COUNTER,COUNT,AVG,NON_NEGATIVE_DERIVATIVE APPLICATION,txn.restarts.unknown,Number of restarts due to a unknown reasons,Restarted Transactions,COUNTER,COUNT,AVG,NON_NEGATIVE_DERIVATIVE APPLICATION,txn.restarts.writetooold,Number of restarts due to a concurrent writer committing first,Restarted Transactions,COUNTER,COUNT,AVG,NON_NEGATIVE_DERIVATIVE -APPLICATION,txn.restarts.writetoooldmulti,Number of restarts due to multiple concurrent writers committing first,Restarted Transactions,COUNTER,COUNT,AVG,NON_NEGATIVE_DERIVATIVE APPLICATION,txn.rollbacks.async.failed,Number of KV transaction that failed to send abort asynchronously which is not always retried,KV Transactions,COUNTER,COUNT,AVG,NON_NEGATIVE_DERIVATIVE APPLICATION,txn.rollbacks.failed,Number of KV transaction that failed to send final abort,KV Transactions,COUNTER,COUNT,AVG,NON_NEGATIVE_DERIVATIVE SERVER,build.timestamp,Build information,Build Time,GAUGE,TIMESTAMP_SEC,AVG,NONE diff --git a/src/current/_includes/v24.3/metric-names.md b/src/current/_includes/v24.3/metric-names.md index 9bbe10b49d5..c72864fd149 100644 --- a/src/current/_includes/v24.3/metric-names.md +++ b/src/current/_includes/v24.3/metric-names.md @@ -1,331 +1,29 @@ -Name | Description ------|------------ -`addsstable.applications` | Number of SSTable ingestions applied (i.e., applied by Replicas) -`addsstable.copies` | Number of SSTable ingestions that required copying files during application -`addsstable.proposals` | Number of SSTable ingestions proposed (i.e., sent to Raft by lease holders) -`build.timestamp` | Build information -`capacity.available` | Available storage capacity -`capacity.reserved` | Capacity reserved for snapshots -`capacity.used` | Used storage capacity -`capacity` | Total storage capacity -`changefeed.aggregator_progress` | The earliest timestamp up to which any [aggregator]({% link {{ page.version.version }}/how-does-an-enterprise-changefeed-work.md %}) is guaranteed to have emitted all values for which it is responsible. **Note:** This metric may regress when a changefeed restarts due to a transient error. Consider tracking the `changefeed.checkpoint_progress` metric, which will not regress. -`changefeed.checkpoint_progress` | The earliest timestamp of any changefeed's persisted checkpoint (values prior to this timestamp will never need to be re-emitted). -`changefeed.failures` | Total number of [changefeed jobs]({% link {{ page.version.version }}/show-jobs.md %}#show-changefeed-jobs) which have failed. -`changefeed.lagging_ranges` | Number of ranges which are behind in a changefeed. This is calculated based on the changefeed options:
  • [`lagging_ranges_threshold`]({% link {{ page.version.version }}/create-changefeed.md %}#lagging-ranges-threshold), which is the amount of time that a range checkpoint needs to be in the past to be considered lagging.
  • [`lagging_ranges_polling_interval`]({% link {{ page.version.version }}/create-changefeed.md %}#lagging-ranges-polling-interval), which is the frequency at which lagging ranges are polled and the metric is updated.
-`changefeed.running` | Number of currently running changefeeds, including sinkless. -`clock-offset.meannanos` | Mean clock offset with other nodes in nanoseconds -`clock-offset.stddevnanos` | Std dev clock offset with other nodes in nanoseconds -`cluster.preserve-downgrade-option.last-updated` | Unix timestamp of last updated time for cluster.preserve_downgrade_option -`compactor.compactingnanos` | Number of nanoseconds spent compacting ranges -`compactor.compactions.failure` | Number of failed compaction requests sent to the storage engine -`compactor.compactions.success` | Number of successful compaction requests sent to the storage engine -`compactor.suggestionbytes.compacted` | Number of logical bytes compacted from suggested compactions -`compactor.suggestionbytes.queued` | Number of logical bytes in suggested compactions in the queue -`compactor.suggestionbytes.skipped` | Number of logical bytes in suggested compactions which were not compacted -`distsender.batches.partial` | Number of partial batches processed -`distsender.batches` | Number of batches processed -`distsender.errors.notleaseholder` | Number of NotLeaseHolderErrors encountered -`distsender.rpc.sent.local` | Number of local RPCs sent -`distsender.rpc.sent.nextreplicaerror` | Number of RPCs sent due to per-replica errors -`distsender.rpc.sent` | Number of RPCs sent -`exec.error` | Number of batch KV requests that failed to execute on this node -`exec.latency` | Latency in nanoseconds of batch KV requests executed on this node -`exec.success` | Number of batch KV requests executed successfully on this node -`gcbytesage` | Cumulative age of non-live data in seconds -`gossip.bytes.received` | Number of received gossip bytes -`gossip.bytes.sent` | Number of sent gossip bytes -`gossip.connections.incoming` | Number of active incoming gossip connections -`gossip.connections.outgoing` | Number of active outgoing gossip connections -`gossip.connections.refused` | Number of refused incoming gossip connections -`gossip.infos.received` | Number of received gossip Info objects -`gossip.infos.sent` | Number of sent gossip Info objects -`intentage` | Cumulative age of intents in seconds -`intentbytes` | Number of bytes in intent KV pairs -`intentcount` | Count of intent keys -`jobs.changefeed.expired_pts_records` | Number of expired [protected timestamp]({% link {{ page.version.version }}/architecture/storage-layer.md %}#protected-timestamps) records owned by [changefeed jobs]({% link {{ page.version.version }}/show-jobs.md %}#show-changefeed-jobs). -`jobs.{job_type}.currently_paused` | Number of `{job_type}` [jobs]({% link {{ page.version.version }}/show-jobs.md %}) currently considered paused. See the [`/_status/vars`]({% link {{ page.version.version }}/monitoring-and-alerting.md %}#prometheus-endpoint) endpoint for all job types. -`jobs.{job_type}.protected_age_sec` | The age of the oldest [protected timestamp]({% link {{ page.version.version }}/architecture/storage-layer.md %}#protected-timestamps) record protecting `{job_type}` [jobs]({% link {{ page.version.version }}/show-jobs.md %}). See the [`/_status/vars`]({% link {{ page.version.version }}/monitoring-and-alerting.md %}#prometheus-endpoint) endpoint for all job types. -`jobs.{job_type}.protected_record_count` | Number of [protected timestamp]({% link {{ page.version.version }}/architecture/storage-layer.md %}#protected-timestamps) records held by `{job_type}` [jobs]({% link {{ page.version.version }}/show-jobs.md %}). See the [`/_status/vars`]({% link {{ page.version.version }}/monitoring-and-alerting.md %}#prometheus-endpoint) endpoint for all job types. -`jobs.row_level_ttl.num_active_spans` | Number of active spans the TTL job is deleting from -`jobs.row_level_ttl.span_total_duration` | Duration for processing a span during row level TTL -`keybytes` | Number of bytes taken up by keys -`keycount` | Count of all keys -`lastupdatenanos` | Time in nanoseconds since Unix epoch at which bytes/keys/intents metrics were last updated -`leases.epoch` | Number of replica leaseholders using epoch-based leases -`leases.error` | Number of failed lease requests -`leases.expiration` | Number of replica leaseholders using expiration-based leases -`leases.success` | Number of successful lease requests -`leases.transfers.error` | Number of failed lease transfers -`leases.transfers.success` | Number of successful lease transfers -`livebytes` | Number of bytes of live data (keys plus values), including unreplicated data -`livecount` | Count of live keys -`liveness.epochincrements` | Number of times this node has incremented its liveness epoch -`liveness.heartbeatfailures` | Number of failed node liveness heartbeats from this node -`liveness.heartbeatlatency` | Node liveness heartbeat latency in nanoseconds -`liveness.heartbeatsuccesses` | Number of successful node liveness heartbeats from this node -`liveness.livenodes` | Number of live nodes in the cluster (will be 0 if this node is not itself live) -`node-id` | node ID with labels for advertised RPC and HTTP addresses -`queue.consistency.pending` | Number of pending replicas in the consistency checker queue -`queue.consistency.process.failure` | Number of replicas which failed processing in the consistency checker queue -`queue.consistency.process.success` | Number of replicas successfully processed by the consistency checker queue -`queue.consistency.processingnanos` | Nanoseconds spent processing replicas in the consistency checker queue -`queue.gc.info.abortspanconsidered` | Number of AbortSpan entries eligible for removal based on their ages -`queue.gc.info.abortspangcnum` | Number of AbortSpan entries fit for removal -`queue.gc.info.abortspanscanned` | Number of transactions present in the AbortSpan scanned from the engine -`queue.gc.info.clearrangefailed` | Number of failed ClearRange operations during GC -`queue.gc.info.clearrangesuccess` | Number of successful ClearRange operations during GC -`queue.gc.info.intentsconsidered` | Number of intents eligible to be considered because they are at least two hours old -`queue.gc.info.intenttxns` | Number of associated distinct transactions -`queue.gc.info.numkeysaffected` | Number of keys with GC'able data -`queue.gc.info.pushtxn` | Number of attempted pushes -`queue.gc.info.resolvesuccess` | Number of successful intent resolutions -`queue.gc.info.resolvetotal` | Number of attempted intent resolutions -`queue.gc.info.transactionspangcaborted` | Number of GC'able entries corresponding to aborted txns -`queue.gc.info.transactionspangccommitted` | Number of GC'able entries corresponding to committed txns -`queue.gc.info.transactionspangcpending` | Number of GC'able entries corresponding to pending txns -`queue.gc.info.transactionspanscanned` | Number of entries in transaction spans scanned from the engine -`queue.gc.pending` | Number of pending replicas in the GC queue -`queue.gc.process.failure` | Number of replicas which failed processing in the GC queue -`queue.gc.process.success` | Number of replicas successfully processed by the GC queue -`queue.gc.processingnanos` | Nanoseconds spent processing replicas in the GC queue -`queue.raftlog.pending` | Number of pending replicas in the Raft log queue -`queue.raftlog.process.failure` | Number of replicas which failed processing in the Raft log queue -`queue.raftlog.process.success` | Number of replicas successfully processed by the Raft log queue -`queue.raftlog.processingnanos` | Nanoseconds spent processing replicas in the Raft log queue -`queue.raftsnapshot.pending` | Number of pending replicas in the Raft repair queue -`queue.raftsnapshot.process.failure` | Number of replicas which failed processing in the Raft repair queue -`queue.raftsnapshot.process.success` | Number of replicas successfully processed by the Raft repair queue -`queue.raftsnapshot.processingnanos` | Nanoseconds spent processing replicas in the Raft repair queue -`queue.replicagc.pending` | Number of pending replicas in the replica GC queue -`queue.replicagc.process.failure` | Number of replicas which failed processing in the replica GC queue -`queue.replicagc.process.success` | Number of replicas successfully processed by the replica GC queue -`queue.replicagc.processingnanos` | Nanoseconds spent processing replicas in the replica GC queue -`queue.replicagc.removereplica` | Number of replica removals attempted by the replica gc queue -`queue.replicate.addreplica` | Number of replica additions attempted by the replicate queue -`queue.replicate.addreplica.error` | Number of failed replica additions processed by the replicate queue -`queue.replicate.addreplica.success` | Number of successful replica additions processed by the replicate queue -`queue.replicate.pending` | Number of pending replicas in the replicate queue -`queue.replicate.process.failure` | Number of replicas which failed processing in the replicate queue -`queue.replicate.process.success` | Number of replicas successfully processed by the replicate queue -`queue.replicate.processingnanos` | Nanoseconds spent processing replicas in the replicate queue -`queue.replicate.purgatory` | Number of replicas in the replicate queue's purgatory, awaiting allocation options -`queue.replicate.rebalancereplica` | Number of replica rebalancer-initiated additions attempted by the replicate queue -`queue.replicate.removedeadreplica` | Number of dead replica removals attempted by the replicate queue (typically in response to a node outage) -`queue.replicate.removedeadreplica.error` | Number of failed dead replica removals processed by the replicate queue -`queue.replicate.removedeadreplica.success` | Number of successful dead replica removals processed by the replicate queue -`queue.replicate.removedecommissioningreplica.error` | Number of failed decommissioning replica removals processed by the replicate queue -`queue.replicate.removedecommissioningreplica.success` | Number of successful decommissioning replica removals processed by the replicate queue -`queue.replicate.removereplica` | Number of replica removals attempted by the replicate queue (typically in response to a rebalancer-initiated addition) -`queue.replicate.removereplica.error` | Number of failed replica removals processed by the replicate queue -`queue.replicate.removereplica.success` | Number of successful replica removals processed by the replicate queue -`queue.replicate.replacedeadreplica.error` | Number of failed dead replica replacements processed by the replicate queue -`queue.replicate.replacedeadreplica.success` | Number of successful dead replica replacements processed by the replicate queue -`queue.replicate.replacedecommissioningreplica.error` | Number of failed decommissioning replica replacements processed by the replicate queue -`queue.replicate.replacedecommissioningreplica.success` | Number of successful decommissioning replica replacements processed by the replicate queue -`queue.replicate.transferlease` | Number of range lease transfers attempted by the replicate queue -`queue.split.pending` | Number of pending replicas in the split queue -`queue.split.process.failure` | Number of replicas which failed processing in the split queue -`queue.split.process.success` | Number of replicas successfully processed by the split queue -`queue.split.processingnanos` | Nanoseconds spent processing replicas in the split queue -`queue.tsmaintenance.pending` | Number of pending replicas in the time series maintenance queue -`queue.tsmaintenance.process.failure` | Number of replicas which failed processing in the time series maintenance queue -`queue.tsmaintenance.process.success` | Number of replicas successfully processed by the time series maintenance queue -`queue.tsmaintenance.processingnanos` | Nanoseconds spent processing replicas in the time series maintenance queue -`raft.commandsapplied` | Count of Raft commands applied -`raft.enqueued.pending` | Number of pending outgoing messages in the Raft Transport queue -`raft.heartbeats.pending` | Number of pending heartbeats and responses waiting to be coalesced -`raft.process.commandcommit.latency` | Latency histogram in nanoseconds for committing Raft commands -`raft.process.logcommit.latency` | Latency histogram in nanoseconds for committing Raft log entries -`raft.process.tickingnanos` | Nanoseconds spent in store.processRaft() processing replica.Tick() -`raft.process.workingnanos` | Nanoseconds spent in store.processRaft() working -`raft.rcvd.app` | Number of MsgApp messages received by this store -`raft.rcvd.appresp` | Number of MsgAppResp messages received by this store -`raft.rcvd.dropped` | Number of dropped incoming Raft messages -`raft.rcvd.heartbeat` | Number of (coalesced, if enabled) MsgHeartbeat messages received by this store -`raft.rcvd.heartbeatresp` | Number of (coalesced, if enabled) MsgHeartbeatResp messages received by this store -`raft.rcvd.prevote` | Number of MsgPreVote messages received by this store -`raft.rcvd.prevoteresp` | Number of MsgPreVoteResp messages received by this store -`raft.rcvd.prop` | Number of MsgProp messages received by this store -`raft.rcvd.snap` | Number of MsgSnap messages received by this store -`raft.rcvd.timeoutnow` | Number of MsgTimeoutNow messages received by this store -`raft.rcvd.transferleader` | Number of MsgTransferLeader messages received by this store -`raft.rcvd.vote` | Number of MsgVote messages received by this store -`raft.rcvd.voteresp` | Number of MsgVoteResp messages received by this store -`raft.ticks` | Number of Raft ticks queued -`raftlog.behind` | Number of Raft log entries followers on other stores are behind -`raftlog.truncated` | Number of Raft log entries truncated -`range.adds` | Number of range additions -`range.raftleadertransfers` | Number of Raft leader transfers -`range.removes` | Number of range removals -`range.snapshots.recv-in-progress` | Number of non-empty snapshots in progress on a receiver store -`range.snapshots.recv-queue` | Number of queued non-empty snapshots on a receiver store -`range.snapshots.recv-total-in-progress` | Number of empty and non-empty snapshots in progress on a receiver store -`range.snapshots.send-in-progress` | Number of non-empty snapshots in progress on a sender store -`range.snapshots.send-queue` | Number of queued non-empty snapshots on a sender store -`range.snapshots.send-total-in-progress` | Number of empty and non-empty in-progress snapshots on a sender store -`range.snapshots.generated` | Number of generated snapshots -`range.snapshots.normal-applied` | Number of applied snapshots -`range.snapshots.preemptive-applied` | Number of applied preemptive snapshots -`range.snapshots.rcvd-bytes` | Number of snapshot bytes received -`range.snapshots.rebalancing.rcvd-bytes` | Number of rebalancing snapshot bytes received -`range.snapshots.rebalancing.sent-bytes` | Number of rebalancing snapshot bytes sent -`range.snapshots.recovery.rcvd-bytes` | Number of recovery snapshot bytes received -`range.snapshots.recovery.sent-bytes` | Number of recovery snapshot bytes sent -`range.snapshots.recv-in-progress` | Number of non-empty snapshots being received -`range.snapshots.recv-queue` | Number of snapshots queued to receive -`range.snapshots.recv-total-in-progress` | Number of total snapshots being received -`range.snapshots.send-in-progress` | Number of non-empty snapshots being sent -`range.snapshots.send-queue` | Number of snapshots queued to send -`range.snapshots.send-total-in-progress` | Number of total snapshots being sent -`range.snapshots.sent-bytes` | Number of snapshot bytes sent -`range.snapshots.unknown.rcvd-bytes` | Number of unknown snapshot bytes received -`range.snapshots.unknown.sent-bytes` | Number of unknown snapshot bytes sent -`range.splits` | Number of range splits -`rangekeybytes` | Number of bytes taken up by range keys (e.g., MVCC range tombstones) -`rangekeycount` | Count of all range keys (e.g., MVCC range tombstones) -`ranges.unavailable` | Number of ranges with fewer live replicas than needed for quorum -`ranges.underreplicated` | Number of ranges with fewer live replicas than the replication target -`ranges` | Number of ranges -`rangevalbytes` | Number of bytes taken up by range key values (e.g., MVCC range tombstones) -`rangevalcount` | Count of all range key values (e.g., MVCC range tombstones) -`rebalancing.queriespersecond` | Number of kv-level requests received per second by the store, considering the last 30 minutes, as used in rebalancing decisions. -`rebalancing.readbytespersecond` | Number of bytes written per second, considering the last 30 minutes. -`rebalancing.readspersecond` | Number of keys read recently per second, considering the last 30 minutes. -`rebalancing.requestspersecond` | Number of requests received recently per second, considering the last 30 minutes. -`rebalancing.writebytespersecond` | Number of bytes read recently per second, considering the last 30 minutes. -`rebalancing.writespersecond` | Number of keys written (i.e. applied by Raft) per second to the store, considering the last 30 minutes. -`replicas.commandqueue.combinedqueuesize` | Number of commands in all CommandQueues combined -`replicas.commandqueue.combinedreadcount` | Number of read-only commands in all CommandQueues combined -`replicas.commandqueue.combinedwritecount` | Number of read-write commands in all CommandQueues combined -`replicas.commandqueue.maxoverlaps` | Largest number of overlapping commands seen when adding to any CommandQueue -`replicas.commandqueue.maxreadcount` | Largest number of read-only commands in any CommandQueue -`replicas.commandqueue.maxsize` | Largest number of commands in any CommandQueue -`replicas.commandqueue.maxtreesize` | Largest number of intervals in any CommandQueue's interval tree -`replicas.commandqueue.maxwritecount` | Largest number of read-write commands in any CommandQueue -`replicas.leaders_invalid_lease` | Number of replicas that are Raft leaders whose lease is invalid -`replicas.leaders_not_leaseholders` | Number of replicas that are Raft leaders whose range lease is held by another store -`replicas.leaders` | Number of Raft leaders -`replicas.leaseholders` | Number of lease holders -`replicas.quiescent` | Number of quiesced replicas -`replicas.reserved` | Number of replicas reserved for snapshots -`replicas` | Number of replicas -`requests.backpressure.split` | Number of backpressured writes waiting on a Range split -`requests.slow.commandqueue` | Number of requests that have been stuck for a long time in the command queue -`requests.slow.distsender` | Number of requests that have been stuck for a long time in the dist sender -`requests.slow.lease` | Number of requests that have been stuck for a long time acquiring a lease -`requests.slow.raft` | Number of requests that have been stuck for a long time in Raft -`rocksdb.block.cache.hits` | Count of block cache hits -`rocksdb.block.cache.misses` | Count of block cache misses -`rocksdb.block.cache.pinned-usage` | Bytes pinned by the block cache -`rocksdb.block.cache.usage` | Bytes used by the block cache -`rocksdb.bloom.filter.prefix.checked` | Number of times the bloom filter was checked -`rocksdb.bloom.filter.prefix.useful` | Number of times the bloom filter helped avoid iterator creation -`rocksdb.compactions` | Number of table compactions -`rocksdb.flushes` | Number of table flushes -`rocksdb.memtable.total-size` | Current size of memtable in bytes -`rocksdb.num-sstables` | Number of storage engine SSTables -`rocksdb.read-amplification` | Number of disk reads per query -`rocksdb.table-readers-mem-estimate` | Memory used by index and filter blocks -`round-trip-latency` | Distribution of round-trip latencies with other nodes in nanoseconds -`security.certificate.expiration.ca` | Expiration timestamp in seconds since Unix epoch for the CA certificate. 0 means no certificate or error. -`security.certificate.expiration.node` | Expiration timestamp in seconds since Unix epoch for the node certificate. 0 means no certificate or error. -`schedules.BACKUP.protected_age_sec` | The age of the oldest [protected timestamp]({% link {{ page.version.version }}/architecture/storage-layer.md %}#protected-timestamps) record protected by `BACKUP` schedules. -`schedules.BACKUP.protected_record_count` | Number of [protected timestamp]({% link {{ page.version.version }}/architecture/storage-layer.md %}#protected-timestamps) records held by `BACKUP` schedules. -`sql.bytesin` | Number of SQL bytes received -`sql.bytesout` | Number of SQL bytes sent -`sql.conns` | Number of active SQL connections. For new recent connections, refer to `sql.new_conns`. -`sql.ddl.count` | Number of SQL DDL statements -`sql.delete.count` | Number of SQL DELETE statements -`sql.distsql.exec.latency` | Latency in nanoseconds of SQL statement executions running on the distributed execution engine. This metric does not include the time to parse and plan the statement. -`sql.distsql.flows.active` | Number of distributed SQL flows currently active -`sql.distsql.flows.total` | Number of distributed SQL flows executed -`sql.distsql.queries.active` | Number of distributed SQL queries currently active -`sql.distsql.queries.total` | Number of distributed SQL queries executed -`sql.distsql.select.count` | Number of DistSQL SELECT statements -`sql.distsql.service.latency` | Latency in nanoseconds of SQL statement executions running on the distributed execution engine, including the time to parse and plan the statement. -`sql.exec.latency` | Latency in nanoseconds of all SQL statement executions. This metric does not include the time to parse and plan the statement. -`sql.guardrails.max_row_size_err.count` | Number of times a large row violates the corresponding `sql.guardrails.max_row_size_err` limit. -`sql.guardrails.max_row_size_log.count` | Number of times a large row violates the corresponding `sql.guardrails.max_row_size_log` limit. -`sql.insert.count` | Number of SQL INSERT statements -`sql.mem.current` | Current sql statement memory usage -`sql.mem.distsql.current` | Current sql statement memory usage for distsql -`sql.mem.distsql.max` | Memory usage per sql statement for distsql -`sql.mem.max` | Memory usage per sql statement -`sql.mem.root.current` | Current sql statement memory usage for root -`sql.mem.root.max` | Memory usage per sql statement for root -`sql.mem.session.current` | Current sql session memory usage -`sql.mem.session.max` | Memory usage per sql session -`sql.mem.txn.current` | Current sql transaction memory usage -`sql.mem.txn.max` | Memory usage per sql transaction -`sql.misc.count` | Number of other SQL statements -`sql.new_conns` | Number of new SQL connections in the previous second. For all connections, refer to `sql.conns`. -`sql.pgwire_cancel.total` | Counter of the number of pgwire query cancel requests -`sql.pgwire_cancel.ignored` | Counter of the number of pgwire query cancel requests that were ignored due to rate limiting -`sql.pgwire_cancel.successful` | Counter of the number of pgwire query cancel requests that were successful -`sql.query.count` | Number of SQL queries -`sql.select.count` | Number of SQL SELECT statements -`sql.service.latency` | Latency in nanoseconds of SQL request execution, including the time to parse and plan the statement. -`sql.txn.abort.count` | Number of SQL transaction ABORT statements -`sql.txn.begin.count` | Number of SQL transaction BEGIN statements -`sql.txn.commit.count` | Number of SQL transaction COMMIT statements -`sql.txn.contended.count` | Number of SQL transactions that experienced contention -`sql.txn.isolation.executed_at.read_committed` | Number of times a [`READ COMMITTED`]({% link {{ page.version.version }}/read-committed.md %}) transaction was executed. -`sql.txn.isolation.upgraded_from.read_committed` | Number of times a [`READ COMMITTED`]({% link {{ page.version.version }}/read-committed.md %}) transaction was automatically upgraded to a stronger isolation level. -`sql.txn.rollback.count` | Number of SQL transaction ROLLBACK statements -`sql.update.count` | Number of SQL UPDATE statements -`storage.l0-level-score` | Compaction score of level 0 -`storage.l1-level-score` | Compaction score of level 1 -`storage.l2-level-score` | Compaction score of level 2 -`storage.l3-level-score` | Compaction score of level 3 -`storage.l4-level-score` | Compaction score of level 4 -`storage.l5-level-score` | Compaction score of level 5 -`storage.l6-level-score` | Compaction score of level 6 -`storage.l0-level-size` | Size of the SSTables in level 0 -`storage.l1-level-size` | Size of the SSTables in level 1 -`storage.l2-level-size` | Size of the SSTables in level 2 -`storage.l3-level-size` | Size of the SSTables in level 3 -`storage.l4-level-size` | Size of the SSTables in level 4 -`storage.l5-level-size` | Size of the SSTables in level 5 -`storage.l6-level-size` | Size of the SSTables in level 6 -`storage.keys.range-key-set.count` | Approximate count of RangeKeySet internal keys across the storage engine. -`storage.marked-for-compaction-files` | Count of SSTables marked for compaction -`sys.cgo.allocbytes` | Current bytes of memory allocated by cgo -`sys.cgo.totalbytes` | Total bytes of memory allocated by cgo, but not released -`sys.cgocalls` | Total number of cgo call -`sys.cpu.sys.ns` | Total system cpu time in nanoseconds -`sys.cpu.sys.percent` | Current system cpu percentage -`sys.cpu.user.ns` | Total user cpu time in nanoseconds -`sys.cpu.user.percent` | Current user cpu percentage -`sys.fd.open` | Process open file descriptors -`sys.fd.softlimit` | Process open FD soft limit -`sys.gc.count` | Total number of GC runs -`sys.gc.pause.ns` | Total GC pause in nanoseconds -`sys.gc.pause.percent` | Current GC pause percentage -`sys.go.allocbytes` | Current bytes of memory allocated by go -`sys.go.totalbytes` | Total bytes of memory allocated by go, but not released -`sys.goroutines` | Current number of goroutines -`sys.rss` | Current process RSS -`sys.uptime` | Process uptime in seconds -`sysbytes` | Number of bytes in system KV pairs -`syscount` | Count of system KV pairs -`timeseries.write.bytes` | Total size in bytes of metric samples written to disk -`timeseries.write.errors` | Total errors encountered while attempting to write metrics to disk -`timeseries.write.samples` | Total number of metric samples written to disk -`totalbytes` | Total number of bytes taken up by keys and values including non-live data -`tscache.skl.read.pages` | Number of pages in the read timestamp cache -`tscache.skl.read.rotations` | Number of page rotations in the read timestamp cache -`tscache.skl.write.pages` | Number of pages in the write timestamp cache -`tscache.skl.write.rotations` | Number of page rotations in the write timestamp cache -`txn.abandons` | Number of abandoned KV transactions -`txn.aborts` | Number of aborted KV transactions -`txn.autoretries` | Number of automatic retries to avoid serializable restarts -`txn.commits1PC` | Number of committed one-phase KV transactions -`txn.commits` | Number of committed KV transactions (including 1PC) -`txn.durations` | KV transaction durations in nanoseconds -`txn.restarts.deleterange` | Number of restarts due to a forwarded commit timestamp and a DeleteRange command -`txn.restarts.possiblereplay` | Number of restarts due to possible replays of command batches at the storage layer -`txn.restarts.serializable` | Number of restarts due to a forwarded commit timestamp and isolation=SERIALIZABLE -`txn.restarts.writetooold` | Number of restarts due to a concurrent writer committing first -`txn.restarts` | Number of restarted KV transactions -`valbytes` | Number of bytes taken up by values -`valcount` | Count of all values +{% assign list1 = site.data.metrics.available-metrics-in-metrics-list %} +{% assign list2 = site.data.metrics.available-metrics-not-in-metrics-list %} + +{% assign available_metrics_combined = list1 | concat: list2 %} +{% assign available_metrics_sorted = available_metrics_combined | sort: "metric_id" %} + + + + + + + + + + + + {% for m in available_metrics_sorted %} {% comment %} Iterate through the available_metrics. {% endcomment %} + {% assign metrics-list = site.data.metrics.metrics-list | where: "metric", m.metric_id %} + {% comment %} Get the row from the metrics-list with the given metric_id. {% endcomment %} + + + {% comment %} Use the value from the metrics-list, if any, followed by the value in the available-metrics-not-in-metrics-list, if any. {% endcomment %} + + + + + {% endfor %} {% comment %} metrics {% endcomment %} + +
CockroachDB Metric NameDescriptionTypeUnit
{{ m.metric_id }}
{{ metrics-list[0].description }}{{ m.description }}{{ metrics-list[0].type }}{{ m.type }}{{ metrics-list[0].unit }}{{ m.unit }}
From fbf1c768449d51a1db550c760768e3a93b25f179 Mon Sep 17 00:00:00 2001 From: Kathryn May <44557882+kathancox@users.noreply.github.com> Date: Thu, 12 Dec 2024 13:04:49 -0500 Subject: [PATCH 07/36] v24.2.6 release notes (Cloud-first) (#19204) --- src/current/_data/releases.yml | 36 ++++++++- src/current/_data/versions.csv | 36 ++++----- .../_includes/releases/v24.2/v24.2.6.md | 78 +++++++++++++++++++ 3 files changed, 131 insertions(+), 19 deletions(-) create mode 100644 src/current/_includes/releases/v24.2/v24.2.6.md diff --git a/src/current/_data/releases.yml b/src/current/_data/releases.yml index 4d4b4a42498..62ace1147d0 100644 --- a/src/current/_data/releases.yml +++ b/src/current/_data/releases.yml @@ -7460,4 +7460,38 @@ docker_arm_experimental: false docker_arm_limited_access: false source: true - previous_release: v24.3.0-beta.3 \ No newline at end of file + previous_release: v24.3.0-beta.3 + +- release_name: v24.2.6 + major_version: v24.2 + release_date: '2024-12-12' + release_type: Production + go_version: go1.22.8 + sha: 74f60821a82c570ab58b8d7716c1d74b85e87df0 + has_sql_only: true + has_sha256sum: true + mac: + mac_arm: true + mac_arm_experimental: true + mac_arm_limited_access: false + windows: true + linux: + linux_arm: true + linux_arm_experimental: false + linux_arm_limited_access: false + linux_intel_fips: true + linux_arm_fips: false + docker: + docker_image: cockroachdb/cockroach + docker_arm: true + docker_arm_experimental: false + docker_arm_limited_access: false + source: true + previous_release: v24.2.5 + cloud_only: true + cloud_only_message_short: 'Available only for select CockroachDB Cloud clusters' + cloud_only_message: > + This version is currently available only for select + CockroachDB Cloud clusters. To request to upgrade + a CockroachDB self-hosted cluster to this version, + [contact support](https://support.cockroachlabs.com/hc/requests/new). \ No newline at end of file diff --git a/src/current/_data/versions.csv b/src/current/_data/versions.csv index e0330551c68..ae829fd5ef4 100644 --- a/src/current/_data/versions.csv +++ b/src/current/_data/versions.csv @@ -1,18 +1,18 @@ -major_version,release_date,maint_supp_exp_date,asst_supp_exp_date,last_ga_patch,initial_lts_patch,initial_lts_release_date,lts_maint_supp_exp_date,lts_asst_supp_exp_date,previous_version,crdb_branch_name -v1.0,2017-05-10,2018-05-10,2018-11-10,N/A,N/A,N/A,N/A,N/A,N/A,release-1.0 -v1.1,2017-10-12,2018-10-12,2019-04-12,N/A,N/A,N/A,N/A,N/A,v1.0,release-1.1 -v2.0,2018-04-04,2019-04-04,2019-10-04,N/A,N/A,N/A,N/A,N/A,v1.1,release-2.0 -v2.1,2018-10-30,2019-10-30,2020-04-30,N/A,N/A,N/A,N/A,N/A,v2.0,release-2.1 -v19.1,2019-04-30,2020-04-30,2020-10-30,N/A,N/A,N/A,N/A,N/A,v2.1,release-19.1 -v19.2,2019-11-12,2020-11-12,2021-05-12,N/A,N/A,N/A,N/A,N/A,v19.1,release-19.2 -v20.1,2020-05-12,2021-05-12,2021-11-12,N/A,N/A,N/A,N/A,N/A,v19.2,release-20.1 -v20.2,2020-11-10,2021-11-10,2022-05-10,N/A,N/A,N/A,N/A,N/A,v20.1,release-20.2 -v21.1,2021-05-18,2022-05-18,2022-11-18,N/A,N/A,N/A,N/A,N/A,v20.2,release-21.1 -v21.2,2021-11-16,2022-11-16,2023-05-16,N/A,N/A,N/A,N/A,N/A,v21.1,release-21.2 -v22.1,2022-05-24,2023-05-24,2023-11-24,N/A,N/A,N/A,N/A,N/A,v21.2,release-22.1 -v22.2,2022-12-05,2023-12-05,2024-06-05,N/A,N/A,N/A,N/A,N/A,v22.1,release-22.2 -v23.1,2023-05-15,2024-05-15,2024-11-15,23.1.11,23.1.12,2023-11-13,2024-11-13,2025-11-13,v22.2,release-23.1 -v23.2,2024-02-05,2025-02-05,2025-08-05,23.2.6,23.2.7,2024-07-08,2025-07-08,2026-07-08,v23.1,release-23.2 -v24.1,2024-05-20,2025-05-20,2025-11-20,24.1.5,24.1.6,2024-10-21,2025-10-21,2026-10-21,v23.2,release-24.1 -v24.2,2024-08-12,2025-02-12,N/A,N/A,N/A,N/A,N/A,N/A,v24.1,release-24.2 -v24.3,2024-11-18,2025-11-18,2026-05-18,N/A,N/A,N/A,N/A,N/A,v24.2,release-24.3 +major_version,release_date,maint_supp_exp_date,asst_supp_exp_date,last_ga_patch,initial_lts_patch,initial_lts_release_date,lts_maint_supp_exp_date,lts_asst_supp_exp_date,previous_version,crdb_branch_name +v1.0,2017-05-10,2018-05-10,2018-11-10,N/A,N/A,N/A,N/A,N/A,N/A,release-1.0 +v1.1,2017-10-12,2018-10-12,2019-04-12,N/A,N/A,N/A,N/A,N/A,v1.0,release-1.1 +v2.0,2018-04-04,2019-04-04,2019-10-04,N/A,N/A,N/A,N/A,N/A,v1.1,release-2.0 +v2.1,2018-10-30,2019-10-30,2020-04-30,N/A,N/A,N/A,N/A,N/A,v2.0,release-2.1 +v19.1,2019-04-30,2020-04-30,2020-10-30,N/A,N/A,N/A,N/A,N/A,v2.1,release-19.1 +v19.2,2019-11-12,2020-11-12,2021-05-12,N/A,N/A,N/A,N/A,N/A,v19.1,release-19.2 +v20.1,2020-05-12,2021-05-12,2021-11-12,N/A,N/A,N/A,N/A,N/A,v19.2,release-20.1 +v20.2,2020-11-10,2021-11-10,2022-05-10,N/A,N/A,N/A,N/A,N/A,v20.1,release-20.2 +v21.1,2021-05-18,2022-05-18,2022-11-18,N/A,N/A,N/A,N/A,N/A,v20.2,release-21.1 +v21.2,2021-11-16,2022-11-16,2023-05-16,N/A,N/A,N/A,N/A,N/A,v21.1,release-21.2 +v22.1,2022-05-24,2023-05-24,2023-11-24,N/A,N/A,N/A,N/A,N/A,v21.2,release-22.1 +v22.2,2022-12-05,2023-12-05,2024-06-05,N/A,N/A,N/A,N/A,N/A,v22.1,release-22.2 +v23.1,2023-05-15,2024-05-15,2024-11-15,23.1.11,23.1.12,2023-11-13,2024-11-13,2025-11-13,v22.2,release-23.1 +v23.2,2024-02-05,2025-02-05,2025-08-05,23.2.6,23.2.7,2024-07-08,2025-07-08,2026-07-08,v23.1,release-23.2 +v24.1,2024-05-20,2025-05-20,2025-11-20,24.1.5,24.1.6,2024-10-21,2025-10-21,2026-10-21,v23.2,release-24.1 +v24.2,2024-08-12,2025-02-12,N/A,N/A,N/A,N/A,N/A,N/A,v24.1,release-24.2 +v24.3,2024-11-18,2025-11-18,2026-05-18,N/A,N/A,N/A,N/A,N/A,v24.2,release-24.3 diff --git a/src/current/_includes/releases/v24.2/v24.2.6.md b/src/current/_includes/releases/v24.2/v24.2.6.md new file mode 100644 index 00000000000..3cedad5d3a0 --- /dev/null +++ b/src/current/_includes/releases/v24.2/v24.2.6.md @@ -0,0 +1,78 @@ +## v24.2.6 + +Release Date: December 12, 2024 + +{% include releases/new-release-downloads-docker-image.md release=include.release %} + +

Security updates

+ +- All cluster settings that accept strings are now fully redacted when transmitted as part of diagnostics telemetry. This payload includes a record of modified cluster settings and their values when they are not strings. Customers who previously applied the mitigations in [Technical Advisory 133479]({% link advisories/a133479.md %}) can safely set the value of cluster setting `server.redact_sensitive_settings.enabled` to false and turn on diagnostic reporting via the `diagnostics.reporting.enabled` cluster setting without leaking sensitive cluster settings values. [#134017][#134017] + +

General changes

+ +- `COCKROACH_SKIP_ENABLING_DIAGNOSTIC_REPORTING` is no longer mentioned in the `cockroach demo` command. [#134088][#134088] +- Added `system.users` to the list of system tables that changefeeds protect with protected timestamps. This table is required for change data capture queries. [#134837][#134837] + +

Operational changes

+ +- The `goschedstats.always_use_short_sample_period.enabled` setting should be set to true for any production cluster, to prevent unnecessary queuing in admission control CPU queues. [#133584][#133584] +- Added a new cluster setting `ui.database_locality_metadata.enabled` that allows operators to disable loading extended database and table region information in the DB Console Database and Table pages. This information can cause significant CPU load on large clusters with many ranges. Versions of this page from v24.3 and later do not have this problem. If customers require this data, they can use the `SHOW RANGES FROM {DATABASE| TABLE}` query via SQL to compute on-demand. [#134095][#134095] +- Row-level TTL jobs now periodically log progress by showing the number of table spans that have been processed so far. [#135170][#135170] + +

Bug fixes

+ +- Fixed a bug that caused non-reusable query plans, e.g., plans for DDL and `SHOW ...` statements, to be cached and reused in future executions, possibly causing stale results to be returned. This bug only occurred when `plan_cache_mode` was set to `auto` or `force_generic_plan`, both of which are not currently the default settings. [#133074][#133074] +- Previously, CockroachDB could encounter an internal error of the form `interface conversion: coldata.Column is` in an edge case and this is now fixed. The bug is present in v22.2.13+, v23.1.9+, v23.2+. [#133761][#133761] +- Fixed a bug that caused incorrect `NOT NULL` constraint violation errors on `UPSERT` and `INSERT .. ON CONFLICT .. DO UPDATE` statements when those statements updated an existing row and a subset of columns that did not include a `NOT NULL` column of the table. This bug has been present since at least v20.1.0. [#133821][#133821] +- Fixed an unhandled error that could occur when using `REVOKE ... ON SEQUENCE FROM ... user` on an object that is not a sequence. [#133709][#133709] +- Addressed a panic inside `CREATE TABLE AS` if sequence builtin expressions had invalid function overloads. [#133869][#133869] +- `STRING`constants can now be compared against collated strings. [#134084][#134084] +- When executing queries with index / lookup joins when the ordering needs to be maintained, previously CockroachDB could experience increased query latency, possibly by several orders of magnitude. This bug was introduced in v22.2 and is now fixed. [#134366][#134366] +- Fixed a minor bug where `DISCARD ALL` statements were counted under the `sql.ddl.count` metric. Now these will be counted under the `sql.misc.count` metric. [#134509][#134509] +- Addressed a bug with `DROP CASCADE` that would occasionally panic with an undropped `backref` message on partitioned tables. [#134472][#134472] +- Reduced the duration of partitions in the gossip network when a node crashes in order to eliminate false positives in the `ranges.unavailable` metric. [#134600][#134600] +- Non-`admin` users that run `DROP ROLE IF EXISTS` on a user that does not exist will no longer receive an error message. [#134969][#134969] +- Fixed a bug that caused quotes around the name of a routine to be dropped when it was called within another routine. This could prevent the correct routine from being resolved if the nested routine name was case sensitive. The bug has existed since v24.1, when nested routines were introduced. [#134000][#134000] +- Fixed a bug that could cause incorrect query results when the optimizer planned a lookup join on an index containing a column of type `CHAR(N)`, `VARCHAR(N)`, `BIT(N)`, `VARBIT(N)`, or `DECIMAL(M, N)`, and the query held that column constant to a single value (e.g., with an equality filter). [#135076][#135076] +- Fixed an unhandled error that would occur if `DROP SCHEMA` was executed on the `public` schema of the `system` database, or on an internal schema like `pg_catalog` or `information_schema`. [#135180][#135180] +- Fixed a bug that caused incorrect evaluation of some binary expressions involving `CHAR(N)` values and untyped string literals with trailing whitespace characters. For example, the expression `'f'::CHAR = 'f '` now correctly evaluates to `true`. [#135689][#135689] +- Fixed a bug where `ALTER DATABASE` operations that modify the zone config would hang if an invalid zone config already exists. [#135215][#135215] +- `CREATE SCHEMA` now returns the correct error if a the schema name is missing. [#135927][#135927] +- Using more than one `DECLARE` statment in the definition of a user-defined function now correctly declares additional variables. [#135738][#135738] +- Fixed a bug where CockroachDB would encounter an internal error when evaluating `FETCH ABSOLUTE 0` statements. The bug has been present since v22.1. [#134992][#134992] +- Fixed an issue where corrupted table statistics could cause the `cockroach` process to crash. [#136041][#136041] +- Fixed a bug that causes the optimizer to use stale table statistics after altering an `ENUM` type used in the table. [#136812][#136812] + +[#133074]: https://github.com/cockroachdb/cockroach/pull/133074 +[#133584]: https://github.com/cockroachdb/cockroach/pull/133584 +[#133709]: https://github.com/cockroachdb/cockroach/pull/133709 +[#133761]: https://github.com/cockroachdb/cockroach/pull/133761 +[#133821]: https://github.com/cockroachdb/cockroach/pull/133821 +[#133869]: https://github.com/cockroachdb/cockroach/pull/133869 +[#134000]: https://github.com/cockroachdb/cockroach/pull/134000 +[#134017]: https://github.com/cockroachdb/cockroach/pull/134017 +[#134084]: https://github.com/cockroachdb/cockroach/pull/134084 +[#134088]: https://github.com/cockroachdb/cockroach/pull/134088 +[#134095]: https://github.com/cockroachdb/cockroach/pull/134095 +[#134099]: https://github.com/cockroachdb/cockroach/pull/134099 +[#134366]: https://github.com/cockroachdb/cockroach/pull/134366 +[#134447]: https://github.com/cockroachdb/cockroach/pull/134447 +[#134472]: https://github.com/cockroachdb/cockroach/pull/134472 +[#134509]: https://github.com/cockroachdb/cockroach/pull/134509 +[#134600]: https://github.com/cockroachdb/cockroach/pull/134600 +[#134646]: https://github.com/cockroachdb/cockroach/pull/134646 +[#134730]: https://github.com/cockroachdb/cockroach/pull/134730 +[#134837]: https://github.com/cockroachdb/cockroach/pull/134837 +[#134969]: https://github.com/cockroachdb/cockroach/pull/134969 +[#134992]: https://github.com/cockroachdb/cockroach/pull/134992 +[#135076]: https://github.com/cockroachdb/cockroach/pull/135076 +[#135170]: https://github.com/cockroachdb/cockroach/pull/135170 +[#135180]: https://github.com/cockroachdb/cockroach/pull/135180 +[#135215]: https://github.com/cockroachdb/cockroach/pull/135215 +[#135611]: https://github.com/cockroachdb/cockroach/pull/135611 +[#135689]: https://github.com/cockroachdb/cockroach/pull/135689 +[#135738]: https://github.com/cockroachdb/cockroach/pull/135738 +[#135927]: https://github.com/cockroachdb/cockroach/pull/135927 +[#136010]: https://github.com/cockroachdb/cockroach/pull/136010 +[#136041]: https://github.com/cockroachdb/cockroach/pull/136041 +[#136812]: https://github.com/cockroachdb/cockroach/pull/136812 From fb9e5b10cbe18516e4887e5397014e9386698cee Mon Sep 17 00:00:00 2001 From: Ryan Kuo <8740013+taroface@users.noreply.github.com> Date: Thu, 12 Dec 2024 13:23:28 -0500 Subject: [PATCH 08/36] update path to BEGIN diagram (#19201) * update path to BEGIN diagram * amend retry loops callout --- src/current/v23.2/begin-transaction.md | 5 ++--- src/current/v24.1/begin-transaction.md | 7 +++---- src/current/v24.2/begin-transaction.md | 7 +++---- src/current/v24.3/begin-transaction.md | 7 +++---- 4 files changed, 11 insertions(+), 15 deletions(-) diff --git a/src/current/v23.2/begin-transaction.md b/src/current/v23.2/begin-transaction.md index 8c159dac361..abee0930220 100644 --- a/src/current/v23.2/begin-transaction.md +++ b/src/current/v23.2/begin-transaction.md @@ -7,11 +7,10 @@ docs_area: reference.sql The `BEGIN` [statement]({% link {{ page.version.version }}/sql-statements.md %}) initiates a [transaction]({% link {{ page.version.version }}/transactions.md %}), which either successfully executes all of the statements it contains or none at all. -{{site.data.alerts.callout_danger}} -When using transactions, your application should include logic to [retry transactions]({% link {{ page.version.version }}/transactions.md %}#transaction-retries) that are aborted to break a dependency cycle between concurrent transactions. +{{site.data.alerts.callout_info}} +When running under the default [`SERIALIZABLE`]({% link {{ page.version.version }}/demo-serializable.md %}) isolation level, your application should [use a retry loop to handle transaction retry errors]({% link {{ page.version.version }}/query-behavior-troubleshooting.md %}#transaction-retry-errors) that can occur under [contention]({{ link_prefix }}performance-best-practices-overview.html#transaction-contention). {{site.data.alerts.end}} - ## Synopsis
diff --git a/src/current/v24.1/begin-transaction.md b/src/current/v24.1/begin-transaction.md index 8c159dac361..6c50b135e95 100644 --- a/src/current/v24.1/begin-transaction.md +++ b/src/current/v24.1/begin-transaction.md @@ -7,15 +7,14 @@ docs_area: reference.sql The `BEGIN` [statement]({% link {{ page.version.version }}/sql-statements.md %}) initiates a [transaction]({% link {{ page.version.version }}/transactions.md %}), which either successfully executes all of the statements it contains or none at all. -{{site.data.alerts.callout_danger}} -When using transactions, your application should include logic to [retry transactions]({% link {{ page.version.version }}/transactions.md %}#transaction-retries) that are aborted to break a dependency cycle between concurrent transactions. +{{site.data.alerts.callout_info}} +When running under the default [`SERIALIZABLE`]({% link {{ page.version.version }}/demo-serializable.md %}) isolation level, your application should [use a retry loop to handle transaction retry errors]({% link {{ page.version.version }}/query-behavior-troubleshooting.md %}#transaction-retry-errors) that can occur under [contention]({{ link_prefix }}performance-best-practices-overview.html#transaction-contention). {{site.data.alerts.end}} - ## Synopsis
-{% remote_include https://raw.githubusercontent.com/cockroachdb/generated-diagrams/{{ page.release_info.crdb_branch_name }}/grammar_svg/begin.html %} +{% remote_include https://raw.githubusercontent.com/cockroachdb/generated-diagrams/{{ page.release_info.crdb_branch_name }}/grammar_svg/legacy_begin.html %}
## Required privileges diff --git a/src/current/v24.2/begin-transaction.md b/src/current/v24.2/begin-transaction.md index 8c159dac361..6c50b135e95 100644 --- a/src/current/v24.2/begin-transaction.md +++ b/src/current/v24.2/begin-transaction.md @@ -7,15 +7,14 @@ docs_area: reference.sql The `BEGIN` [statement]({% link {{ page.version.version }}/sql-statements.md %}) initiates a [transaction]({% link {{ page.version.version }}/transactions.md %}), which either successfully executes all of the statements it contains or none at all. -{{site.data.alerts.callout_danger}} -When using transactions, your application should include logic to [retry transactions]({% link {{ page.version.version }}/transactions.md %}#transaction-retries) that are aborted to break a dependency cycle between concurrent transactions. +{{site.data.alerts.callout_info}} +When running under the default [`SERIALIZABLE`]({% link {{ page.version.version }}/demo-serializable.md %}) isolation level, your application should [use a retry loop to handle transaction retry errors]({% link {{ page.version.version }}/query-behavior-troubleshooting.md %}#transaction-retry-errors) that can occur under [contention]({{ link_prefix }}performance-best-practices-overview.html#transaction-contention). {{site.data.alerts.end}} - ## Synopsis
-{% remote_include https://raw.githubusercontent.com/cockroachdb/generated-diagrams/{{ page.release_info.crdb_branch_name }}/grammar_svg/begin.html %} +{% remote_include https://raw.githubusercontent.com/cockroachdb/generated-diagrams/{{ page.release_info.crdb_branch_name }}/grammar_svg/legacy_begin.html %}
## Required privileges diff --git a/src/current/v24.3/begin-transaction.md b/src/current/v24.3/begin-transaction.md index 8c159dac361..6c50b135e95 100644 --- a/src/current/v24.3/begin-transaction.md +++ b/src/current/v24.3/begin-transaction.md @@ -7,15 +7,14 @@ docs_area: reference.sql The `BEGIN` [statement]({% link {{ page.version.version }}/sql-statements.md %}) initiates a [transaction]({% link {{ page.version.version }}/transactions.md %}), which either successfully executes all of the statements it contains or none at all. -{{site.data.alerts.callout_danger}} -When using transactions, your application should include logic to [retry transactions]({% link {{ page.version.version }}/transactions.md %}#transaction-retries) that are aborted to break a dependency cycle between concurrent transactions. +{{site.data.alerts.callout_info}} +When running under the default [`SERIALIZABLE`]({% link {{ page.version.version }}/demo-serializable.md %}) isolation level, your application should [use a retry loop to handle transaction retry errors]({% link {{ page.version.version }}/query-behavior-troubleshooting.md %}#transaction-retry-errors) that can occur under [contention]({{ link_prefix }}performance-best-practices-overview.html#transaction-contention). {{site.data.alerts.end}} - ## Synopsis
-{% remote_include https://raw.githubusercontent.com/cockroachdb/generated-diagrams/{{ page.release_info.crdb_branch_name }}/grammar_svg/begin.html %} +{% remote_include https://raw.githubusercontent.com/cockroachdb/generated-diagrams/{{ page.release_info.crdb_branch_name }}/grammar_svg/legacy_begin.html %}
## Required privileges From c45495d49270f2afd1c45e6030a7b7025f83470a Mon Sep 17 00:00:00 2001 From: Florence Morris Date: Thu, 12 Dec 2024 13:39:27 -0500 Subject: [PATCH 09/36] v24.1.8: [Docs] Generate release notes for Cloud (#19205) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * edited notes * incorporated Kathryn’s feedback --- src/current/_data/releases.yml | 34 +++++++++ .../_includes/releases/v24.1/v24.1.8.md | 73 +++++++++++++++++++ 2 files changed, 107 insertions(+) create mode 100644 src/current/_includes/releases/v24.1/v24.1.8.md diff --git a/src/current/_data/releases.yml b/src/current/_data/releases.yml index 62ace1147d0..8c4814e5f17 100644 --- a/src/current/_data/releases.yml +++ b/src/current/_data/releases.yml @@ -7490,6 +7490,40 @@ previous_release: v24.2.5 cloud_only: true cloud_only_message_short: 'Available only for select CockroachDB Cloud clusters' + cloud_only_message: > + This version is currently available only for select + CockroachDB Cloud clusters. To request to upgrade + a CockroachDB self-hosted cluster to this version, + [contact support](https://support.cockroachlabs.com/hc/requests/new). + +- release_name: v24.1.8 + major_version: v24.1 + release_date: '2024-12-12' + release_type: Production + go_version: go1.22.8 + sha: 05e5df3c04a577643a59b78f574b0084832e67e6 + has_sql_only: true + has_sha256sum: true + mac: + mac_arm: true + mac_arm_experimental: true + mac_arm_limited_access: false + windows: true + linux: + linux_arm: true + linux_arm_experimental: false + linux_arm_limited_access: false + linux_intel_fips: true + linux_arm_fips: false + docker: + docker_image: cockroachdb/cockroach + docker_arm: true + docker_arm_experimental: false + docker_arm_limited_access: false + source: true + previous_release: v24.1.7 + cloud_only: true + cloud_only_message_short: 'Available only for select CockroachDB Cloud clusters' cloud_only_message: > This version is currently available only for select CockroachDB Cloud clusters. To request to upgrade diff --git a/src/current/_includes/releases/v24.1/v24.1.8.md b/src/current/_includes/releases/v24.1/v24.1.8.md new file mode 100644 index 00000000000..a6b427cf04b --- /dev/null +++ b/src/current/_includes/releases/v24.1/v24.1.8.md @@ -0,0 +1,73 @@ +## v24.1.8 + +Release Date: December 12, 2024 + +{% include releases/new-release-downloads-docker-image.md release=include.release %} + +

Security updates

+ +- All cluster settings that accept strings are now fully redacted when transmitted as part of our diagnostics telemetry. The transmitted payload includes a record of modified cluster settings and their values when they are not strings. If you previously applied the mitigations in [Technical Advisory 133479]({% link advisories/a133479.md %}), you can safely set the value of cluster setting `server.redact_sensitive_settings.enabled` to false and turn on diagnostic reporting via the `diagnostics.reporting.enabled` cluster setting without leaking sensitive cluster settings values. [#134016][#134016] + +

General changes

+ +- `COCKROACH_SKIP_ENABLING_DIAGNOSTIC_REPORTING` is no longer mentioned in the `cockroach demo` command. [#134087][#134087] +- Added `system.users` to the list of system tables that changefeeds protect with protected timestamps (PTS). This table is required for change data capture queries. [#134836][#134836] +- Added changefeed support for the `mvcc_timestamp` option with the `avro` format. If both options are specified, the `avro` schema includes an `mvcc_timestamp` metadata field and emits the row's MVCC timestamp with the row data. [#136482][#136482] + +

Operational changes

+ +- To prevent unnecessary queuing in admission control CPU queues, the `goschedstats.always_use_short_sample_period.enabled` setting should be set to `true` for any production cluster. [#133583][#133583] +- A new cluster setting `ui.database_locality_metadata.enabled`, when set to `true`, disables loading database and table region information in the DB Console Database and Table pages. This information can cause significant CPU load on large clusters with many ranges. The Database and Table pages in v24.3 onwards do not have this problem. If you require this data, use the `SHOW RANGES FROM {DATABASE|TABLE}` SQL statement to compute this information on-demand. [#134094][#134094] +- The row-level TTL job now will periodically log progress by showing the number of table spans that have been processed so far. [#135179][#135179] + +

Bug fixes

+ +- Fixed a bug where CockroachDB could encounter an internal error `interface conversion: coldata.Column is` in an edge case. The bug was present in v22.2.13+, v23.1.9+, and v23.2+. [#133760][#133760] +- Fixed an unhandled error that could occur when using `REVOKE ... ON SEQUENCE ... FROM user` on an object that is not a sequence. [#133708][#133708] +- Fixed a bug that caused incorrect `NOT NULL` constraint violation errors on `UPSERT` and `INSERT ... ON CONFLICT ... DO UPDATE` statements when those statements updated an existing row and a subset of columns that did not include a `NOT NULL` column of the table. This bug had been present since at least v20.1.0. [#133822][#133822] +- Addressed a panic that could occur inside `CREATE TABLE AS` if sequence builtin expressions had invalid function overloads. [#133868][#133868] +- String constants can now be compared against collated strings. [#134105][#134105] +- Previously, when executing queries with index / lookup joins where ordering needed to be maintained, CockroachDB's behavior could lead to increased query latency, potentially by several orders of magnitude. This bug was introduced in v22.2, and is now fixed. [#134365][#134365] +- Fixed a bug where `DISCARD ALL` statements were counted under the `sql.ddl.count` metric. Now these statements will be counted under the `sql.misc.count` metric. [#134508][#134508] +- Fixed a bug where `DROP CASCADE` would occasionally panic with an `un-dropped backref` message on partitioned tables. [#134522][#134522] +- Reduced the duration of partitions in the gossip network when a node crashes. This eliminates false positives in the `ranges.unavailable` metric. [#134601][#134601] +- As a non-admin user who runs `DROP ROLE IF EXISTS` on a user that does not exist, you no longer get an error message. [#134968][#134968] +- Fixed a bug that caused quotes around the name of a routine to be dropped when the routine was called within another routine. This could prevent the correct routine name from being resolved if the nested routine name was case-sensitive. The bug has existed since v24.1, when nested routines were introduced. [#134001][#134001] +- Fixed a bug that could cause incorrect query results when the optimizer planned a lookup join on an index containing a column of type `CHAR(N)`, `VARCHAR(N)`, `BIT(N)`, `VARBIT(N)`, or `DECIMAL(M, N)`, and the query held that column constant to a single value (for example, with an equality filter). [#135111][#135111] +- Fixed an unhandled error that would occur if `DROP SCHEMA` was executed on the `public` schema of the `system` database, or on an internal schema, such as `pg_catalog` or `information_schema`. [#135195][#135195] +- Fixed a bug that caused incorrect evaluation of some binary expressions involving `CHAR(N)` values and untyped string literals with trailing whitespace characters. For example, the expression `'f'::CHAR = 'f '` now correctly evaluates to `true`. [#135690][#135690] +- `CREATE SCHEMA` now returns the correct error if a the schema name is missing. [#135926][#135926] + +

Performance improvements

+ +- Unnecessary block loads of SSTable files are now avoided in some rare cases after a replica rebalance. [#134541][#134541] + +[#133583]: https://github.com/cockroachdb/cockroach/pull/133583 +[#133708]: https://github.com/cockroachdb/cockroach/pull/133708 +[#133760]: https://github.com/cockroachdb/cockroach/pull/133760 +[#133822]: https://github.com/cockroachdb/cockroach/pull/133822 +[#133868]: https://github.com/cockroachdb/cockroach/pull/133868 +[#134001]: https://github.com/cockroachdb/cockroach/pull/134001 +[#134016]: https://github.com/cockroachdb/cockroach/pull/134016 +[#134087]: https://github.com/cockroachdb/cockroach/pull/134087 +[#134094]: https://github.com/cockroachdb/cockroach/pull/134094 +[#134100]: https://github.com/cockroachdb/cockroach/pull/134100 +[#134105]: https://github.com/cockroachdb/cockroach/pull/134105 +[#134365]: https://github.com/cockroachdb/cockroach/pull/134365 +[#134446]: https://github.com/cockroachdb/cockroach/pull/134446 +[#134508]: https://github.com/cockroachdb/cockroach/pull/134508 +[#134522]: https://github.com/cockroachdb/cockroach/pull/134522 +[#134541]: https://github.com/cockroachdb/cockroach/pull/134541 +[#134601]: https://github.com/cockroachdb/cockroach/pull/134601 +[#134648]: https://github.com/cockroachdb/cockroach/pull/134648 +[#134731]: https://github.com/cockroachdb/cockroach/pull/134731 +[#134836]: https://github.com/cockroachdb/cockroach/pull/134836 +[#134968]: https://github.com/cockroachdb/cockroach/pull/134968 +[#135111]: https://github.com/cockroachdb/cockroach/pull/135111 +[#135179]: https://github.com/cockroachdb/cockroach/pull/135179 +[#135195]: https://github.com/cockroachdb/cockroach/pull/135195 +[#135614]: https://github.com/cockroachdb/cockroach/pull/135614 +[#135690]: https://github.com/cockroachdb/cockroach/pull/135690 +[#135926]: https://github.com/cockroachdb/cockroach/pull/135926 +[#136008]: https://github.com/cockroachdb/cockroach/pull/136008 +[#136482]: https://github.com/cockroachdb/cockroach/pull/136482 From e8353a41528137f276e9aa0e72e66eb8e2265d26 Mon Sep 17 00:00:00 2001 From: Ryan Kuo <8740013+taroface@users.noreply.github.com> Date: Thu, 12 Dec 2024 14:00:11 -0500 Subject: [PATCH 10/36] v23.1.30 release notes (Cloud-first) (#19197) --- src/current/_data/releases.yml | 34 ++++++++++++++++++ .../_includes/releases/v23.1/v23.1.30.md | 35 +++++++++++++++++++ 2 files changed, 69 insertions(+) create mode 100644 src/current/_includes/releases/v23.1/v23.1.30.md diff --git a/src/current/_data/releases.yml b/src/current/_data/releases.yml index 8c4814e5f17..0f67f03e811 100644 --- a/src/current/_data/releases.yml +++ b/src/current/_data/releases.yml @@ -7524,6 +7524,40 @@ previous_release: v24.1.7 cloud_only: true cloud_only_message_short: 'Available only for select CockroachDB Cloud clusters' + cloud_only_message: > + This version is currently available only for select + CockroachDB Cloud clusters. To request to upgrade + a CockroachDB self-hosted cluster to this version, + [contact support](https://support.cockroachlabs.com/hc/requests/new). + +- release_name: v23.1.30 + major_version: v23.1 + release_date: '2024-12-12' + release_type: Production + go_version: go1.22.8 + sha: 2ee70ad63870ea3b44cb8593d0ac84d379547fc6 + has_sql_only: true + has_sha256sum: true + mac: + mac_arm: true + mac_arm_experimental: true + mac_arm_limited_access: true + windows: true + linux: + linux_arm: true + linux_arm_experimental: false + linux_arm_limited_access: false + linux_intel_fips: true + linux_arm_fips: false + docker: + docker_image: cockroachdb/cockroach + docker_arm: true + docker_arm_experimental: false + docker_arm_limited_access: false + source: true + previous_release: v23.1.29 + cloud_only: true + cloud_only_message_short: 'Available only for select CockroachDB Cloud clusters' cloud_only_message: > This version is currently available only for select CockroachDB Cloud clusters. To request to upgrade diff --git a/src/current/_includes/releases/v23.1/v23.1.30.md b/src/current/_includes/releases/v23.1/v23.1.30.md new file mode 100644 index 00000000000..84ca49d3100 --- /dev/null +++ b/src/current/_includes/releases/v23.1/v23.1.30.md @@ -0,0 +1,35 @@ +## v23.1.30 + +Release Date: December 12, 2024 + +{% include releases/new-release-downloads-docker-image.md release=include.release %} + +

Security updates

+ +- All cluster settings that accept strings are now fully redacted when transmitted as part of CockroachDB's diagnostics telemetry. The transmitted payload includes a record of modified cluster settings and their values when they are not strings. If you previously applied the mitigations in [Technical Advisory 133479]({% link advisories/a133479.md %}), you can safely turn on diagnostic reporting via the `diagnostics.reporting.enabled` cluster setting without leaking sensitive cluster settings values. [#134063][#134063] + +

General changes

+ +- `COCKROACH_SKIP_ENABLING_DIAGNOSTIC_REPORTING` is no longer mentioned in the `cockroach demo` command. [#134083][#134083] + +

Bug fixes

+ +- Fixed a bug where CockroachDB could encounter an internal error `interface conversion: coldata.Column is` in an edge case. The bug was present in v22.2.13+, v23.1.9+, and v23.2+. [#133758][#133758] +- Fixed a bug that caused incorrect `NOT NULL` constraint violation errors on `UPSERT` and `INSERT ... ON CONFLICT ... DO UPDATE` statements when those statements updated an existing row and a subset of columns that did not include a `NOT NULL` column of the table. This bug had been present since at least v20.1.0. [#133824][#133824] +- Fixed an unhandled error that could occur when using `REVOKE ... ON SEQUENCE FROM ... user` on an object that is not a sequence. [#133706][#133706] +- Addressed a panic that could occur inside `CREATE TABLE AS` if sequence builtin expressions had invalid function overloads. [#133866][#133866] +- Previously, when executing queries with index / lookup joins where ordering needed to be maintained, CockroachDB's behavior could lead to increased query latency, potentially by several orders of magnitude. This bug was introduced in v22.2, and is now fixed. [#134363][#134363] +- Fixed a bug where `DROP CASCADE` would occasionally panic with an `un-dropped backref` message on partitioned tables. [#134524][#134524] +- Reduced the duration of partitions in the gossip network when a node crashes. This eliminates false positives in the `ranges.unavailable` metric. [#134603][#134603] + +[#133706]: https://github.com/cockroachdb/cockroach/pull/133706 +[#133758]: https://github.com/cockroachdb/cockroach/pull/133758 +[#133824]: https://github.com/cockroachdb/cockroach/pull/133824 +[#133866]: https://github.com/cockroachdb/cockroach/pull/133866 +[#134063]: https://github.com/cockroachdb/cockroach/pull/134063 +[#134083]: https://github.com/cockroachdb/cockroach/pull/134083 +[#134363]: https://github.com/cockroachdb/cockroach/pull/134363 +[#134524]: https://github.com/cockroachdb/cockroach/pull/134524 +[#134603]: https://github.com/cockroachdb/cockroach/pull/134603 +[#134649]: https://github.com/cockroachdb/cockroach/pull/134649 +[154e9f0e0]: https://github.com/cockroachdb/cockroach/commit/154e9f0e0 From f58938b697dbac54873ce9fca307eb071a8360ba Mon Sep 17 00:00:00 2001 From: Mike Lewis <76072290+mikeCRL@users.noreply.github.com> Date: Thu, 12 Dec 2024 14:07:58 -0500 Subject: [PATCH 11/36] Temporarily disable Algolia builds in prod (#19225) --- src/current/netlify/build.sh | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/src/current/netlify/build.sh b/src/current/netlify/build.sh index cc69afa0680..c447d97999e 100755 --- a/src/current/netlify/build.sh +++ b/src/current/netlify/build.sh @@ -54,8 +54,9 @@ fi # Run Algolia if building main if [ "$CONTEXT" == "production" ]; then - echo "Building Algolia index..." - ALGOLIA_API_KEY=${PROD_ALGOLIA_API_KEY} bundle exec jekyll algolia --config _config_base.yml,_config_url.yml --builds-config _config_cockroachdb.yml + echo "Temporarily skipping the Algolia index build" + # echo "Building Algolia index..." + # ALGOLIA_API_KEY=${PROD_ALGOLIA_API_KEY} bundle exec jekyll algolia --config _config_base.yml,_config_url.yml --builds-config _config_cockroachdb.yml else echo "Not building Algolia index for context $CONTEXT" fi From 574ce1777323e0a9159644c2ee6f8bc9c05b3eff Mon Sep 17 00:00:00 2001 From: Florence Morris Date: Thu, 12 Dec 2024 14:37:18 -0500 Subject: [PATCH 12/36] DOC-11721 Create a Logical Data Replication Metrics Dashboard page (#19218) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit (1) Added ui-logical-data-replication-dashboard.md. (2) In self-hosted-deployments.json, updated Metrics Dashboards by adding a link to Logical Data Replication and removing the word Dashboard from all other link text. (3) In logical-data-replication-monitoring, replaced links to ui-overview-dashboard.md with ui-logical-data-replication-dashboard.md. (4) In set-up-logical-data-replication.md, (a) corrected syntax by replacing backtick with single quote and (b) added link to ui-logical-data-replication-dashboard.md. * Incorporated Kathryn’s feedback. --- .../sidebar-data/self-hosted-deployments.json | 34 +++--- .../logical-data-replication-monitoring.md | 10 +- .../v24.3/set-up-logical-data-replication.md | 4 +- .../ui-logical-data-replication-dashboard.md | 107 ++++++++++++++++++ 4 files changed, 133 insertions(+), 22 deletions(-) create mode 100644 src/current/v24.3/ui-logical-data-replication-dashboard.md diff --git a/src/current/_includes/v24.3/sidebar-data/self-hosted-deployments.json b/src/current/_includes/v24.3/sidebar-data/self-hosted-deployments.json index 1e48de5aeff..76608d8939a 100644 --- a/src/current/_includes/v24.3/sidebar-data/self-hosted-deployments.json +++ b/src/current/_includes/v24.3/sidebar-data/self-hosted-deployments.json @@ -406,89 +406,95 @@ "title": "Metrics Dashboards", "items": [ { - "title": "Overview Dashboard", + "title": "Overview", "urls": [ "/${VERSION}/ui-overview-dashboard.html" ] }, { - "title": "Hardware Dashboard", + "title": "Hardware", "urls": [ "/${VERSION}/ui-hardware-dashboard.html" ] }, { - "title": "Runtime Dashboard", + "title": "Runtime", "urls": [ "/${VERSION}/ui-runtime-dashboard.html" ] }, { - "title": "Networking Dashboard", + "title": "Networking", "urls": [ "/${VERSION}/ui-networking-dashboard.html" ] }, { - "title": "SQL Dashboard", + "title": "SQL", "urls": [ "/${VERSION}/ui-sql-dashboard.html" ] }, { - "title": "Storage Dashboard", + "title": "Storage", "urls": [ "/${VERSION}/ui-storage-dashboard.html" ] }, { - "title": "Replication Dashboard", + "title": "Replication", "urls": [ "/${VERSION}/ui-replication-dashboard.html" ] }, { - "title": "Distributed Dashboard", + "title": "Distributed", "urls": [ "/${VERSION}/ui-distributed-dashboard.html" ] }, { - "title": "Queues Dashboard", + "title": "Queues", "urls": [ "/${VERSION}/ui-queues-dashboard.html" ] }, { - "title": "Slow Requests Dashboard", + "title": "Slow Requests", "urls": [ "/${VERSION}/ui-slow-requests-dashboard.html" ] }, { - "title": "Changefeeds Dashboard", + "title": "Changefeeds", "urls": [ "/${VERSION}/ui-cdc-dashboard.html" ] }, { - "title": "Overload Dashboard", + "title": "Overload", "urls": [ "/${VERSION}/ui-overload-dashboard.html" ] }, { - "title": "TTL Dashboard", + "title": "TTL", "urls": [ "/${VERSION}/ui-ttl-dashboard.html" ] }, { - "title": "Physical Cluster Replication Dashboard", + "title": "Physical Cluster Replication", "urls": [ "/${VERSION}/ui-physical-cluster-replication-dashboard.html" ] }, + { + "title": "Logical Data Replication", + "urls": [ + "/${VERSION}/ui-logical-data-replication-dashboard.html" + ] + }, { "title": "Custom Chart", "urls": [ diff --git a/src/current/v24.3/logical-data-replication-monitoring.md b/src/current/v24.3/logical-data-replication-monitoring.md index 88c2b919354..77730488cc9 100644 --- a/src/current/v24.3/logical-data-replication-monitoring.md +++ b/src/current/v24.3/logical-data-replication-monitoring.md @@ -14,7 +14,7 @@ Logical data replication is only supported in CockroachDB {{ site.data.products. You can monitor [**logical data replication (LDR)**]({% link {{ page.version.version }}/logical-data-replication-overview.md %}) using: - [`SHOW LOGICAL REPLICATION JOBS`](#sql-shell) in the SQL shell to view a list of LDR jobs on the cluster. -- The **Logical Data Replication** dashboard on the [DB Console](#db-console) to view metrics at the cluster level. {% comment %}To add link later to dashboard page{% endcomment %} +- The [**Logical Data Replication** dashboard]({% link {{ page.version.version }}/ui-logical-data-replication-dashboard.md %}) on the [DB Console](#db-console) to view metrics at the cluster level. - [Prometheus and Alertmanager](#prometheus) to track and alert on LDR metrics. - Metrics export with [Datadog](#datadog). - [Metrics labels](#metrics-labels) to view metrics at the job level. @@ -77,26 +77,24 @@ You can also use [`SHOW JOBS`]({% link {{ page.version.version }}/show-jobs.md % In the DB Console, you can use: -- The [**Metrics** dashboard]({% link {{ page.version.version }}/ui-overview-dashboard.md %}) for LDR to view metrics for the job on the destination cluster. +- The [**Metrics** dashboard]({% link {{ page.version.version }}/ui-logical-data-replication-dashboard.md %}) for LDR to view metrics for the job on the destination cluster. - The [**Jobs** page]({% link {{ page.version.version }}/ui-jobs-page.md %}) to view the history retention job on the source cluster and the LDR job on the destination cluster The metrics for LDR in the DB Console metrics are at the **cluster** level. This means that if there are multiple LDR jobs running on a cluster the DB Console will show the average metrics across jobs. ### Metrics dashboard -You can use the [**Logical Data Replication** dashboard]({% link {{ page.version.version }}/ui-overview-dashboard.md %}) of the destination cluster to monitor the following metric graphs at the **cluster** level: +You can use the [**Logical Data Replication** dashboard]({% link {{ page.version.version }}/ui-logical-data-replication-dashboard.md %}) of the destination cluster to monitor the following metric graphs at the **cluster** level: - Replication latency - Replication lag - Row updates applied -- Logical bytes reviewed +- Logical bytes received - Batch application processing time: 50th percentile - Batch application processing time: 99th percentile - DLQ causes - Retry queue size -{% comment %}Dashboard page in the DB Console docs to be added with more information per other dashboards. Link to there from this section.{% endcomment %} - To track replicated time, ingested events, and events added to the DLQ at the **job** level, refer to [Metrics labels](#metrics-labels). ### Jobs page diff --git a/src/current/v24.3/set-up-logical-data-replication.md b/src/current/v24.3/set-up-logical-data-replication.md index 1aa4ad90cca..9dfcf183ab2 100644 --- a/src/current/v24.3/set-up-logical-data-replication.md +++ b/src/current/v24.3/set-up-logical-data-replication.md @@ -110,7 +110,7 @@ You can use the `cockroach encode-uri` command to generate a connection string c {% include_cached copy-clipboard.html %} ~~~ sql - CREATE EXTERNAL CONNECTION {source} AS 'postgresql://{replication user}:{password}@{node IP}:26257?options=-ccluster%3Dsystem&sslinline=true&sslmode=verify-full&sslrootcert=-----BEGIN+CERTIFICATE-----{encoded certificate}-----END+CERTIFICATE-----%0A`; + CREATE EXTERNAL CONNECTION {source} AS 'postgresql://{replication user}:{password}@{node IP}:26257?options=-ccluster%3Dsystem&sslinline=true&sslmode=verify-full&sslrootcert=-----BEGIN+CERTIFICATE-----{encoded certificate}-----END+CERTIFICATE-----%0A'; ~~~ ## Step 3. Start LDR @@ -180,7 +180,7 @@ For a full reference on monitoring LDR, refer to [Logical Data Replication Monit 1. Access the DB Console at `http://{node IP or hostname}:8080` and enter your user's credentials. 1. On the source cluster, navigate to the [**Jobs** page]({% link {{ page.version.version }}/ui-jobs-page.md %}) to view a list of all jobs. Use the job **Type** dropdown and select **Replication Producer**. This will display the history retention job. This will run while the LDR job is active to protect changes to the table from [garbage collection]({% link {{ page.version.version }}/architecture/storage-layer.md %}#garbage-collection) until they have been replicated to the destination cluster. 1. On the destination cluster, use the job **Type** dropdown and select **Logical Replication Ingestion**. This page will display the logical replication stream job. There will be a progress bar in the **Status** column when LDR is replicating a table with existing data. This progress bar shows the status of the initial scan, which backfills the destination table with the existing data. -1. On the destination cluster, click on **Metrics** in the left-hand navigation menu. Use the **Dashboard** dropdown to select **Logical Data Replication**. This page shows graphs for monitoring LDR. +1. On the destination cluster, click on **Metrics** in the left-hand navigation menu. Use the **Dashboard** dropdown to select [**Logical Data Replication**]({% link {{ page.version.version }}/ui-logical-data-replication-dashboard.md %}). This page shows graphs for monitoring LDR. ## What's next diff --git a/src/current/v24.3/ui-logical-data-replication-dashboard.md b/src/current/v24.3/ui-logical-data-replication-dashboard.md new file mode 100644 index 00000000000..468ac5d380e --- /dev/null +++ b/src/current/v24.3/ui-logical-data-replication-dashboard.md @@ -0,0 +1,107 @@ +--- +title: Logical Data Replication Dashboard +summary: The Physical Cluster Replication Dashboard lets you monitor and observe replication streams between a primary and standby cluster. +toc: true +docs_area: reference.db_console +--- + +The **Logical Data Replication** dashboard in the DB Console lets you monitor metrics related to the [logical data replication]({% link {{ page.version.version }}/logical-data-replication-overview.md %}) (LDR) jobs on the destination cluster. These metrics are at the **cluster** level. This means that if there are multiple LDR jobs running on a cluster the DB Console will show the average metrics across jobs. + +To view this dashboard, [access the DB Console]({% link {{ page.version.version }}/ui-overview.md %}#db-console-access) for the destination cluster, click **Metrics** on the left-hand navigation bar, and select **Logical Data Replication** from the **Dashboard** dropdown. + +{{site.data.alerts.callout_info}} +The **Logical Data Replication** dashboard is distinct from the [**Replication** dashboard]({% link {{ page.version.version }}/ui-replication-dashboard.md %}), which tracks metrics related to how data is replicated across the cluster, e.g., range status, replicas per store, and replica quiescence. +{{site.data.alerts.end}} + +## Dashboard navigation + +{% include {{ page.version.version }}/ui/ui-metrics-navigation.md %} + +--- + +The **Logical Data Replication** dashboard displays the following time-series graphs: + +{{site.data.alerts.callout_info}} +The specific node views do not apply for LDR metrics. +{{site.data.alerts.end}} + +## Replication Latency + +The graph shows the difference in commit times between the source cluster and the destination cluster. + +Metric | CockroachDB Metric Name | Description +-------|-------------------------|------------- +p50,
p99 | `logical_replication.commit_latency-p50`,
`logical_replication.commit_latency-p99` | Event commit latency: a difference between event MVCC timestamp and the time it was flushed into disk. If we batch events, then the difference between the oldest event in the batch and flush is recorded. + +## Replication Lag + +The graph shows the age of the oldest row on the source cluster that has yet to replicate to the destination cluster. + +Metric | CockroachDB Metric Name | Description +-------|-------------------------|------------- +Replication Lag | `logical_replication.replicated_time_seconds` | The replicated time of the logical replication stream in seconds since the unix epoch. + +## Row Updates Applied + +The graph shows the rate at which row updates are applied by all logical replication jobs. + +Metric | CockroachDB Metric Name | Description +-------|-------------------------|------------- +Row Updates Applied | `logical_replication.events_ingested` | Events ingested by all logical replication jobs. +Row Updates sent to DLQ | `logical_replication.events_dlqed` | Row update events sent to the dead letter queue (DLQ). + +## Logical Bytes Received + +- In the cluster view, the graph shows the rate at which the logical bytes (sum of keys + values) are received by all logical replication jobs across all nodes. +- In the node view, the graph shows the rate at which the logical bytes (sum of keys + values) are received by all logical replication jobs on the node. + +Metric | CockroachDB Metric Name | Description +-------|-------------------------|------------- +logical bytes | `logical_replication.logical_bytes` | Logical bytes (sum of keys + values) received by all replication jobs. + +## Batch Application Processing Time: 50th percentile + +- In the cluster view, the graph shows the 50th percentile in the time it takes to write a batch of row updates across all nodes. +- In the node view, the graph shows the 50th percentile in the time it takes to write a batch of row updates on the node. + +Metric | CockroachDB Metric Name | Description +-------|-------------------------|------------- +processing time | `logical_replication.batch_hist_nanos-p50` | Time spent flushing a batch. + + +## Batch Application Processing Time: 99th percentile + +- In the cluster view, the graph shows the 99th percentile in the time it takes to write a batch of row updates across all nodes. +- In the node view, the graph shows the 99th percentile in the time it takes to write a batch of row updates on the node. + +Metric | CockroachDB Metric Name | Description +-------|-------------------------|------------- +processing time | `logical_replication.batch_hist_nanos-p99` | Time spent flushing a batch. + +## DLQ Causes + +The graph shows the reasons why events were sent to the [dead letter queue (DLQ)]({% link {{ page.version.version }}/manage-logical-data-replication.md %}#dead-letter-queue-dlq) + +Metric | CockroachDB Metric Name | Description +-------|-------------------------|------------- +Retry Duration Expired | `logical_replication.events_dlqed_age` | Row update events sent to DLQ due to reaching the maximum time allowed in the retry queue. +Retry Queue Full | `logical_replication.events_dlqed_space` | Row update events sent to DLQ due to capacity of the retry queue. +Non-retryable | `logical_replication.events_dlqed_errtype` | Row update events sent to DLQ due to an error not considered retryable. + +## Retry Queue Size + +- In the cluster view, the graph shows the total size of the retry queues across all processors in all LDR jobs across all nodes. +- In the node view, the graph shows the total size of the retry queues across all processors in all LDR jobs on the node. + +Metric | CockroachDB Metric Name | Description +-------|-------------------------|------------- +retry queue bytes | `logical_replication.retry_queue_bytes` | The size of the retry queue of the logical replication stream. + +{% include {{ page.version.version }}/ui/ui-summary-events.md %} + +## See also + +- [Logical Data Replication Overview]({% link {{ page.version.version }}/logical-data-replication-overview.md %}) +- [Logical Data Replication Monitoring]({% link {{ page.version.version }}/logical-data-replication-monitoring.md %}) +- [Troubleshooting Overview]({% link {{ page.version.version }}/troubleshooting-overview.md %}) +- [Support Resources]({% link {{ page.version.version }}/support-resources.md %}) \ No newline at end of file From 3b7c1b47dcdeeef55907889554e69e18520d489c Mon Sep 17 00:00:00 2001 From: shannonbradshaw Date: Thu, 12 Dec 2024 14:52:54 -0500 Subject: [PATCH 13/36] update release dates to Nov 18 --- src/current/_data/releases.yml | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/src/current/_data/releases.yml b/src/current/_data/releases.yml index 0f67f03e811..2b8fc98dc05 100644 --- a/src/current/_data/releases.yml +++ b/src/current/_data/releases.yml @@ -7301,7 +7301,7 @@ - release_name: v24.2.5 major_version: v24.2 - release_date: '2024-11-14' + release_date: '2024-11-18' release_type: Production go_version: go1.22.8 sha: beddec16a5a438d66bd3f5f4202085cf21c0973c @@ -7329,7 +7329,7 @@ - release_name: v23.1.29 major_version: v23.1 - release_date: '2024-11-14' + release_date: '2024-11-18' release_type: Production go_version: go1.22.8 sha: cb0646dc26328963ca2c89d7a8d105d166836d6d @@ -7356,7 +7356,7 @@ - release_name: v24.1.7 major_version: v24.1 - release_date: '2024-11-14' + release_date: '2024-11-18' release_type: Production go_version: go1.22.8 sha: 3613a619e327fb412ac62036ce2c7060c25e48db @@ -7410,7 +7410,7 @@ - release_name: v23.2.16 major_version: v23.2 - release_date: '2024-11-16' + release_date: '2024-11-18' release_type: Production go_version: go1.22.8 sha: 720d706874e2424b32c7c9347f6d52140f83beee From 793c14708c5c7acc5e8521529b4225c7a92174f3 Mon Sep 17 00:00:00 2001 From: Ryan Kuo <8740013+taroface@users.noreply.github.com> Date: Thu, 12 Dec 2024 14:56:53 -0500 Subject: [PATCH 14/36] update SERIALIZABLE aliases (#19200) --- src/current/v24.1/transactions.md | 6 ++++-- src/current/v24.2/transactions.md | 6 ++++-- src/current/v24.3/transactions.md | 6 ++++-- 3 files changed, 12 insertions(+), 6 deletions(-) diff --git a/src/current/v24.1/transactions.md b/src/current/v24.1/transactions.md index 81dd2e35914..a3d1fcd091c 100644 --- a/src/current/v24.1/transactions.md +++ b/src/current/v24.1/transactions.md @@ -212,9 +212,11 @@ CockroachDB uses slightly different isolation levels than [ANSI SQL isolation le #### Aliases -`SNAPSHOT`, `READ UNCOMMITTED`, `READ COMMITTED`, and `REPEATABLE READ` are aliases for `SERIALIZABLE`. +`SNAPSHOT` and `REPEATABLE READ` are aliases for `SERIALIZABLE`. -If [`READ COMMITTED` isolation is enabled]({% link {{ page.version.version }}/read-committed.md %}#enable-read-committed-isolation) using the `sql.txn.read_committed_isolation.enabled` [cluster setting]({% link {{ page.version.version }}/cluster-settings.md %}), `READ COMMITTED` is no longer an alias for `SERIALIZABLE`, and `READ UNCOMMITTED` becomes an alias for `READ COMMITTED`. +`READ UNCOMMITTED` is an alias for `READ COMMITTED`. + +If `READ COMMITTED` isolation is disabled using the `sql.txn.read_committed_isolation.enabled` [cluster setting]({% link {{ page.version.version }}/cluster-settings.md %}#setting-sql-txn-read-committed-isolation-enabled), `READ COMMITTED` and `READ UNCOMMITTED` become aliases for `SERIALIZABLE`. #### Comparison diff --git a/src/current/v24.2/transactions.md b/src/current/v24.2/transactions.md index 81dd2e35914..a3d1fcd091c 100644 --- a/src/current/v24.2/transactions.md +++ b/src/current/v24.2/transactions.md @@ -212,9 +212,11 @@ CockroachDB uses slightly different isolation levels than [ANSI SQL isolation le #### Aliases -`SNAPSHOT`, `READ UNCOMMITTED`, `READ COMMITTED`, and `REPEATABLE READ` are aliases for `SERIALIZABLE`. +`SNAPSHOT` and `REPEATABLE READ` are aliases for `SERIALIZABLE`. -If [`READ COMMITTED` isolation is enabled]({% link {{ page.version.version }}/read-committed.md %}#enable-read-committed-isolation) using the `sql.txn.read_committed_isolation.enabled` [cluster setting]({% link {{ page.version.version }}/cluster-settings.md %}), `READ COMMITTED` is no longer an alias for `SERIALIZABLE`, and `READ UNCOMMITTED` becomes an alias for `READ COMMITTED`. +`READ UNCOMMITTED` is an alias for `READ COMMITTED`. + +If `READ COMMITTED` isolation is disabled using the `sql.txn.read_committed_isolation.enabled` [cluster setting]({% link {{ page.version.version }}/cluster-settings.md %}#setting-sql-txn-read-committed-isolation-enabled), `READ COMMITTED` and `READ UNCOMMITTED` become aliases for `SERIALIZABLE`. #### Comparison diff --git a/src/current/v24.3/transactions.md b/src/current/v24.3/transactions.md index 81dd2e35914..a3d1fcd091c 100644 --- a/src/current/v24.3/transactions.md +++ b/src/current/v24.3/transactions.md @@ -212,9 +212,11 @@ CockroachDB uses slightly different isolation levels than [ANSI SQL isolation le #### Aliases -`SNAPSHOT`, `READ UNCOMMITTED`, `READ COMMITTED`, and `REPEATABLE READ` are aliases for `SERIALIZABLE`. +`SNAPSHOT` and `REPEATABLE READ` are aliases for `SERIALIZABLE`. -If [`READ COMMITTED` isolation is enabled]({% link {{ page.version.version }}/read-committed.md %}#enable-read-committed-isolation) using the `sql.txn.read_committed_isolation.enabled` [cluster setting]({% link {{ page.version.version }}/cluster-settings.md %}), `READ COMMITTED` is no longer an alias for `SERIALIZABLE`, and `READ UNCOMMITTED` becomes an alias for `READ COMMITTED`. +`READ UNCOMMITTED` is an alias for `READ COMMITTED`. + +If `READ COMMITTED` isolation is disabled using the `sql.txn.read_committed_isolation.enabled` [cluster setting]({% link {{ page.version.version }}/cluster-settings.md %}#setting-sql-txn-read-committed-isolation-enabled), `READ COMMITTED` and `READ UNCOMMITTED` become aliases for `SERIALIZABLE`. #### Comparison From ebfd645aa8a8585f898e0d60a67d24aaf4e65d43 Mon Sep 17 00:00:00 2001 From: Rich Loveland Date: Thu, 12 Dec 2024 15:10:45 -0500 Subject: [PATCH 15/36] Add cloud-first release notes for v23.2.17 (#19206) * Add cloud-first release notes for v23.2.17 Fixes REL-1674 --- src/current/_data/releases.yml | 36 ++++++++++- .../_includes/releases/v23.2/v23.2.17.md | 63 +++++++++++++++++++ 2 files changed, 98 insertions(+), 1 deletion(-) create mode 100644 src/current/_includes/releases/v23.2/v23.2.17.md diff --git a/src/current/_data/releases.yml b/src/current/_data/releases.yml index 0f67f03e811..1107507ca03 100644 --- a/src/current/_data/releases.yml +++ b/src/current/_data/releases.yml @@ -7562,4 +7562,38 @@ This version is currently available only for select CockroachDB Cloud clusters. To request to upgrade a CockroachDB self-hosted cluster to this version, - [contact support](https://support.cockroachlabs.com/hc/requests/new). \ No newline at end of file + [contact support](https://support.cockroachlabs.com/hc/requests/new). + +- release_name: v23.2.17 + major_version: v23.2 + release_date: '2024-12-12' + release_type: Production + go_version: go1.22.8 + sha: 1a524cd3ced15426926fb4898162ab43c7484d92 + has_sql_only: true + has_sha256sum: true + mac: + mac_arm: true + mac_arm_experimental: true + mac_arm_limited_access: false + windows: true + linux: + linux_arm: true + linux_arm_experimental: false + linux_arm_limited_access: false + linux_intel_fips: true + linux_arm_fips: false + docker: + docker_image: cockroachdb/cockroach + docker_arm: true + docker_arm_experimental: false + docker_arm_limited_access: false + source: true + previous_release: v23.2.16 + cloud_only: true + cloud_only_message_short: 'Available only for select CockroachDB Cloud clusters' + cloud_only_message: > + This version is currently available only for select + CockroachDB Cloud clusters. To request to upgrade + a CockroachDB self-hosted cluster to this version, + [contact support](https://support.cockroachlabs.com/hc/requests/new). diff --git a/src/current/_includes/releases/v23.2/v23.2.17.md b/src/current/_includes/releases/v23.2/v23.2.17.md new file mode 100644 index 00000000000..29855bc2638 --- /dev/null +++ b/src/current/_includes/releases/v23.2/v23.2.17.md @@ -0,0 +1,63 @@ +## v23.2.17 + +Release Date: December 12, 2024 + +{% include releases/new-release-downloads-docker-image.md release=include.release %} + +

Security updates

+ +- All cluster settings that accept strings are now fully redacted when transmitted as part of Cockroach Labs' diagnostics telemetry. The payload includes a record of modified cluster settings and their values when they are not strings. If you previously applied the mitigations in Technical Advisory 133479, you can safely set the value of cluster setting `server.redact_sensitive_settings.enabled` to `false` and turn on diagnostic reporting via the `diagnostics.reporting.enabled` cluster setting without leaking sensitive cluster setting values. [#134015][#134015] + +

General changes

+ +- `COCKROACH_SKIP_ENABLING_DIAGNOSTIC_REPORTING` is no longer mentioned in the `cockroach demo` command. [#134085][#134085] + +

{{ site.data.products.enterprise }} edition changes

+ +- Added `system.users` to the list of system tables that changefeeds protect with protected timestamps (PTS). This table is required for CDC queries. [#134233][#134233] + +

Operational changes

+ +- Added a new cluster setting `ui.database_locality_metadata.enabled`, which allows operators to disable loading extended database and table region information in the DB Console's Databases and Table Details pages. This information can cause significant CPU load on large clusters with many ranges. Versions of this page from v24.3 onwards do not have this problem. If you require this data, you can use the `SHOW RANGES FROM {DATABASE| TABLE}` query via SQL to compute on-demand. [#134093][#134093] + +

Bug fixes

+ +- Previously, CockroachDB could encounter an internal error of the form `interface conversion: coldata.Column is` in an edge case. This is now fixed. The bug was present in versions v22.2.13 and later, v23.1.9 and later, and v23.2 and later. [#133759][#133759] +- Fixed a bug that caused incorrect `NOT NULL` constraint violation errors on `UPSERT` and `INSERT ... ON CONFLICT ... DO UPDATE` statements when those statements updated an existing row and a subset of columns that did not include a `NOT NULL` column of the table. This bug had been present since at least v20.1.0. [#133823][#133823] +- Fixed an unhandled error that could occur when using `REVOKE ... ON SEQUENCE ... FROM user` on an object that was not a sequence. [#133707][#133707] +- Addressed a panic that could occur inside `CREATE TABLE AS` that occurred if sequence builtin expressions had invalid function overloads. [#133867][#133867] +- String constants can now be compared against collated strings. [#134114][#134114] +- Previously, when executing queries with index or lookup joins when the ordering needed to be maintained, CockroachDB in some cases could get into a pathological state which would lead to increased query latency, possibly by several orders of magnitude. This bug was introduced in v22.2 and is now fixed. [#134364][#134364] +- Addressed a bug with `DROP CASCADE` that would occasionally panic with an `un-dropped backref` message on partitioned tables. [#134523][#134523] +- Reduced the duration of partitions in the gossip network when a node crashes. This eliminates false positives in the `ranges.unavailable` metric. [#134602][#134602] +- An error message is no longer returned when a non-admin user runs `DROP ROLE IF EXISTS` on a user that does not exist. [#134967][#134967] +- Fixed a bug that could cause incorrect query results when the optimizer planned a lookup join on an index containing a column of type `CHAR(N)`, `VARCHAR(N)`, `BIT(N)`, `VARBIT(N)`, or `DECIMAL(M, N)`, and the query held that column constant to a single value (e.g., with an equality filter). [#135113][#135113] +- Fixed an unhandled error that would occur if `DROP SCHEMA` was executed on the `public` schema of the `system` database, or on an internal schema like `pg_catalog` or `information_schema`. [#135196][#135196] +- Fixed a bug that caused incorrect evaluation of some binary expressions involving `CHAR(N)` values and untyped string literals with trailing whitespace characters. For example, the expression `'f'::CHAR = 'f '` now correctly evaluates to `true`. [#135691][#135691] + +

Performance improvements

+ +- CockroachDB now avoids loading unnecessary file blocks shortly after a rebalance in a rare case. [#134526][#134526] [#135303][#135303] [#135577][#135577] +- Reduced the write-amplification impact of rebalances by splitting snapshot sstable files into smaller ones before ingesting them into Pebble. [#134526][#134526] [#135303][#135303] [#135577][#135577] + +[#133707]: https://github.com/cockroachdb/cockroach/pull/133707 +[#133759]: https://github.com/cockroachdb/cockroach/pull/133759 +[#133823]: https://github.com/cockroachdb/cockroach/pull/133823 +[#133867]: https://github.com/cockroachdb/cockroach/pull/133867 +[#134015]: https://github.com/cockroachdb/cockroach/pull/134015 +[#134085]: https://github.com/cockroachdb/cockroach/pull/134085 +[#134093]: https://github.com/cockroachdb/cockroach/pull/134093 +[#134114]: https://github.com/cockroachdb/cockroach/pull/134114 +[#134233]: https://github.com/cockroachdb/cockroach/pull/134233 +[#134364]: https://github.com/cockroachdb/cockroach/pull/134364 +[#134523]: https://github.com/cockroachdb/cockroach/pull/134523 +[#134526]: https://github.com/cockroachdb/cockroach/pull/134526 +[#134602]: https://github.com/cockroachdb/cockroach/pull/134602 +[#134647]: https://github.com/cockroachdb/cockroach/pull/134647 +[#134967]: https://github.com/cockroachdb/cockroach/pull/134967 +[#135113]: https://github.com/cockroachdb/cockroach/pull/135113 +[#135196]: https://github.com/cockroachdb/cockroach/pull/135196 +[#135303]: https://github.com/cockroachdb/cockroach/pull/135303 +[#135577]: https://github.com/cockroachdb/cockroach/pull/135577 +[#135691]: https://github.com/cockroachdb/cockroach/pull/135691 +[#136006]: https://github.com/cockroachdb/cockroach/pull/136006 From e72d41528f664ad75bf77593b45ada3f6db4c554 Mon Sep 17 00:00:00 2001 From: Mike Lewis <76072290+mikeCRL@users.noreply.github.com> Date: Thu, 12 Dec 2024 17:26:19 -0500 Subject: [PATCH 16/36] v24.3.1 release notes (#19217) * v24.3.1 release notes * Release note edits * Revert versions.csv change --- src/current/_data/releases.yml | 34 +++++++++ .../_includes/releases/v24.3/v24.3.1.md | 76 +++++++++++++++++++ 2 files changed, 110 insertions(+) create mode 100644 src/current/_includes/releases/v24.3/v24.3.1.md diff --git a/src/current/_data/releases.yml b/src/current/_data/releases.yml index 1107507ca03..991ab13d9c4 100644 --- a/src/current/_data/releases.yml +++ b/src/current/_data/releases.yml @@ -7597,3 +7597,37 @@ CockroachDB Cloud clusters. To request to upgrade a CockroachDB self-hosted cluster to this version, [contact support](https://support.cockroachlabs.com/hc/requests/new). + +- release_name: v24.3.1 + major_version: v24.3 + release_date: '2024-12-12' + release_type: Production + go_version: go1.22.8 + sha: f9b40bb0eb7ed20acd68f24afc4c402614a9274b + has_sql_only: true + has_sha256sum: true + mac: + mac_arm: true + mac_arm_experimental: true + mac_arm_limited_access: false + windows: true + linux: + linux_arm: true + linux_arm_experimental: false + linux_arm_limited_access: false + linux_intel_fips: true + linux_arm_fips: false + docker: + docker_image: cockroachdb/cockroach + docker_arm: true + docker_arm_experimental: false + docker_arm_limited_access: false + source: true + previous_release: v24.3.0 + cloud_only: true + cloud_only_message_short: 'Available only for select CockroachDB Cloud clusters' + cloud_only_message: > + This version is currently available only for select + CockroachDB Cloud clusters. To request to upgrade + a CockroachDB self-hosted cluster to this version, + [contact support](https://support.cockroachlabs.com/hc/requests/new). \ No newline at end of file diff --git a/src/current/_includes/releases/v24.3/v24.3.1.md b/src/current/_includes/releases/v24.3/v24.3.1.md new file mode 100644 index 00000000000..e5529f6e4b8 --- /dev/null +++ b/src/current/_includes/releases/v24.3/v24.3.1.md @@ -0,0 +1,76 @@ +## v24.3.1 + +Release Date: December 12, 2024 + +{% include releases/new-release-downloads-docker-image.md release=include.release %} + +

SQL language changes

+ +- When triggers fire one another cyclically, the new `recursion_depth_limit` setting now limits the depth of the recursion. By default, the limit is `1000` nested trigger executions. [#135046][#135046] + +

Operational changes

+ +- The metrics scrape HTTP endpoint at `/ _status/vars` will now truncate HELP text at the first sentence, reducing the metadata for metrics with large descriptions. Customers can still access these descriptions via our docs. [#135021][#135021] +- The row-level TTL job now periodically updates the progress meter in the jobs introspection interfaces, including `SHOW JOBS` and the Jobs page in the DB console. [#135171][#135171] +- Telemetry delivery is now considered successful even in cases where we experience a network timeout. This will prevent throttling in cases outside an operator's control. [#136481][#136481] + +

DB Console changes

+ +- When activating statement diagnostics in the DB Console, users now have the option to produce a redacted bundle as output. This bundle will omit sensitive data. [#134993][#134993] + +

Other changes

+ +- Protected timestamp records for changefeeds now include the `system.users` table. This ensures that user information remains available when running CDC queries against historical data. [#134238][#134238] + +

Bug fixes

+ +- Fixed a bug that could cause `DELETE` triggers not to fire on cascading delete, and which could cause `INSERT` triggers to match incorrectly in the same scenario. [#134896][#134896] +- Reduced the duration of partitions in the gossip network when a node crashes in order to eliminate false positives in the `ranges.unavailable` metric. [#134480][#134480] +- When a non-admin user runs `DROP ROLE IF EXISTS` on a user that does not exist, an error is no longer returned. [#134970][#134970] +- Fixed a bug that could cause incorrect query results when the optimizer planned a lookup join on an index containing a column of type `CHAR(N)`, `VARCHAR(N)`, `BIT(N)`, `VARBIT(N)`, or `DECIMAL(M, N)`, and the query held that column constant to a single value (e.g. with an equality filter). [#135037][#135037] +- Fixed an unhandled error that would occur if `DROP SCHEMA` was executed on the `public` schema of the `system` database, or on an internal schema like `pg_catalog` or `information_schema`. [#135181][#135181] +- A bug has been fixed that caused incorrect evaluation of some binary expressions involving `CHAR(N)` values and untyped string literals with trailing whitespace characters. For example, the expression `'f'::CHAR = 'f '` now correctly evaluates to `true`. [#134710][#134710] +- Prevent `ALTER DATABASE` operations that modify the zone config from hanging if an invalid zone config already exists. [#135216][#135216] +- `CREATE SCHEMA` now returns the correct error if a schema name is missing. [#135928][#135928] +- `percentile_cont` and `percentile_disc` aggregate functions now support `float4` data type inputs. Previously, these functions would return an error when used with `float4` values. [#135764][#135764] +- `security.certificate.*` metrics now update correctly when certificates are reloaded during node runtime. Previously, these metrics would not reflect changes to certificates after node startup. [#136227][#136227] +- SQL roles created from LDAP groups that contain periods (.) or hyphens (-) in their Common Names (CN) no longer result in authorization failures. [#134942][#134942] +- LDAP authorization now supports partial group mapping, allowing users to authenticate even when some LDAP groups do not have corresponding CockroachDB roles. Previously, authentication would fail if any LDAP group lacked a matching database role. [#135587][#135587] + +

Performance improvements

+ +- The `_status/nodes_ui` API no longer returns unnecessary metrics in its response. This decreases the payload size of the API and improves the load time of various DB Console pages and components. [#135209][#135209] +- PL/pgSQL loops now execute up to 3-4x faster through improved optimization, particularly when they contain subqueries. This enhancement improves performance for routines with many iterations or nested operations. [#135648][#135648] + +[#133230]: https://github.com/cockroachdb/cockroach/pull/133230 +[#134238]: https://github.com/cockroachdb/cockroach/pull/134238 +[#134480]: https://github.com/cockroachdb/cockroach/pull/134480 +[#134710]: https://github.com/cockroachdb/cockroach/pull/134710 +[#134729]: https://github.com/cockroachdb/cockroach/pull/134729 +[#134896]: https://github.com/cockroachdb/cockroach/pull/134896 +[#134942]: https://github.com/cockroachdb/cockroach/pull/134942 +[#134970]: https://github.com/cockroachdb/cockroach/pull/134970 +[#134993]: https://github.com/cockroachdb/cockroach/pull/134993 +[#135021]: https://github.com/cockroachdb/cockroach/pull/135021 +[#135037]: https://github.com/cockroachdb/cockroach/pull/135037 +[#135046]: https://github.com/cockroachdb/cockroach/pull/135046 +[#135094]: https://github.com/cockroachdb/cockroach/pull/135094 +[#135120]: https://github.com/cockroachdb/cockroach/pull/135120 +[#135171]: https://github.com/cockroachdb/cockroach/pull/135171 +[#135181]: https://github.com/cockroachdb/cockroach/pull/135181 +[#135209]: https://github.com/cockroachdb/cockroach/pull/135209 +[#135216]: https://github.com/cockroachdb/cockroach/pull/135216 +[#135587]: https://github.com/cockroachdb/cockroach/pull/135587 +[#135648]: https://github.com/cockroachdb/cockroach/pull/135648 +[#135764]: https://github.com/cockroachdb/cockroach/pull/135764 +[#135928]: https://github.com/cockroachdb/cockroach/pull/135928 +[#136011]: https://github.com/cockroachdb/cockroach/pull/136011 +[#136227]: https://github.com/cockroachdb/cockroach/pull/136227 +[#136481]: https://github.com/cockroachdb/cockroach/pull/136481 +[0d7f6eed3]: https://github.com/cockroachdb/cockroach/commit/0d7f6eed3 +[1f2b1b084]: https://github.com/cockroachdb/cockroach/commit/1f2b1b084 +[3cbd07fbd]: https://github.com/cockroachdb/cockroach/commit/3cbd07fbd +[3f5305a4c]: https://github.com/cockroachdb/cockroach/commit/3f5305a4c +[965dded2a]: https://github.com/cockroachdb/cockroach/commit/965dded2a +[989a49c3f]: https://github.com/cockroachdb/cockroach/commit/989a49c3f +[9951e3e61]: https://github.com/cockroachdb/cockroach/commit/9951e3e61 From cc608abafa6d9c3c235a858913fd9895a9888adc Mon Sep 17 00:00:00 2001 From: Kathryn May <44557882+kathancox@users.noreply.github.com> Date: Fri, 13 Dec 2024 11:55:43 -0500 Subject: [PATCH 17/36] Remove KL for changefeed schedule on different cluster (#19190) --- .../known-limitations/pcr-scheduled-changefeeds.md | 1 - .../known-limitations/pcr-scheduled-changefeeds.md | 1 - src/current/v24.2/create-and-configure-changefeeds.md | 1 - src/current/v24.2/create-schedule-for-changefeed.md | 4 ++++ src/current/v24.2/cutover-replication.md | 10 +++++++--- src/current/v24.2/known-limitations.md | 2 -- .../v24.2/physical-cluster-replication-overview.md | 1 - src/current/v24.3/create-and-configure-changefeeds.md | 1 - src/current/v24.3/create-schedule-for-changefeed.md | 4 ++++ src/current/v24.3/failover-replication.md | 10 +++++++--- src/current/v24.3/known-limitations.md | 2 -- .../v24.3/physical-cluster-replication-overview.md | 1 - 12 files changed, 22 insertions(+), 16 deletions(-) delete mode 100644 src/current/_includes/v24.2/known-limitations/pcr-scheduled-changefeeds.md delete mode 100644 src/current/_includes/v24.3/known-limitations/pcr-scheduled-changefeeds.md diff --git a/src/current/_includes/v24.2/known-limitations/pcr-scheduled-changefeeds.md b/src/current/_includes/v24.2/known-limitations/pcr-scheduled-changefeeds.md deleted file mode 100644 index 31fbf83187c..00000000000 --- a/src/current/_includes/v24.2/known-limitations/pcr-scheduled-changefeeds.md +++ /dev/null @@ -1 +0,0 @@ -After the [cutover process]({% link {{ page.version.version }}/cutover-replication.md %}) for [physical cluster replication]({% link {{ page.version.version }}/physical-cluster-replication-overview.md %}), [scheduled changefeeds]({% link {{ page.version.version }}/create-schedule-for-changefeed.md %}) will continue on the promoted cluster. You will need to manage [pausing]({% link {{ page.version.version }}/pause-schedules.md %}) or [canceling]({% link {{ page.version.version }}/drop-schedules.md %}) the schedule on the promoted standby cluster to avoid two clusters running the same changefeed to one sink. [#123776](https://github.com/cockroachdb/cockroach/issues/123776) \ No newline at end of file diff --git a/src/current/_includes/v24.3/known-limitations/pcr-scheduled-changefeeds.md b/src/current/_includes/v24.3/known-limitations/pcr-scheduled-changefeeds.md deleted file mode 100644 index 3d6b8aa8628..00000000000 --- a/src/current/_includes/v24.3/known-limitations/pcr-scheduled-changefeeds.md +++ /dev/null @@ -1 +0,0 @@ -After the [failover process]({% link {{ page.version.version }}/failover-replication.md %}) for [physical cluster replication]({% link {{ page.version.version }}/physical-cluster-replication-overview.md %}), [scheduled changefeeds]({% link {{ page.version.version }}/create-schedule-for-changefeed.md %}) will continue on the promoted cluster. You will need to manage [pausing]({% link {{ page.version.version }}/pause-schedules.md %}) or [canceling]({% link {{ page.version.version }}/drop-schedules.md %}) the schedule on the promoted standby cluster to avoid two clusters running the same changefeed to one sink. [#123776](https://github.com/cockroachdb/cockroach/issues/123776) \ No newline at end of file diff --git a/src/current/v24.2/create-and-configure-changefeeds.md b/src/current/v24.2/create-and-configure-changefeeds.md index 69c3a1b8e47..c2815931249 100644 --- a/src/current/v24.2/create-and-configure-changefeeds.md +++ b/src/current/v24.2/create-and-configure-changefeeds.md @@ -186,7 +186,6 @@ For more information, see [`EXPERIMENTAL CHANGEFEED FOR`]({% link {{ page.versio ## Known limitations {% include {{ page.version.version }}/known-limitations/cdc.md %} -- {% include {{ page.version.version }}/known-limitations/pcr-scheduled-changefeeds.md %} - {% include {{ page.version.version }}/known-limitations/alter-changefeed-cdc-queries.md %} - {% include {{ page.version.version }}/known-limitations/cdc-queries-column-families.md %} - {% include {{ page.version.version }}/known-limitations/changefeed-column-family-message.md %} diff --git a/src/current/v24.2/create-schedule-for-changefeed.md b/src/current/v24.2/create-schedule-for-changefeed.md index 6902e2cc135..6c3730172dc 100644 --- a/src/current/v24.2/create-schedule-for-changefeed.md +++ b/src/current/v24.2/create-schedule-for-changefeed.md @@ -58,6 +58,10 @@ Option | Value | Description `on_execution_failure` | `retry` / `reschedule` / `pause` | Determine how the schedule handles an error.

`retry`: Retry the changefeed immediately.

`reschedule`: Reschedule the changefeed based on the `RECURRING` expression.

`pause`: Pause the schedule. This requires that you [resume the schedule]({% link {{ page.version.version }}/resume-schedules.md %}) manually.

**Default:** `reschedule` `on_previous_running` | `start` / `skip` / `wait` | Control whether the changefeed schedule should start a changefeed if the previous scheduled changefeed is still running.

`start`: Start the new changefeed anyway, even if the previous one is running.

`skip`: Skip the new changefeed and run the next changefeed based on the `RECURRING` expression.

`wait`: Wait for the previous changefeed to complete.

**Default:** `wait` +{{site.data.alerts.callout_info}} +{% include_cached new-in.html version="v24.2" %} To avoid multiple clusters running the same schedule concurrently, changefeed schedules will [pause]({% link {{ page.version.version }}/pause-schedules.md %}) when [restored]({% link {{ page.version.version }}/restore.md %}) onto a different cluster or after [physical cluster replication]({% link {{ page.version.version }}/cutover-replication.md %}) has completed. +{{site.data.alerts.end}} + ## Examples Before running any of the examples in this section, it is necessary to enable the `kv.rangefeed.enabled` cluster setting. If you are working on a CockroachDB {{ site.data.products.standard }} or {{ site.data.products.basic }} cluster, this cluster setting is enabled by default. diff --git a/src/current/v24.2/cutover-replication.md b/src/current/v24.2/cutover-replication.md index 08a6b44e058..a70e693bf17 100644 --- a/src/current/v24.2/cutover-replication.md +++ b/src/current/v24.2/cutover-replication.md @@ -167,17 +167,21 @@ During a replication stream, jobs running on the primary cluster will replicate [Backup schedules]({% link {{ page.version.version }}/manage-a-backup-schedule.md %}) will pause after cutover on the promoted cluster. Take the following steps to resume jobs: 1. Verify that there are no other schedules running backups to the same [collection of backups]({% link {{ page.version.version }}/take-full-and-incremental-backups.md %}#backup-collections), i.e., the schedule that was running on the original primary cluster. -1. Resume the backup schedule on the promoted cluster. +1. [Resume]({% link {{ page.version.version }}/resume-schedules.md %}) the backup schedule on the promoted cluster. {{site.data.alerts.callout_info}} -If your backup schedule was created on a cluster in v23.1 or earlier, it will **not** pause automatically on the promoted cluster after cutover. In this case, you must pause the schedule manually on the promoted cluster and then take the outlined steps. +If your backup schedule was created on a cluster in v23.1 or earlier, it will **not** pause automatically on the promoted cluster after cutover. In this case, you must [pause]({% link {{ page.version.version }}/pause-schedules.md %}) the schedule manually on the promoted cluster and then take the outlined steps. {{site.data.alerts.end}} ### Changefeeds [Changefeeds]({% link {{ page.version.version }}/change-data-capture-overview.md %}) will fail on the promoted cluster immediately after cutover to avoid two clusters running the same changefeed to one sink. We recommend that you recreate changefeeds on the promoted cluster. -[Scheduled changefeeds]({% link {{ page.version.version }}/create-schedule-for-changefeed.md %}) will continue on the promoted cluster. You will need to manage [pausing]({% link {{ page.version.version }}/pause-schedules.md %}) or [canceling]({% link {{ page.version.version }}/drop-schedules.md %}) the schedule on the promoted standby cluster to avoid two clusters running the same changefeed to one sink. +{% include_cached new-in.html version="v24.2" %} To avoid multiple clusters running the same schedule concurrently, [changefeed schedules]({% link {{ page.version.version }}/create-schedule-for-changefeed.md %}) will [pause]({% link {{ page.version.version }}/pause-schedules.md %}) after physical cluster replication has completed. + +{{site.data.alerts.callout_info}} +If your changefeed schedule was created on a cluster in v24.1 or earlier, it will **not** pause automatically on the promoted cluster after cutover. In this case, you will need to manage [pausing]({% link {{ page.version.version }}/pause-schedules.md %}) or [canceling]({% link {{ page.version.version }}/drop-schedules.md %}) the schedule on the promoted standby cluster to avoid two clusters running the same changefeed to one sink. +{{site.data.alerts.end}} ## Cut back to the primary cluster diff --git a/src/current/v24.2/known-limitations.md b/src/current/v24.2/known-limitations.md index d7dccc618a5..b6c00e689f2 100644 --- a/src/current/v24.2/known-limitations.md +++ b/src/current/v24.2/known-limitations.md @@ -454,7 +454,6 @@ Accessing the DB Console for a secure cluster now requires login information (i. #### Physical cluster replication {% include {{ page.version.version }}/known-limitations/physical-cluster-replication.md %} -- {% include {{ page.version.version }}/known-limitations/pcr-scheduled-changefeeds.md %} - {% include {{ page.version.version }}/known-limitations/cutover-stop-application.md %} #### `RESTORE` limitations @@ -478,7 +477,6 @@ As a workaround, take a cluster backup instead, as the `system.comments` table i Change data capture (CDC) provides efficient, distributed, row-level changefeeds into Apache Kafka for downstream processing such as reporting, caching, or full-text indexing. It has the following known limitations: {% include {{ page.version.version }}/known-limitations/cdc.md %} -- {% include {{ page.version.version }}/known-limitations/pcr-scheduled-changefeeds.md %} {% include {{ page.version.version }}/known-limitations/cdc-queries.md %} - {% include {{ page.version.version }}/known-limitations/cdc-queries-column-families.md %} - {% include {{ page.version.version }}/known-limitations/changefeed-column-family-message.md %} diff --git a/src/current/v24.2/physical-cluster-replication-overview.md b/src/current/v24.2/physical-cluster-replication-overview.md index c0b699faf1d..41259a7b312 100644 --- a/src/current/v24.2/physical-cluster-replication-overview.md +++ b/src/current/v24.2/physical-cluster-replication-overview.md @@ -35,7 +35,6 @@ You can use PCR in a disaster recovery plan to: ## Known limitations {% include {{ page.version.version }}/known-limitations/physical-cluster-replication.md %} -- {% include {{ page.version.version }}/known-limitations/pcr-scheduled-changefeeds.md %} - {% include {{ page.version.version }}/known-limitations/cutover-stop-application.md %} ## Performance diff --git a/src/current/v24.3/create-and-configure-changefeeds.md b/src/current/v24.3/create-and-configure-changefeeds.md index 112f2a32410..7ea21b2ece6 100644 --- a/src/current/v24.3/create-and-configure-changefeeds.md +++ b/src/current/v24.3/create-and-configure-changefeeds.md @@ -164,7 +164,6 @@ For more information, see [`EXPERIMENTAL CHANGEFEED FOR`]({% link {{ page.versio ## Known limitations {% include {{ page.version.version }}/known-limitations/cdc.md %} -- {% include {{ page.version.version }}/known-limitations/pcr-scheduled-changefeeds.md %} - {% include {{ page.version.version }}/known-limitations/alter-changefeed-cdc-queries.md %} - {% include {{ page.version.version }}/known-limitations/cdc-queries-column-families.md %} - {% include {{ page.version.version }}/known-limitations/changefeed-column-family-message.md %} diff --git a/src/current/v24.3/create-schedule-for-changefeed.md b/src/current/v24.3/create-schedule-for-changefeed.md index 6902e2cc135..5822569b0dc 100644 --- a/src/current/v24.3/create-schedule-for-changefeed.md +++ b/src/current/v24.3/create-schedule-for-changefeed.md @@ -58,6 +58,10 @@ Option | Value | Description `on_execution_failure` | `retry` / `reschedule` / `pause` | Determine how the schedule handles an error.

`retry`: Retry the changefeed immediately.

`reschedule`: Reschedule the changefeed based on the `RECURRING` expression.

`pause`: Pause the schedule. This requires that you [resume the schedule]({% link {{ page.version.version }}/resume-schedules.md %}) manually.

**Default:** `reschedule` `on_previous_running` | `start` / `skip` / `wait` | Control whether the changefeed schedule should start a changefeed if the previous scheduled changefeed is still running.

`start`: Start the new changefeed anyway, even if the previous one is running.

`skip`: Skip the new changefeed and run the next changefeed based on the `RECURRING` expression.

`wait`: Wait for the previous changefeed to complete.

**Default:** `wait` +{{site.data.alerts.callout_info}} +To avoid multiple clusters running the same schedule concurrently, changefeed schedules will [pause]({% link {{ page.version.version }}/pause-schedules.md %}) when [restored]({% link {{ page.version.version }}/restore.md %}) onto a different cluster or after [physical cluster replication]({% link {{ page.version.version }}/failover-replication.md %}) has completed. +{{site.data.alerts.end}} + ## Examples Before running any of the examples in this section, it is necessary to enable the `kv.rangefeed.enabled` cluster setting. If you are working on a CockroachDB {{ site.data.products.standard }} or {{ site.data.products.basic }} cluster, this cluster setting is enabled by default. diff --git a/src/current/v24.3/failover-replication.md b/src/current/v24.3/failover-replication.md index acce31d45c8..394d234fe29 100644 --- a/src/current/v24.3/failover-replication.md +++ b/src/current/v24.3/failover-replication.md @@ -167,17 +167,21 @@ During a replication stream, jobs running on the primary cluster will replicate [Backup schedules]({% link {{ page.version.version }}/manage-a-backup-schedule.md %}) will pause after failover on the promoted cluster. Take the following steps to resume jobs: 1. Verify that there are no other schedules running backups to the same [collection of backups]({% link {{ page.version.version }}/take-full-and-incremental-backups.md %}#backup-collections), i.e., the schedule that was running on the original primary cluster. -1. Resume the backup schedule on the promoted cluster. +1. [Resume]({% link {{ page.version.version }}/resume-schedules.md %}) the backup schedule on the promoted cluster. {{site.data.alerts.callout_info}} -If your backup schedule was created on a cluster in v23.1 or earlier, it will **not** pause automatically on the promoted cluster after failover. In this case, you must pause the schedule manually on the promoted cluster and then take the outlined steps. +If your backup schedule was created on a cluster in v23.1 or earlier, it will **not** pause automatically on the promoted cluster after failover. In this case, you must [pause]({% link {{ page.version.version }}/pause-schedules.md %}) the schedule manually on the promoted cluster and then take the outlined steps. {{site.data.alerts.end}} ### Changefeeds [Changefeeds]({% link {{ page.version.version }}/change-data-capture-overview.md %}) will fail on the promoted cluster immediately after failover to avoid two clusters running the same changefeed to one sink. We recommend that you recreate changefeeds on the promoted cluster. -[Scheduled changefeeds]({% link {{ page.version.version }}/create-schedule-for-changefeed.md %}) will continue on the promoted cluster. You will need to manage [pausing]({% link {{ page.version.version }}/pause-schedules.md %}) or [canceling]({% link {{ page.version.version }}/drop-schedules.md %}) the schedule on the promoted standby cluster to avoid two clusters running the same changefeed to one sink. +To avoid multiple clusters running the same schedule concurrently, [changefeed schedules]({% link {{ page.version.version }}/create-schedule-for-changefeed.md %}) will [pause]({% link {{ page.version.version }}/pause-schedules.md %}) after physical cluster replication has completed. + +{{site.data.alerts.callout_info}} +If your changefeed schedule was created on a cluster in v24.1 or earlier, it will **not** pause automatically on the promoted cluster after failover. In this case, you will need to manage [pausing]({% link {{ page.version.version }}/pause-schedules.md %}) or [canceling]({% link {{ page.version.version }}/drop-schedules.md %}) the schedule on the promoted standby cluster to avoid two clusters running the same changefeed to one sink. +{{site.data.alerts.end}} ## Fail back to the primary cluster diff --git a/src/current/v24.3/known-limitations.md b/src/current/v24.3/known-limitations.md index 792ba38c91f..184ff766a49 100644 --- a/src/current/v24.3/known-limitations.md +++ b/src/current/v24.3/known-limitations.md @@ -465,7 +465,6 @@ Accessing the DB Console for a secure cluster now requires login information (i. #### Physical cluster replication {% include {{ page.version.version }}/known-limitations/physical-cluster-replication.md %} -- {% include {{ page.version.version }}/known-limitations/pcr-scheduled-changefeeds.md %} - {% include {{ page.version.version }}/known-limitations/failover-stop-application.md %} #### `RESTORE` limitations @@ -489,7 +488,6 @@ As a workaround, take a cluster backup instead, as the `system.comments` table i Change data capture (CDC) provides efficient, distributed, row-level changefeeds into Apache Kafka for downstream processing such as reporting, caching, or full-text indexing. It has the following known limitations: {% include {{ page.version.version }}/known-limitations/cdc.md %} -- {% include {{ page.version.version }}/known-limitations/pcr-scheduled-changefeeds.md %} {% include {{ page.version.version }}/known-limitations/cdc-queries.md %} - {% include {{ page.version.version }}/known-limitations/cdc-queries-column-families.md %} - {% include {{ page.version.version }}/known-limitations/changefeed-column-family-message.md %} diff --git a/src/current/v24.3/physical-cluster-replication-overview.md b/src/current/v24.3/physical-cluster-replication-overview.md index 8ad79e3d6d0..d0e4f831a4f 100644 --- a/src/current/v24.3/physical-cluster-replication-overview.md +++ b/src/current/v24.3/physical-cluster-replication-overview.md @@ -39,7 +39,6 @@ You can use PCR in a disaster recovery plan to: ## Known limitations {% include {{ page.version.version }}/known-limitations/physical-cluster-replication.md %} -- {% include {{ page.version.version }}/known-limitations/pcr-scheduled-changefeeds.md %} - {% include {{ page.version.version }}/known-limitations/failover-stop-application.md %} ## Performance From 61507169bf2a3de3380fd3461e4f6d8298cf5a0e Mon Sep 17 00:00:00 2001 From: Kathryn May <44557882+kathancox@users.noreply.github.com> Date: Fri, 13 Dec 2024 12:03:08 -0500 Subject: [PATCH 18/36] Update descriptions of modes and ttl deletes option for LDR (#19198) --- .../_includes/v24.3/ldr/immediate-description.md | 1 + .../_includes/v24.3/ldr/validated-description.md | 1 + src/current/v24.3/create-logical-replication-stream.md | 10 +++++----- src/current/v24.3/manage-logical-data-replication.md | 4 ++-- src/current/v24.3/set-up-logical-data-replication.md | 8 ++------ 5 files changed, 11 insertions(+), 13 deletions(-) create mode 100644 src/current/_includes/v24.3/ldr/immediate-description.md create mode 100644 src/current/_includes/v24.3/ldr/validated-description.md diff --git a/src/current/_includes/v24.3/ldr/immediate-description.md b/src/current/_includes/v24.3/ldr/immediate-description.md new file mode 100644 index 00000000000..eb87361a009 --- /dev/null +++ b/src/current/_includes/v24.3/ldr/immediate-description.md @@ -0,0 +1 @@ +Attempts to replicate the changed row directly into the destination table, without re-running constraint validations. It does not support writing into tables with [foreign key]({% link {{ page.version.version }}/foreign-key.md %}) constraints. \ No newline at end of file diff --git a/src/current/_includes/v24.3/ldr/validated-description.md b/src/current/_includes/v24.3/ldr/validated-description.md new file mode 100644 index 00000000000..4b7bb9a8b18 --- /dev/null +++ b/src/current/_includes/v24.3/ldr/validated-description.md @@ -0,0 +1 @@ +Attempts to apply the write in a similar way to a user-run query, which would re-run all constraint validations relevant to the destination table(s). If the change violates foreign key dependencies, unique constraints, or other constraints, the row will be put in the [dead letter queue (DLQ)]({% link {{ page.version.version }}/manage-logical-data-replication.md %}#dead-letter-queue-dlq) instead. Like the [SQL layer]({% link {{ page.version.version }}/architecture/sql-layer.md %}), `validated` mode does not recognize deletion tombstones. As a result, an update to the same key from cluster A will successfully apply on cluster B, even if that key was deleted on cluster B before the LDR job streamed the cluster A update to the key. \ No newline at end of file diff --git a/src/current/v24.3/create-logical-replication-stream.md b/src/current/v24.3/create-logical-replication-stream.md index f09afad5917..8c3832678e5 100644 --- a/src/current/v24.3/create-logical-replication-stream.md +++ b/src/current/v24.3/create-logical-replication-stream.md @@ -48,7 +48,7 @@ Parameter | Description Option | Description -------+------------ `cursor` | Emits any changes after the specified timestamp. LDR will not perform an initial backfill with the `cursor` option, it will stream any changes after the specified timestamp. The LDR job will encounter an error if you specify a `cursor` timestamp that is before the configured [garbage collection]({% link {{ page.version.version }}/architecture/storage-layer.md %}#garbage-collection) window for that table. **Warning:** Apply the `cursor` option carefully to LDR streams. Using a timestamp in error could cause data loss. -`discard` | Ignore [TTL deletes]({% link {{ page.version.version }}/row-level-ttl.md %}) in an LDR stream. Use `discard = ttl-deletes`. **Note**: To ignore row-level TTL deletes in an LDR stream, it is necessary to set the [`ttl_disable_changefeed_replication`]({% link {{ page.version.version }}/row-level-ttl.md %}#ttl-storage-parameters) storage parameter on the source table. Refer to the [Ignore row-level TTL deletes](#ignore-row-level-ttl-deletes) example. +`discard` | (**Unidirectional LDR only**) Ignore [TTL deletes]({% link {{ page.version.version }}/row-level-ttl.md %}) in an LDR stream with `discard = ttl-deletes`. **Note**: To ignore row-level TTL deletes in an LDR stream, it is necessary to set the [`ttl_disable_changefeed_replication`]({% link {{ page.version.version }}/row-level-ttl.md %}#ttl-storage-parameters) storage parameter on the source table. Refer to the [Ignore row-level TTL deletes](#ignore-row-level-ttl-deletes) example. `label` | Tracks LDR metrics at the job level. Add a user-specified string with `label`. Refer to [Metrics labels]({% link {{ page.version.version }}/logical-data-replication-monitoring.md %}#metrics-labels). `mode` | Determines how LDR replicates the data to the destination cluster. Possible values: `immediate`, `validated`. For more details refer to [LDR modes](#ldr-modes). @@ -56,8 +56,8 @@ Option | Description _Modes_ determine how LDR replicates the data to the destination cluster. There are two modes: -- `immediate` (default): Attempts to replicate the changed row directly into the destination table, without re-running constraint validations. It does not support writing into tables with [foreign key]({% link {{ page.version.version }}/foreign-key.md %}) constraints. -- `validated`: Attempts to apply the write in a similar way to a user-run query, which would re-run all constraint validations or triggers relevant to the destination table(s). It will potentially reject the change if it fails rather than writing it, which will cause the row change to enter the DLQ instead. +- `immediate` (default): {% include {{ page.version.version }}/ldr/immediate-description.md %} +- `validated`: {% include {{ page.version.version }}/ldr/validated-description.md %} ## Bidirectional LDR @@ -89,14 +89,14 @@ CREATE LOGICAL REPLICATION STREAM FROM TABLES ({database.public.table_name},{dat ### Ignore row-level TTL deletes -If you would like to ignore [row-level TTL]({% link {{ page.version.version }}/row-level-ttl.md %}) deletes in the LDR stream, set the [`ttl_disable_changefeed_replication` storage parameter]({% link {{ page.version.version }}/row-level-ttl.md %}#ttl-storage-parameters) on the table. On the **source** cluster, alter the table to set the table storage parameter: +If you would like to ignore [row-level TTL]({% link {{ page.version.version }}/row-level-ttl.md %}) deletes in a **unidirectional** LDR stream, set the [`ttl_disable_changefeed_replication` storage parameter]({% link {{ page.version.version }}/row-level-ttl.md %}#ttl-storage-parameters) on the table. On the **source** cluster, alter the table to set the table storage parameter: {% include_cached copy-clipboard.html %} ~~~ sql ALTER TABLE {table_name} SET (ttl_disable_changefeed_replication = 'true'); ~~~ -When you start LDR on the **destination cluster**, include the `discard = ttl-deletes` option in the statement: +When you start LDR on the **destination cluster**, include the [`discard = ttl-deletes` option](#discard-ttl-deletes-option) in the statement: {% include_cached copy-clipboard.html %} ~~~ sql diff --git a/src/current/v24.3/manage-logical-data-replication.md b/src/current/v24.3/manage-logical-data-replication.md index 6e9b47fa1cf..eb43b0a1932 100644 --- a/src/current/v24.3/manage-logical-data-replication.md +++ b/src/current/v24.3/manage-logical-data-replication.md @@ -18,11 +18,11 @@ Once you have **logical data replication (LDR)** running, you will need to track ## Conflict resolution -Conflict resolution in LDR differs depending on whether the conflict occurs at the [KV]({% link {{ page.version.version }}/architecture/storage-layer.md %}) level or the [SQL]({% link {{ page.version.version }}/architecture/sql-layer.md %}) level. +In LDR, conflicts are detected at both the [KV]({% link {{ page.version.version }}/architecture/storage-layer.md %}) and the [SQL]({% link {{ page.version.version }}/architecture/sql-layer.md %}) level, which determines how LDR resolves the conflict. ### KV level conflicts -LDR uses _last write wins (LWW)_ conflict resolution based on the [MVCC timestamp]({% link {{ page.version.version }}/architecture/storage-layer.md %}#mvcc) of the replicating write. LDR will resolve conflicts by inserting the row with the latest MVCC timestamp. +LDR uses _last write wins (LWW)_ conflict resolution based on the [MVCC timestamp]({% link {{ page.version.version }}/architecture/storage-layer.md %}#mvcc) of the replicating write. LDR will resolve conflicts by inserting the row with the latest MVCC timestamp. Conflicts at the KV level are detected in both `immediate` and `validated` mode. Conflicts at the KV level are detected when there is either: diff --git a/src/current/v24.3/set-up-logical-data-replication.md b/src/current/v24.3/set-up-logical-data-replication.md index 9dfcf183ab2..be66a1ee062 100644 --- a/src/current/v24.3/set-up-logical-data-replication.md +++ b/src/current/v24.3/set-up-logical-data-replication.md @@ -119,12 +119,8 @@ In this step, you'll start the LDR job from the destination cluster. You can rep _Modes_ determine how LDR replicates the data to the destination cluster. There are two modes: -- `immediate` (default): Attempts to replicate the changed row directly into the destination table, without re-running constraint validations. It does not support writing into tables with [foreign key]({% link {{ page.version.version }}/foreign-key.md %}) constraints. -- `validated`: Attempts to apply the write in a similar way to a user-run query, which would re-run all constraint validations relevant to the destination table(s). If the change violates foreign key dependencies, unique constraints, or other constraints, the row will be put in the [dead letter queue (DLQ)]({% link {{ page.version.version }}/manage-logical-data-replication.md %}#dead-letter-queue-dlq) instead. - -{{site.data.alerts.callout_info}} -If you would like to ignore TTL deletes in LDR, you can use the `discard = ttl-deletes` option in the `CREATE LOGICAL REPLICATION STREAM` statement. For an example, refer to [Ignore row-level TTL deletes]({% link {{ page.version.version }}/create-logical-replication-stream.md %}#ignore-row-level-ttl-deletes). -{{site.data.alerts.end}} +- `immediate` (default): {% include {{ page.version.version }}/ldr/immediate-description.md %} +- `validated`: {% include {{ page.version.version }}/ldr/validated-description.md %} 1. From the **destination** cluster, start LDR. Use the fully qualified table name for the source and destination tables: From a5d0b13cd6ec8bec428770d343a6cbad92945933 Mon Sep 17 00:00:00 2001 From: Kathryn May <44557882+kathancox@users.noreply.github.com> Date: Fri, 13 Dec 2024 12:11:21 -0500 Subject: [PATCH 19/36] Update the PCR mr known limitation and remove read on standby KL (#19199) --- .../v23.2/known-limitations/physical-cluster-replication.md | 2 +- .../v24.1/known-limitations/physical-cluster-replication.md | 2 +- .../v24.2/known-limitations/physical-cluster-replication.md | 2 +- .../v24.3/known-limitations/physical-cluster-replication.md | 3 +-- src/current/v24.3/physical-cluster-replication-overview.md | 3 ++- 5 files changed, 6 insertions(+), 6 deletions(-) diff --git a/src/current/_includes/v23.2/known-limitations/physical-cluster-replication.md b/src/current/_includes/v23.2/known-limitations/physical-cluster-replication.md index 0a9fc624cf2..7d0d79a79b5 100644 --- a/src/current/_includes/v23.2/known-limitations/physical-cluster-replication.md +++ b/src/current/_includes/v23.2/known-limitations/physical-cluster-replication.md @@ -1,6 +1,6 @@ - Physical cluster replication is supported only on CockroachDB {{ site.data.products.core }} in new v23.2 clusters. Physical Cluster Replication cannot be enabled on clusters that have been upgraded from a previous version of CockroachDB. - Read queries are not supported on the standby cluster before [cutover]({% link {{ page.version.version }}/cutover-replication.md %}). -- The primary and standby cluster **cannot have different [region topology]({% link {{ page.version.version }}/topology-patterns.md %})**. For example, replicating a multi-region primary cluster to a single-region standby cluster is not supported. Mismatching regions between a multi-region primary and standby cluster is also not supported. +- The primary and standby clusters must have the same [zone configurations]({% link {{ page.version.version }}/configure-replication-zones.md %}). - Cutting back to the primary cluster after a cutover is a manual process. Refer to [Cut back to the primary cluster]({% link {{ page.version.version }}/cutover-replication.md %}#cut-back-to-the-primary-cluster). In addition, after cutover, to continue using physical cluster replication, you must configure it again. - Before cutover to the standby, the standby cluster does not support running [backups]({% link {{ page.version.version }}/backup-and-restore-overview.md %}) or [changefeeds]({% link {{ page.version.version }}/change-data-capture-overview.md %}). - After a cutover, there is no mechanism to stop applications from connecting to the original primary cluster. It is necessary to redirect application traffic manually, such as by using a network load balancer or adjusting DNS records. diff --git a/src/current/_includes/v24.1/known-limitations/physical-cluster-replication.md b/src/current/_includes/v24.1/known-limitations/physical-cluster-replication.md index f914c7eced2..a6c7edf2f32 100644 --- a/src/current/_includes/v24.1/known-limitations/physical-cluster-replication.md +++ b/src/current/_includes/v24.1/known-limitations/physical-cluster-replication.md @@ -1,6 +1,6 @@ - Physical cluster replication is supported only on CockroachDB {{ site.data.products.core }} in new clusters on v23.2 or above. Physical Cluster Replication cannot be enabled on clusters that have been upgraded from a previous version of CockroachDB. - Read queries are not supported on the standby cluster before [cutover]({% link {{ page.version.version }}/cutover-replication.md %}). -- The primary and standby cluster **cannot have different [region topology]({% link {{ page.version.version }}/topology-patterns.md %})**. For example, replicating a multi-region primary cluster to a single-region standby cluster is not supported. Mismatching regions between a multi-region primary and standby cluster is also not supported. +- The primary and standby clusters must have the same [zone configurations]({% link {{ page.version.version }}/configure-replication-zones.md %}). - Cutting back to the primary cluster after a cutover is a manual process. Refer to [Cut back to the primary cluster]({% link {{ page.version.version }}/cutover-replication.md %}#cut-back-to-the-primary-cluster). In addition, after cutover, to continue using physical cluster replication, you must configure it again. - Before cutover to the standby, the standby cluster does not support running [backups]({% link {{ page.version.version }}/backup-and-restore-overview.md %}) or [changefeeds]({% link {{ page.version.version }}/change-data-capture-overview.md %}). - Large data imports, such as those produced by [`RESTORE`]({% link {{ page.version.version }}/restore.md %}) or [`IMPORT INTO`]({% link {{ page.version.version }}/import-into.md %}), may dramatically increase [replication lag]({% link {{ page.version.version }}/physical-cluster-replication-technical-overview.md %}#cutover-and-promotion-process). diff --git a/src/current/_includes/v24.2/known-limitations/physical-cluster-replication.md b/src/current/_includes/v24.2/known-limitations/physical-cluster-replication.md index f914c7eced2..a6c7edf2f32 100644 --- a/src/current/_includes/v24.2/known-limitations/physical-cluster-replication.md +++ b/src/current/_includes/v24.2/known-limitations/physical-cluster-replication.md @@ -1,6 +1,6 @@ - Physical cluster replication is supported only on CockroachDB {{ site.data.products.core }} in new clusters on v23.2 or above. Physical Cluster Replication cannot be enabled on clusters that have been upgraded from a previous version of CockroachDB. - Read queries are not supported on the standby cluster before [cutover]({% link {{ page.version.version }}/cutover-replication.md %}). -- The primary and standby cluster **cannot have different [region topology]({% link {{ page.version.version }}/topology-patterns.md %})**. For example, replicating a multi-region primary cluster to a single-region standby cluster is not supported. Mismatching regions between a multi-region primary and standby cluster is also not supported. +- The primary and standby clusters must have the same [zone configurations]({% link {{ page.version.version }}/configure-replication-zones.md %}). - Cutting back to the primary cluster after a cutover is a manual process. Refer to [Cut back to the primary cluster]({% link {{ page.version.version }}/cutover-replication.md %}#cut-back-to-the-primary-cluster). In addition, after cutover, to continue using physical cluster replication, you must configure it again. - Before cutover to the standby, the standby cluster does not support running [backups]({% link {{ page.version.version }}/backup-and-restore-overview.md %}) or [changefeeds]({% link {{ page.version.version }}/change-data-capture-overview.md %}). - Large data imports, such as those produced by [`RESTORE`]({% link {{ page.version.version }}/restore.md %}) or [`IMPORT INTO`]({% link {{ page.version.version }}/import-into.md %}), may dramatically increase [replication lag]({% link {{ page.version.version }}/physical-cluster-replication-technical-overview.md %}#cutover-and-promotion-process). diff --git a/src/current/_includes/v24.3/known-limitations/physical-cluster-replication.md b/src/current/_includes/v24.3/known-limitations/physical-cluster-replication.md index 1371e178a25..abd0fc2b10b 100644 --- a/src/current/_includes/v24.3/known-limitations/physical-cluster-replication.md +++ b/src/current/_includes/v24.3/known-limitations/physical-cluster-replication.md @@ -1,6 +1,5 @@ - Physical cluster replication is supported only on CockroachDB {{ site.data.products.core }} in new clusters on v23.2 or above. Physical Cluster Replication cannot be enabled on clusters that have been upgraded from a previous version of CockroachDB. -- Read queries are not supported on the standby cluster before [failover]({% link {{ page.version.version }}/failover-replication.md %}). -- The primary and standby cluster **cannot have different [region topology]({% link {{ page.version.version }}/topology-patterns.md %})**. For example, replicating a multi-region primary cluster to a single-region standby cluster is not supported. Mismatching regions between a multi-region primary and standby cluster is also not supported. +- The primary and standby clusters must have the same [zone configurations]({% link {{ page.version.version }}/configure-replication-zones.md %}). - Failing back to the primary cluster after a failover is a manual process. Refer to [Fail back to the primary cluster]({% link {{ page.version.version }}/failover-replication.md %}#fail-back-to-the-primary-cluster). In addition, after failover, to continue using physical cluster replication, you must configure it again. - Before failover to the standby, the standby cluster does not support running [backups]({% link {{ page.version.version }}/backup-and-restore-overview.md %}) or [changefeeds]({% link {{ page.version.version }}/change-data-capture-overview.md %}). - Large data imports, such as those produced by [`RESTORE`]({% link {{ page.version.version }}/restore.md %}) or [`IMPORT INTO`]({% link {{ page.version.version }}/import-into.md %}), may dramatically increase [replication lag]({% link {{ page.version.version }}/physical-cluster-replication-technical-overview.md %}#failover-and-promotion-process). diff --git a/src/current/v24.3/physical-cluster-replication-overview.md b/src/current/v24.3/physical-cluster-replication-overview.md index d0e4f831a4f..179281f9b57 100644 --- a/src/current/v24.3/physical-cluster-replication-overview.md +++ b/src/current/v24.3/physical-cluster-replication-overview.md @@ -30,7 +30,8 @@ You can use PCR in a disaster recovery plan to: - **Transactional consistency**: You can fail over to the standby cluster at the [`LATEST` timestamp]({% link {{ page.version.version }}/failover-replication.md %}#fail-over-to-the-most-recent-replicated-time) or a [point of time]({% link {{ page.version.version }}/failover-replication.md %}#fail-over-to-a-point-in-time) in the past or the future. When the failover process completes, the standby cluster will be in a transactionally consistent state as of the point in time you specified. - **Maintained/improved RPO and RTO**: Depending on workload and deployment configuration, [replication lag]({% link {{ page.version.version }}/physical-cluster-replication-technical-overview.md %}) between the primary and standby is generally in the tens-of-seconds range. The failover process from the primary cluster to the standby should typically happen within five minutes when completing a failover to the latest replicated time using `LATEST`. - **Failover to a timestamp in the past or the future**: In the case of logical disasters or mistakes, you can [fail over]({% link {{ page.version.version }}/failover-replication.md %}) from the primary to the standby cluster to a timestamp in the past. This means that you can return the standby to a timestamp before the mistake was replicated to the standby. You can also configure the [`WITH RETENTION`]({% link {{ page.version.version }}/alter-virtual-cluster.md %}#set-a-retention-window) option to control how far in the past you can fail over to. Furthermore, you can plan a failover by specifying a timestamp in the future. -- **Monitoring**: To monitor the replication's initial progress, current status, and performance, you can use metrics available in the [DB Console]({% link {{ page.version.version }}/ui-overview.md %}) and [Prometheus]({% link {{ page.version.version }}/monitor-cockroachdb-with-prometheus.md %}). For more detail, refer to [Physical Cluster Replication Monitoring]({% link {{ page.version.version }}/physical-cluster-replication-monitoring.md %}). +- {% include_cached new-in.html version="v24.3" %} **Read from standby cluster**: You can configure PCR to allow read queries on the standby cluster. For more details, refer to [Start a PCR stream with read from standby]({% link {{ page.version.version }}/create-virtual-cluster.md %}#start-a-pcr-stream-with-read-from-standby). +- **Monitoring**: To monitor the replication's initial progress, current status, and performance, you can use metrics available in the [DB Console]({% link {{ page.version.version }}/ui-overview.md %}) and [Prometheus]({% link {{ page.version.version }}/monitor-cockroachdb-with-prometheus.md %}). For more details, refer to [Physical Cluster Replication Monitoring]({% link {{ page.version.version }}/physical-cluster-replication-monitoring.md %}). {{site.data.alerts.callout_info}} [Failing over to a timestamp in the past]({% link {{ page.version.version }}/failover-replication.md %}#fail-over-to-a-point-in-time) involves reverting data on the standby cluster. As a result, this type of failover takes longer to complete than failover to the [latest replicated time]({% link {{ page.version.version }}/failover-replication.md %}#fail-over-to-the-most-recent-replicated-time). The increase in failover time will correlate to how much data you are reverting from the standby. For more detail, refer to the [Technical Overview]({% link {{ page.version.version }}/physical-cluster-replication-technical-overview.md %}) page for PCR. From 9f9d961c401ac4dd60a260f7ceb570d97bfcc603 Mon Sep 17 00:00:00 2001 From: shannonbradshaw Date: Fri, 13 Dec 2024 15:33:29 -0500 Subject: [PATCH 20/36] update release dates on release pages --- src/current/_includes/releases/v23.1/v23.1.29.md | 2 +- src/current/_includes/releases/v23.2/v23.2.16.md | 2 +- src/current/_includes/releases/v24.1/v24.1.7.md | 2 +- src/current/_includes/releases/v24.2/v24.2.5.md | 2 +- 4 files changed, 4 insertions(+), 4 deletions(-) diff --git a/src/current/_includes/releases/v23.1/v23.1.29.md b/src/current/_includes/releases/v23.1/v23.1.29.md index 00a857cf34c..a69b6cd42d9 100644 --- a/src/current/_includes/releases/v23.1/v23.1.29.md +++ b/src/current/_includes/releases/v23.1/v23.1.29.md @@ -1,6 +1,6 @@ ## v23.1.29 -Release Date: November 14, 2024 +Release Date: November 18, 2024 {% include releases/new-release-downloads-docker-image.md release=include.release %} diff --git a/src/current/_includes/releases/v23.2/v23.2.16.md b/src/current/_includes/releases/v23.2/v23.2.16.md index 5793f589357..6e205c889b3 100644 --- a/src/current/_includes/releases/v23.2/v23.2.16.md +++ b/src/current/_includes/releases/v23.2/v23.2.16.md @@ -1,6 +1,6 @@ ## v23.2.16 -Release Date: November 15, 2024 +Release Date: November 18, 2024 {% include releases/new-release-downloads-docker-image.md release=include.release %} diff --git a/src/current/_includes/releases/v24.1/v24.1.7.md b/src/current/_includes/releases/v24.1/v24.1.7.md index 9517b2ec9bb..f0d339f9c81 100644 --- a/src/current/_includes/releases/v24.1/v24.1.7.md +++ b/src/current/_includes/releases/v24.1/v24.1.7.md @@ -1,6 +1,6 @@ ## v24.1.7 -Release Date: November 14, 2024 +Release Date: November 18, 2024 {% include releases/new-release-downloads-docker-image.md release=include.release %} diff --git a/src/current/_includes/releases/v24.2/v24.2.5.md b/src/current/_includes/releases/v24.2/v24.2.5.md index 643d7a89006..844401e23df 100644 --- a/src/current/_includes/releases/v24.2/v24.2.5.md +++ b/src/current/_includes/releases/v24.2/v24.2.5.md @@ -1,6 +1,6 @@ ## v24.2.5 -Release Date: November 14, 2024 +Release Date: November 18, 2024 {% include releases/new-release-downloads-docker-image.md release=include.release %} From c516ab9f5cf5bf2b37f0a717e90cbbbe5d4fc394 Mon Sep 17 00:00:00 2001 From: shannonbradshaw Date: Fri, 13 Dec 2024 15:49:25 -0500 Subject: [PATCH 21/36] fix bullets --- src/current/_includes/v24.3/upgrade-requirements.md | 1 + 1 file changed, 1 insertion(+) diff --git a/src/current/_includes/v24.3/upgrade-requirements.md b/src/current/_includes/v24.3/upgrade-requirements.md index f1111c9393d..d729ee9c6ee 100644 --- a/src/current/_includes/v24.3/upgrade-requirements.md +++ b/src/current/_includes/v24.3/upgrade-requirements.md @@ -1,4 +1,5 @@ CockroachDB v24.3 is a Regular release. To upgrade to it, you must be running either: + - [v24.2]({% link v24.2/upgrade-cockroach-version.md %}), the previous Innovation release. - [v24.1]({% link v24.1/upgrade-cockroach-version.md %}), the previous Regular release. From e927b010f07ec87c3954735028c5e290310e03a9 Mon Sep 17 00:00:00 2001 From: Kathryn May <44557882+kathancox@users.noreply.github.com> Date: Fri, 13 Dec 2024 16:28:36 -0500 Subject: [PATCH 22/36] Correct rendering + managed backup mention on self-hosted page (#19236) --- .../common/upgrade/prepare-to-upgrade-self-hosted.md | 8 +++----- .../v24.3/backups/recommend-backups-for-upgrade.md | 5 +---- 2 files changed, 4 insertions(+), 9 deletions(-) diff --git a/src/current/_includes/common/upgrade/prepare-to-upgrade-self-hosted.md b/src/current/_includes/common/upgrade/prepare-to-upgrade-self-hosted.md index 5875f314b32..afd1fb5bf02 100644 --- a/src/current/_includes/common/upgrade/prepare-to-upgrade-self-hosted.md +++ b/src/current/_includes/common/upgrade/prepare-to-upgrade-self-hosted.md @@ -1,11 +1,9 @@ Before beginning a major-version or patch upgrade: 1. Verify the overall health of your cluster using the [DB Console]({% link {{ page.version.version }}/ui-cluster-overview-page.md %}): - - Under **Node Status**, make sure all nodes that should be live are listed as such. If any nodes are unexpectedly listed as `SUSPECT` or `DEAD`, identify why the nodes are offline and either restart them or [decommission]({% link {{ page.version.version }}/node-shutdown.md %}?filters=decommission#remove-nodes) them before beginning your upgrade. If there are `DEAD` and non-decommissioned nodes in your cluster, the upgrade cannot be finalized. - - If any node is not fully decommissioned, try the following: - 1. First, reissue the [decommission command]({% link {{ page.version.version }}/node-shutdown.md %}?filters=decommission#decommission-the-node). The second command typically succeeds within a few minutes. - 1. If the second decommission command does not succeed, [recommission]({% link {{ page.version.version }}/node-shutdown.md %}?filters=decommission#recommission-nodes) and then decommission it again. Before continuing the upgrade, the node must be marked as `decommissioned`. + - Under **Node Status**, make sure all nodes that should be live are listed as such. If any nodes are unexpectedly listed as `SUSPECT` or `DEAD`, identify why the nodes are offline and either restart them or [decommission]({% link {{ page.version.version }}/node-shutdown.md %}?filters=decommission#remove-nodes) them before beginning your upgrade. If there are `DEAD` and non-decommissioned nodes in your cluster, the upgrade cannot be finalized. If any node is not fully decommissioned, try the following: + 1. First, reissue the [decommission command]({% link {{ page.version.version }}/node-shutdown.md %}?filters=decommission#decommission-the-node). The second command typically succeeds within a few minutes. + 1. If the second decommission command does not succeed, [recommission]({% link {{ page.version.version }}/node-shutdown.md %}?filters=decommission#recommission-nodes) and then decommission it again. Before continuing the upgrade, the node must be marked as `decommissioned`. - Under **Replication Status**, make sure there are `0` under-replicated and unavailable ranges. Otherwise, performing a rolling upgrade increases the risk that ranges will lose a majority of their replicas and cause cluster unavailability. Therefore, it's important to identify and resolve the cause of range under-replication and/or unavailability before beginning your upgrade. - In the **Node List**, make sure all nodes are on the same version. Upgrade them to the cluster's current version before continuing. If any nodes are behind, this also indicates that the previous major-version upgrade may not be finalized. - In the **Metrics** dashboards, make sure [CPU]({% link {{ page.version.version }}/common-issues-to-monitor.md %}#cpu-usage), [memory]({% link {{ page.version.version }}/common-issues-to-monitor.md %}#database-memory-usage), and [storage]({% link {{ page.version.version }}/common-issues-to-monitor.md %}#storage-capacity) capacity are within acceptable values for each node. Nodes must be able to tolerate some increase in case the new version uses more resources for your workload. If any of these metrics is above healthy limits, consider [adding nodes]({% link {{ page.version.version }}/cockroach-start.md %}) to your cluster before beginning your upgrade. diff --git a/src/current/_includes/v24.3/backups/recommend-backups-for-upgrade.md b/src/current/_includes/v24.3/backups/recommend-backups-for-upgrade.md index 375f4489914..2ef075abf9c 100644 --- a/src/current/_includes/v24.3/backups/recommend-backups-for-upgrade.md +++ b/src/current/_includes/v24.3/backups/recommend-backups-for-upgrade.md @@ -3,7 +3,4 @@ When upgrading to a major release, you can optionally [take a self-managed backup]({% link cockroachcloud/take-and-restore-self-managed-backups.md %}) of your cluster to your own cloud storage, as an extra layer of protection in case the upgrade leads to issues. {% else %} -CockroachDB is designed with high fault tolerance. However, taking regular backups of your data is an operational best practice for [disaster recovery]({% link {{ page.version.version }}/disaster-recovery-planning.md %}) planning. - -We recommend that you enable [managed backups]({% link cockroachcloud/managed-backups.md %}#managed-backup-settings) and confirm that the cluster is backed up before beginning a major-version upgrade. This provides an extra layer of protection in case the upgrade leads to issues. -{% endif %} +CockroachDB is designed with high fault tolerance. However, taking regular backups of your data is an operational best practice for [disaster recovery]({% link {{ page.version.version }}/disaster-recovery-planning.md %}) planning.{% endif %} From a8e85b3dd4e41205ebcc924a54a552cd9aef4575 Mon Sep 17 00:00:00 2001 From: Kathryn May <44557882+kathancox@users.noreply.github.com> Date: Mon, 16 Dec 2024 10:59:57 -0500 Subject: [PATCH 23/36] Change deprecation warning for old backup syntax to removal note in v24.3+ (#19192) --- src/current/_includes/v24.3/backups/backup-to-deprec.md | 7 ------- src/current/_includes/v24.3/backups/old-syntax-removed.md | 5 +++++ src/current/v24.3/backup.md | 8 ++------ src/current/v24.3/restore.md | 2 +- src/current/v24.3/show-backup.md | 6 ++---- src/current/v24.3/take-full-and-incremental-backups.md | 2 +- 6 files changed, 11 insertions(+), 19 deletions(-) delete mode 100644 src/current/_includes/v24.3/backups/backup-to-deprec.md create mode 100644 src/current/_includes/v24.3/backups/old-syntax-removed.md diff --git a/src/current/_includes/v24.3/backups/backup-to-deprec.md b/src/current/_includes/v24.3/backups/backup-to-deprec.md deleted file mode 100644 index 77a2b6cae18..00000000000 --- a/src/current/_includes/v24.3/backups/backup-to-deprec.md +++ /dev/null @@ -1,7 +0,0 @@ -{{site.data.alerts.callout_danger}} -The `BACKUP ... TO` and `RESTORE ... FROM` syntax is **deprecated** as of v22.1 and will be removed in a future release. - -We recommend using the `BACKUP ... INTO {collectionURI}` syntax, which creates or adds to a [backup collection]({% link {{ page.version.version }}/take-full-and-incremental-backups.md %}#backup-collections) in your storage location. For restoring backups, we recommend using `RESTORE FROM {backup} IN {collectionURI}` with `{backup}` being [`LATEST`]({% link {{ page.version.version }}/restore.md %}#restore-the-most-recent-full-or-incremental-backup) or a specific [subdirectory]({% link {{ page.version.version }}/restore.md %}#subdir-param). - -For guidance on the syntax for backups and restores, see the [`BACKUP`]({% link {{ page.version.version }}/backup.md %}#examples) and [`RESTORE`]({% link {{ page.version.version }}/restore.md %}#examples) examples. -{{site.data.alerts.end}} diff --git a/src/current/_includes/v24.3/backups/old-syntax-removed.md b/src/current/_includes/v24.3/backups/old-syntax-removed.md new file mode 100644 index 00000000000..7052fe0d3af --- /dev/null +++ b/src/current/_includes/v24.3/backups/old-syntax-removed.md @@ -0,0 +1,5 @@ +{{site.data.alerts.callout_danger}} +The `BACKUP ... TO` and `RESTORE ... FROM {storage_uri}` syntax has been removed from CockroachDB v24.3 and later. + +For details on the syntax to run `BACKUP` and `RESTORE`, refer to the {% if page.name == "backup.md" %} [backup](#examples) {% else %} [backup]({% link {{ page.version.version }}/backup.md %}#examples) {% endif %} and {% if page.name == "restore.md" %} [restore](#examples) {% else %} [restore]({% link {{ page.version.version }}/restore.md %}#examples) {% endif %} examples. +{{site.data.alerts.end}} \ No newline at end of file diff --git a/src/current/v24.3/backup.md b/src/current/v24.3/backup.md index b5b6f8cceef..7d5b004a881 100644 --- a/src/current/v24.3/backup.md +++ b/src/current/v24.3/backup.md @@ -28,7 +28,7 @@ To view the contents of an backup created with the `BACKUP` statement, use [`SHO {% include {{ page.version.version }}/backups/scheduled-backups-tip.md %} -{% include {{ page.version.version }}/backups/backup-to-deprec.md %} +{% include {{ page.version.version }}/backups/old-syntax-removed.md %} ## Considerations @@ -236,11 +236,7 @@ Per our guidance in the [Performance](#performance) section, we recommend starti If you need to limit the control specific users have over your storage buckets, see [Assume role authentication]({% link {{ page.version.version }}/cloud-storage-authentication.md %}) for setup instructions. -{{site.data.alerts.callout_info}} -The `BACKUP ... TO` syntax is **deprecated** as of v22.1 and will be removed in a future release. - -Cockroach Labs recommends using the `BACKUP ... INTO {collectionURI}` syntax shown in the following examples. -{{site.data.alerts.end}} +{% include {{ page.version.version }}/backups/old-syntax-removed.md %} ### Back up a cluster diff --git a/src/current/v24.3/restore.md b/src/current/v24.3/restore.md index 7ac3e82afe9..e8aac279600 100644 --- a/src/current/v24.3/restore.md +++ b/src/current/v24.3/restore.md @@ -17,7 +17,7 @@ You can restore: For details on restoring across versions of CockroachDB, see [Restoring Backups Across Versions]({% link {{ page.version.version }}/restoring-backups-across-versions.md %}). -{% include {{ page.version.version }}/backups/backup-to-deprec.md %} +{% include {{ page.version.version }}/backups/old-syntax-removed.md %} ## Considerations diff --git a/src/current/v24.3/show-backup.md b/src/current/v24.3/show-backup.md index e529a0a05f5..d7ecd78a5d9 100644 --- a/src/current/v24.3/show-backup.md +++ b/src/current/v24.3/show-backup.md @@ -8,11 +8,9 @@ docs_area: reference.sql The `SHOW BACKUP` [statement]({% link {{ page.version.version }}/sql-statements.md %}) lists the contents of a backup created with the [`BACKUP`]({% link {{ page.version.version }}/backup.md %}) statement. {{site.data.alerts.callout_danger}} -The `SHOW BACKUP` syntax **without** the `IN` keyword is **deprecated** as of v22.1 and will be removed in a future release. +The `SHOW BACKUP` syntax **without** the `IN` keyword has been removed from CockroachDB v24.3 and later. -We recommend using `SHOW BACKUP FROM {subdirectory} IN {collectionURI}` to view backups in a subdirectory at a collection's URI. - -For guidance on the syntax for `SHOW BACKUP FROM`, see the [examples](#examples). +For guidance on the syntax for `SHOW BACKUP FROM`, refer to the [Synopsis](#synopsis) and [examples](#examples) on this page. {{site.data.alerts.end}} ## Required privileges diff --git a/src/current/v24.3/take-full-and-incremental-backups.md b/src/current/v24.3/take-full-and-incremental-backups.md index 89afa7d088f..09862b25947 100644 --- a/src/current/v24.3/take-full-and-incremental-backups.md +++ b/src/current/v24.3/take-full-and-incremental-backups.md @@ -14,7 +14,7 @@ There are two main types of backups: You can use the [`BACKUP`]({% link {{ page.version.version }}/backup.md %}) statement to efficiently back up your cluster's schemas and data to popular cloud services such as AWS S3, Google Cloud Storage, or NFS, and the [`RESTORE`]({% link {{ page.version.version }}/restore.md %}) statement to efficiently restore schema and data as necessary. For more information, see [Use Cloud Storage]({% link {{ page.version.version }}/use-cloud-storage.md %}). -{% include {{ page.version.version }}/backups/backup-to-deprec.md %} +{% include {{ page.version.version }}/backups/old-syntax-removed.md %} {% include {{ page.version.version }}/backups/scheduled-backups-tip.md %} From c777649c35b9cfe9bec6114bf74a7809e9bd2ea2 Mon Sep 17 00:00:00 2001 From: Ryan Kuo <8740013+taroface@users.noreply.github.com> Date: Mon, 16 Dec 2024 14:16:15 -0500 Subject: [PATCH 24/36] publish v23.1.30 binary links (#19242) --- src/current/_data/releases.yml | 7 ------- 1 file changed, 7 deletions(-) diff --git a/src/current/_data/releases.yml b/src/current/_data/releases.yml index 6df205d6971..6b699723a56 100644 --- a/src/current/_data/releases.yml +++ b/src/current/_data/releases.yml @@ -7556,13 +7556,6 @@ docker_arm_limited_access: false source: true previous_release: v23.1.29 - cloud_only: true - cloud_only_message_short: 'Available only for select CockroachDB Cloud clusters' - cloud_only_message: > - This version is currently available only for select - CockroachDB Cloud clusters. To request to upgrade - a CockroachDB self-hosted cluster to this version, - [contact support](https://support.cockroachlabs.com/hc/requests/new). - release_name: v23.2.17 major_version: v23.2 From b672e6b0ea23abf6c35194dcb4c5d3a5ded3bacc Mon Sep 17 00:00:00 2001 From: Kathryn May <44557882+kathancox@users.noreply.github.com> Date: Mon, 16 Dec 2024 14:31:04 -0500 Subject: [PATCH 25/36] Add references to the Cloud costs page from cdc docs (#19223) --- .../_includes/common/cdc-cloud-costs-link.md | 1 + .../_includes/common/define-watched-cdc.md | 1 + src/current/cockroachcloud/costs.md | 32 +++++++++++-------- .../v23.2/create-and-configure-changefeeds.md | 1 + .../v24.1/create-and-configure-changefeeds.md | 1 + .../v24.2/create-and-configure-changefeeds.md | 1 + .../v24.3/change-data-capture-overview.md | 2 +- .../v24.3/changefeed-best-practices.md | 4 +++ .../v24.3/create-and-configure-changefeeds.md | 1 + 9 files changed, 29 insertions(+), 15 deletions(-) create mode 100644 src/current/_includes/common/cdc-cloud-costs-link.md create mode 100644 src/current/_includes/common/define-watched-cdc.md diff --git a/src/current/_includes/common/cdc-cloud-costs-link.md b/src/current/_includes/common/cdc-cloud-costs-link.md new file mode 100644 index 00000000000..5f0b169325d --- /dev/null +++ b/src/current/_includes/common/cdc-cloud-costs-link.md @@ -0,0 +1 @@ +If you're using a CockroachDB {{ site.data.products.cloud }} cluster, refer to [Understand CockroachDB Cloud Costs]({% link cockroachcloud/costs.md %}) for detail on how CDC is billed monthly based on usage. \ No newline at end of file diff --git a/src/current/_includes/common/define-watched-cdc.md b/src/current/_includes/common/define-watched-cdc.md new file mode 100644 index 00000000000..20fb749c5e2 --- /dev/null +++ b/src/current/_includes/common/define-watched-cdc.md @@ -0,0 +1 @@ +The main feature of {% if page.name == "change-data-capture-overview.md" %} CockroachDB CDC {% else %} [CockroachDB CDC]({% link {{site.current_cloud_version}}/change-data-capture-overview.md %}) {% endif %} is the _changefeed_, which targets an allowlist of tables, known as _watched tables_. diff --git a/src/current/cockroachcloud/costs.md b/src/current/cockroachcloud/costs.md index 653b11f2ae6..158d9c48516 100644 --- a/src/current/cockroachcloud/costs.md +++ b/src/current/cockroachcloud/costs.md @@ -34,7 +34,7 @@ This table summarizes key details about how costs are calculated for each plan t CockroachDB Basic

Usage-based billing only - CockroachDB Standard

Provisioned compute, usage-based storage, data transfer, backups, and Change Data Capture (CDC) + CockroachDB Standard

Provisioned compute, usage-based storage, data transfer, backups, and Change data capture (CDC) CockroachDB Advanced

Provisioned compute, storage, and IOPS with usage-based billing for data transfer, backups, and CDC @@ -60,8 +60,8 @@ This table summarizes key details about how costs are calculated for each plan t Backups Usage-based via Request Units. - Usage-based.

Managed Backups storage rates vary per cloud provider, region, and backup frequency.

Self-Managed Backups fee incurred per GiB transferred to your own object storage. - Usage-based.

Managed Backups storage rates vary per cloud provider, region, backup frequency, and whether security add-on is enabled.

Self-Managed Backups fee incurred per GiB transferred to your own object storage. Rate varies depending on whether security add-on is enabled. + Usage-based.

Managed backups storage rates vary per cloud provider, region, and backup frequency.

Self-managed backups fee incurred per GiB transferred to your own object storage. + Usage-based.

Managed backups storage rates vary per cloud provider, region, backup frequency, and whether security add-on is enabled.

Self-managed backups fee incurred per GiB transferred to your own object storage. Rate varies depending on whether security add-on is enabled. Data transfer @@ -70,7 +70,7 @@ This table summarizes key details about how costs are calculated for each plan t Usage-based; rates vary per cloud provider and data transfer type. - Change Data Capture + Change data capture Usage-based via Request Units. Usage-based, per GiB-hour watched across all changefeeds. Usage-based, per GiB-hour watched across all changefeeds. Rates vary depending on whether security add-on is enabled. @@ -187,9 +187,9 @@ Backups on Basic clusters are included in the Request Unit costs. Managed backup
-[Managed Backups]({% link cockroachcloud/managed-backups.md %}) are charged per-GiB storage rates that vary per cloud provider, region, and backup [frequency]({% link cockroachcloud/managed-backups.md %}#frequency) (daily vs. more than daily). The per-GiB unit prices are tiered, based on the amount of backup data stored: Less than 5 GiB-Month, 5 to 100 GiB-Month, 100 to 500 GiB-Month, 500 to 1000 GiB-Month, or 1000 GiB-Month and higher. +[Managed backups]({% link cockroachcloud/managed-backups.md %}) are charged per-GiB storage rates that vary per cloud provider, region, and backup [frequency]({% link cockroachcloud/managed-backups.md %}#frequency) (daily vs. more than daily). The per-GiB unit prices are tiered, based on the amount of backup data stored: Less than 5 GiB-Month, 5 to 100 GiB-Month, 100 to 500 GiB-Month, 500 to 1000 GiB-Month, or 1000 GiB-Month and higher. -[Self-Managed Backups]({% link cockroachcloud/take-and-restore-self-managed-backups.md %}) to your own object storage are charged a per-GiB fee for the data transferred. This option provides an advanced backup scheduler and additional control over backup storage placement. +[Self-managed backups]({% link cockroachcloud/take-and-restore-self-managed-backups.md %}) to your own object storage are charged a per-GiB fee for the data transferred. This option provides an advanced backup scheduler and additional control over backup storage placement. For further details, refer to CockroachDB Cloud [Pricing](https://www.cockroachlabs.com/pricing/new/). @@ -197,9 +197,9 @@ For further details, refer to CockroachDB Cloud [Pricing](https://www.cockroachl
-[Managed Backups]({% link cockroachcloud/managed-backups.md %}) are charged per-GiB storage rates that vary per cloud provider, region, backup [frequency]({% link cockroachcloud/managed-backups.md %}#frequency) (daily vs. more than daily), and whether the Advanced security add-on is enabled. The per-GiB unit prices are tiered, based on the amount of backup data stored: Less than 5 GiB-Month, 5 to 100 GiB-Month, 100 to 500 GiB-Month, 500 to 1000 GiB-Month, or 1000 GiB-Month and higher. +[Managed backups]({% link cockroachcloud/managed-backups.md %}) are charged per-GiB storage rates that vary per cloud provider, region, backup [frequency]({% link cockroachcloud/managed-backups.md %}#frequency) (daily vs. more than daily), and whether the Advanced security add-on is enabled. The per-GiB unit prices are tiered, based on the amount of backup data stored: Less than 5 GiB-Month, 5 to 100 GiB-Month, 100 to 500 GiB-Month, 500 to 1000 GiB-Month, or 1000 GiB-Month and higher. -[Self-Managed Backups]({% link cockroachcloud/take-and-restore-self-managed-backups.md %}) to your own object storage are charged a per-GiB fee for the data transferred. The rate varies depending on whether the Advanced security add-on is enabled. Self-Managed Backups offer additional control over backup storage placement, and an advanced backup scheduler. +[Self-managed backups]({% link cockroachcloud/take-and-restore-self-managed-backups.md %}) to your own object storage are charged a per-GiB fee for the data transferred. The rate varies depending on whether the Advanced security add-on is enabled. Self-managed backups offer additional control over backup storage placement, and an advanced backup scheduler. For further details, refer to CockroachDB Cloud [Pricing](https://www.cockroachlabs.com/pricing/new/).
@@ -232,7 +232,7 @@ Cross-region data transfer includes: - CockroachDB replication across nodes that are in different regions. - Data egress from the CockroachDB Cloud cluster via supported [private connectivity]({% link cockroachcloud/connect-to-your-cluster.md %}#establish-private-connectivity) services to a private endpoint in another region. - [Managed backup]({% link cockroachcloud/backup-and-restore-overview.md %}#managed-backups) and [Self-managed backup]({% link cockroachcloud/take-and-restore-self-managed-backups.md %}) data transfer to another region. -- Change Data Capture (changefeed) data transfer to another region. +- Change data capture (changefeed) data transfer to another region. Customers are charged the cloud service provider’s (CSP) list price for metered cross-region data transfer. @@ -266,7 +266,7 @@ Cross-region data transfer includes: - CockroachDB replication across nodes that are in different regions. - Data egress from the CockroachDB Cloud cluster via supported [private connectivity]({% link cockroachcloud/connect-to-your-cluster.md %}#establish-private-connectivity) services to a private endpoint in another region. - [Managed backup]({% link cockroachcloud/backup-and-restore-overview.md %}#managed-backups) and [Self-managed backup]({% link cockroachcloud/take-and-restore-self-managed-backups.md %}) data transfer to another region. -- Change Data Capture (changefeed) data transfer to another region. +- Change data capture (changefeed) data transfer to another region. Customers are charged the cloud service provider’s (CSP) list price for metered cross-region data transfer. @@ -278,9 +278,9 @@ This is the usage for any data leaving CockroachDB such as SQL data being sent t
-### Change Data Capture (Changefeeds) +### Change data capture (changefeeds) -For Change Data Capture (CDC), all CockroachDB {{ site.data.products.cloud }} clusters can use [Enterprise Changefeeds]({% link {{ site.current_cloud_version}}/how-does-an-enterprise-changefeed-work.md %}). +For change data capture (CDC), all CockroachDB {{ site.data.products.cloud }} clusters can use [Enterprise changefeeds]({% link {{ site.current_cloud_version}}/how-does-an-enterprise-changefeed-work.md %}).
@@ -289,13 +289,17 @@ In CockroachDB {{ site.data.products.basic }}, CDC cost is usage-based via Reque
-In CockroachDB {{ site.data.products.standard }}, CDC is billed monthly based on usage, determined by the total GiB-Month watched across all of a cluster’s changefeeds. The per-GiB unit price is tiered, based on the total watched: Less than 5 GiB-Month, 5 to 100 GiB-Month, 100 to 250 GiB-Month, 250 to 500 GiB-Month, or 500 GiB-Month and higher. +{% include common/define-watched-cdc.md %} + +In CockroachDB {{ site.data.products.standard }}, CDC is billed monthly based on the total size in GiB-Month of all of a cluster's watched tables. The per-GiB unit price is tiered, based on the size of watched data: Less than 5 GiB-Month, 5 to 100 GiB-Month, 100 to 250 GiB-Month, 250 to 500 GiB-Month, or 500 GiB-Month and higher.
-In CockroachDB {{ site.data.products.advanced }}, CDC is billed monthly based on usage, determined by the total GiB-Month watched across all of a cluster’s changefeeds and whether the Advanced security add-on is enabled. The per-GiB unit price is tiered, based on the total watched: Less than 5 GiB-Month, 5 to 100 GiB-Month, 100 to 250 GiB-Month, 250 to 500 GiB-Month, or 500 GiB-Month and higher. +{% include common/define-watched-cdc.md %} + +In CockroachDB {{ site.data.products.advanced }}, CDC is billed monthly based on the total size of a cluster's watched tables and whether the {{ site.data.products.advanced }} security add-on is enabled. The per-GiB unit price is tiered, based on the size of watched data: Less than 5 GiB-Month, 5 to 100 GiB-Month, 100 to 250 GiB-Month, 250 to 500 GiB-Month, or 500 GiB-Month and higher.
diff --git a/src/current/v23.2/create-and-configure-changefeeds.md b/src/current/v23.2/create-and-configure-changefeeds.md index 53fab8d09f8..496a810426b 100644 --- a/src/current/v23.2/create-and-configure-changefeeds.md +++ b/src/current/v23.2/create-and-configure-changefeeds.md @@ -17,6 +17,7 @@ This page describes: 1. Enable rangefeeds on CockroachDB {{ site.data.products.advanced }} and CockroachDB {{ site.data.products.core }}. Refer to [Enable rangefeeds](#enable-rangefeeds) for instructions. 1. Decide on whether you will run an {{ site.data.products.enterprise }} or basic changefeed. Refer to the [Overview]({% link {{ page.version.version }}/change-data-capture-overview.md %}) page for a comparative capability table. 1. Plan the number of changefeeds versus the number of tables to include in a single changefeed for your cluster. {% include {{ page.version.version }}/cdc/changefeed-number-limit.md %} Refer to [System resources and running changefeeds](#system-resources-and-running-changefeeds) and [Recommendations for the number of target tables](#recommendations-for-the-number-of-target-tables). + - {% include common/cdc-cloud-costs-link.md %} 1. Consider whether your {{ site.data.products.enterprise }} [changefeed use case](#create) would be better served by [change data capture queries]({% link {{ page.version.version }}/cdc-queries.md %}) that can filter data on a single table. CDC queries can improve the efficiency of changefeeds because the job will not need to encode as much change data. 1. Read the [Considerations](#considerations) section that provides information on changefeed interactions that could affect how you configure or run your changefeed. diff --git a/src/current/v24.1/create-and-configure-changefeeds.md b/src/current/v24.1/create-and-configure-changefeeds.md index b56237149a4..d7b0d5ad034 100644 --- a/src/current/v24.1/create-and-configure-changefeeds.md +++ b/src/current/v24.1/create-and-configure-changefeeds.md @@ -17,6 +17,7 @@ This page describes: 1. Enable rangefeeds on CockroachDB {{ site.data.products.advanced }} and CockroachDB {{ site.data.products.core }}. Refer to [Enable rangefeeds](#enable-rangefeeds) for instructions. 1. Decide on whether you will run an {{ site.data.products.enterprise }} or basic changefeed. Refer to the [Overview]({% link {{ page.version.version }}/change-data-capture-overview.md %}) page for a comparative capability table. 1. Plan the number of changefeeds versus the number of tables to include in a single changefeed for your cluster. {% include {{ page.version.version }}/cdc/changefeed-number-limit.md %} Refer to [System resources and running changefeeds](#system-resources-and-running-changefeeds) and [Recommendations for the number of target tables](#recommendations-for-the-number-of-target-tables). + - {% include common/cdc-cloud-costs-link.md %} 1. Consider whether your {{ site.data.products.enterprise }} [changefeed use case](#create) would be better served by [change data capture queries]({% link {{ page.version.version }}/cdc-queries.md %}) that can filter data on a single table. CDC queries can improve the efficiency of changefeeds because the job will not need to encode as much change data. 1. Read the [Considerations](#considerations) section that provides information on changefeed interactions that could affect how you configure or run your changefeed. diff --git a/src/current/v24.2/create-and-configure-changefeeds.md b/src/current/v24.2/create-and-configure-changefeeds.md index c2815931249..f61991da802 100644 --- a/src/current/v24.2/create-and-configure-changefeeds.md +++ b/src/current/v24.2/create-and-configure-changefeeds.md @@ -17,6 +17,7 @@ This page describes: 1. Enable rangefeeds on CockroachDB {{ site.data.products.advanced }} and CockroachDB {{ site.data.products.core }}. Refer to [Enable rangefeeds](#enable-rangefeeds) for instructions. 1. Decide on whether you will run an {{ site.data.products.enterprise }} or basic changefeed. Refer to the [Overview]({% link {{ page.version.version }}/change-data-capture-overview.md %}) page for a comparative capability table. 1. Plan the number of changefeeds versus the number of tables to include in a single changefeed for your cluster. {% include {{ page.version.version }}/cdc/changefeed-number-limit.md %} Refer to [System resources and running changefeeds](#system-resources-and-running-changefeeds) and [Recommendations for the number of target tables](#recommendations-for-the-number-of-target-tables). + - {% include common/cdc-cloud-costs-link.md %} 1. Consider whether your {{ site.data.products.enterprise }} [changefeed use case](#create) would be better served by [change data capture queries]({% link {{ page.version.version }}/cdc-queries.md %}) that can filter data on a single table. CDC queries can improve the efficiency of changefeeds because the job will not need to encode as much change data. 1. Read the [Considerations](#considerations) section that provides information on changefeed interactions that could affect how you configure or run your changefeed. diff --git a/src/current/v24.3/change-data-capture-overview.md b/src/current/v24.3/change-data-capture-overview.md index 7997456e941..64fa3ea148b 100644 --- a/src/current/v24.3/change-data-capture-overview.md +++ b/src/current/v24.3/change-data-capture-overview.md @@ -15,7 +15,7 @@ For example, you might want to: - Export a snaphot of tables to backfill new applications. - Send updates to data stores for machine learning models. -The main feature of CockroachDB CDC is the _changefeed_, which targets an allowlist of tables, or "watched tables". +{% include common/define-watched-cdc.md %} ## Stream row-level changes with changefeeds diff --git a/src/current/v24.3/changefeed-best-practices.md b/src/current/v24.3/changefeed-best-practices.md index 391018ffabe..3e61a629a48 100644 --- a/src/current/v24.3/changefeed-best-practices.md +++ b/src/current/v24.3/changefeed-best-practices.md @@ -6,6 +6,10 @@ toc: true This page describes best practices to consider when starting changefeeds on a CockroachDB cluster. We recommend referring to this information while planning your cluster's changefeeds and following the links in each of the sections for more details on a topic. +{{site.data.alerts.callout_info}} +To help in planning your cluster's changefeeds on CockroachDB {{ site.data.products.cloud }} clusters, refer to the [Understand CockroachDB Cloud Costs]({% link cockroachcloud/costs.md %}) page for detail on how CDC is billed monthly based on usage. +{{site.data.alerts.end}} + ## Plan the number of watched tables for a single changefeed When creating a changefeed, it's important to consider the number of changefeeds versus the number of tables to include in a single changefeed: diff --git a/src/current/v24.3/create-and-configure-changefeeds.md b/src/current/v24.3/create-and-configure-changefeeds.md index 7ea21b2ece6..4662b2451ba 100644 --- a/src/current/v24.3/create-and-configure-changefeeds.md +++ b/src/current/v24.3/create-and-configure-changefeeds.md @@ -17,6 +17,7 @@ This page describes: 1. Enable rangefeeds on CockroachDB {{ site.data.products.advanced }} and CockroachDB {{ site.data.products.core }}. Refer to [Enable rangefeeds](#enable-rangefeeds) for instructions. 1. Decide on whether you will run an {{ site.data.products.enterprise }} or basic changefeed. Refer to the [Overview]({% link {{ page.version.version }}/change-data-capture-overview.md %}) page for a comparative capability table. 1. Plan the number of changefeeds versus the number of tables to include in a single changefeed for your cluster. {% include {{ page.version.version }}/cdc/changefeed-number-limit.md %} Refer to [System resources and running changefeeds]({% link {{ page.version.version }}/changefeed-best-practices.md %}#maintain-system-resources-and-running-changefeeds) and [Recommendations for the number of target tables]({% link {{ page.version.version }}/changefeed-best-practices.md %}#plan-the-number-of-watched-tables-for-a-single-changefeed). + - {% include common/cdc-cloud-costs-link.md %} 1. Consider whether your {{ site.data.products.enterprise }} [changefeed use case](#create) would be better served by [change data capture queries]({% link {{ page.version.version }}/cdc-queries.md %}) that can filter data on a single table. CDC queries can improve the efficiency of changefeeds because the job will not need to encode as much change data. 1. Read the following: - The [Changefeed Best Practices]({% link {{ page.version.version }}/changefeed-best-practices.md %}) reference for details on planning changefeeds, monitoring basics, and schema changes. From 2636d63c5f7e4cf567a082ade9c79588058930b1 Mon Sep 17 00:00:00 2001 From: Mike Lewis <76072290+mikeCRL@users.noreply.github.com> Date: Mon, 16 Dec 2024 14:37:53 -0500 Subject: [PATCH 26/36] v24.3.1 binaries and release note from an additional commit (#19241) --- src/current/_data/releases.yml | 9 +-------- src/current/_includes/releases/v24.3/v24.3.1.md | 2 ++ 2 files changed, 3 insertions(+), 8 deletions(-) diff --git a/src/current/_data/releases.yml b/src/current/_data/releases.yml index 6b699723a56..0dc6e51e735 100644 --- a/src/current/_data/releases.yml +++ b/src/current/_data/releases.yml @@ -7616,11 +7616,4 @@ docker_arm_experimental: false docker_arm_limited_access: false source: true - previous_release: v24.3.0 - cloud_only: true - cloud_only_message_short: 'Available only for select CockroachDB Cloud clusters' - cloud_only_message: > - This version is currently available only for select - CockroachDB Cloud clusters. To request to upgrade - a CockroachDB self-hosted cluster to this version, - [contact support](https://support.cockroachlabs.com/hc/requests/new). \ No newline at end of file + previous_release: v24.3.0 \ No newline at end of file diff --git a/src/current/_includes/releases/v24.3/v24.3.1.md b/src/current/_includes/releases/v24.3/v24.3.1.md index e5529f6e4b8..4db85eb32a8 100644 --- a/src/current/_includes/releases/v24.3/v24.3.1.md +++ b/src/current/_includes/releases/v24.3/v24.3.1.md @@ -36,6 +36,7 @@ Release Date: December 12, 2024 - `security.certificate.*` metrics now update correctly when certificates are reloaded during node runtime. Previously, these metrics would not reflect changes to certificates after node startup. [#136227][#136227] - SQL roles created from LDAP groups that contain periods (.) or hyphens (-) in their Common Names (CN) no longer result in authorization failures. [#134942][#134942] - LDAP authorization now supports partial group mapping, allowing users to authenticate even when some LDAP groups do not have corresponding CockroachDB roles. Previously, authentication would fail if any LDAP group lacked a matching database role. [#135587][#135587] +- Regional by row tables with uniqueness constraints where the region is not part of those uniqueness constraints and which also contain non-unique indices will now have those constraints properly enforced when modified at `READ COMMITTED` isolation. This bug was introduced in v24.3.0. [#137367][#137367]

Performance improvements

@@ -67,6 +68,7 @@ Release Date: December 12, 2024 [#136011]: https://github.com/cockroachdb/cockroach/pull/136011 [#136227]: https://github.com/cockroachdb/cockroach/pull/136227 [#136481]: https://github.com/cockroachdb/cockroach/pull/136481 +[#137367]: https://github.com/cockroachdb/cockroach/pull/137367 [0d7f6eed3]: https://github.com/cockroachdb/cockroach/commit/0d7f6eed3 [1f2b1b084]: https://github.com/cockroachdb/cockroach/commit/1f2b1b084 [3cbd07fbd]: https://github.com/cockroachdb/cockroach/commit/3cbd07fbd From fef0a386f3224717261a5abc09c5acc533abdf03 Mon Sep 17 00:00:00 2001 From: Mike Lewis <76072290+mikeCRL@users.noreply.github.com> Date: Mon, 16 Dec 2024 14:53:28 -0500 Subject: [PATCH 27/36] Release binaries for v23.2.17, v24.1.8, 24.2.6 (#19244) --- src/current/_data/releases.yml | 21 --------------------- 1 file changed, 21 deletions(-) diff --git a/src/current/_data/releases.yml b/src/current/_data/releases.yml index 0dc6e51e735..3efcc9d248c 100644 --- a/src/current/_data/releases.yml +++ b/src/current/_data/releases.yml @@ -7488,13 +7488,6 @@ docker_arm_limited_access: false source: true previous_release: v24.2.5 - cloud_only: true - cloud_only_message_short: 'Available only for select CockroachDB Cloud clusters' - cloud_only_message: > - This version is currently available only for select - CockroachDB Cloud clusters. To request to upgrade - a CockroachDB self-hosted cluster to this version, - [contact support](https://support.cockroachlabs.com/hc/requests/new). - release_name: v24.1.8 major_version: v24.1 @@ -7522,13 +7515,6 @@ docker_arm_limited_access: false source: true previous_release: v24.1.7 - cloud_only: true - cloud_only_message_short: 'Available only for select CockroachDB Cloud clusters' - cloud_only_message: > - This version is currently available only for select - CockroachDB Cloud clusters. To request to upgrade - a CockroachDB self-hosted cluster to this version, - [contact support](https://support.cockroachlabs.com/hc/requests/new). - release_name: v23.1.30 major_version: v23.1 @@ -7583,13 +7569,6 @@ docker_arm_limited_access: false source: true previous_release: v23.2.16 - cloud_only: true - cloud_only_message_short: 'Available only for select CockroachDB Cloud clusters' - cloud_only_message: > - This version is currently available only for select - CockroachDB Cloud clusters. To request to upgrade - a CockroachDB self-hosted cluster to this version, - [contact support](https://support.cockroachlabs.com/hc/requests/new). - release_name: v24.3.1 major_version: v24.3 From 6c1734c9b6f36e24861d88947049f87067837c91 Mon Sep 17 00:00:00 2001 From: Florence Morris Date: Mon, 16 Dec 2024 15:23:06 -0500 Subject: [PATCH 28/36] DOC-11722 Create a section on the Child Metrics page for LDR (#19230) (1) In child-metrics.md, added section for Clusters with logical data replication jobs. (2) In child-metrics.yml, added ldr metrics. (3) In metrics-list.csv, with v24.3.0 binary, ran cockroach gen metric-list --format=csv > metrics-list.csv and manually lower-cased column headers and corrected spelling of inaccurate and changed COUNTER to GAUGE for five _by_label metrics. (4) In logical-data-replication-monitoring, (a) updated link to new section in child-metrics.md and (b) updated metrics to _by_label names. (5) In child-metric-table.md include file, modified liquid code to display Description as column header for ldr feature. (6) In ui-logical-data-replication-dashboard.md, corrected the summary in the frontmatter. --- src/current/_data/metrics/child-metrics.yml | 18 +++++++++- src/current/_data/metrics/metrics-list.csv | 13 +++++-- .../_includes/v24.3/child-metrics-table.md | 2 +- src/current/v24.3/child-metrics.md | 35 +++++++++++++++++++ .../logical-data-replication-monitoring.md | 12 +++---- .../ui-logical-data-replication-dashboard.md | 2 +- 6 files changed, 70 insertions(+), 12 deletions(-) diff --git a/src/current/_data/metrics/child-metrics.yml b/src/current/_data/metrics/child-metrics.yml index e6393213331..afe92da0139 100644 --- a/src/current/_data/metrics/child-metrics.yml +++ b/src/current/_data/metrics/child-metrics.yml @@ -233,4 +233,20 @@ feature: all - child_metric_id: rpc.connection.avg_round_trip_latency - feature: all \ No newline at end of file + feature: all + +- child_metric_id: logical_replication.catchup_ranges_by_label + feature: ldr + +- child_metric_id: logical_replication.events_dlqed_by_label + feature: ldr + +- child_metric_id: logical_replication.events_ingested_by_label + feature: ldr + +- child_metric_id: logical_replication.replicated_time_by_label + feature: ldr + +- child_metric_id: logical_replication.scanning_ranges_by_label + feature: ldr + diff --git a/src/current/_data/metrics/metrics-list.csv b/src/current/_data/metrics/metrics-list.csv index 6b195f89ac2..0792493fc70 100644 --- a/src/current/_data/metrics/metrics-list.csv +++ b/src/current/_data/metrics/metrics-list.csv @@ -978,6 +978,8 @@ A likely cause of having a checkpoint is that one of the ranges in this store had inconsistent data among its replicas. Such checkpoint directories are located in auxiliary/checkpoints/rN_at_M, where N is the range ID, and M is the Raft applied index at which this checkpoint was taken.",Directories,GAUGE,COUNT,AVG,NONE +STORAGE,storage.compactions.cancelled.bytes,Cumulative volume of data written to sstables during compactions that were ultimately cancelled due to a conflicting operation.,Bytes,COUNTER,BYTES,AVG,NON_NEGATIVE_DERIVATIVE +STORAGE,storage.compactions.cancelled.count,Cumulative count of compactions that were cancelled before they completed due to a conflicting operation.,Compactions,COUNTER,COUNT,AVG,NON_NEGATIVE_DERIVATIVE STORAGE,storage.compactions.duration,"Cumulative sum of all compaction durations. The rate of this value provides the effective compaction concurrency of a store, @@ -1308,6 +1310,7 @@ APPLICATION,distsender.batch_responses.cross_zone.bytes,"Total byte count of rep monitor the data transmitted.",Bytes,COUNTER,BYTES,AVG,NON_NEGATIVE_DERIVATIVE APPLICATION,distsender.batch_responses.replica_addressed.bytes,Total byte count of replica-addressed batch responses received,Bytes,COUNTER,BYTES,AVG,NON_NEGATIVE_DERIVATIVE APPLICATION,distsender.batches,Number of batches processed,Batches,COUNTER,COUNT,AVG,NON_NEGATIVE_DERIVATIVE +APPLICATION,distsender.batches.async.in_progress,Number of partial batches currently being executed asynchronously,Partial Batches,GAUGE,COUNT,AVG,NONE APPLICATION,distsender.batches.async.sent,Number of partial batches sent asynchronously,Partial Batches,COUNTER,COUNT,AVG,NON_NEGATIVE_DERIVATIVE APPLICATION,distsender.batches.async.throttled,Number of partial batches not sent asynchronously due to throttling,Partial Batches,COUNTER,COUNT,AVG,NON_NEGATIVE_DERIVATIVE APPLICATION,distsender.batches.partial,Number of partial batches processed after being divided on range boundaries,Partial Batches,COUNTER,COUNT,AVG,NON_NEGATIVE_DERIVATIVE @@ -2274,15 +2277,17 @@ APPLICATION,kv.protectedts.reconciliation.num_runs,number of successful reconcil APPLICATION,kv.protectedts.reconciliation.records_processed,number of records processed without error during reconciliation on this node,Count,COUNTER,COUNT,AVG,NON_NEGATIVE_DERIVATIVE APPLICATION,kv.protectedts.reconciliation.records_removed,number of records removed during reconciliation runs on this node,Count,COUNTER,COUNT,AVG,NON_NEGATIVE_DERIVATIVE APPLICATION,logical_replication.batch_hist_nanos,Time spent flushing a batch,Nanoseconds,HISTOGRAM,NANOSECONDS,AVG,NONE +APPLICATION,logical_replication.catchup_ranges,Source side ranges undergoing catch up scans (inaccurate with multiple LDR jobs),Ranges,GAUGE,COUNT,AVG,NONE +APPLICATION,logical_replication.catchup_ranges_by_label,Source side ranges undergoing catch up scans,Ranges,GAUGE,COUNT,AVG,NON_NEGATIVE_DERIVATIVE APPLICATION,logical_replication.checkpoint_events_ingested,Checkpoint events ingested by all replication jobs,Events,COUNTER,COUNT,AVG,NON_NEGATIVE_DERIVATIVE APPLICATION,logical_replication.commit_latency,"Event commit latency: a difference between event MVCC timestamp and the time it was flushed into disk. If we batch events, then the difference between the oldest event in the batch and flush is recorded",Nanoseconds,HISTOGRAM,NANOSECONDS,AVG,NONE APPLICATION,logical_replication.events_dlqed,Row update events sent to DLQ,Failures,COUNTER,COUNT,AVG,NON_NEGATIVE_DERIVATIVE APPLICATION,logical_replication.events_dlqed_age,Row update events sent to DLQ due to reaching the maximum time allowed in the retry queue,Failures,COUNTER,COUNT,AVG,NON_NEGATIVE_DERIVATIVE -APPLICATION,logical_replication.events_dlqed_by_label,Row update events sent to DLQ by label,Failures,COUNTER,COUNT,AVG,NON_NEGATIVE_DERIVATIVE +APPLICATION,logical_replication.events_dlqed_by_label,Row update events sent to DLQ by label,Failures,GAUGE,COUNT,AVG,NON_NEGATIVE_DERIVATIVE APPLICATION,logical_replication.events_dlqed_errtype,Row update events sent to DLQ due to an error not considered retryable,Failures,COUNTER,COUNT,AVG,NON_NEGATIVE_DERIVATIVE APPLICATION,logical_replication.events_dlqed_space,Row update events sent to DLQ due to capacity of the retry queue,Failures,COUNTER,COUNT,AVG,NON_NEGATIVE_DERIVATIVE APPLICATION,logical_replication.events_ingested,Events ingested by all replication jobs,Events,COUNTER,COUNT,AVG,NON_NEGATIVE_DERIVATIVE -APPLICATION,logical_replication.events_ingested_by_label,Events ingested by all replication jobs by label,Events,COUNTER,COUNT,AVG,NON_NEGATIVE_DERIVATIVE +APPLICATION,logical_replication.events_ingested_by_label,Events ingested by all replication jobs by label,Events,GAUGE,COUNT,AVG,NON_NEGATIVE_DERIVATIVE APPLICATION,logical_replication.events_initial_failure,Failed attempts to apply an incoming row update,Failures,COUNTER,COUNT,AVG,NON_NEGATIVE_DERIVATIVE APPLICATION,logical_replication.events_initial_success,Successful applications of an incoming row update,Failures,COUNTER,COUNT,AVG,NON_NEGATIVE_DERIVATIVE APPLICATION,logical_replication.events_retry_failure,Failed re-attempts to apply a row update,Failures,COUNTER,COUNT,AVG,NON_NEGATIVE_DERIVATIVE @@ -2291,10 +2296,12 @@ APPLICATION,logical_replication.kv.update_too_old,Total number of updates that w APPLICATION,logical_replication.kv.value_refreshes,Total number of batches that refreshed the previous value,Events,COUNTER,COUNT,AVG,NON_NEGATIVE_DERIVATIVE APPLICATION,logical_replication.logical_bytes,Logical bytes (sum of keys + values) received by all replication jobs,Bytes,COUNTER,BYTES,AVG,NON_NEGATIVE_DERIVATIVE APPLICATION,logical_replication.replan_count,Total number of dist sql replanning events,Events,COUNTER,COUNT,AVG,NON_NEGATIVE_DERIVATIVE -APPLICATION,logical_replication.replicated_time_by_label,Replicated time of the logical replication stream by label,Seconds,COUNTER,SECONDS,AVG,NON_NEGATIVE_DERIVATIVE +APPLICATION,logical_replication.replicated_time_by_label,Replicated time of the logical replication stream by label,Seconds,GAUGE,SECONDS,AVG,NON_NEGATIVE_DERIVATIVE APPLICATION,logical_replication.replicated_time_seconds,The replicated time of the logical replication stream in seconds since the unix epoch.,Seconds,GAUGE,SECONDS,AVG,NONE APPLICATION,logical_replication.retry_queue_bytes,The replicated time of the logical replication stream in seconds since the unix epoch.,Bytes,GAUGE,BYTES,AVG,NONE APPLICATION,logical_replication.retry_queue_events,The replicated time of the logical replication stream in seconds since the unix epoch.,Events,GAUGE,COUNT,AVG,NONE +APPLICATION,logical_replication.scanning_ranges,Source side ranges undergoing an initial scan (inaccurate with multiple LDR jobs),Ranges,GAUGE,COUNT,AVG,NONE +APPLICATION,logical_replication.scanning_ranges_by_label,Source side ranges undergoing an initial scan,Ranges,GAUGE,COUNT,AVG,NON_NEGATIVE_DERIVATIVE APPLICATION,obs.tablemetadata.update_job.duration,Time spent running the update table metadata job.,Duration,HISTOGRAM,NANOSECONDS,AVG,NONE APPLICATION,obs.tablemetadata.update_job.errors,The total number of errors that have been emitted from the update table metadata job.,Errors,COUNTER,COUNT,AVG,NON_NEGATIVE_DERIVATIVE APPLICATION,obs.tablemetadata.update_job.runs,The total number of runs of the update table metadata job.,Executions,COUNTER,COUNT,AVG,NON_NEGATIVE_DERIVATIVE diff --git a/src/current/_includes/v24.3/child-metrics-table.md b/src/current/_includes/v24.3/child-metrics-table.md index 05d57c55453..2b1a16f092a 100644 --- a/src/current/_includes/v24.3/child-metrics-table.md +++ b/src/current/_includes/v24.3/child-metrics-table.md @@ -7,7 +7,7 @@ Following is a list of the metrics that have child metrics: CockroachDB Metric Name - Description When Aggregated + {% if feature == "ldr" %}Description{% else %}Description When Aggregated{% endif %} Type Unit diff --git a/src/current/v24.3/child-metrics.md b/src/current/v24.3/child-metrics.md index 3618ec7d5ad..8fee7bd1a6a 100644 --- a/src/current/v24.3/child-metrics.md +++ b/src/current/v24.3/child-metrics.md @@ -110,6 +110,41 @@ changefeed_error_retries{node_id="1",scope="office_dogs"} 0 {% assign feature = "changefeed" %} {% include {{ page.version.version }}/child-metrics-table.md %} +## Clusters with logical data replication jobs + +When child metrics is enabled and [logical data replication (LDR) jobs with metrics labels]({% link {{ page.version.version }}/logical-data-replication-monitoring.md %}#metrics-labels) are created on the cluster, the `logical_replication_*_by_label` metrics are exported per LDR metric label. The `label` may have the values set using the `label` option. The cardinality increases with the number of LDR metric labels. + +For example, when you create two LDR jobs with the metrics labels `ldr_job1` and `ldr_job2`, the metrics `logical_replication_*_by_label` export child metrics with a `label` for `ldr_job1` and `ldr_job2`. + +~~~ +# HELP logical_replication_replicated_time_by_label Replicated time of the logical replication stream by label +# TYPE logical_replication_replicated_time_by_label gauge +logical_replication_replicated_time_by_label{label="ldr_job2",node_id="2"} 1.73411035e+09 +logical_replication_replicated_time_by_label{label="ldr_job1",node_id="2"} 1.73411035e+09 +# HELP logical_replication_catchup_ranges_by_label Source side ranges undergoing catch up scans +# TYPE logical_replication_catchup_ranges_by_label gauge +logical_replication_catchup_ranges_by_label{label="ldr_job1",node_id="2"} 0 +logical_replication_catchup_ranges_by_label{label="ldr_job2",node_id="2"} 0 +# HELP logical_replication_scanning_ranges_by_label Source side ranges undergoing an initial scan +# TYPE logical_replication_scanning_ranges_by_label gauge +logical_replication_scanning_ranges_by_label{label="ldr_job1",node_id="2"} 0 +logical_replication_scanning_ranges_by_label{label="ldr_job2",node_id="2"} 0 +~~~ + +Note that the `logical_replication_*` metrics without the `_by_label` suffix may be `inaccurate with multiple LDR jobs`. + +~~~ +# HELP logical_replication_catchup_ranges Source side ranges undergoing catch up scans (inaccurate with multiple LDR jobs) +# TYPE logical_replication_catchup_ranges gauge +logical_replication_catchup_ranges{node_id="2"} 0 +# HELP logical_replication_scanning_ranges Source side ranges undergoing an initial scan (inaccurate with multiple LDR jobs) +# TYPE logical_replication_scanning_ranges gauge +logical_replication_scanning_ranges{node_id="2"} 0 +~~~ + +{% assign feature = "ldr" %} +{% include {{ page.version.version }}/child-metrics-table.md %} + ## Clusters with row-level TTL jobs When child metrics is enabled and [row-level TTL jobs]({% link {{ page.version.version }}/row-level-ttl.md %}) are created on the cluster with the [`ttl_label_metrics` storage parameter enabled]({% link {{ page.version.version }}/row-level-ttl.md %}#ttl-metrics), the `jobs.row_level_ttl.*` metrics are exported per TTL job with `ttl_label_metrics` enabled with a label for `relation`. The value of the `relation` label may have the format: `{database}_{schema}_{table}_{primary key}`. The cardinality increases with the number of TTL jobs with `ttl_label_metrics` enabled. An aggregated metric is also included. diff --git a/src/current/v24.3/logical-data-replication-monitoring.md b/src/current/v24.3/logical-data-replication-monitoring.md index 77730488cc9..240744851fa 100644 --- a/src/current/v24.3/logical-data-replication-monitoring.md +++ b/src/current/v24.3/logical-data-replication-monitoring.md @@ -114,11 +114,11 @@ You can use Prometheus and Alertmanager to track and alert on LDR metrics. Refer To view metrics at the job level, you can use the `label` option when you start LDR to add a metrics label to the LDR job. This enables [child metric]({% link {{ page.version.version }}/child-metrics.md %}) export, which are Prometheus time series with extra labels. You can track the following metrics for an LDR job with labels: -- `logical_replication.replicated_time_seconds` -- `logical_replication.events_ingested` -- `logical_replication.events_dlqed` -- `logical_replication.scanning_ranges` -- `logical_replication.catchup_ranges` +- `logical_replication.catchup_ranges_by_label` +- `logical_replication.events_dlqed_by_label` +- `logical_replication.events_ingested_by_label` +- `logical_replication.replicated_time_by_label` +- `logical_replication.scanning_ranges_by_label` To use metrics labels, ensure you have enabled the child metrics cluster setting: @@ -136,7 +136,7 @@ ON 'external://{source_external_connection}' INTO TABLE {database.public.table_name} WITH label=ldr_job; ~~~ -For a full reference on tracking metrics with labels, refer to the [Child Metrics]({% link {{ page.version.version }}/child-metrics.md %}) page. +For a full reference on tracking metrics with labels, refer to the [Child Metrics]({% link {{ page.version.version }}/child-metrics.md %}#clusters-with-logical-data-replication-jobs) page. ### Datadog diff --git a/src/current/v24.3/ui-logical-data-replication-dashboard.md b/src/current/v24.3/ui-logical-data-replication-dashboard.md index 468ac5d380e..40ec394da3e 100644 --- a/src/current/v24.3/ui-logical-data-replication-dashboard.md +++ b/src/current/v24.3/ui-logical-data-replication-dashboard.md @@ -1,6 +1,6 @@ --- title: Logical Data Replication Dashboard -summary: The Physical Cluster Replication Dashboard lets you monitor and observe replication streams between a primary and standby cluster. +summary: The Logical Data Replication Dashboard lets you monitor and observe logical data replication jobs on the destination cluster. toc: true docs_area: reference.db_console --- From 2e191f8aa6a9b976ad3d72d4376e47225a217ff2 Mon Sep 17 00:00:00 2001 From: Ryan Kuo <8740013+taroface@users.noreply.github.com> Date: Tue, 17 Dec 2024 13:25:54 -0500 Subject: [PATCH 29/36] update path to BEGIN diagram in 22.2-23.2 (#19229) --- src/current/v22.2/begin-transaction.md | 2 +- src/current/v23.1/begin-transaction.md | 7 +++---- src/current/v23.2/begin-transaction.md | 2 +- 3 files changed, 5 insertions(+), 6 deletions(-) diff --git a/src/current/v22.2/begin-transaction.md b/src/current/v22.2/begin-transaction.md index d071eea8165..3e9966642f4 100644 --- a/src/current/v22.2/begin-transaction.md +++ b/src/current/v22.2/begin-transaction.md @@ -15,7 +15,7 @@ When using transactions, your application should include logic to [retry transac ## Synopsis
-{% remote_include https://raw.githubusercontent.com/cockroachdb/generated-diagrams/{{ page.release_info.crdb_branch_name }}/grammar_svg/begin.html %} +{% remote_include https://raw.githubusercontent.com/cockroachdb/generated-diagrams/{{ page.release_info.crdb_branch_name }}/grammar_svg/legacy_begin.html %}
## Required privileges diff --git a/src/current/v23.1/begin-transaction.md b/src/current/v23.1/begin-transaction.md index 685a395a2f1..b2cbef9a048 100644 --- a/src/current/v23.1/begin-transaction.md +++ b/src/current/v23.1/begin-transaction.md @@ -7,15 +7,14 @@ docs_area: reference.sql The `BEGIN` [statement]({% link {{ page.version.version }}/sql-statements.md %}) initiates a [transaction]({% link {{ page.version.version }}/transactions.md %}), which either successfully executes all of the statements it contains or none at all. -{{site.data.alerts.callout_danger}} -When using transactions, your application should include logic to [retry transactions]({% link {{ page.version.version }}/transactions.md %}#transaction-retries) that are aborted to break a dependency cycle between concurrent transactions. +{{site.data.alerts.callout_info}} +When running under the default [`SERIALIZABLE`]({% link {{ page.version.version }}/demo-serializable.md %}) isolation level, your application should [use a retry loop to handle transaction retry errors]({% link {{ page.version.version }}/query-behavior-troubleshooting.md %}#transaction-retry-errors) that can occur under [contention]({{ link_prefix }}performance-best-practices-overview.html#transaction-contention). {{site.data.alerts.end}} - ## Synopsis
-{% remote_include https://raw.githubusercontent.com/cockroachdb/generated-diagrams/{{ page.release_info.crdb_branch_name }}/grammar_svg/begin.html %} +{% remote_include https://raw.githubusercontent.com/cockroachdb/generated-diagrams/{{ page.release_info.crdb_branch_name }}/grammar_svg/legacy_begin.html %}
## Required privileges diff --git a/src/current/v23.2/begin-transaction.md b/src/current/v23.2/begin-transaction.md index abee0930220..6c50b135e95 100644 --- a/src/current/v23.2/begin-transaction.md +++ b/src/current/v23.2/begin-transaction.md @@ -14,7 +14,7 @@ When running under the default [`SERIALIZABLE`]({% link {{ page.version.version ## Synopsis
-{% remote_include https://raw.githubusercontent.com/cockroachdb/generated-diagrams/{{ page.release_info.crdb_branch_name }}/grammar_svg/begin.html %} +{% remote_include https://raw.githubusercontent.com/cockroachdb/generated-diagrams/{{ page.release_info.crdb_branch_name }}/grammar_svg/legacy_begin.html %}
## Required privileges From 7c3056cf42e3e01c30bee01fcba855cdf1d7c2e5 Mon Sep 17 00:00:00 2001 From: Ryan Kuo <8740013+taroface@users.noreply.github.com> Date: Tue, 17 Dec 2024 15:22:37 -0500 Subject: [PATCH 30/36] update SQL Feature Support page (#19247) --- src/current/v24.3/sql-feature-support.md | 10 ++++------ 1 file changed, 4 insertions(+), 6 deletions(-) diff --git a/src/current/v24.3/sql-feature-support.md b/src/current/v24.3/sql-feature-support.md index 3ca62dbbc71..df9fa0711d8 100644 --- a/src/current/v24.3/sql-feature-support.md +++ b/src/current/v24.3/sql-feature-support.md @@ -1,18 +1,16 @@ --- -title: SQL Feature Support in CockroachDB v24.2 +title: SQL Feature Support in CockroachDB summary: Summary of CockroachDB's conformance to the SQL standard and which common extensions it supports. toc: true keywords: gin, gin index, gin indexes, inverted index, inverted indexes, accelerated index, accelerated indexes docs_area: reference.sql --- -Making CockroachDB easy to use is a top priority for us, so we chose to implement SQL. However, even though SQL has a standard, no database implements all of it, nor do any of them have standard implementations of all features. - -To understand which standard SQL features we support (as well as common extensions to the standard), use the table below. +CockroachDB {{ page.version.version }} supports the following standard SQL features and common extensions. - **Component** lists the components that are commonly considered part of SQL. - **Supported** shows CockroachDB's level of support for the component. -- **Type** indicates whether the component is part of the SQL *Standard* or is an *Extension* created by ourselves or others. +- **Type** indicates whether the component is part of the SQL *Standard* or is an *Extension* created by Cockroach Labs or others. - **Details** provides greater context about the component.