From 896f538ac80c5285e988de36e01e080a292c6902 Mon Sep 17 00:00:00 2001 From: Shubh Sahu Date: Thu, 2 May 2024 20:41:08 +0530 Subject: [PATCH 1/5] Adding document for setting introduced : cluster.remote_store.translog.max_readers Signed-off-by: Shubh Sahu --- .../availability-and-recovery/remote-store/index.md | 12 +++++++----- 1 file changed, 7 insertions(+), 5 deletions(-) diff --git a/_tuning-your-cluster/availability-and-recovery/remote-store/index.md b/_tuning-your-cluster/availability-and-recovery/remote-store/index.md index 5fd19f5cc2..6670d2bb53 100644 --- a/_tuning-your-cluster/availability-and-recovery/remote-store/index.md +++ b/_tuning-your-cluster/availability-and-recovery/remote-store/index.md @@ -62,11 +62,13 @@ During the bootstrapping process, the remote-backed repositories listed in `open You can use the following [cluster settings]({{site.url}}{{site.baseurl}}//api-reference/cluster-api/cluster-settings/) to tune how remote-backed clusters handle each workload. -| Field | Data type | Description | -| :--- | :--- | :--- | -| cluster.default.index.refresh_interval | Time unit | Sets the refresh interval when the `index.refresh_interval` setting is not provided. This setting can be useful when you want to set a default refresh interval across all indexes in a cluster and also support the `searchIdle` setting. You cannot set the interval lower than the `cluster.minimum.index.refresh_interval` setting. | -| cluster.minimum.index.refresh_interval | Time unit | Sets the minimum refresh interval and applies it to all indexes in the cluster. The `cluster.default.index.refresh_interval` setting should be higher than this setting's value. If, during index creation, the `index.refresh_interval` setting is lower than the minimum, index creation fails. | -| cluster.remote_store.translog.buffer_interval | Time unit | The default value of the translog buffer interval used when performing periodic translog updates. This setting is only effective when the index setting `index.remote_store.translog.buffer_interval` is not present. | +| Field | Data type | Description | +|:----------------------------------------------|:----------|:----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| +| cluster.default.index.refresh_interval | Time unit | Sets the refresh interval when the `index.refresh_interval` setting is not provided. This setting can be useful when you want to set a default refresh interval across all indexes in a cluster and also support the `searchIdle` setting. You cannot set the interval lower than the `cluster.minimum.index.refresh_interval` setting. | +| cluster.minimum.index.refresh_interval | Time unit | Sets the minimum refresh interval and applies it to all indexes in the cluster. The `cluster.default.index.refresh_interval` setting should be higher than this setting's value. If, during index creation, the `index.refresh_interval` setting is lower than the minimum, index creation fails. | +| cluster.remote_store.translog.buffer_interval | Time unit | The default value of the translog buffer interval used when performing periodic translog updates. This setting is only effective when the index setting `index.remote_store.translog.buffer_interval` is not present. | +| cluster.remote_store.translog.max_readers | Integer | Sets the maximum number of open translog files for remote backed indices. This setting is introduced to limit the total number of translog files per shard. Upon breaching which, flush will be triggered and translog files will be trimmed. | + ## Restoring from a backup From 3e24d1fbbd51f5cd0c1e4315775b6e5114241480 Mon Sep 17 00:00:00 2001 From: Shubh Sahu Date: Thu, 2 May 2024 21:00:53 +0530 Subject: [PATCH 2/5] resolving vale error Signed-off-by: Shubh Sahu --- .../availability-and-recovery/remote-store/index.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/_tuning-your-cluster/availability-and-recovery/remote-store/index.md b/_tuning-your-cluster/availability-and-recovery/remote-store/index.md index 6670d2bb53..4757ec8f9c 100644 --- a/_tuning-your-cluster/availability-and-recovery/remote-store/index.md +++ b/_tuning-your-cluster/availability-and-recovery/remote-store/index.md @@ -67,7 +67,7 @@ You can use the following [cluster settings]({{site.url}}{{site.baseurl}}//api-r | cluster.default.index.refresh_interval | Time unit | Sets the refresh interval when the `index.refresh_interval` setting is not provided. This setting can be useful when you want to set a default refresh interval across all indexes in a cluster and also support the `searchIdle` setting. You cannot set the interval lower than the `cluster.minimum.index.refresh_interval` setting. | | cluster.minimum.index.refresh_interval | Time unit | Sets the minimum refresh interval and applies it to all indexes in the cluster. The `cluster.default.index.refresh_interval` setting should be higher than this setting's value. If, during index creation, the `index.refresh_interval` setting is lower than the minimum, index creation fails. | | cluster.remote_store.translog.buffer_interval | Time unit | The default value of the translog buffer interval used when performing periodic translog updates. This setting is only effective when the index setting `index.remote_store.translog.buffer_interval` is not present. | -| cluster.remote_store.translog.max_readers | Integer | Sets the maximum number of open translog files for remote backed indices. This setting is introduced to limit the total number of translog files per shard. Upon breaching which, flush will be triggered and translog files will be trimmed. | +| cluster.remote_store.translog.max_readers | Integer | Sets the maximum number of open translog files for remote backed indexes. This setting is introduced to limit the total number of translog files per shard. Upon breaching which, flush will be triggered and translog files will be trimmed. | From aaa1766bf99e37d046d8dd2e889daaf983fa6b5b Mon Sep 17 00:00:00 2001 From: Shubh Sahu Date: Mon, 6 May 2024 15:43:17 +0530 Subject: [PATCH 3/5] Addressed comment on PR Signed-off-by: Shubh Sahu --- .../remote-store/index.md | 15 ++++++--------- 1 file changed, 6 insertions(+), 9 deletions(-) diff --git a/_tuning-your-cluster/availability-and-recovery/remote-store/index.md b/_tuning-your-cluster/availability-and-recovery/remote-store/index.md index 4757ec8f9c..52a37a66d6 100644 --- a/_tuning-your-cluster/availability-and-recovery/remote-store/index.md +++ b/_tuning-your-cluster/availability-and-recovery/remote-store/index.md @@ -62,15 +62,12 @@ During the bootstrapping process, the remote-backed repositories listed in `open You can use the following [cluster settings]({{site.url}}{{site.baseurl}}//api-reference/cluster-api/cluster-settings/) to tune how remote-backed clusters handle each workload. -| Field | Data type | Description | -|:----------------------------------------------|:----------|:----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| -| cluster.default.index.refresh_interval | Time unit | Sets the refresh interval when the `index.refresh_interval` setting is not provided. This setting can be useful when you want to set a default refresh interval across all indexes in a cluster and also support the `searchIdle` setting. You cannot set the interval lower than the `cluster.minimum.index.refresh_interval` setting. | -| cluster.minimum.index.refresh_interval | Time unit | Sets the minimum refresh interval and applies it to all indexes in the cluster. The `cluster.default.index.refresh_interval` setting should be higher than this setting's value. If, during index creation, the `index.refresh_interval` setting is lower than the minimum, index creation fails. | -| cluster.remote_store.translog.buffer_interval | Time unit | The default value of the translog buffer interval used when performing periodic translog updates. This setting is only effective when the index setting `index.remote_store.translog.buffer_interval` is not present. | -| cluster.remote_store.translog.max_readers | Integer | Sets the maximum number of open translog files for remote backed indexes. This setting is introduced to limit the total number of translog files per shard. Upon breaching which, flush will be triggered and translog files will be trimmed. | - - - +| Field | Data type | Description | +| :--- | :--- | :--- | +| cluster.default.index.refresh_interval | Time unit | Sets the refresh interval when the `index.refresh_interval` setting is not provided. This setting can be useful when you want to set a default refresh interval across all indexes in a cluster and also support the `searchIdle` setting. You cannot set the interval lower than the `cluster.minimum.index.refresh_interval` setting. | +| cluster.minimum.index.refresh_interval | Time unit | Sets the minimum refresh interval and applies it to all indexes in the cluster. The `cluster.default.index.refresh_interval` setting should be higher than this setting's value. If, during index creation, the `index.refresh_interval` setting is lower than the minimum, index creation fails. | +| cluster.remote_store.translog.buffer_interval | Time unit | The default value of the translog buffer interval used when performing periodic translog updates. This setting is only effective when the index setting `index.remote_store.translog.buffer_interval` is not present. | +| cluster.remote_store.translog.max_readers | Integer | Sets the maximum number of open translog files for remote-backed indexes. This limits the total number of translog files per shard. Upon breaching this limit, a flush will be triggered and the translog files will be trimmed. Default is `1000` and the minimum required is `100`.| ## Restoring from a backup To restore an index from a remote backup, such as in the event of a node failure, use one of the following options: From 697c386f939e4d498151d76eba3fc5db2a3db790 Mon Sep 17 00:00:00 2001 From: Shubh Sahu Date: Mon, 6 May 2024 15:46:08 +0530 Subject: [PATCH 4/5] Minor correction Signed-off-by: Shubh Sahu --- .../availability-and-recovery/remote-store/index.md | 1 + 1 file changed, 1 insertion(+) diff --git a/_tuning-your-cluster/availability-and-recovery/remote-store/index.md b/_tuning-your-cluster/availability-and-recovery/remote-store/index.md index 52a37a66d6..ae922ed71f 100644 --- a/_tuning-your-cluster/availability-and-recovery/remote-store/index.md +++ b/_tuning-your-cluster/availability-and-recovery/remote-store/index.md @@ -68,6 +68,7 @@ You can use the following [cluster settings]({{site.url}}{{site.baseurl}}//api-r | cluster.minimum.index.refresh_interval | Time unit | Sets the minimum refresh interval and applies it to all indexes in the cluster. The `cluster.default.index.refresh_interval` setting should be higher than this setting's value. If, during index creation, the `index.refresh_interval` setting is lower than the minimum, index creation fails. | | cluster.remote_store.translog.buffer_interval | Time unit | The default value of the translog buffer interval used when performing periodic translog updates. This setting is only effective when the index setting `index.remote_store.translog.buffer_interval` is not present. | | cluster.remote_store.translog.max_readers | Integer | Sets the maximum number of open translog files for remote-backed indexes. This limits the total number of translog files per shard. Upon breaching this limit, a flush will be triggered and the translog files will be trimmed. Default is `1000` and the minimum required is `100`.| + ## Restoring from a backup To restore an index from a remote backup, such as in the event of a node failure, use one of the following options: From 373686f40a88e87e06324b7e9168a93ac4a0791a Mon Sep 17 00:00:00 2001 From: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com> Date: Tue, 7 May 2024 10:42:06 -0500 Subject: [PATCH 5/5] Update _tuning-your-cluster/availability-and-recovery/remote-store/index.md Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com> --- .../availability-and-recovery/remote-store/index.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/_tuning-your-cluster/availability-and-recovery/remote-store/index.md b/_tuning-your-cluster/availability-and-recovery/remote-store/index.md index ae922ed71f..d5dc99d5fe 100644 --- a/_tuning-your-cluster/availability-and-recovery/remote-store/index.md +++ b/_tuning-your-cluster/availability-and-recovery/remote-store/index.md @@ -67,7 +67,7 @@ You can use the following [cluster settings]({{site.url}}{{site.baseurl}}//api-r | cluster.default.index.refresh_interval | Time unit | Sets the refresh interval when the `index.refresh_interval` setting is not provided. This setting can be useful when you want to set a default refresh interval across all indexes in a cluster and also support the `searchIdle` setting. You cannot set the interval lower than the `cluster.minimum.index.refresh_interval` setting. | | cluster.minimum.index.refresh_interval | Time unit | Sets the minimum refresh interval and applies it to all indexes in the cluster. The `cluster.default.index.refresh_interval` setting should be higher than this setting's value. If, during index creation, the `index.refresh_interval` setting is lower than the minimum, index creation fails. | | cluster.remote_store.translog.buffer_interval | Time unit | The default value of the translog buffer interval used when performing periodic translog updates. This setting is only effective when the index setting `index.remote_store.translog.buffer_interval` is not present. | -| cluster.remote_store.translog.max_readers | Integer | Sets the maximum number of open translog files for remote-backed indexes. This limits the total number of translog files per shard. Upon breaching this limit, a flush will be triggered and the translog files will be trimmed. Default is `1000` and the minimum required is `100`.| +| cluster.remote_store.translog.max_readers | Integer | Sets the maximum number of open translog files for remote-backed indexes. This limits the total number of translog files per shard. After reaching this limit, the remote store flushes the translog files. Default is `1000`. The minimum required is `100`. | ## Restoring from a backup