Command Manager configuration file

[AlexRuiz7]

Description

Aside from using the OpenSearch's keystore for storing sensitive data, the Command Manager needs to expose several non-sensitive settings that modify its behavior. The values for these properties are currently set by code, so the CM plugin needs to expose these settings to a configuration file, read, validate and apply these settings correctly.

A proposition of the settings to be exposed, and their accepted values, must be provided.

wazuh/wazuh#27158 can be used as reference.

Functional requirements

• The CM plugin reads and applies valid settings from the configuration file.
• The CM plugins rejects invalid settings, raising an error. In this case, default values must be used.
• The configuration file is created / updated and added to the wazuh-indexer package.

Implementation restrictions

• opensearch.yml is preferred.

Plan

• Spike. Check if custom settings can be added to opensearch.yml.
• Identify settings to expose.
• Register the settings.
• Load values from the settings file at start up.
• Unit tests.
• Documentation.

Related issues

• Add Management API CA file field to opensearch.yml #171

[QU3B1M]

Spike. Check if custom settings can be added to opensearch.yml

OpenSearch supports adding custom configurations to the opensearch.yml, and it can be done in a friendly way, as we researched in #86, and as we can see from the code of this third-party plugin as reference.

---

Proof of concept

To validate the implementation is possible, I've generated a custom package with the changes required to read the settings m_api.auth.username, m_api.auth.password & m_api.urifrom the configuration file:

• Convert the settings from SecureSetting.secureString to Setting.simpleString

  public class PluginSettings {
      public static final Setting<String> M_API_AUTH_USERNAME = Setting.simpleString("m_api.auth.username", Setting.Property.NodeScope, Setting.Property.Filtered);
      public static final Setting<String> M_API_AUTH_PASSWORD = Setting.simpleString("m_api.auth.password", Setting.Property.NodeScope, Setting.Property.Filtered);
      public static final Setting<String> M_API_URI = Setting.simpleString("m_api.uri", Setting.Property.NodeScope, Setting.Property.Filtered);
      ...
  }

• On the build.gradle for the integTest section define the values storage to be on setting instead of keystore

  testClusters.integTest {
       ...
       setting 'm_api.auth.username', 'admin'
       setting 'm_api.auth.password', 'test'
       setting 'm_api.uri', 'https://127.0.0.1:55000' 
       ...
  }

• Add some logs to check the values at wazuh-indexer startup

      private PluginSettings(@NonNull final Settings settings) {
          ...
          log.info("[SETTINGS] Username: {}", this.authUsername);
          log.info("[SETTINGS] URI: {}", this.uri);
      }

Functionality validation

1. Install the custom wazuh-indexer package with the POC changes
2. Configure the settings on the opensearch.yml

   m_api:
     auth:
       username: "test_user"
       password: "test_pwd"
     uri: "test_uri"

3. Restart wazuh-indexer and check the logs

   [2024-12-17T10:02:38,276][INFO ][c.w.c.s.PluginSettings   ] [node-1] [SETTINGS] Username: test_user
   [2024-12-17T10:02:38,276][INFO ][c.w.c.s.PluginSettings   ] [node-1] [SETTINGS] URI: test_uri

[AlexRuiz7]

Identify settings to expose

• management_api.auth.username: secure string
• management_api.auth.password: secure string
• management_api.host: simple string
• management_api.retries: integer
• management_api.timeout: integer (seconds)
• management_api.page_size: ???
• command_manager.job.schedule: integer (seconds/cron expression)
• command_manager.job.page_size: integer
• command_manager.job.pit_keep_alive: integer (seconds)
• command_manager.logging: simple string [debug, warn, error, info]
• command_manager.timeout: integer (seconds)

Settings definitions

management_api.auth.username:
  description: Username for the management API
  definition: com/wazuh/commandmanager/settings/PluginSettings.java. Line 34. M_API_AUTH_USERNAME
  getter: getAuthUsername()
  getter-usage:
    - com/wazuh/commandmanager/utils/httpclient/AuthHttpRestClient.java. Line 52
  constant-usage:
    - com/wazuh/commandmanager/CommandManagerPlugin.java. Line 174

management_api.auth.password:
  description: Password for the management API
  definition: com/wazuh/commandmanager/settings/PluginSettings.java. Line 38. M_API_AUTH_PASSWORD
  getter: getAuthPassword()
  getter-usage:
    - com/wazuh/commandmanager/utils/httpclient/AuthHttpRestClient.java. Line 53
  constant-usage:
    - com/wazuh/commandmanager/CommandManagerPlugin.java. Line 175

management_api.host:
  description: URI of the management API
  definition: com/wazuh/commandmanager/settings/PluginSettings.java. Line 43. M_API_URI
  getter: getUri()
  getter-usage:
    - com/wazuh/commandmanager/jobscheduler/SearchThread.java. Line 157
    - com/wazuh/commandmanager/utils/httpclient/AuthHttpRestClient.java. Line 133
    - com/wazuh/commandmanager/utils/httpclient/HttpRestClientDemo.java. Line 74
  constant-usage:
    - com/wazuh/commandmanager/CommandManagerPlugin.java. Line 176

management_api.retries:
  description: Maximum retries for requests from client to management API
  definition: com/wazuh/commandmanager/utils/httpclient/HttpRestClient.java. Line 47. MAX_RETRIES
  constant-usage:
    - com/wazuh/commandmanager/utils/httpclient/HttpRestClient.java. Line 88 & 94

management_api.timeout:
  description: Timeout for requests from client to management API
  definition: com/wazuh/commandmanager/utils/httpclient/HttpRestClient.java. Line 45. TIMEOUT
  constant-usage:
    - com/wazuh/commandmanager/utils/httpclient/HttpRestClient.java. Line 143

management_api.page_size:
  description: Size of page from client to management API (>=CommandManager.job.page_size)
  definition: TODO

---

command_manager.job.schedule:
  description: Interval between jobs, time in minutes (passed to JobDocument.create interval) must be migrated to cron
  definition: com/wazuh/commandmanager/CommandManagerPlugin.java. Line 87. JOB_PERIOD_MINUTES
  constant-usage:
    - com/wazuh/commandmanager/CommandManagerPlugin.java. Line 147

command_manager.job.page_size:
  description: Size of the page requested to Job Scheduler
  definition: com/wazuh/commandmanager/CommandManagerPlugin.java. Line 88. PAGE_SIZE
  constant-usage:
    - com/wazuh/commandmanager/jobscheduler/SearchThread.java. Line 211

command_manager.job.index.name:
  description: 
  definition: com/wazuh/commandmanager/CommandManagerPlugin.java. Line 85. JOB_INDEX_NAME
  constant-usage:
    - com/wazuh/commandmanager/jobscheduler/JobDocument.java. Line 76
    - com/wazuh/commandmanager/jobscheduler/JobDocument.java. Line 91
    - com/wazuh/commandmanager/CommandManagerPlugin.java. Line 195

command_manager.job.index.template:
  description: 
  definition: com/wazuh/commandmanager/CommandManagerPlugin.java. Line 86. JOB_INDEX_TEMPLATE_NAME
  constant-usage:
    - com/wazuh/commandmanager/jobscheduler/JobDocument.java. Line 85
    - com/wazuh/commandmanager/jobscheduler/JobDocument.java. Line 87

command_manager.job.keep_alive:
  description: Interval between keep alive signals to PIT query, time in seconds
  definition: com/wazuh/commandmanager/CommandManagerPlugin.java. Line 90. PIT_KEEP_ALIVE_SECONDS
  constant-usage:
    - com/wazuh/commandmanager/jobscheduler/SearchThread.java. Line 313

command_manager.timeout:
  description: Timeout for commands PIT query
  definition: com/wazuh/commandmanager/CommandManagerPlugin.java. Line 89. DEFAULT_TIMEOUT_SECONDS
  constant-usage:
    - com/wazuh/commandmanager/jobscheduler/SearchThread.java. Line 190 & 208 

command_manager.index.name:
  description: 
  definition: com/wazuh/commandmanager/CommandManagerPlugin.java. Line 82. COMMAND_MANAGER_INDEX_NAME
  constant-usage:
    - com/wazuh/commandmanager/index/CommandIndex.java. Line 213
    - com/wazuh/commandmanager/jobscheduler/CommandManagerJobRunner.java. Line 67
    - com/wazuh/commandmanager/jobscheduler/CommandManagerJobRunner.java. Line 75
    - com/wazuh/commandmanager/jobscheduler/SearchThread.java. Line 185
    - com/wazuh/commandmanager/jobscheduler/SearchThread.java. Line 204
    - com/wazuh/commandmanager/jobscheduler/SearchThread.java. Line 315
    - com/wazuh/commandmanager/rest/RestPostCommandAction.java. Line 165

command_manager.index.template:
  description: 
  definition: com/wazuh/commandmanager/CommandManagerPlugin.java. Line 86. JOB_INDEX_TEMPLATE_NAME
  constant-usage:
    - com/wazuh/commandmanager/index/CommandIndex.java. Line 87
    - com/wazuh/commandmanager/index/CommandIndex.java. Line 90
    - com/wazuh/commandmanager/index/CommandIndex.java. Line 94
    - com/wazuh/commandmanager/index/CommandIndex.java. Line 141
    - com/wazuh/commandmanager/index/CommandIndex.java. Line 144
    - com/wazuh/commandmanager/index/CommandIndex.java. Line 148

---

Based on the previous definitions, we have analyzed a potential structure for the command-manager configuration. Here is a proposed YAML structure:

management_api:
  host: string                    # Default: https://localhost:55000/
  retries: int                    # Default: 3
  timeout: int                    # Default: 10 (seconds)
  page_size: ???                  # Default: 100                                - Needs further analysis + development

command_manager:
  timeout: int                    # Default: 30 (seconds)
  job:
    schedule: string              # Default: 1 (minute)                         - Cron compatible
    page_size: int                # Default: 100
    pit_keep_alive: int           # Default: 10 (seconds)                       - Needs further analysis
    index:
      name: string                # Default: .scheduled-commands
      template: string            # Default: index-template-scheduled-commands
  api:
    prefix: string                # Default /_command_manager
    endpoint: string              # Default /commands
  index:
    name: string                  # Default: .commands
    template: string              # Default: index-template-commands

For the key store we have this settings

management_api:
  auth:
    username: secure string       # Default: admin
    password: secure string       # Default: admin

Reduced version without non-mandatory value

management_api:
  host: string                    # Default: https://localhost:55000/

Important

In this example, all time-related values are in seconds. This is part of our proposal to standardize time units. Currently, we have mixed time unit definitions, and it would be beneficial to normalize them to a single metric (seconds).

Further research and testing is required to validate if this approach is possible, for now we only tested for "simple" key values on the yaml. On the POC from the previous comment we have validated this configuration format in a minor scale.

[QU3B1M]

We need to evaluate the possibility of supporting multiple management_api connections. Currently, the configuration supports only one host with a username and password, making it incompatible with multiple connections. To address this, we should consider modifying the configuration to use an array for each API node, which will require additional development on the plugin.

The updated configuration might look like this:

management_api:
  nodes:
    - auth:
        username: secure string
        password: secure string
      host: string