Integrations maintenance request

[QU3B1M]

Maintenance need detected by: https://github.com/wazuh/internal-devel-requests/issues/1736

Description

The Wazuh Indexer team is responsible for the maintenance of the third-party integrations hosted in the wazuh/wazuh-indexer repository. We must ensure these integrations work under new releases of the third-party software (Splunk, Elastic, Logstash, …) and our own.

For that, we need to:

• Create a pull request that upgrades the components to the latest version.
• Update our testing environments to verify the integrations work under new versions.
• Test the integrations, checking that:
• The Docker Compose project starts without errors.
• The data arrives to the destination.
• All the dashboards can be imported successfully.
• All the dashboards are populated with data.
• Finally, upgrade the compatibility matrix in integrations/README.md with the new versions.

Note

• For Logstash, we use the logstash-oss image.
• For Wazuh Indexer and Wazuh Dashboard, we use the opensearch and opensearch-dashboards images. These must match the opensearch version that we support (e.g: for Wazuh 4.9.0 it is OpenSearch 2.13.0).

Issues

• List here the detected issues

[QU3B1M]

We detected a missing "build context" on the the splunk's integration docker compose file, more specifically on the Logstash-oss component, we must fix it since it could generate inconsistencies on the build process when the Logstash image has not been previously built.

[QU3B1M]

Wazuh Indexer Splunk integration

Built Docker compose

docker compose -f compose.indexer-splunk.yml up -d                   
[+] Running 43/12
 ✔ splunk Pulled                                                                                                                                                                                  101.6s 
 ✔ generator Pulled                                                                                                                                                                               102.8s 
 ✔ wazuh.indexer Pulled                                                                                                                                                                            61.1s 
 ! logstash Warning         pull access denied for logstash-oss, repository does not exist or may require 'docker login': denied: requested access to the resource is denied                        2.4s 
 ! events-generator Warning pull access denied for wazuh/indexer-events-generator, repository does not exist or may require 'docker login': denied: requested access t...                           2.4s 
 ✔ wazuh-certs-generator Pulled                                                                                                                                                                     6.7s 
 ✔ generate-certs-config Pulled                                                                                                                                                                     3.7s 
 ✔ wazuh.dashboard Pulled                                                                                                                                                                          54.9s 
                                                                                                                                                                                                         
                                                                                                                                                                                                         
                                                                                                                                                                                                         
                                                                                                                                                                                                         
[+] Building 58.8s (22/22) FINISHED                                                                                                                                                 docker:desktop-linux
 => [logstash internal] load build definition from Dockerfile                                                                                                                                       0.0s
 => => transferring dockerfile: 794B                                                                                                                                                                0.0s
 => WARN: InvalidDefaultArgInFrom: Default value for ARG opensearchproject/logstash-oss-with-opensearch-output-plugin:${LOGSTASH_OSS_VERSION} results in empty or invalid base image name (line 2)  0.0s
 => [events-generator internal] load build definition from Dockerfile                                                                                                                               0.0s
 => => transferring dockerfile: 187B                                                                                                                                                                0.0s
 => [events-generator internal] load metadata for docker.io/library/python:3.9                                                                                                                      2.3s
 => [logstash internal] load metadata for docker.io/opensearchproject/logstash-oss-with-opensearch-output-plugin:8.9.0                                                                              1.4s
 => [logstash internal] load .dockerignore                                                                                                                                                          0.0s
 => => transferring context: 2B                                                                                                                                                                     0.0s
 => [logstash 1/6] FROM docker.io/opensearchproject/logstash-oss-with-opensearch-output-plugin:8.9.0@sha256:43b0cdaf26edca812a9eafb684ae6101d8f2fd27165345737c39c04a6c7430eb                        0.0s
 => [logstash internal] load build context                                                                                                                                                          0.0s
 => => transferring context: 527B                                                                                                                                                                   0.0s
 => CACHED [logstash 2/6] RUN LS_JAVA_OPTS="-Xms1024m -Xmx1024m" logstash-plugin install logstash-input-opensearch                                                                                  0.0s
 => [logstash 3/6] COPY --chown=logstash:logstash ./setup.sh /usr/share/logstash/bin/setup.sh                                                                                                       0.1s
 => [logstash 4/6] RUN bash /usr/share/logstash/bin/setup.sh                                                                                                                                       57.1s
 => [events-generator internal] load .dockerignore                                                                                                                                                  0.0s
 => => transferring context: 93B                                                                                                                                                                    0.0s
 => [events-generator internal] load build context                                                                                                                                                  0.0s
 => => transferring context: 904.76kB                                                                                                                                                               0.0s
 => [events-generator 1/4] FROM docker.io/library/python:3.9@sha256:ed8b9dd4e9f89c111f4bdb85a55f8c9f0e22796a298449380b15f627d9914095                                                               17.5s
 => => resolve docker.io/library/python:3.9@sha256:ed8b9dd4e9f89c111f4bdb85a55f8c9f0e22796a298449380b15f627d9914095                                                                                 0.0s
 => => sha256:7d98d813d54f6207a57721008a4081378343ad8f1b2db66c121406019171805b 49.56MB / 49.56MB                                                                                                    3.0s
 => => sha256:ed8b9dd4e9f89c111f4bdb85a55f8c9f0e22796a298449380b15f627d9914095 10.35kB / 10.35kB                                                                                                    0.0s
 => => sha256:980816fef8aa0a957c9ac5bc4502951bdca106c77f1578d529431cc30c5c61b4 2.32kB / 2.32kB                                                                                                      0.0s
 => => sha256:24e94023d80962f8ad8e7a968fad0ea47c4fd9992938e18270f8f49859a35a62 6.30kB / 6.30kB                                                                                                      0.0s
 => => sha256:da802df85c965baeca9d39869f9e2cbb3dc844d4627f413bfbb2f2c3d6055988 24.05MB / 24.05MB                                                                                                    5.2s
 => => sha256:7aadc5092c3b7a865666b14bef3d4d038282b19b124542f1a158c98ea8c1ed1b 64.39MB / 64.39MB                                                                                                    7.5s
 => => extracting sha256:7d98d813d54f6207a57721008a4081378343ad8f1b2db66c121406019171805b                                                                                                           1.2s
 => => sha256:ad1c7cfc347f5c86fc2678b58f6a8fb6c6003471405760532fc3240b9eb1b343 211.27MB / 211.27MB                                                                                                 13.4s
 => => extracting sha256:da802df85c965baeca9d39869f9e2cbb3dc844d4627f413bfbb2f2c3d6055988                                                                                                           0.3s
 => => sha256:4eb48115a0423399a647666a3212b3977f31d779480dca8d8d8f9bbfb35f92e4 6.16MB / 6.16MB                                                                                                      7.0s
 => => sha256:ccecc6c1c4bf5a8539f053b01dda5a0fba46a5b04afdd30fb30dcaf526778824 19.84MB / 19.84MB                                                                                                    9.6s
 => => extracting sha256:7aadc5092c3b7a865666b14bef3d4d038282b19b124542f1a158c98ea8c1ed1b                                                                                                           2.0s
 => => sha256:63dc518f902b82e47f42908845205bcbdd2bea1a70e1f13f4fb0859fbfd91671 250B / 250B                                                                                                          7.8s
 => => extracting sha256:ad1c7cfc347f5c86fc2678b58f6a8fb6c6003471405760532fc3240b9eb1b343                                                                                                           3.3s
 => => extracting sha256:4eb48115a0423399a647666a3212b3977f31d779480dca8d8d8f9bbfb35f92e4                                                                                                           0.1s
 => => extracting sha256:ccecc6c1c4bf5a8539f053b01dda5a0fba46a5b04afdd30fb30dcaf526778824                                                                                                           0.4s
 => => extracting sha256:63dc518f902b82e47f42908845205bcbdd2bea1a70e1f13f4fb0859fbfd91671                                                                                                           0.0s
 => [events-generator 2/4] COPY . /home/events-generator/                                                                                                                                           0.2s
 => [events-generator 3/4] WORKDIR /home/events-generator                                                                                                                                           0.0s
 => [events-generator 4/4] RUN pip install -r requirements.txt                                                                                                                                      2.9s
 => [events-generator] exporting to image                                                                                                                                                           0.0s
 => => exporting layers                                                                                                                                                                             0.0s
 => => writing image sha256:d8e325e71971b259c343b8c8b9a064f7bda4c5923c38a7f3316d2a074a1b22df                                                                                                        0.0s
 => => naming to docker.io/wazuh/indexer-events-generator                                                                                                                                           0.0s
 => [events-generator] resolving provenance for metadata file                                                                                                                                       0.0s
 => [logstash 5/6] RUN `echo "pipeline.ecs_compatibility: disabled" >> /usr/share/logstash/config/logstash.yml`                                                                                     0.1s
 => [logstash 6/6] WORKDIR /usr/share/logstash                                                                                                                                                      0.0s
 => [logstash] exporting to image                                                                                                                                                                   0.0s
 => => exporting layers                                                                                                                                                                             0.0s
 => => writing image sha256:58a31fe58c5227c3f0a7be69c8ec36c8282c14d802dbb035d2b5a585d09385a2                                                                                                        0.0s
 => => naming to docker.io/library/logstash-oss:8.9.0                                                                                                                                               0.0s
 => [logstash] resolving provenance for metadata file                                                                                                                                               0.0s
[+] Running 9/9
 ✔ Network splunk-integration_default                    Created                                                                                                                                    0.0s 
 ✔ Container splunk-integration-generate-certs-config-1  Exited                                                                                                                                     0.7s 
 ✔ Container splunk-integration-wazuh-certs-generator-1  Exited                                                                                                                                     5.4s 
 ✔ Container splunk-integration-generator-1              Exited                                                                                                                                     5.4s 
 ✔ Container splunk-integration-wazuh.indexer-1          Healthy                                                                                                                                   47.9s 
 ✔ Container splunk-integration-wazuh.dashboard-1        Started                                                                                                                                    5.0s 
 ✔ Container splunk-integration-events-generator-1       Started                                                                                                                                   48.0s 
 ✔ Container splunk-integration-splunk-1                 Healthy                                                                                                                                  106.5s 
 ✔ Container splunk-integration-logstash-1               Started                                                                                                                                  106.6s 

• Wazuh-amazon-aws-v1.0
• Wazuh-docker-listener-v1.0
• Wazuh-incident-response-v1.0
• Wazuh-malware-detection-v1.0
• Wazuh-pci-dss-v1.0
• Wazuh-security-events-v1.0
• Wazuh-vulnerabilities-v1.0

[QU3B1M]

Wazuh Indexer Elastic Stack integration

• Wazuh-amazon-aws-v1.0
• Wazuh-docker-listener-v1.0
• Wazuh-incident-response-v1.0
• Wazuh-malware-detection-v1.0
• Wazuh-pci-dss-v1.0
• Wazuh-security-events-v1.0
• Wazuh-vulnerabilities-v1.0

[QU3B1M]

Wazuh indexer OpenSearch integration

• OpenSearch Version!
  

• Wazuh-amazon-aws-v1.0

• Wazuh-docker-listener-v1.0

• Wazuh-incident-response-v1.0

• Wazuh-malware-detection-v1.0

• Wazuh-pci-dss-v1.0

• Wazuh-security-events-v1.0

• Wazuh-vulnerabilities-v1.0