Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[BUG] opensearch/opensearch - "Could not load codec 'Lucene912.'" #623

Open
williamtrelawny opened this issue Nov 28, 2024 · 4 comments
Open

[BUG] opensearch/opensearch - "Could not load codec 'Lucene912.'" #623

williamtrelawny opened this issue Nov 28, 2024 · 4 comments
Labels
bug Something isn't working

Comments

@williamtrelawny
Copy link

Describe the bug
The above error appears in my pod log during chart install. Here is my full startup pod log:

Defaulted container "opensearch" out of: opensearch, fsgroup-volume (init), configfile (init)
Enabling OpenSearch Security Plugin
Enabling execution of install_demo_configuration.sh for OpenSearch Security Plugin
OpenSearch 2.12.0 onwards, the OpenSearch Security Plugin a change that requires an initial password for 'admin' user.
Please define an environment variable 'OPENSEARCH_INITIAL_ADMIN_PASSWORD' with a strong password string.
If a password is not provided, the setup will quit.
 For more details, please visit: https://opensearch.org/docs/latest/install-and-configure/install-opensearch/docker/
### OpenSearch Security Demo Installer
### ** Warning: Do not use on production or public reachable systems **
OpenSearch install type: rpm/deb on Linux 6.8.0-49-generic amd64
OpenSearch config dir: /usr/share/opensearch/config/
OpenSearch config file: /usr/share/opensearch/config/opensearch.yml
OpenSearch bin dir: /usr/share/opensearch/bin/
OpenSearch plugins dir: /usr/share/opensearch/plugins/
OpenSearch lib dir: /usr/share/opensearch/lib/
Detected OpenSearch Version: 2.15.0
Detected OpenSearch Security Version: 2.15.0.0
Admin password set successfully.
### Success
### Execute this script now on all your nodes and then start all nodes
### OpenSearch Security will be automatically initialized.
### If you like to change the runtime configuration
### change the files in ../../../config/opensearch-security and execute:
sudo "/usr/share/opensearch/plugins/opensearch-security/tools/securityadmin.sh" -cd "/usr/share/opensearch/config/opensearch-security" -icl -key "/usr/share/opensearch/config/kirk-key.pem" -cert "/usr/share/opensearch/config/kirk.pem" -cacert "/usr/share/opensearch/config/root-ca.pem" -nhnv
### or run ./securityadmin_demo.sh
### To use the Security Plugin ConfigurationGUI
### To access your secured cluster open https://<hostname>:<HTTP port> and log in with admin/<your-custom-admin-password>.
### (Ignore the SSL certificate warning because we installed self-signed demo certificates)
Enabling execution of OPENSEARCH_HOME/bin/opensearch-performance-analyzer/performance-analyzer-agent-cli for OpenSearch Performance Analyzer Plugin
WARNING: Using incubator modules: jdk.incubator.vector
WARNING: A terminally deprecated method in java.lang.System has been called
WARNING: System::setSecurityManager has been called by org.opensearch.bootstrap.OpenSearch (file:/usr/share/opensearch/lib/opensearch-2.15.0.jar)
WARNING: Please consider reporting this to the maintainers of org.opensearch.bootstrap.OpenSearch
WARNING: System::setSecurityManager will be removed in a future release
Nov 28, 2024 1:46:37 AM sun.util.locale.provider.LocaleProviderAdapter <clinit>
WARNING: COMPAT locale provider will be removed in a future release
WARNING: A terminally deprecated method in java.lang.System has been called
WARNING: System::setSecurityManager has been called by org.opensearch.bootstrap.Security (file:/usr/share/opensearch/lib/opensearch-2.15.0.jar)
WARNING: Please consider reporting this to the maintainers of org.opensearch.bootstrap.Security
WARNING: System::setSecurityManager will be removed in a future release
[2024-11-28T01:46:37,720][INFO ][o.o.n.Node               ] [opensearch-cluster-master-0] version[2.15.0], pid[1], build[tar/61dbcd0795c9bfe9b81e5762175414bc38bbcadf/2024-06-20T03:26:49.193630411Z], OS[Linux/6.8.0-49-generic/amd64], JVM[Eclipse Adoptium/OpenJDK 64-Bit Server VM/21.0.3/21.0.3+9-LTS]
[2024-11-28T01:46:37,724][INFO ][o.o.n.Node               ] [opensearch-cluster-master-0] JVM home [/usr/share/opensearch/jdk], using bundled JDK/JRE [true]
[2024-11-28T01:46:37,724][INFO ][o.o.n.Node               ] [opensearch-cluster-master-0] JVM arguments [-Xshare:auto, -Dopensearch.networkaddress.cache.ttl=60, -Dopensearch.networkaddress.cache.negative.ttl=10, -XX:+AlwaysPreTouch, -Xss1m, -Djava.awt.headless=true, -Dfile.encoding=UTF-8, -Djna.nosys=true, -XX:-OmitStackTraceInFastThrow, -XX:+ShowCodeDetailsInExceptionMessages, -Dio.netty.noUnsafe=true, -Dio.netty.noKeySetOptimization=true, -Dio.netty.recycler.maxCapacityPerThread=0, -Dio.netty.allocator.numDirectArenas=0, -Dlog4j.shutdownHookEnabled=false, -Dlog4j2.disable.jmx=true, -Djava.security.manager=allow, -Djava.locale.providers=SPI,COMPAT, -Xms1g, -Xmx1g, -XX:+UseG1GC, -XX:G1ReservePercent=25, -XX:InitiatingHeapOccupancyPercent=30, -Djava.io.tmpdir=/tmp/opensearch-15466128336574912065, -XX:+HeapDumpOnOutOfMemoryError, -XX:HeapDumpPath=data, -XX:ErrorFile=logs/hs_err_pid%p.log, -Xlog:gc*,gc+age=trace,safepoint:file=logs/gc.log:utctime,pid,tags:filecount=32,filesize=64m, -Djava.security.manager=allow, --add-modules=jdk.incubator.vector, -Djava.util.concurrent.ForkJoinPool.common.threadFactory=org.opensearch.secure_sm.SecuredForkJoinWorkerThreadFactory, -Dclk.tck=100, -Djdk.attach.allowAttachSelf=true, -Djava.security.policy=/usr/share/opensearch/config/opensearch-performance-analyzer/opensearch_security.policy, --add-opens=jdk.attach/sun.tools.attach=ALL-UNNAMED, -Dopensearch.cgroups.hierarchy.override=/, -Xmx512M, -Xms512M, -XX:MaxDirectMemorySize=268435456, -Dopensearch.path.home=/usr/share/opensearch, -Dopensearch.path.conf=/usr/share/opensearch/config, -Dopensearch.distribution.type=tar, -Dopensearch.bundled_jdk=true]
[2024-11-28T01:46:39,470][INFO ][o.o.s.s.t.SSLConfig      ] [opensearch-cluster-master-0] SSL dual mode is disabled
[2024-11-28T01:46:39,470][INFO ][o.o.s.OpenSearchSecurityPlugin] [opensearch-cluster-master-0] OpenSearch Config path is /usr/share/opensearch/config
[2024-11-28T01:46:39,949][INFO ][o.o.s.s.DefaultSecurityKeyStore] [opensearch-cluster-master-0] JVM supports TLSv1.3
[2024-11-28T01:46:39,951][INFO ][o.o.s.s.DefaultSecurityKeyStore] [opensearch-cluster-master-0] Config directory is /usr/share/opensearch/config/, from there the key- and truststore files are resolved relatively
[2024-11-28T01:46:40,798][INFO ][o.o.s.s.DefaultSecurityKeyStore] [opensearch-cluster-master-0] TLS Transport Client Provider : JDK
[2024-11-28T01:46:40,798][INFO ][o.o.s.s.DefaultSecurityKeyStore] [opensearch-cluster-master-0] TLS Transport Server Provider : JDK
[2024-11-28T01:46:40,798][INFO ][o.o.s.s.DefaultSecurityKeyStore] [opensearch-cluster-master-0] TLS HTTP Provider             : JDK
[2024-11-28T01:46:40,799][INFO ][o.o.s.s.DefaultSecurityKeyStore] [opensearch-cluster-master-0] Enabled TLS protocols for transport layer : [TLSv1.3, TLSv1.2]
[2024-11-28T01:46:40,799][INFO ][o.o.s.s.DefaultSecurityKeyStore] [opensearch-cluster-master-0] Enabled TLS protocols for HTTP layer      : [TLSv1.3, TLSv1.2]
[2024-11-28T01:46:40,875][INFO ][o.o.s.OpenSearchSecurityPlugin] [opensearch-cluster-master-0] Clustername: opensearch-cluster
[2024-11-28T01:46:40,893][WARN ][o.o.s.OpenSearchSecurityPlugin] [opensearch-cluster-master-0] Directory /usr/share/opensearch/config has insecure file permissions (should be 0700)
[2024-11-28T01:46:40,893][WARN ][o.o.s.OpenSearchSecurityPlugin] [opensearch-cluster-master-0] File /usr/share/opensearch/config/opensearch.yml has insecure file permissions (should be 0600)
[2024-11-28T01:46:40,893][WARN ][o.o.s.OpenSearchSecurityPlugin] [opensearch-cluster-master-0] File /usr/share/opensearch/config/esnode.pem has insecure file permissions (should be 0600)
[2024-11-28T01:46:40,894][WARN ][o.o.s.OpenSearchSecurityPlugin] [opensearch-cluster-master-0] File /usr/share/opensearch/config/kirk-key.pem has insecure file permissions (should be 0600)
[2024-11-28T01:46:40,894][WARN ][o.o.s.OpenSearchSecurityPlugin] [opensearch-cluster-master-0] File /usr/share/opensearch/config/kirk.pem has insecure file permissions (should be 0600)
[2024-11-28T01:46:40,894][WARN ][o.o.s.OpenSearchSecurityPlugin] [opensearch-cluster-master-0] File /usr/share/opensearch/config/esnode-key.pem has insecure file permissions (should be 0600)
[2024-11-28T01:46:40,894][WARN ][o.o.s.OpenSearchSecurityPlugin] [opensearch-cluster-master-0] File /usr/share/opensearch/config/securityadmin_demo.sh has insecure file permissions (should be 0600)
[2024-11-28T01:46:40,894][WARN ][o.o.s.OpenSearchSecurityPlugin] [opensearch-cluster-master-0] File /usr/share/opensearch/config/root-ca.pem has insecure file permissions (should be 0600)
[2024-11-28T01:46:42,139][INFO ][o.o.p.c.c.PluginSettings ] [opensearch-cluster-master-0] Config: metricsLocation: /dev/shm/performanceanalyzer/, metricsDeletionInterval: 1, httpsEnabled: false, cleanup-metrics-db-files: true, batch-metrics-retention-period-minutes: 7, rpc-port: 9650, webservice-port 9600
[2024-11-28T01:46:42,987][INFO ][o.o.i.r.ReindexPlugin    ] [opensearch-cluster-master-0] ReindexPlugin reloadSPI called
[2024-11-28T01:46:42,988][INFO ][o.o.i.r.ReindexPlugin    ] [opensearch-cluster-master-0] Unable to find any implementation for RemoteReindexExtension
[2024-11-28T01:46:43,105][INFO ][o.o.j.JobSchedulerPlugin ] [opensearch-cluster-master-0] Loaded scheduler extension: reports-scheduler, index: .opendistro-reports-definitions
[2024-11-28T01:46:43,108][INFO ][o.o.j.JobSchedulerPlugin ] [opensearch-cluster-master-0] Loaded scheduler extension: opensearch_time_series_analytics, index: .opendistro-anomaly-detector-jobs
[2024-11-28T01:46:43,109][INFO ][o.o.j.JobSchedulerPlugin ] [opensearch-cluster-master-0] Loaded scheduler extension: opendistro-index-management, index: .opendistro-ism-config
[2024-11-28T01:46:43,111][INFO ][o.o.j.JobSchedulerPlugin ] [opensearch-cluster-master-0] Loaded scheduler extension: scheduler_geospatial_ip2geo_datasource, index: .scheduler-geospatial-ip2geo-datasource
[2024-11-28T01:46:43,113][INFO ][o.o.j.JobSchedulerPlugin ] [opensearch-cluster-master-0] Loaded scheduler extension: opensearch_sap_job, index: .opensearch-sap--job
[2024-11-28T01:46:43,141][INFO ][o.o.p.PluginsService     ] [opensearch-cluster-master-0] loaded module [aggs-matrix-stats]
[2024-11-28T01:46:43,142][INFO ][o.o.p.PluginsService     ] [opensearch-cluster-master-0] loaded module [analysis-common]
[2024-11-28T01:46:43,142][INFO ][o.o.p.PluginsService     ] [opensearch-cluster-master-0] loaded module [cache-common]
[2024-11-28T01:46:43,142][INFO ][o.o.p.PluginsService     ] [opensearch-cluster-master-0] loaded module [geo]
[2024-11-28T01:46:43,142][INFO ][o.o.p.PluginsService     ] [opensearch-cluster-master-0] loaded module [ingest-common]
[2024-11-28T01:46:43,142][INFO ][o.o.p.PluginsService     ] [opensearch-cluster-master-0] loaded module [ingest-geoip]
[2024-11-28T01:46:43,142][INFO ][o.o.p.PluginsService     ] [opensearch-cluster-master-0] loaded module [ingest-user-agent]
[2024-11-28T01:46:43,142][INFO ][o.o.p.PluginsService     ] [opensearch-cluster-master-0] loaded module [lang-expression]
[2024-11-28T01:46:43,142][INFO ][o.o.p.PluginsService     ] [opensearch-cluster-master-0] loaded module [lang-mustache]
[2024-11-28T01:46:43,143][INFO ][o.o.p.PluginsService     ] [opensearch-cluster-master-0] loaded module [lang-painless]
[2024-11-28T01:46:43,143][INFO ][o.o.p.PluginsService     ] [opensearch-cluster-master-0] loaded module [mapper-extras]
[2024-11-28T01:46:43,143][INFO ][o.o.p.PluginsService     ] [opensearch-cluster-master-0] loaded module [opensearch-dashboards]
[2024-11-28T01:46:43,143][INFO ][o.o.p.PluginsService     ] [opensearch-cluster-master-0] loaded module [parent-join]
[2024-11-28T01:46:43,143][INFO ][o.o.p.PluginsService     ] [opensearch-cluster-master-0] loaded module [percolator]
[2024-11-28T01:46:43,143][INFO ][o.o.p.PluginsService     ] [opensearch-cluster-master-0] loaded module [rank-eval]
[2024-11-28T01:46:43,143][INFO ][o.o.p.PluginsService     ] [opensearch-cluster-master-0] loaded module [reindex]
[2024-11-28T01:46:43,143][INFO ][o.o.p.PluginsService     ] [opensearch-cluster-master-0] loaded module [repository-url]
[2024-11-28T01:46:43,144][INFO ][o.o.p.PluginsService     ] [opensearch-cluster-master-0] loaded module [search-pipeline-common]
[2024-11-28T01:46:43,144][INFO ][o.o.p.PluginsService     ] [opensearch-cluster-master-0] loaded module [systemd]
[2024-11-28T01:46:43,144][INFO ][o.o.p.PluginsService     ] [opensearch-cluster-master-0] loaded module [transport-netty4]
[2024-11-28T01:46:43,144][INFO ][o.o.p.PluginsService     ] [opensearch-cluster-master-0] loaded plugin [opensearch-alerting]
[2024-11-28T01:46:43,144][INFO ][o.o.p.PluginsService     ] [opensearch-cluster-master-0] loaded plugin [opensearch-anomaly-detection]
[2024-11-28T01:46:43,144][INFO ][o.o.p.PluginsService     ] [opensearch-cluster-master-0] loaded plugin [opensearch-asynchronous-search]
[2024-11-28T01:46:43,145][INFO ][o.o.p.PluginsService     ] [opensearch-cluster-master-0] loaded plugin [opensearch-cross-cluster-replication]
[2024-11-28T01:46:43,145][INFO ][o.o.p.PluginsService     ] [opensearch-cluster-master-0] loaded plugin [opensearch-custom-codecs]
[2024-11-28T01:46:43,145][INFO ][o.o.p.PluginsService     ] [opensearch-cluster-master-0] loaded plugin [opensearch-flow-framework]
[2024-11-28T01:46:43,145][INFO ][o.o.p.PluginsService     ] [opensearch-cluster-master-0] loaded plugin [opensearch-geospatial]
[2024-11-28T01:46:43,145][INFO ][o.o.p.PluginsService     ] [opensearch-cluster-master-0] loaded plugin [opensearch-index-management]
[2024-11-28T01:46:43,145][INFO ][o.o.p.PluginsService     ] [opensearch-cluster-master-0] loaded plugin [opensearch-job-scheduler]
[2024-11-28T01:46:43,145][INFO ][o.o.p.PluginsService     ] [opensearch-cluster-master-0] loaded plugin [opensearch-knn]
[2024-11-28T01:46:43,145][INFO ][o.o.p.PluginsService     ] [opensearch-cluster-master-0] loaded plugin [opensearch-ml]
[2024-11-28T01:46:43,145][INFO ][o.o.p.PluginsService     ] [opensearch-cluster-master-0] loaded plugin [opensearch-neural-search]
[2024-11-28T01:46:43,146][INFO ][o.o.p.PluginsService     ] [opensearch-cluster-master-0] loaded plugin [opensearch-notifications]
[2024-11-28T01:46:43,146][INFO ][o.o.p.PluginsService     ] [opensearch-cluster-master-0] loaded plugin [opensearch-notifications-core]
[2024-11-28T01:46:43,146][INFO ][o.o.p.PluginsService     ] [opensearch-cluster-master-0] loaded plugin [opensearch-observability]
[2024-11-28T01:46:43,146][INFO ][o.o.p.PluginsService     ] [opensearch-cluster-master-0] loaded plugin [opensearch-performance-analyzer]
[2024-11-28T01:46:43,146][INFO ][o.o.p.PluginsService     ] [opensearch-cluster-master-0] loaded plugin [opensearch-reports-scheduler]
[2024-11-28T01:46:43,146][INFO ][o.o.p.PluginsService     ] [opensearch-cluster-master-0] loaded plugin [opensearch-security]
[2024-11-28T01:46:43,146][INFO ][o.o.p.PluginsService     ] [opensearch-cluster-master-0] loaded plugin [opensearch-security-analytics]
[2024-11-28T01:46:43,146][INFO ][o.o.p.PluginsService     ] [opensearch-cluster-master-0] loaded plugin [opensearch-skills]
[2024-11-28T01:46:43,146][INFO ][o.o.p.PluginsService     ] [opensearch-cluster-master-0] loaded plugin [opensearch-sql]
[2024-11-28T01:46:43,171][INFO ][o.o.s.OpenSearchSecurityPlugin] [opensearch-cluster-master-0] Disabled https compression by default to mitigate BREACH attacks. You can enable it by setting 'http.compression: true' in opensearch.yml
[2024-11-28T01:46:43,233][INFO ][o.o.e.ExtensionsManager  ] [opensearch-cluster-master-0] ExtensionsManager initialized
[2024-11-28T01:46:43,250][DEPRECATION][o.o.d.c.s.Settings       ] [opensearch-cluster-master-0] [node.max_local_storage_nodes] setting was deprecated in OpenSearch and will be removed in a future release! See the breaking changes documentation for the next major version.
[2024-11-28T01:46:43,257][INFO ][o.a.l.s.MemorySegmentIndexInputProvider] [opensearch-cluster-master-0] Using MemorySegmentIndexInput with Java 21 or later; to disable start with -Dorg.apache.lucene.store.MMapDirectory.enableMemorySegments=false
[2024-11-28T01:46:43,270][INFO ][o.o.e.NodeEnvironment    ] [opensearch-cluster-master-0] using [1] data paths, mounts [[/usr/share/opensearch/data (/dev/sda2)]], net usable_space [10.7gb], net total_space [31.3gb], types [ext4]
[2024-11-28T01:46:43,271][INFO ][o.o.e.NodeEnvironment    ] [opensearch-cluster-master-0] heap size [512mb], compressed ordinary object pointers [true]
[2024-11-28T01:46:43,287][DEPRECATION][o.o.d.c.n.DiscoveryNodeRole] [opensearch-cluster-master-0] Assigning [master] role in setting [node.roles] is deprecated. To promote inclusive language, please use [cluster_manager] role instead.
[2024-11-28T01:46:43,339][ERROR][o.o.b.OpenSearchUncaughtExceptionHandler] [opensearch-cluster-master-0] uncaught exception in thread [main]
org.opensearch.bootstrap.StartupException: java.lang.IllegalArgumentException: Could not load codec 'Lucene912'. Did you forget to add lucene-backward-codecs.jar?
        at org.opensearch.bootstrap.OpenSearch.init(OpenSearch.java:185) ~[opensearch-2.15.0.jar:2.15.0]
        at org.opensearch.bootstrap.OpenSearch.execute(OpenSearch.java:172) ~[opensearch-2.15.0.jar:2.15.0]
        at org.opensearch.cli.EnvironmentAwareCommand.execute(EnvironmentAwareCommand.java:104) ~[opensearch-2.15.0.jar:2.15.0]
        at org.opensearch.cli.Command.mainWithoutErrorHandling(Command.java:138) ~[opensearch-cli-2.15.0.jar:2.15.0]
        at org.opensearch.cli.Command.main(Command.java:101) ~[opensearch-cli-2.15.0.jar:2.15.0]
        at org.opensearch.bootstrap.OpenSearch.main(OpenSearch.java:138) ~[opensearch-2.15.0.jar:2.15.0]
        at org.opensearch.bootstrap.OpenSearch.main(OpenSearch.java:104) ~[opensearch-2.15.0.jar:2.15.0]
Caused by: java.lang.IllegalArgumentException: Could not load codec 'Lucene912'. Did you forget to add lucene-backward-codecs.jar?
        at org.apache.lucene.index.SegmentInfos.readCodec(SegmentInfos.java:520) ~[lucene-core-9.10.0.jar:9.10.0 695c0ac84508438302cd346a812cfa2fdc5a10df - 2024-02-14 16:48:06]
        at org.apache.lucene.index.SegmentInfos.parseSegmentInfos(SegmentInfos.java:409) ~[lucene-core-9.10.0.jar:9.10.0 695c0ac84508438302cd346a812cfa2fdc5a10df - 2024-02-14 16:48:06]
        at org.apache.lucene.index.SegmentInfos.readCommit(SegmentInfos.java:368) ~[lucene-core-9.10.0.jar:9.10.0 695c0ac84508438302cd346a812cfa2fdc5a10df - 2024-02-14 16:48:06]
        at org.apache.lucene.index.SegmentInfos.readCommit(SegmentInfos.java:304) ~[lucene-core-9.10.0.jar:9.10.0 695c0ac84508438302cd346a812cfa2fdc5a10df - 2024-02-14 16:48:06]
        at org.apache.lucene.index.StandardDirectoryReader$1.doBody(StandardDirectoryReader.java:88) ~[lucene-core-9.10.0.jar:9.10.0 695c0ac84508438302cd346a812cfa2fdc5a10df - 2024-02-14 16:48:06]
        at org.apache.lucene.index.StandardDirectoryReader$1.doBody(StandardDirectoryReader.java:77) ~[lucene-core-9.10.0.jar:9.10.0 695c0ac84508438302cd346a812cfa2fdc5a10df - 2024-02-14 16:48:06]
        at org.apache.lucene.index.SegmentInfos$FindSegmentsFile.run(SegmentInfos.java:820) ~[lucene-core-9.10.0.jar:9.10.0 695c0ac84508438302cd346a812cfa2fdc5a10df - 2024-02-14 16:48:06]
        at org.apache.lucene.index.StandardDirectoryReader.open(StandardDirectoryReader.java:109) ~[lucene-core-9.10.0.jar:9.10.0 695c0ac84508438302cd346a812cfa2fdc5a10df - 2024-02-14 16:48:06]
        at org.apache.lucene.index.StandardDirectoryReader.open(StandardDirectoryReader.java:67) ~[lucene-core-9.10.0.jar:9.10.0 695c0ac84508438302cd346a812cfa2fdc5a10df - 2024-02-14 16:48:06]
        at org.apache.lucene.index.DirectoryReader.open(DirectoryReader.java:60) ~[lucene-core-9.10.0.jar:9.10.0 695c0ac84508438302cd346a812cfa2fdc5a10df - 2024-02-14 16:48:06]
        at org.opensearch.gateway.PersistedClusterStateService.nodeMetadata(PersistedClusterStateService.java:309) ~[opensearch-2.15.0.jar:2.15.0]
        at org.opensearch.env.NodeEnvironment.loadNodeMetadata(NodeEnvironment.java:479) ~[opensearch-2.15.0.jar:2.15.0]
        at org.opensearch.env.NodeEnvironment.<init>(NodeEnvironment.java:394) ~[opensearch-2.15.0.jar:2.15.0]
        at org.opensearch.env.NodeEnvironment.<init>(NodeEnvironment.java:301) ~[opensearch-2.15.0.jar:2.15.0]
        at org.opensearch.node.Node.<init>(Node.java:543) ~[opensearch-2.15.0.jar:2.15.0]
        at org.opensearch.node.Node.<init>(Node.java:425) ~[opensearch-2.15.0.jar:2.15.0]
        at org.opensearch.bootstrap.Bootstrap$5.<init>(Bootstrap.java:242) ~[opensearch-2.15.0.jar:2.15.0]
        at org.opensearch.bootstrap.Bootstrap.setup(Bootstrap.java:242) ~[opensearch-2.15.0.jar:2.15.0]
        at org.opensearch.bootstrap.Bootstrap.init(Bootstrap.java:404) ~[opensearch-2.15.0.jar:2.15.0]
        at org.opensearch.bootstrap.OpenSearch.init(OpenSearch.java:181) ~[opensearch-2.15.0.jar:2.15.0]
        ... 6 more
        Suppressed: org.apache.lucene.index.CorruptIndexException: checksum passed (e86ef5ee). possibly transient resource issue, or a Lucene or JVM bug (resource=BufferedChecksumIndexInput(NIOFSIndexInput(path="/usr/share/opensearch/data/nodes/0/_state/segments_r")))
                at org.apache.lucene.codecs.CodecUtil.checkFooter(CodecUtil.java:501) ~[lucene-core-9.10.0.jar:9.10.0 695c0ac84508438302cd346a812cfa2fdc5a10df - 2024-02-14 16:48:06]
                at org.apache.lucene.index.SegmentInfos.readCommit(SegmentInfos.java:375) ~[lucene-core-9.10.0.jar:9.10.0 695c0ac84508438302cd346a812cfa2fdc5a10df - 2024-02-14 16:48:06]
                at org.apache.lucene.index.SegmentInfos.readCommit(SegmentInfos.java:304) ~[lucene-core-9.10.0.jar:9.10.0 695c0ac84508438302cd346a812cfa2fdc5a10df - 2024-02-14 16:48:06]
                at org.apache.lucene.index.StandardDirectoryReader$1.doBody(StandardDirectoryReader.java:88) ~[lucene-core-9.10.0.jar:9.10.0 695c0ac84508438302cd346a812cfa2fdc5a10df - 2024-02-14 16:48:06]
                at org.apache.lucene.index.StandardDirectoryReader$1.doBody(StandardDirectoryReader.java:77) ~[lucene-core-9.10.0.jar:9.10.0 695c0ac84508438302cd346a812cfa2fdc5a10df - 2024-02-14 16:48:06]
                at org.apache.lucene.index.SegmentInfos$FindSegmentsFile.run(SegmentInfos.java:820) ~[lucene-core-9.10.0.jar:9.10.0 695c0ac84508438302cd346a812cfa2fdc5a10df - 2024-02-14 16:48:06]
                at org.apache.lucene.index.StandardDirectoryReader.open(StandardDirectoryReader.java:109) ~[lucene-core-9.10.0.jar:9.10.0 695c0ac84508438302cd346a812cfa2fdc5a10df - 2024-02-14 16:48:06]
                at org.apache.lucene.index.StandardDirectoryReader.open(StandardDirectoryReader.java:67) ~[lucene-core-9.10.0.jar:9.10.0 695c0ac84508438302cd346a812cfa2fdc5a10df - 2024-02-14 16:48:06]
                at org.apache.lucene.index.DirectoryReader.open(DirectoryReader.java:60) ~[lucene-core-9.10.0.jar:9.10.0 695c0ac84508438302cd346a812cfa2fdc5a10df - 2024-02-14 16:48:06]
                at org.opensearch.gateway.PersistedClusterStateService.nodeMetadata(PersistedClusterStateService.java:309) ~[opensearch-2.15.0.jar:2.15.0]
                at org.opensearch.env.NodeEnvironment.loadNodeMetadata(NodeEnvironment.java:479) ~[opensearch-2.15.0.jar:2.15.0]
                at org.opensearch.env.NodeEnvironment.<init>(NodeEnvironment.java:394) ~[opensearch-2.15.0.jar:2.15.0]
                at org.opensearch.env.NodeEnvironment.<init>(NodeEnvironment.java:301) ~[opensearch-2.15.0.jar:2.15.0]
                at org.opensearch.node.Node.<init>(Node.java:543) ~[opensearch-2.15.0.jar:2.15.0]
                at org.opensearch.node.Node.<init>(Node.java:425) ~[opensearch-2.15.0.jar:2.15.0]
                at org.opensearch.bootstrap.Bootstrap$5.<init>(Bootstrap.java:242) ~[opensearch-2.15.0.jar:2.15.0]
                at org.opensearch.bootstrap.Bootstrap.setup(Bootstrap.java:242) ~[opensearch-2.15.0.jar:2.15.0]
                at org.opensearch.bootstrap.Bootstrap.init(Bootstrap.java:404) ~[opensearch-2.15.0.jar:2.15.0]
                at org.opensearch.bootstrap.OpenSearch.init(OpenSearch.java:181) ~[opensearch-2.15.0.jar:2.15.0]
                at org.opensearch.bootstrap.OpenSearch.execute(OpenSearch.java:172) ~[opensearch-2.15.0.jar:2.15.0]
                at org.opensearch.cli.EnvironmentAwareCommand.execute(EnvironmentAwareCommand.java:104) ~[opensearch-2.15.0.jar:2.15.0]
                at org.opensearch.cli.Command.mainWithoutErrorHandling(Command.java:138) ~[opensearch-cli-2.15.0.jar:2.15.0]
                at org.opensearch.cli.Command.main(Command.java:101) ~[opensearch-cli-2.15.0.jar:2.15.0]
                at org.opensearch.bootstrap.OpenSearch.main(OpenSearch.java:138) ~[opensearch-2.15.0.jar:2.15.0]
                at org.opensearch.bootstrap.OpenSearch.main(OpenSearch.java:104) ~[opensearch-2.15.0.jar:2.15.0]
Caused by: java.lang.IllegalArgumentException: An SPI class of type org.apache.lucene.codecs.Codec with name 'Lucene912' does not exist.  You need to add the corresponding JAR file supporting this SPI to your classpath.  The current classpath supports the following names: [Lucene99, Lucene80, Lucene84, Lucene86, Lucene87, Lucene70, Lucene90, Lucene91, Lucene92, Lucene94, Lucene95, KNN80Codec, KNN84Codec, KNN86Codec, KNN87Codec, KNN910Codec, KNN920Codec, KNN940Codec, KNN950Codec, KNN990Codec, ZSTD, ZSTDNODICT, Lucene95CustomCodec, ZSTD99, ZSTDNODICT99, QATDEFLATE99, QATLZ499, CorrelationCodec950, CorrelationCodec990]
        at org.apache.lucene.util.NamedSPILoader.lookup(NamedSPILoader.java:113) ~[lucene-core-9.10.0.jar:9.10.0 695c0ac84508438302cd346a812cfa2fdc5a10df - 2024-02-14 16:48:06]
        at org.apache.lucene.codecs.Codec.forName(Codec.java:118) ~[lucene-core-9.10.0.jar:9.10.0 695c0ac84508438302cd346a812cfa2fdc5a10df - 2024-02-14 16:48:06]
        at org.apache.lucene.index.SegmentInfos.readCodec(SegmentInfos.java:516) ~[lucene-core-9.10.0.jar:9.10.0 695c0ac84508438302cd346a812cfa2fdc5a10df - 2024-02-14 16:48:06]
        at org.apache.lucene.index.SegmentInfos.parseSegmentInfos(SegmentInfos.java:409) ~[lucene-core-9.10.0.jar:9.10.0 695c0ac84508438302cd346a812cfa2fdc5a10df - 2024-02-14 16:48:06]
        at org.apache.lucene.index.SegmentInfos.readCommit(SegmentInfos.java:368) ~[lucene-core-9.10.0.jar:9.10.0 695c0ac84508438302cd346a812cfa2fdc5a10df - 2024-02-14 16:48:06]
        at org.apache.lucene.index.SegmentInfos.readCommit(SegmentInfos.java:304) ~[lucene-core-9.10.0.jar:9.10.0 695c0ac84508438302cd346a812cfa2fdc5a10df - 2024-02-14 16:48:06]
        at org.apache.lucene.index.StandardDirectoryReader$1.doBody(StandardDirectoryReader.java:88) ~[lucene-core-9.10.0.jar:9.10.0 695c0ac84508438302cd346a812cfa2fdc5a10df - 2024-02-14 16:48:06]
        at org.apache.lucene.index.StandardDirectoryReader$1.doBody(StandardDirectoryReader.java:77) ~[lucene-core-9.10.0.jar:9.10.0 695c0ac84508438302cd346a812cfa2fdc5a10df - 2024-02-14 16:48:06]
        at org.apache.lucene.index.SegmentInfos$FindSegmentsFile.run(SegmentInfos.java:820) ~[lucene-core-9.10.0.jar:9.10.0 695c0ac84508438302cd346a812cfa2fdc5a10df - 2024-02-14 16:48:06]
        at org.apache.lucene.index.StandardDirectoryReader.open(StandardDirectoryReader.java:109) ~[lucene-core-9.10.0.jar:9.10.0 695c0ac84508438302cd346a812cfa2fdc5a10df - 2024-02-14 16:48:06]
        at org.apache.lucene.index.StandardDirectoryReader.open(StandardDirectoryReader.java:67) ~[lucene-core-9.10.0.jar:9.10.0 695c0ac84508438302cd346a812cfa2fdc5a10df - 2024-02-14 16:48:06]
        at org.apache.lucene.index.DirectoryReader.open(DirectoryReader.java:60) ~[lucene-core-9.10.0.jar:9.10.0 695c0ac84508438302cd346a812cfa2fdc5a10df - 2024-02-14 16:48:06]
        at org.opensearch.gateway.PersistedClusterStateService.nodeMetadata(PersistedClusterStateService.java:309) ~[opensearch-2.15.0.jar:2.15.0]
        at org.opensearch.env.NodeEnvironment.loadNodeMetadata(NodeEnvironment.java:479) ~[opensearch-2.15.0.jar:2.15.0]
        at org.opensearch.env.NodeEnvironment.<init>(NodeEnvironment.java:394) ~[opensearch-2.15.0.jar:2.15.0]
        at org.opensearch.env.NodeEnvironment.<init>(NodeEnvironment.java:301) ~[opensearch-2.15.0.jar:2.15.0]
        at org.opensearch.node.Node.<init>(Node.java:543) ~[opensearch-2.15.0.jar:2.15.0]
        at org.opensearch.node.Node.<init>(Node.java:425) ~[opensearch-2.15.0.jar:2.15.0]
        at org.opensearch.bootstrap.Bootstrap$5.<init>(Bootstrap.java:242) ~[opensearch-2.15.0.jar:2.15.0]
        at org.opensearch.bootstrap.Bootstrap.setup(Bootstrap.java:242) ~[opensearch-2.15.0.jar:2.15.0]
        at org.opensearch.bootstrap.Bootstrap.init(Bootstrap.java:404) ~[opensearch-2.15.0.jar:2.15.0]
        at org.opensearch.bootstrap.OpenSearch.init(OpenSearch.java:181) ~[opensearch-2.15.0.jar:2.15.0]
        ... 6 more
uncaught exception in thread [main]
java.lang.IllegalArgumentException: Could not load codec 'Lucene912'. Did you forget to add lucene-backward-codecs.jar?
Likely root cause: java.lang.IllegalArgumentException: An SPI class of type org.apache.lucene.codecs.Codec with name 'Lucene912' does not exist.  You need to add the corresponding JAR file supporting this SPI to your classpath.  The current classpath supports the following names: [Lucene99, Lucene80, Lucene84, Lucene86, Lucene87, Lucene70, Lucene90, Lucene91, Lucene92, Lucene94, Lucene95, KNN80Codec, KNN84Codec, KNN86Codec, KNN87Codec, KNN910Codec, KNN920Codec, KNN940Codec, KNN950Codec, KNN990Codec, ZSTD, ZSTDNODICT, Lucene95CustomCodec, ZSTD99, ZSTDNODICT99, QATDEFLATE99, QATLZ499, CorrelationCodec950, CorrelationCodec990]
        at org.apache.lucene.util.NamedSPILoader.lookup(NamedSPILoader.java:113)
        at org.apache.lucene.codecs.Codec.forName(Codec.java:118)
        at org.apache.lucene.index.SegmentInfos.readCodec(SegmentInfos.java:516)
        at org.apache.lucene.index.SegmentInfos.parseSegmentInfos(SegmentInfos.java:409)
        at org.apache.lucene.index.SegmentInfos.readCommit(SegmentInfos.java:368)
        at org.apache.lucene.index.SegmentInfos.readCommit(SegmentInfos.java:304)
        at org.apache.lucene.index.StandardDirectoryReader$1.doBody(StandardDirectoryReader.java:88)
        at org.apache.lucene.index.StandardDirectoryReader$1.doBody(StandardDirectoryReader.java:77)
        at org.apache.lucene.index.SegmentInfos$FindSegmentsFile.run(SegmentInfos.java:820)
        at org.apache.lucene.index.StandardDirectoryReader.open(StandardDirectoryReader.java:109)
        at org.apache.lucene.index.StandardDirectoryReader.open(StandardDirectoryReader.java:67)
        at org.apache.lucene.index.DirectoryReader.open(DirectoryReader.java:60)
        at org.opensearch.gateway.PersistedClusterStateService.nodeMetadata(PersistedClusterStateService.java:309)
        at org.opensearch.env.NodeEnvironment.loadNodeMetadata(NodeEnvironment.java:479)
        at org.opensearch.env.NodeEnvironment.<init>(NodeEnvironment.java:394)
        at org.opensearch.env.NodeEnvironment.<init>(NodeEnvironment.java:301)
        at org.opensearch.node.Node.<init>(Node.java:543)
        at org.opensearch.node.Node.<init>(Node.java:425)
        at org.opensearch.bootstrap.Bootstrap$5.<init>(Bootstrap.java:242)
        at org.opensearch.bootstrap.Bootstrap.setup(Bootstrap.java:242)
        at org.opensearch.bootstrap.Bootstrap.init(Bootstrap.java:404)
        at org.opensearch.bootstrap.OpenSearch.init(OpenSearch.java:181)
        at org.opensearch.bootstrap.OpenSearch.execute(OpenSearch.java:172)
        at org.opensearch.cli.EnvironmentAwareCommand.execute(EnvironmentAwareCommand.java:104)
        at org.opensearch.cli.Command.mainWithoutErrorHandling(Command.java:138)
        at org.opensearch.cli.Command.main(Command.java:101)
        at org.opensearch.bootstrap.OpenSearch.main(OpenSearch.java:138)
        at org.opensearch.bootstrap.OpenSearch.main(OpenSearch.java:104)
For complete error details, refer to the log at /usr/share/opensearch/logs/opensearch-cluster.log

To Reproduce
Steps to reproduce the behavior:

  1. Use the following values-override.yaml file:
singleNode: true
image:
  tag: "2.15.0"
extraEnvs:
  - name: OPENSEARCH_INITIAL_ADMIN_PASSWORD
    value: 'Yabba dabba d00!'
  1. Install chart passing the above values-override.yaml file.
  2. Observe pod logs

Expected behavior
Pod starts healthy

Chart Name
opensearch/opensearch Chart

Host/Environment (please complete the following information):

  • Helm Version: 3.16.2
  • Kubernetes Version: 1.31.2

Additional context
Add any other context about the problem here.
@williamtrelawny williamtrelawny added bug Something isn't working untriaged Issues that have not yet been triaged labels Nov 28, 2024
@williamtrelawny
Copy link
Author

williamtrelawny commented Nov 28, 2024 •
edited
Loading

Only thing awry I noticed is that even though I specified .Values.image.tag: "2.15.0", various annotations still show .Chart.appVersion: 2.18.0. Could this be causing some sort of conflict?

@prudhvigodithi
Copy link
Member

Only thing awry I noticed is that even though I specified .Values.image.tag: "2.15.0", various annotations still show .Chart.appVersion: 2.18.0. Could this be causing some sort of conflict?

[Triage]
Looks like yes there is a version conflict, are you looking to start OpenSearch with 2.15.0 or 2.18.0?, can you also try a clean install with latest helm chart https://artifacthub.io/packages/helm/opensearch-project-helm-charts/opensearch.
Thanks

@prudhvigodithi prudhvigodithi removed the untriaged Issues that have not yet been triaged label Dec 5, 2024
@williamtrelawny
Copy link
Author

Yes, trying to do a new install with v2.15.0. Clean install with latest version works fine.

@prudhvigodithi
Copy link
Member

Thanks in that case, can you try to update the appVersion with 2.15.0 and try to install?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug Something isn't working
Projects
Engineering Effectiveness Board
Status: 🆕 New
Milestone
No milestone
Development

No branches or pull requests
2 participants
@williamtrelawny @prudhvigodithi