Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Unable to run dependencies-report -- reporting cannot load (ext) (org.jruby.ext.psych.PsychLibrary) #15427

Closed
dliappis opened this issue Oct 11, 2023 · 1 comment · Fixed by #15433
Closed

Unable to run dependencies-report -- reporting cannot load (ext) (org.jruby.ext.psych.PsychLibrary) #15427

dliappis opened this issue Oct 11, 2023 · 1 comment · Fixed by #15433
Labels
bug

Comments

@dliappis
Copy link
Contributor

Logstash information:

Latest checkout from main

JVM (e.g. java -version):

# java -version
openjdk version "17.0.8.1" 2023-08-24
OpenJDK Runtime Environment (build 17.0.8.1+1-Ubuntu-0ubuntu122.04)
OpenJDK 64-Bit Server VM (build 17.0.8.1+1-Ubuntu-0ubuntu122.04, mixed mode, sharing)

OS version (uname -a if on a Unix-like system):

Ubuntu 22.04.3 LTS (using Docker ubuntu:jammy)

Description of the problem including expected versus actual behavior:

When running the dependencies-report from the latest commit on the main branch of Logstash on a docker container based on Ubuntu 22.04, we are getting a Bundler::GemspecError: mentioning
error while loading logstash-core-plugin-api.gemspec: cannot load (ext) (org.jruby.ext.psych.PsychLibrary). Bundler cannot continue.

We've hit something similar in #15356 (see PR description) which forced us to switch to VMs.

Steps to reproduce:

Use the following Dockerfile:

FROM ubuntu:jammy

RUN apt-get update && \
    apt-get install -y zlib1g-dev build-essential vim rake git curl libssl-dev libreadline-dev libyaml-dev  \
      libxml2-dev libxslt-dev openjdk-17-jdk-headless curl iputils-ping netcat git && \
    apt-get clean

WORKDIR /root

RUN git clone https://github.com/elastic/logstash
RUN export GRADLE_OPTS="-Xmx2g -Dorg.gradle.daemon=false -Dorg.gradle.logging.level=info -Dfile.encoding=UTF-8" && \
    cd logstash && \
    ./gradlew installDefaultGems && \
    bin/dependencies-report --csv report.csv

and then run docker build .

this will result in:

295.8 Using system java: /usr/bin/java
298.2 logstash-core/lib/logstash/dependency_report_runner.rb:32: warning: already initialized constant Bundler::GemfileNotFound
298.2 No Gemfile found. Maybe you need to run `rake artifact:tar`?
298.2 Bundler::GemspecError: 
298.2 [!] There was an error while loading `logstash-core-plugin-api.gemspec`: cannot load (ext) (org.jruby.ext.psych.PsychLibrary). Bundler cannot continue.
298.2 
298.2  #  from /root/logstash/logstash-core-plugin-api/logstash-core-plugin-api.gemspec:24
298.2  #  -------------------------------------------
298.2  #  
298.2  >  require "logstash-core-plugin-api/version"
298.2  #  
298.2  #  -------------------------------------------
298.2 
298.2                eval_gemspec at /root/logstash/vendor/jruby/lib/ruby/stdlib/bundler.rb:679
298.2       load_gemspec_uncached at /root/logstash/vendor/jruby/lib/ruby/stdlib/bundler.rb:596
298.2                       chdir at org/jruby/RubyDir.java:452
298.2                       chdir at /root/logstash/vendor/jruby/lib/ruby/stdlib/bundler/shared_helpers.rb:52
298.2                 synchronize at org/jruby/ext/monitor/Monitor.java:82
298.2                       chdir at /root/logstash/vendor/jruby/lib/ruby/stdlib/bundler/shared_helpers.rb:51
298.2       load_gemspec_uncached at /root/logstash/vendor/jruby/lib/ruby/stdlib/bundler.rb:595
298.2                load_gemspec at /root/logstash/vendor/jruby/lib/ruby/stdlib/bundler.rb:581
298.2                load_gemspec at /root/logstash/vendor/jruby/lib/ruby/stdlib/bundler/source/path.rb:162
298.2             load_spec_files at /root/logstash/vendor/jruby/lib/ruby/stdlib/bundler/source/path.rb:177
298.2                        each at org/jruby/RubyArray.java:1987
298.2             load_spec_files at /root/logstash/vendor/jruby/lib/ruby/stdlib/bundler/source/path.rb:176
298.2                 local_specs at /root/logstash/vendor/jruby/lib/ruby/stdlib/bundler/source/path.rb:107
298.2                       specs at /root/logstash/vendor/jruby/lib/ruby/stdlib/bundler/source/path.rb:115
298.2   specs_for_source_changed? at /root/logstash/vendor/jruby/lib/ruby/stdlib/bundler/definition.rb:604
298.2              specs_changed? at /root/logstash/vendor/jruby/lib/ruby/stdlib/bundler/definition.rb:589
298.2              converge_paths at /root/logstash/vendor/jruby/lib/ruby/stdlib/bundler/definition.rb:633
298.2                        any? at org/jruby/RubyArray.java:4958
298.2              converge_paths at /root/logstash/vendor/jruby/lib/ruby/stdlib/bundler/definition.rb:632
298.2                  initialize at /root/logstash/vendor/jruby/lib/ruby/stdlib/bundler/definition.rb:136
298.2                         new at org/jruby/RubyClass.java:931
298.2               to_definition at /root/logstash/vendor/jruby/lib/ruby/stdlib/bundler/dsl.rb:218
298.2                    evaluate at /root/logstash/vendor/jruby/lib/ruby/stdlib/bundler/dsl.rb:13
298.2                       build at /root/logstash/vendor/jruby/lib/ruby/stdlib/bundler/definition.rb:38
298.2                  definition at /root/logstash/vendor/jruby/lib/ruby/stdlib/bundler.rb:207
298.2                       setup at /root/logstash/vendor/jruby/lib/ruby/stdlib/bundler.rb:155
298.2                      setup! at /root/logstash/lib/bootstrap/bundler.rb:83
298.2                      <main> at logstash-core/lib/logstash/dependency_report_runner.rb:22
298.2 NameError: cannot load (ext) (org.jruby.ext.psych.PsychLibrary)
298.2                    load_ext at org/jruby/ext/jruby/JRubyUtilLibrary.java:216
298.2                      <main> at /root/logstash/vendor/bundle/jruby/3.1.0/gems/psych-5.1.1-java/lib/psych.rb:7
298.2                     require at org/jruby/RubyKernel.java:1057
298.2                     require at /root/logstash/vendor/jruby/lib/ruby/stdlib/rubygems/core_ext/kernel_require.rb:85
298.2                      <main> at /root/logstash/vendor/jruby/lib/ruby/stdlib/yaml.rb:4
298.2                     require at org/jruby/RubyKernel.java:1057
298.2                     require at /root/logstash/vendor/jruby/lib/ruby/stdlib/rubygems/core_ext/kernel_require.rb:85
298.2                      <main> at /root/logstash/logstash-core-plugin-api/lib/logstash-core-plugin-api/version.rb:22
298.2                     require at org/jruby/RubyKernel.java:1057
298.2                     require at /root/logstash/vendor/jruby/lib/ruby/stdlib/rubygems/core_ext/kernel_require.rb:85
298.2                eval_gemspec at /root/logstash/logstash-core-plugin-api/logstash-core-plugin-api.gemspec:24
298.2                        eval at org/jruby/RubyKernel.java:1143
298.2                eval_gemspec at /root/logstash/vendor/jruby/lib/ruby/stdlib/bundler.rb:675
298.2       load_gemspec_uncached at /root/logstash/vendor/jruby/lib/ruby/stdlib/bundler.rb:596
298.2                       chdir at org/jruby/RubyDir.java:452
298.2                       chdir at /root/logstash/vendor/jruby/lib/ruby/stdlib/bundler/shared_helpers.rb:52
298.2                 synchronize at org/jruby/ext/monitor/Monitor.java:82
298.2                       chdir at /root/logstash/vendor/jruby/lib/ruby/stdlib/bundler/shared_helpers.rb:51
298.2       load_gemspec_uncached at /root/logstash/vendor/jruby/lib/ruby/stdlib/bundler.rb:595
298.2                load_gemspec at /root/logstash/vendor/jruby/lib/ruby/stdlib/bundler.rb:581
298.2                load_gemspec at /root/logstash/vendor/jruby/lib/ruby/stdlib/bundler/source/path.rb:162
298.2             load_spec_files at /root/logstash/vendor/jruby/lib/ruby/stdlib/bundler/source/path.rb:177
298.2                        each at org/jruby/RubyArray.java:1987
298.2             load_spec_files at /root/logstash/vendor/jruby/lib/ruby/stdlib/bundler/source/path.rb:176
298.2                 local_specs at /root/logstash/vendor/jruby/lib/ruby/stdlib/bundler/source/path.rb:107
298.2                       specs at /root/logstash/vendor/jruby/lib/ruby/stdlib/bundler/source/path.rb:115
298.2   specs_for_source_changed? at /root/logstash/vendor/jruby/lib/ruby/stdlib/bundler/definition.rb:604
298.2              specs_changed? at /root/logstash/vendor/jruby/lib/ruby/stdlib/bundler/definition.rb:589
298.2              converge_paths at /root/logstash/vendor/jruby/lib/ruby/stdlib/bundler/definition.rb:633
298.2                        any? at org/jruby/RubyArray.java:4958
298.2              converge_paths at /root/logstash/vendor/jruby/lib/ruby/stdlib/bundler/definition.rb:632
298.2                  initialize at /root/logstash/vendor/jruby/lib/ruby/stdlib/bundler/definition.rb:136
298.2                         new at org/jruby/RubyClass.java:931
298.2               to_definition at /root/logstash/vendor/jruby/lib/ruby/stdlib/bundler/dsl.rb:218
298.2                    evaluate at /root/logstash/vendor/jruby/lib/ruby/stdlib/bundler/dsl.rb:13
298.2                       build at /root/logstash/vendor/jruby/lib/ruby/stdlib/bundler/definition.rb:38
298.2                  definition at /root/logstash/vendor/jruby/lib/ruby/stdlib/bundler.rb:207
298.2                       setup at /root/logstash/vendor/jruby/lib/ruby/stdlib/bundler.rb:155
298.2                      setup! at /root/logstash/lib/bootstrap/bundler.rb:83
298.2                      <main> at logstash-core/lib/logstash/dependency_report_runner.rb:22
------
Dockerfile:11
--------------------
  10 |     RUN git clone https://github.com/elastic/logstash
  11 | >>> RUN export GRADLE_OPTS="-Xmx2g -Dorg.gradle.daemon=false -Dorg.gradle.logging.level=info -Dfile.encoding=UTF-8" && \
  12 | >>>     cd logstash && \
  13 | >>>     ./gradlew installDefaultGems && \
  14 | >>>     bin/dependencies-report --csv report.csv
  15 |     
--------------------
ERROR: failed to solve: process "/bin/sh -c export GRADLE_OPTS=\"-Xmx2g -Dorg.gradle.daemon=false -Dorg.gradle.logging.level=info -Dfile.encoding=UTF-8\" &&     cd logstash &&     ./gradlew installDefaultGems &&     bin/dependencies-report --csv report.csv" did not complete successfully: exit code: 1
dliappis added a commit to dliappis/logstash that referenced this issue Oct 11, 2023
@dliappis
[ci] Use VM for PR license check on Buildkite
9a954ca 
This commit switches the license check to be run using a VM (as its
currently done in Jenkins) due to the problem described in
elastic#15427.

This is a bit unfortunate because the Jenkins workflow
builds a Docker image (based on Ubuntu 18.04) within the VM and then
runs the check inside a container. We will address this issue and
remove this unnecessary VM -> build docker image -> run inside Docker
in a future PR.
@dliappis dliappis mentioned this issue Oct 11, 2023
Closed
@dliappis
Copy link
Contributor Author

dliappis commented Oct 11, 2023 •
edited
Loading

Actually it appears that something may have changed since yesterday and this problem is also affecting Jenkins. In other words, this is not specific to Ubuntu 22.04.

See:

@dliappis dliappis changed the title Unable to run dependencies-report on Docker using Ubuntu:22.04 -- reporting cannot load (ext) (org.jruby.ext.psych.PsychLibrary) Unable to run dependencies-report -- reporting cannot load (ext) (org.jruby.ext.psych.PsychLibrary) Oct 11, 2023
@dliappis dliappis mentioned this issue Oct 11, 2023
Merged
@github-actions github-actions bot mentioned this issue Oct 11, 2023
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

pin psych 5.1.0 due to ruby/psych#655 jsvd/logstash
2 participants
@dliappis @roaksoax