LogStash::ConfigManagement::BootstrapCheck raises a LogStash::BootstrapCheckError when running ruby tests in isolation on Windows

[dliappis]

Background

While working on the ability to run separately the Java and Ruby unit tests on Windows (#15861 and #15888)) I noticed that when running both tests with .\gradlew test --console=plain --no-daemon --info everything succeeds.

However, if we run the rubyTests specifically with: .\gradlew rubyTests --console=plain --no-daemon --info, the test LogStash::ConfigManagement::BootstrapCheck fails consistently (on all Windows variants).

Buildkite failure link: https://buildkite.com/elastic/logstash-windows-jdk-matrix-pipeline/builds/98#018d63dd-bfe0-425b-a822-e8e3110eba76/56-8127

Interestingly enough, running the javaTests in isolation works.
I also tested, unsuccessfully, if running the entire suite first makes a subsequent run of rubyTests work.

Failure excerpt

      1) LogStash::ConfigManagement::BootstrapCheck when `xpack.management.enabled` is `TRUE` when `path.config` is given when a configuration file exists in the specified location raises a `LogStash::BootstrapCheckError` error
         Failure/Error: expect { subject.check(settings) }.to raise_error LogStash::BootstrapCheckError
           expected LogStash::BootstrapCheckError but nothing was raised
         # ./spec/config_management/bootstrap_check_spec.rb:125:in `block in <main>'
         # A:/vendor/bundle/jruby/3.1.0/gems/rspec-wait-0.0.9/lib/rspec/wait.rb:46:in `block in <main>'
         # A:/vendor/bundle/jruby/3.1.0/gems/logstash-devutils-2.6.2-java/lib/logstash/devutils/rspec/spec_helper.rb:47:in `block in <main>'
         # A:\lib\bootstrap\rspec.rb:36:in `<main>'
    Finished in 32.6 seconds (files took 6.42 seconds to load)
    370 examples, 1 failure, 4 pending
    Failed examples:
    rspec ./spec/config_management/bootstrap_check_spec.rb:124 # LogStash::ConfigManagement::BootstrapCheck when `xpack.management.enabled` is `TRUE` when `path.config` is given when a configuration file exists in the specified location raises a `LogStash::BootstrapCheckError` error
    Randomized with seed 56404

Applicable branches

All branches

Analysis

@yaauie noticed that the bootstrap task is a dependency of javaTests, but not of rubyTests and feels that this particular test is failing because we use a library to create a “temporary” file without writing any bytes to it, and then run code that should throw an exception if the file exists. It may be possible that Windows isn’t syncing the file handle that hasn’t been written to, or not syncing it fast enough for it to be present when we do a glob listing to search for it.

Reproduction

Please refer to #15861 on how to run it (see also https://docs.elastic.dev/ingest-dev-docs/logstash/logstash-jdk-matrix-buildkite-ci#troubleshooting-and-reproducing-failures)