fluent-package v5.1.0

[kenhys]

Tracking v5.1.0 issues still in development.

• Fix increased package size
  • rdkafka 0.16.1 causes it (+5MB)
  • confluent test fails with rdkafka 0.16.1
  • Revert to v0.16.0. (RPM: ~+3MB, DEB: ~+2MB)
  • ✅ Upgraded to v0.16.1 again. (No need to relax size limitation and resolved confluent CI failure)
• Fix build failure on AmazonLinux2 with ffi
  • NG: v1.17.0
  • NG: v1.16.3
  • ✅ Suppress byte compile hook.
• Add confluent test case for Red Hat and derivatives.
  • rpm: add confluent (kafka) test case #661
  • Refactor confluent test case.
• Fix build failure on windows
  • Fixed inconsistent webhdfs dependency in Gemfile.lock
• Fix container image name inconsistency.
  • rpm: unify to short name #664
• Improve GitHub Actions logging
  • deb rpm: group logging #665
• Disable macOS GitHub Actions
  • Disabled macOS GitHub Actions workflow #666
• Bundle winevt_c 0.11.0
• Switch package signing key #579
• Add v5.1.0 #659

[kenhys]

bundled gem size was increased. (~5MB)

[kenhys]

mainly rdkafka upgrade affect it.( 2.3MB=>7.2MB )

[kenhys]

need to relax limitation for a while.

[kenhys]

ffi 1.16.3 can't be built on AmazonLinux 2.

https://github.com/fluent/fluent-package-builder/actions/runs/10034848340/job/27729984596

[kenhys]

rdkafka 0.16.1 is not correctly built?

[kenhys]

If rdkafka was reverted to 0.16.0, package size was not so increased. (may be no need to relax limitation )

[kenhys]

rdkafka 0.16.0 is buldable, but not available in colfluent test.

[kenhys]

/opt/fluent/bin/gem list |grep rdkafka
rdkafka (0.16.0)

rdkafka was installed. but not recognized... 🤔

[kenhys]

rdkafka 0.16.0 was installed at:

~$ find /opt/fluent/ -name '*rdkafka*'
/opt/fluent/lib/ruby/gems/3.2.0/specifications/rdkafka-0.16.0.gemspec
/opt/fluent/lib/ruby/gems/3.2.0/gems/rdkafka-0.16.0
/opt/fluent/lib/ruby/gems/3.2.0/gems/rdkafka-0.16.0/lib/rdkafka
/opt/fluent/lib/ruby/gems/3.2.0/gems/rdkafka-0.16.0/lib/rdkafka.rb
/opt/fluent/lib/ruby/gems/3.2.0/gems/rdkafka-0.16.0/ext/librdkafka.so
/opt/fluent/lib/ruby/gems/3.2.0/gems/rdkafka-0.16.0/rdkafka.gemspec
/opt/fluent/lib/ruby/gems/3.2.0/gems/fluent-plugin-kafka-0.19.2/lib/fluent/plugin/in_rdkafka_group.rb
/opt/fluent/lib/ruby/gems/3.2.0/gems/fluent-plugin-kafka-0.19.2/lib/fluent/plugin/out_rdkafka.rb
/opt/fluent/lib/ruby/gems/3.2.0/gems/fluent-plugin-kafka-0.19.2/lib/fluent/plugin/out_rdkafka2.rb
/opt/fluent/lib/ruby/gems/3.2.0/extensions/x86_64-linux/3.2.0/rdkafka-0.16.0

[kenhys]

The rdkafka failure is caused by removing gem.build_complete.

[kenhys]

https://github.com/fluent/fluent-package-builder/actions/runs/10036365251/job/27733978175

windows_platforms = [:mingw, :x64_mingw] # :mswin
gem "ffi", "1.16.3", platforms: windows_platforms

Thus, platforms: does not work as expected on AmazonLinux 2.

[kenhys]

+ /usr/lib/rpm/brp-python-bytecompile /usr/bin/python 1
Compiling /root/rpmbuild/BUILDROOT/fluent-package-5.1.0-1.amzn2.x86_64/opt/fluent/lib/ruby/gems/3.2.0/gems/ffi-1.16.3/ext/ffi_c/libffi/generate-darwin-source-and-headers.py ...

And during building debuinfo?, unexpectedly bytecompile process was kicked.

[kenhys]

And during building debuinfo?, unexpectedly bytecompile process was kicked.

Same result (v1.17.0) on AmazonLinux 2.

[kenhys]

https://fedoraproject.org/wiki/Changes/No_more_automagic_Python_bytecompilation

does not work as expected...

[kenhys]

We should follow https://docs.fedoraproject.org/en-US/packaging-guidelines/#_brp_buildroot_policy_scripts

[kenhys]

container ea60ff1bf4515e0721f4583107bf62c2fee67b8fe97b9435ff8ca5bf4d1a295c encountered an error during hcs::System::Start: context deadline exceeded

it may be resolved with retry.

[kenhys]

Check whether rdkafka 0.16.0 => 0.16.1 upgrade can be bundled or not again.
(as gem build_complete issue was resolved)

[kenhys]

Blocker issue on Windows:

Downloading fluent-plugin-webhdfs-1.6.0 revealed dependencies not in the API or 
the lockfile (webhdfs (>= 0.11.0)).
Either installing with `--full-index` or running `bundle update
fluent-plugin-webhdfs` should fix the problem.
rake aborted!

[kenhys]

Note: confluent community can't be installed on AmazonLinux (via package management)

[kenhys]

Note: confluent community can't be installed on AmazonLinux (via package management)

Precisely, confluent-client repository is not supported.

[kenhys]

build failure on windows was fixed in
https://github.com/fluent/fluent-package-builder/actions/runs/10052696903/job/27784298154

[kenhys]

kafka related test case is bit out-dated. so refactor it now.

[kenhys]

NOTE:

Now macOS workflow always fails because of refreshed runner image

https://github.com/fluent/fluent-package-builder/actions/runs/10105868174/job/27947051208
macos-14-arm64 20240722.3 causes failure.

https://github.com/fluent/fluent-package-builder/actions/runs/10091587427/job/27903442530
until version macos-14-arm64 20240714.2, it succeeds.

[kenhys]

recently nokogiri 1.16.7 was released, catch up it.

[kenhys]

recently nokogiri 1.16.7 was released, catch up it.

done.

[kenhys]

We need to tackle windows build failure...

[kenhys]

Try to update to ruby 3.2.5.

[kenhys]

We need to tackle windows build failure...

Noticed that nokogiri 1.16.7 fails to build on windows, but not reproducible now...

[kenhys]

build process on arm works as expected.

[kenhys]

Even though some fixes for script are needed, but it works well with switching signing key.
checked with test/experimental/5 repository.

[kenhys]

Done!