Berkshelf started failing cannot load such file -- archive/tar/minitar

[johnymachine]

Description

Berkshelf started failing in recent installations

Berkshelf Version

8.0.9 on old system (works)
8.0.9 on new system (fails)

Platform Version

Ubuntu 20.04 on AWS

Replication Case

curl -Ls https://omnitruck.chef.io/install.sh | sudo bash -s -- -v 18.5.0
/opt/chef/embedded/bin/gem install --no-document berkshelf

Berks Output

running simply berks results in:

<internal:/usr/local/lib/ruby/3.2.0/rubygems/core_ext/kernel_require.rb>:85:in `require': cannot load such file -- archive/tar/minitar (LoadError)
	from <internal:/usr/local/lib/ruby/3.2.0/rubygems/core_ext/kernel_require.rb>:85:in `require'
	from /usr/local/lib/ruby/gems/3.2.0/gems/berkshelf-8.0.9/lib/berkshelf/packager.rb:1:in `<top (required)>'
	from /usr/local/lib/ruby/gems/3.2.0/gems/berkshelf-8.0.9/lib/berkshelf/berksfile.rb:1:in `require_relative'
	from /usr/local/lib/ruby/gems/3.2.0/gems/berkshelf-8.0.9/lib/berkshelf/berksfile.rb:1:in `<top (required)>'
	from /usr/local/lib/ruby/gems/3.2.0/gems/berkshelf-8.0.9/lib/berkshelf.rb:222:in `require_relative'
	from /usr/local/lib/ruby/gems/3.2.0/gems/berkshelf-8.0.9/lib/berkshelf.rb:222:in `<top (required)>'
	from /usr/local/lib/ruby/gems/3.2.0/gems/berkshelf-8.0.9/lib/berkshelf/cli.rb:1:in `require_relative'
	from /usr/local/lib/ruby/gems/3.2.0/gems/berkshelf-8.0.9/lib/berkshelf/cli.rb:1:in `<top (required)>'
	from <internal:/usr/local/lib/ruby/3.2.0/rubygems/core_ext/kernel_require.rb>:85:in `require'
	from <internal:/usr/local/lib/ruby/3.2.0/rubygems/core_ext/kernel_require.rb>:85:in `require'
	from /usr/local/lib/ruby/gems/3.2.0/gems/berkshelf-8.0.9/bin/berks:3:in `<top (required)>'
	from /usr/local/bin/berks:25:in `load'
	from /usr/local/bin/berks:25:in `<main>'

Stacktrace

[enrico-usai]

I'm facing the same issue when executing

/opt/cinc/embedded/bin/berks vendor /etc/chef/cookbooks --delete

I noticed a change in the sudo /opt/cinc/embedded/bin/gem install --no-document berkshelf:8.0.7 output.

Previously berskhelf installation log was:

| 2024-08-07 21:42:32.619 | Step InstallBerkshelf |
| 2024-08-07 21:42:32.623 | ExecuteBash: STARTED EXECUTION |
| 2024-08-07 21:42:37.788 | Stdout: Successfully installed molinillo-0.8.0 |
| 2024-08-07 21:42:37.788 | Stdout: Successfully installed solve-4.0.4 |
| 2024-08-07 21:42:37.788 | Stdout: Successfully installed retryable-3.0.5 |
| 2024-08-07 21:42:37.788 | Stdout: Successfully installed sawyer-0.9.2 |
| 2024-08-07 21:42:37.788 | Stdout: Successfully installed octokit-4.25.1 |
| 2024-08-07 21:42:37.788 | Stdout: Successfully installed concurrent-ruby-1.3.3 |
| 2024-08-07 21:42:37.788 | Stdout: Successfully installed minitar-1.0.0 |
| 2024-08-07 21:42:37.788 | Stdout: Successfully installed cleanroom-1.0.0 |
| 2024-08-07 21:42:37.788 | Stdout: Successfully installed aws-sigv4-1.9.1 |
| 2024-08-07 21:42:37.788 | Stdout: Building native extensions. This could take a while... |
| 2024-08-07 21:42:43.193 | Stdout: Successfully installed ffi-1.17.0 |
| 2024-08-07 21:42:43.193 | Stdout: Successfully installed net-ssh-7.2.3 |
| 2024-08-07 21:42:43.193 | Stdout: Successfully installed berkshelf-8.0.7 |
| 2024-08-07 21:42:43.193 | Stdout: 12 gems installed |
| 2024-08-07 21:42:43.205 | Stderr: sudo /opt/cinc/embedded/bin/gem install --no-document berkshelf:8.0.7 |
| 2024-08-07 21:42:43.205 | CmdExecution: Command execution has been completed |
| 2024-08-07 21:42:43.205 | CmdExecution: Command execution was completed successfully |
| 2024-08-07 21:42:43.205 | CmdExecution: Stderr: sudo /opt/cinc/embedded/bin/gem install --no-document berkshelf:8.0.7 |
| 2024-08-07 21:42:43.205 | CmdExecution: ExitCode 0 |
| 2024-08-07 21:42:43.205 | ExecuteBash: FINISHED EXECUTION |

Now is:

| 2024-08-07 16:46:28.831 | Step InstallBerkshelf |
| 2024-08-07 16:46:28.835 | ExecuteBash: STARTED EXECUTION |
| 2024-08-07 16:46:33.875 | Stdout: Building native extensions. This could take a while... |
| 2024-08-07 16:46:33.875 | Stdout: Successfully installed aws-sigv4-1.9.1 |
| 2024-08-07 16:46:33.875 | Stdout: Successfully installed cleanroom-1.0.0 |
| 2024-08-07 16:46:33.875 | Stdout: Successfully installed minitar-0.12 |
| 2024-08-07 16:46:33.875 | Stdout: expecting this executable, make sure you also install `minitar-cli`. |
| 2024-08-07 16:46:33.875 | Stdout: The `minitar` executable is no longer bundled with `minitar`. If you are |
| 2024-08-07 16:46:33.875 | Stdout: Successfully installed concurrent-ruby-1.3.3 |
| 2024-08-07 16:46:33.875 | Stdout: Successfully installed octokit-4.25.1 |
| 2024-08-07 16:46:33.875 | Stdout: Successfully installed sawyer-0.9.2 |
| 2024-08-07 16:46:33.875 | Stdout: Successfully installed retryable-3.0.5 |
| 2024-08-07 16:46:33.875 | Stdout: Successfully installed solve-4.0.4 |
| 2024-08-07 16:46:33.875 | Stdout: Successfully installed molinillo-0.8.0 |
| 2024-08-07 16:46:39.222 | Stdout: 12 gems installed |
| 2024-08-07 16:46:39.222 | Stdout: Successfully installed berkshelf-8.0.7 |
| 2024-08-07 16:46:39.222 | Stdout: Successfully installed net-ssh-7.2.3 |
| 2024-08-07 16:46:39.222 | Stdout: Successfully installed ffi-1.17.0 |
| 2024-08-07 16:46:39.234 | ExecuteBash: FINISHED EXECUTION |

So the workaround, while waiting for a new berkshelf release could be to explicitly install minitar-cli gem (to be verified).

| 2024-08-07 16:46:33.875 | Stdout: expecting this executable, make sure you also install `minitar-cli`. |
| 2024-08-07 16:46:33.875 | Stdout: The `minitar` executable is no longer bundled with `minitar`.

[johnymachine]

There might be simple fix then, possibly just add minitar-cli here:

berkshelf/berkshelf.gemspec

Line 38 in 4d34215

s.add_dependency "minitar", ">= 0.6"

[zaichang]

On August 6, 2024 they published a breaking change in the archive-tar-minitar gem with v0.12 (See halostatue/minitar@4341800)

Fixing this within berkshelf should theoretically mean raising the minimum version of minitar to 0.12 and changing this require statement

berkshelf/lib/berkshelf/packager.rb

Line 1 in 4d34215

require "archive/tar/minitar"

to

require "minitar"

Or, to fix this without changing berkshelf itself is to pin archive-tar-minitar to v0.8 before installing berkshelf:

gem install archive-tar-minitar:0.8 berkshelf:8.0.9

[halostatue]

On August 6, 2024 they published a breaking change in the archive-tar-minitar gem with v0.12 (See halostatue/minitar@4341800)

This is incorrect. On 2024-08-06, I published the last version of minitar v0.12 along with the long deprecated archive-tar-minitar v0.12. The Archive::Tar and archive/tar namespace/require resource have been deprecated since 2017-02-07 (v0.6). On 2024-08-07, I published minitar v1.0 which removed the Archive::Tar namespace and archive/tar path. This was a major version upgrade where I also dropped all support for Ruby 3.0 or older.

Fixing this within berkshelf should theoretically mean raising the minimum version of minitar to 0.12 and changing this require statement

Fixing this within berkshelf can be one of two ways:

1. Change the berkshelf.gemspec specification and modify nothing else.

   -  s.add_dependency "minitar",              ">= 0.6"
   +  s.add_dependency "minitar",              "~> 0.6"

   The gemspec was always incorrect here. This, however, is a bad choice because there will be no further maintenance on the v0 line.

2. Change the berkshelf.gemspec specification and modify packager.rb:1 to use require 'minitar'.

   -  s.add_dependency "minitar",              ">= 0.6"
   +  s.add_dependency "minitar",              "~> 1.0"

   This is the correct fix. I am considering additional changes that would push a minitar v2 that may not maintain any compatibility with the current version (I would like to drop support for creating GNU tar archives and only support extracting from them, like libarchive; the reasons for this are because the only specification for the GNU tar archive extensions is the gnu tar source code, and there are better approaches available).

Or, to fix this without changing berkshelf itself is to pin archive-tar-minitar to v0.8 before installing berkshelf:

gem install archive-tar-minitar:0.8 berkshelf:8.0.9

This is an unnecessary reference as archive-tar-minitar has been a "shadow" gem since 2017 and load downloads minitar of the same version. Simply installing minitar:0.12 will work, although it will not help if there are any tools other than berkshelf which use or install Minitar.

See halostatue/minitar#61.

[zaichang]

@halostatue Thank you for the detailed explanation

[halostatue]

@zaichang Note that #27 has been provided as a fix for Berkshelf.

The changes are slightly larger than I anticipated because they were continuing to use the deprecated Archive::Tar: namespace, and while I could have pushed a minitar hotfix that would add archive/tar/minitar (with a really noisy message), there is no way that I will be adding back the Archive::Tar namespace to the 1.0 line. That has been deprecated for seven years.