ratarmount 1.0.0

ratarmount

Semantic versioning of GUIs and CLIs is not easy to define. The simplest GUI usage has not changed since the first version and even the index file format has been mostly compatible since version 0.2.0 and there is a lot of code for version checking. Based on the semver FAQ, the 1.0.0. release should probably have been a long time ago. Here it is now!

Features

• Bundle modified fusepy in order to add libfuse3 support in case only that one is installed.
  Contributions to the mfusepy fork would be welcome!
• Add message for first time users to show the result mount point.

Performance

• Disable Python-side buffering when opening files via FUSE.
• Forward underlying archive block sizes to statfs and stat implementations.

Fixes

• statfs did not work when using a write overlay and calling it on a file not in the overlay folder.
• Union mounting of inputs with the same name, even if in different folders, ignored all but the first.
• Suppress teardown warning in case the mount source was not even created yet.
• Make mounting work with only a write overlay.
• Avoid hangs and errors caused by non-joined threads before forking into the background by checking for running threads.
• Set locale to C when calling GNU tar to get more reproducible output on --commit-overlay.

ratarmountcore

Features

• Add fsspec implementation and register it as ratar://.
• Add support for new formats: SquashFS, Git, FAT12, FAT16, FAT32.
• Add support for fsspec backends. Archives and even index files can now be specified via URIs:
  dropbox://, ftp://, git://, github://, http://, https://, ipfs://, ipns://, s3://, ssh://, sftp://, smb://, webdav://.
• Add support for remote and compressed index files. Ratarmount will automatically look for
  index files with .gz and other common extensions and extracts these into /tmp/ or RATARMOUNT_INDEX_TMPDIR
  before using them.
• MountSource.open: Add buffering argument to enable/disable buffering or set the buffer size.

Fixes

• Argument to --gzip-seek-point-spacing was ignored when using the rapidgzip backend.
• Index creation did not work with default arguments with an archive in a read-only location.
• Close sqlite3 dummy connection after querying the SQLite version.
• Avoid resource leaks in case a MountSource constructor throws.
• SQLiteIndex: Do note store checkMetadata callback as a member to avoid dependency cycles.
• SQLiteIndex: Ignore errors when the connection is already closed.
• SQLiteIndexedTar: Avoid resource leak when constructor fails.

Performance

• Import compiled zip decrypter for 100x speedup for Standard ZIP 2.0 encrypted ZIP files
• Speed up readdir and therefore simple use cases such as find to iterate all files and folders by 3x.
• Avoid reading the whole appended TAR parts into memory for the check has-been-appended-to check.
• Fix block size being ignored when reading everything via io.BufferedReader.
• Do not use parallelization with possibly huge prefetches for simple file type checks.

ratarmount 0.15.2

ratarmount

The AppImage now uses manylinux_2_28 instead of manylinux2014 because CentOS 7 was EOL 2024-06-30, which broke the Github Actions pipeline for building the AppImage because some of the "actions" such as "checkout" broken because they are using npm, which requires a newer glibc version and does not work anymore inside the manylinux2014 container. I tried to manually check out the repo, which worked, but then I lost my patience when the upload-artifact action also failed.

Fixes

• Setting either the owner or group for a file with --write-overlay would reset the group or user respectively.
• Setting owner or group to 0 (root) with --write-overlay was not working.

ratarmountcore

Fixes

• Fix error when combining --recursive or --recursion-depth with --write-overlay.

ratarmount 0.15.1

ratarmount

Fixes

• Show install suggestions when FUSE is missing.
• File deletion with --commit-overlay did not work.

ratarmountcore

Fixes

• Fix the missing indentation for the index version check.
• Make --help and --version work even if libarchive.so is not installed.
• Fix LibarchiveMountSource compatibility issues with older libarchive versions.
• Fix the exception in StenciledFile when seeking before the file start.

ratarmount 0.15.0

ratarmount

Added

• Print indicators for versions of loaded shared libraries.

ratarmountcore

Added

• Add libarchive backend and detection support for:
  grzip, lrzip, lz4, lzip, lzma, lzop, rpm, uuencode, compress, 7zip, ar, cab, deb, xar, cpio, iso, war, cxar.
• Add --transform option to map each archive entry path via a regex to some user-specified one.
• Add zlib support (RFC1950).
• Store backend name into the index and check that the index fits to the current backend / MountSource.
• Store isGnuIncremental flag in the index.
• Apply specified priorities for opening all archives not just gzip.

Performance

• Determine incremental archives from index rows to avoid seeks.
• Remove indexed_bzip2 dependency in favor of rapidgzip, which in the future should support even more formats.

Fixes

• utils.findModuleVersion: Return version not name if __version__ does not exist.

rapidgzip

Added

• The rapidgzip Python library now also bundles IndexedBzip2File from indexed_bzip2.
• Enable checksum verification by default.
• Support for decompression from non-seekable inputs such as stdin.

Performance

• Windows / seek points are compressed in memory for reduced memory usage.
• Avoid doubling memory usage during index import and export by streaming the data directly to the output file.

ratarmount 0.14.2

ratarmount

Fixes

• Do not check mount point because of faulty os.path.ismount, simply try fusermount.
• Avoid total I/O hang when lazy-mounting a folder with archives onto itself.

ratarmountcore

Fixes

• Return a valid file info and file version count for /.
• Make the original archive viewable as an older file version when using AutoMountLayer.
• Resolve symbolic links pointing outside the source folder given to FolderMountSource to not break them.
• Do not return a valid FileInfo for invalid paths such as ../...
• Make --index-minimum-file-count work for the TAR backend.
• The index should not be created for very small archives.
• Root file info userdata was not initialized correctly.
• Index validation did fail for TAR entries with more than 2 metadata blocks.
• Do not check for folder consistency because parent folders get automatically added to the index.
• Move _createFileInfo out of MountSource class to fix "protected-access" warning.
• Joined files (a.001, a.002) did not work because of an accidentally shared list.
• Do not check file header for zip, only for the footer, to detect self-extracting archives.

ratarmount 0.14.1

ratarmount

Fixes

• Fix AttributeError: module 'fuse' has no attribute 'errno'.
• Fix --commit-overlay, which did not add newly created empty folders to TARs.
• Do not ask for confirmation if there is nothing to commit when using --commit-overlay.
• Improve unmounting with ratarmount -u, especially with the AppImage.

ratarmountcore

Fixes

• Properly close opened file objects in mount sources.
• Fix --disable-union-mount, which returned an I/O error.
• Fix that --use-backend=indexed_gzip still did use rapidgzip. It only worked when both were specified.
• Fix detection of self-extracting RAR files.
• Improve the error message when a file cannot be read because of a missing dependency.
• Improve debug message when the index does not yet contain a gzip index.
• Fix faulty seek forward for files opened via RarMountSource when whence is not io.SEEK_SET.

Performance

• Skip data offset collection for zip files because it takes too long and is unused.

ratarmount 0.14.0

ratarmount

Added

• Add --disable-union-mount option to present multiple archives in subfolders under the mount point.
• Add fine-grained control over the union mount cache:
  • --union-mount-cache-max-depth
  • --union-mount-cache-max-entries
  • --union-mount-cache-timeout
• Only use the number of cores the ratarmount process has affinity for by default.
• Add --index-minimum-file-count with sane default (1000) to avoid creating index files for
  rather small ZIP archives.

Fixes

• Disable auto-detection of GNU incremental TAR archives by default.
  Use --detect-gnu-incremental or better --gnu-incremental/--no-gnu-incremental
  instead.
• Apply name change: pragzip -> rapigzip.
• Fix ambiguous FUSE behavior that resulted in find not descending into recursively
  mounted subarchives.

ratarmountcore

Added

• Use rapidgzip instead of indexed_gzip by default.
• Do not parallelize gzip on slow drives because pread is slower than
  sequential read on those.
• Enable profiling output for rapidgzip with -d 2 or greater.

Fixes

• Do not check for GNU incremental TAR archive after an index has been loaded because
  it is only necessary for index creation. Note that this means thatSQLiteIndexedTar.isGnuIncremental
  member will remain False when an index has been loaded.
• Test for incremental archive after loading the compression index to avoid having
  to recreate it.
• Fix missing warning for multi-frame requirement of Zstandard archives.
• Fix unnecessary warning about mismatching gzip-seek-point-spacing
  when loading an index for
  archives without gzip compression.
• Change the default value of the SQLiteIndexedTar constructor argument
  gzipSeekPointSpacing from 4 MiB tp 16 MiB to make it consistent with the ratarmount CLI option.

ratarmount 0.13.0

Fixes in ratarmountcore

• Use XDG_CACHE_HOME according to FreeDesktop as default fallback
  when the archive folder is not writable.
• Create an SQLite index file for zip archives to speed up metadata lookup.
• Fix the issue with folders for mounted zip files not being traversed with find.

ratarmount 0.12.0

Fixes in ratarmount

• Add --recursion-depth argument for more fine-grained recursion control.
• Add support to show a joined view of split files, e.g., .001, and also support split archives.
• Add --use-backend option to choose between multiple available backends, e.g., 'lzmaffi' and 'xz' for xz files.
• Add support for pragzip when it has been installed and specified with '--use-backend pragzip'.
  Pragzip offers unique parallelized decompression of arbitrary gzip files. Using a 12-core processor,
  it can show speedups of 12 over gzip and speedups of 8 over pigz for sequentially decoding base64 files.
  Furthermore, similar to indexed_bzip2 it is based on a block cache and therefore can substantially speed up
  concurrent access streams as might happen when accessing multiple files through ratarmount concurrently.
  Currently, indexed_gzip will still be used to create the index in the first pass and pragzip is used
  for subsequent accesses with '--use-backend pragzip'.

Fixes in ratarmountcore

• AutoMountLayer now takes a 'recursionDepth' argument instead of 'recursive'.
• Fix handling of paths in zip and rar files that contain '../'.
• Add backend prioritization option to SQLiteIndexedTar.
• Add support for pragzip.

ratarmount 0.11.3

Fixes in ratarmount

• Fix files with leading "./" not being deleted when using --commit-overlay
• Fix files directly under the mount point not being deleted correctly with --write-overlay.

Fixes in ratarmountcore

• Fix exception when trying to mount a RAR archive containing files without timestamps.