Release 2.0.0b13 - beta quality, for testing only.

[ThomasWaldmann]

borgbackup 2.0.0b13 beta release - do not use this on production backup repositories.

Long changelog:
https://borgbackup.readthedocs.io/en/2.0.0b13/changes.html#version-2-0-0b13-2024-10-31

If you use pip to install this, use: pip install "borgbackup==2.0.0b13"

Docs: https://borgbackup.readthedocs.io/en/2.0.0b13/

Discussion:

Notes:

• Binaries will come later.
• Only limited platform testing and binaries this time, see "vagrant" tag in issue tracker.

---

This discussion was created from the release Release 2.0.0b13 - beta quality, for testing only..

[ThomasWaldmann]

First run after upgrade from b12 on existing (bigger and/or remote) repos will be slow.

Due to the new hashtable serialization code, the format changed and I needed to invalidate existing caches (like the cached chunk index in the repo). I will then rebuild the chunk index and cache it in the repo and will be faster next time.

[dietmargoldbeck]

traceback with 2.0.0b13 on 2.0.0b12 repo

I tried last night to use a repo created with 2.0.0b12 (Client is Debian and Server is freebsd 14) and got a traceback.Full log is attached.

Borg server: FileNotFoundError: [Errno 2] No such file or directory: '/usr/home/dietmar/borgtest/repos/git/data/5f/ee/tmpjod03ks6.tmp'
Borg server: Platform: FreeBSD dgo-beastie 14.1-RELEASE-p5 FreeBSD 14.1-RELEASE-p5 GENERIC amd64
Borg server: Borg: 2.0.0b13 Python: CPython 3.12.4 msgpack: 1.1.0 fuse: llfuse 1.5.1 [pyfuse3,llfuse]
Borg server: PID: 20683 CWD: /usr/home/dietmar
Borg server: sys.argv: ['/usr/local/bin/borg-2', 'serve', '--debug']
Borg server: SSH_ORIGINAL_COMMAND: None

[ThomasWaldmann]

Can you try to reproduce with a freshly initialized repo?

The latest borgstore code that comes with b13 has faster code with less storage api calls during operations, but requires pre-created levels dirs (and does pre-create them when initializing the repository).

So:

• 1x borg repo-create will be significantly slower (depends on storage speed and latency)
• 1000x borg create will be faster, because it doesn't have to make sure directories exist all the time.

[dietmargoldbeck]

With two new repos created with 2.0.0b13 running 2.0.0b13 has not given me a traceback. 2.0.0b13 is working slow but reliable for me.

[dietmargoldbeck]

Full Log borg-200b13-using-200b12repo-traceback.log

[dietmargoldbeck]

Files cache not used with 2.0.0b13, backups quite slow

When i run a backup with --debug i get this message
Reading files cache ... The files cache can't be read. [[Errno 2] No such file or directory: '/home/dietmar/.cache/borg/23581be32c02fcf0fec04495a7b3fbe9051ae082a0622274cbd1235d707ca02c/files.7e6fd736306eb290']
and it does not look for the existing files cache. The hash above is the name this run is going to write. After the run
dietmar@eselspinguin-zfs:~/borgbackup/log/borg-200b13$ ls -al ~/.cache/borg/23581be32c02fcf0fec04495a7b3fbe9051ae082a0622274cbd1235d707ca02c/*7e6fd736306eb290* -rw------- 1 dietmar dietmar 23381424 6. Nov 05:59 /home/dietmar/.cache/borg/23581be32c02fcf0fec04495a7b3fbe9051ae082a0622274cbd1235d707ca02c/files.7e6fd736306eb290

At the end the statistics show all files as new and borg did a benchmark of my notebook SSD. Unfortunately my fileserver has a RAID of spinning disks which really need the file cache

Duration: 31 minutes 29.425 seconds Number of files: 155374 Original size: 494.39 GB Deduplicated size: 240.03 MB Time spent in hashing: 14 minutes 37.134 seconds Time spent in chunking: 15 minutes 29.983 seconds Added files: 155374 Unchanged files: 0 Modified files: 0 Error files: 0

[ThomasWaldmann]

after introducing the archive series feature (== always use same archive name, don't put the date into there), the files cache filename suffix was derived from the archive name.

this was done to have a separate files cache per series (== per input data set), which reduces the memory needs of the files cache, so that only the data is stored in there which is actually needed for that series. it also gets rid of ttl issues of the files cache (of borg 1.x) if people have many different data sets.

i suspect you have different archive names (as it was usually the case for borg 1.x), so that would always create a new cache suffix and thus would not be useful.

you could either use the env vars to enforce the files cache name to be always the same or switch to using the archive series way of doing things (just use the same name always, e.g. just "docs" - you will have the timestamp, the user, the host in separate metadata). i'ld recommend using the archive series feature.

[ThomasWaldmann]

if you'ld rather like to continue with the old way of doing things:

https://borgbackup.readthedocs.io/en/2.0.0b13/usage/general.html#environment-variables

export BORG_FILES_CACHE_SUFFIX=myconstantname

[dietmargoldbeck]

borg test restore failed with 2.0.0b13: borg mount problem without unique backup names

After changing my borgbackup shell script to omit date and time and using just hostname.user for backup name i tried to mount the repo with borg mount. This gave me two problems:

An ls of the repo gets quite messy because the top level directory has no sensible sorting, It is not easy to finde the most recent backups. There was a driectory with the backup name and a few dirs with backupname minus hash (archive id).

All those directories were empty. I could only see files in the directory of my first backup with a uniq name (time at the end)

[ThomasWaldmann]

Well, guess the default is order-by-name but ls or your file manager have options to sort by e.g. mtime or maybe also by inode, did you try that?

empty: that's strange and sounds like a bug, will try to reproduce. thanks!