Release 2.0.0b12 - beta quality, only for testing.

[ThomasWaldmann]

borgbackup 2.0.0b12 beta release - do not use this on production backup repositories.

Long changelog:
https://borgbackup.readthedocs.io/en/2.0.0b12/changes.html#version-2-0-0b12-2024-10-03

If you use pip to install this, use: pip install "borgbackup==2.0.0b12"

Docs: https://borgbackup.readthedocs.io/en/2.0.0b12/

Discussion:

Note: Only limited platform testing and binaries this time, see "vagrant" tag in issue tracker.

---

This discussion was created from the release Release 2.0.0b12 - beta quality, only for testing..

[SpiritInAShell]

Question (trying this beta12 right now):
when using borg repo-create --from-borg1:
is BORG_OTHER_REPO / --other-repo allowed/intended to be either ssh:// or sftp:// ?

(in my scenario BORG_REPO is local but that should be storage agnostic?)

If it is intended, I might have a bug, but I also might have a bad configuration... I am not ready to open an issue at the moment.

Also, as borg transfer will be the next step:
the use of borg transfer --upgrader UPGRADER is unclear, as borg transfer --from-borg1 seems to be sufficient (quick test run). Even the examples at the end of borg transfer --help (and the online docs) ommit --upgrader

Is this due to outdated documentation?

[SpiritInAShell]

What I am trying to do is to (for testing only) create a borg2beta12 repo on the server and then transfer the borg1 repo on the same server (actually, both repos are in the same directory on a Hetzner storagebox accessible with sftp and with ssh/borg (v1.2.7))

I imagined that accessing a borg1 repo through borg2beta12 ---from-borg1 --other-repo=ssh:// would be possible.

Of course, I could mount the borg1 repo via sftp (fuse, sshfs). It was my expectation that, doing borg2beta12 transfer "would be more efficient" when using ssh://

[SpiritInAShell]

Hetzner wrote, that they will not provide a borg2 until there was a final version. Actually, they are quite behind with their borg1 version:

# echo "a" for triggering any response of remote server
> echo "a" | ssh ssh://[email protected]:23 borg serve --info --show-version --show-rc
$LOG INFO borg.output.show-version Remote: borgbackup version 1.2.7
$LOG ERROR borg.archiver Remote: Borg 1.2.7: Got unexpected RPC data format from client.
$LOG ERROR borg.output.show-rc Remote: terminating with error status, rc 2

[ThomasWaldmann]

Yeah, I would not expect Hetzner providing a borg beta version.

A remote server you don't control is not good for beta testing either.

The sshfs approach might work (assuming that works good/stable enough).

Or just rsync the repo to a local storage and then do experiments. Don't run borg2 with production repos.

[awgcooper]

Is it worth renting a standard hetzner cloud server which we control to perform remote backups. Anyone who wants access (within reason) can get their own shell account. If too many people are using it, then the account owner can restrict access. The attached 8 vcpu could work. I appreciate, there won't be scope for TB size backups but it will still be useful, right? I don't mind covering the cost for the first three months if we can be sure(-ish) that at least two or three people will use it fairly regularly.

[ThomasWaldmann]

Thanks for offering! Personally, I do some testing with an intranet and also one remote server (and sometimes just ssh://localhost :-) ). But let's see, maybe some beta testers could use it?

[SpiritInAShell]

Silly question, more out of curiosity than out of need:

as I know that internal structures may change and did have changed from borg2 beta9 to borg beta10..12 (change to borgstore):

do you have any concerns doing this?:

borg2beta9 export-tar | borg2beta12 import-tar

[ThomasWaldmann]

That's a lossy pipe, e.g. ACLs, xattrs, bsdflags are not yet supported by the standard tar formats (PAX and GNU). You can try the tar BORG format, that might be better.

[SpiritInAShell]

I am sorry, the command supplied was "symbolic", the exporting commandline should of course be
borg2beta9 export-tar --tar-format=BORG.

Did the tar-export format for type BORG change from beta9 to beta12 in a way, that you would advice against doing that?

[ThomasWaldmann]

there were no recent changes in the tar-related code.

[awgcooper]

From the change list:

analyze: list changed chunks’ sizes per directory

What's the practical use for this.

Separately, I missed in beta11 that you've add rclone address locators. Great functionality, thanks a lot!

[ThomasWaldmann]

from borg analyze --help:

            Analyze archives to find "hot spots".

            Borg analyze relies on the usual archive matching options to select the
            archives that should be considered for analysis (e.g. ``-a series_name``).
            Then it iterates over all matching archives, over all contained files and
            collects information about chunks stored in all directories it encountered.

            It considers chunk IDs and their plaintext sizes (we don't have the compressed
            size in the repository easily available) and adds up added/removed chunks'
            sizes per direct parent directory and outputs a list of "directory: size".

            You can use that list to find directories with a lot of "activity" - maybe
            some of these are temporary or cache directories you did forget to exclude.

            To not have these unwanted directories in your backups, you could carefully
            exclude these in ``borg create`` (for future backups) or use ``borg recreate``
            to re-create existing archives without these.

[ThomasWaldmann]

Most of the rclone work was done by @ncw in borgstore, thanks go to him!

[awgcooper]

Thanks to @ncw then!

[dietmargoldbeck]

Borg Beta not working for larger backups. Missing snapshots cause problems after network connection problems

Hi Thomas,

i upgraded from 2.0.0b9 to 2.0.0b12 (Debian Linux notebook) and created a new repository on my offsite freebsd server.
I did a small backup and restore test without problems and startet a backup of my home directory.
At 3am my DSL connection went down and when restarting the backup in the morning it startet to transfer all files again.

I think having no snapshots at all is a bad idea. I am writing this from a train e.g. and i configured the snapshot time to 720 seconds for borg 1.4. If a backup fails with 2.0.0b12 i cannot restore anything although my git stuff comes first. The failed backup is not even visible with borg repo-list.

I am also concerned that a borg compact run will delete valuable files from interrupted backups but did not test this yet.

Is it possible to add some checkpointing to the new repository structure?
Ciao,
Dietmar

[dietmargoldbeck]

The problem is reproducible for me and i just needed to transfer 400GB upstream the second time to my borg beta test repository.

If i interrupt borg with ctrl-c borg takes a few seconds and generates a new file under .cache/borg//files.
Resuming a backup after ctrl-c does not resend data to the repo. If the ssh process dies due to keepalive problems (e.g. DSL disconnect) there is no new file under .cache/borg//files. and the backup resends all data since the last update of the cache.

Please drop me a note, if You need some debug output.

[ThomasWaldmann]

.cache/borg/files is the files cache - it has a huge impact on speed (as it avoids reading/hashing unchanged input files), but it should have no impact on data transfer to the repository (because chunks which are already in the repo are not transferred again).

How do you know it has transferred 400GB upstream?

[dietmargoldbeck]

because 400GB is the approx maximum my notebook can transfer via ssh to the freebsd borg server within 8 working hours
when i am in the office. I checked frequently and the cpu was always busy with ssh and borg on notebook and borg server.

[ThomasWaldmann]

OK, looks like there is a problem.

As a workaround you could do some smaller temp archives (that can complete before your connection breaks down) with a subset of your data. I guess it can make progress if the transfer completes normally. Afterwards do the full backup (should be quick then) and remove the temp archives again.

[ThomasWaldmann]

Guess I know what it is:

• borg2 uses a ChunkIndex cache since recently, stored in repository/cache/chunks (and a checksum of it in chunks_hash).
• if borg2 create does not complete, it does not update that cache.
• in that case, there is still a valid cache, but it represents the repository contents from when the last backup ended normally and does not know about the chunks transmitted in the interrupted backup run.
• if one would kill that cache, borg would rebuild it by listing all objects in the repository (slow, but without much traffic) and it would then represent all currently present chunks in the repo.