Fix git archive for .gitattributes

[LecrisUT]

This might not be the correct understanding of what this command does, but hopefully it is a good stepping stone to fix this. See #444 for more details on the issue.

Closes #444

[FrostyX]

Thank you for the PR @LecrisUT,
this is the difference between those commands

Currently, we do

[jkadlcik@zeratul tito]$ git archive --format=tar d342fc61:./src/tito/builder --output=/tmp/tito-0.6.22.tar
[jkadlcik@zeratul tito]$ tar -tvf /tmp/tito-0.6.22.tar
-rw-rw-r-- root/root       502 2023-03-01 15:31 __init__.py
-rw-rw-r-- root/root      8043 2023-03-01 15:31 fetch.py
-rw-rw-r-- root/root     57690 2023-03-01 15:31 main.py
-rw-rw-r-- root/root      6776 2023-03-01 15:31 submodule_aware_builder.py

And the proposed change

[jkadlcik@zeratul tito]$ tar -tvf /tmp/tito-0.6.22.tar
drwxrwxr-x root/root         0 2022-11-14 01:10 src/
drwxrwxr-x root/root         0 2022-11-14 01:10 src/tito/
drwxrwxr-x root/root         0 2022-11-14 01:10 src/tito/builder/
-rw-rw-r-- root/root       502 2022-11-14 01:10 src/tito/builder/__init__.py
-rw-rw-r-- root/root      8043 2022-11-14 01:10 src/tito/builder/fetch.py
-rw-rw-r-- root/root     57690 2022-11-14 01:10 src/tito/builder/main.py
-rw-rw-r-- root/root      6776 2022-11-14 01:10 src/tito/builder/submodule_aware_builder.py

so this breaks tito for all projects that have a spec file (or more of them) in some subdirectories. Reproducer:

$ git clone https://github.com/fedora-copr/copr.git
$ cd copr/cli
$ tito build --tgz 
Creating output directory: /tmp/tito
Checking for tag [copr-cli-1.105-1] in git repo [[email protected]:fedora-copr/copr.git]
Building package [copr-cli-1.105-1]
ERROR: Unable to locate files ending with ['.spec', '.spec.tmpl'] in /tmp/tito/rpmbuild-copr-cli7euv01ox/SOURCES/copr-cli-1.105

[LecrisUT]

Indeed I was suspecting there is something I am missing. Thanks for pointing to an example for local debugging. Any ideas of how to fix the main issue of this? (I did try adding --worktree-attributes, but it did not fix the original issue)

Probably related: https://stackoverflow.com/questions/52804334/how-to-ignore-files-directories-in-git-archive-and-only-create-an-archive-of-a

My test repo is https://github.com/scikit-build/scikit-build-core:

$ git archive HEAD:./ --output=test.tar.gz
$ tar -axf test.tar.gz .git_archival.txt -O
node: $Format:%H$
node-date: $Format:%cI$
describe-name: $Format:%(describe:tags=true,match=*[0-9]*)$
ref-names: $Format:%D$

Compared to what it should look like:

$ git archive HEAD --output=test.tar.gz
$ tar -axf test.tar.gz .git_archival.txt -O
node: 745ce26169fb44e04d91d40ee581cccd591c941e
node-date: 2023-03-01T15:22:44+01:00
describe-name: v0.2.0-14-g745ce26
ref-names: HEAD -> rpm-spec, LecrisUT/rpm-spec

[LecrisUT]

How about we use --transform in #443 to strip the relative output?

[FrostyX]

My test repo is https://github.com/scikit-build/scikit-build-core

Thank you. I don't use .gitattributes on any of my projects, so I didn't know how to test. Sooo I may have found at least a partial solution. I don't know if this issue is only for .gitattributes in the root directory and works in subdirectories, or if it doesn't work at all. Can you please try?

The directory structure produced by these two commands is the same:

git archive HEAD:./ --output=test1.tar.gz
git archive HEAD --output=test2.tar.gz
diff <(tar -tvf test1.tar.gz | awk '{print $NF}') <(tar -tvf test2.tar.gz | awk '{print $NF}')

But, in the first case, the .gitattributes was ignored

$ tar -axf test1.tar.gz .git_archival.txt -O
node: $Format:%H$
node-date: $Format:%cI$
describe-name: $Format:%(describe:tags=true,match=*[0-9]*)$
ref-names: $Format:%D$

and in the second, it worked

$ tar -axf test2.tar.gz .git_archival.txt -O
node: 03b90246d3c5396bf8cfda97b5fe7d53d64bf3ce
node-date: 2023-03-02T20:17:36+00:00
describe-name: v0.2.0-10-g03b9024
ref-names: HEAD -> main, origin/main, origin/HEAD

but I don't know how it will work for .gitattributes somewhere in a subdirectory. But if it doesn't, maybe it doesn't mind?

[LecrisUT]

I've just tested locally, .gitattributes works in subdirectories as well, and it works with git archive HEAD sub_dir. What doesn't work in any permutation is git archive HEAD:./ or git archive HEAD:./sub_dir.

To get this to work I think we can do like this:

• git archive ./sub_dir
• tar fixer
• tar -xf | tar -cf --transform "s|sub_dir/|prefix/|g"

[LecrisUT]

Also, I think this might be a bug in git. Do you know how to file a bug report there?

[LecrisUT]

Oh no, this doesn't work. I have uploaded d008bd1 just for reference. The ownership and timestamp of the prefix are broken, but can't figure a way to fix the paths without extraction

[LecrisUT]

@FrostyX If you want to chime in: https://lore.kernel.org/git/[email protected]/T/#t

[LecrisUT]

From the discussion on the mail, it seems that the origin of the tarfixer was because we are using a tree instead of the commit. If we are not using the tree as we do it in this PR, we can avoid it? What other headers do we need to have set? What extract/archive commands could we use to preserve the same tar format?