tito does not find sources it wrote just before

[ibotty]

rpm -q tito: tito-0.6.16-1.fc33.noarch

While trying to build https://gitlab.com/ibotty/winexe/-/commit/54f27f033d5c29e603b4ac158aa4f2d66ff23d2f in a clean checkout I get the following error. Note that tito build --rpm --test works fine!

$ tito  build --rpm
Creating output directory: /tmp/tito
Checking for tag [winexe-1.5-1] in git repo [[email protected]:ibotty/winexe.git]
Building package [winexe-1.5-1]
Wrote: /tmp/tito/winexe-1.5.tar.gz
error: Bad source: /tmp/tito/rpmbuild-winexek8cr0g1n/SOURCES/winexe-1.5.tar.bz2: No such file or directory



ERROR: Error running command: rpmbuild   --eval '%undefine scl' --define "_topdir /tmp/tito/rpmbuild-winexek8cr0g1n" --define "_sourcedir /tmp/tito/rpmbuild-winexek8cr0g1n/SOURCES" --define "_builddir /tmp/tito/rpmbuild-winexek8cr0g1n/BUILD" --define "_srcrpmdir /tmp/tito" --define "_rpmdir /tmp/tito"   --clean  -ba /tmp/tito/rpmbuild-winexek8cr0g1n/SOURCES/winexe-1.5/winexe.spec

Status code: 1

Command output: ['error: Bad source: /tmp/tito/rpmbuild-winexek8cr0g1n/SOURCES/winexe-1.5.tar.bz2: No such file or directory', '']

ERROR: Error running command: rpmbuild   --eval '%undefine scl' --define "_topdir /tmp/tito/rpmbuild-winexek8cr0g1n" --define "_sourcedir /tmp/tito/rpmbuild-winexek8cr0g1n/SOURCES" --define "_builddir /tmp/tito/rpmbuild-winexek8cr0g1n/BUILD" --define "_srcrpmdir /tmp/tito" --define "_rpmdir /tmp/tito"   --clean  -ba /tmp/tito/rpmbuild-winexek8cr0g1n/SOURCES/winexe-1.5/winexe.spec

[bryce-carson]

I have the same issue,

[cyber@fedora rpi-imager]$ tito build --rpm
Creating output directory: /tmp/tito
Checking for tag [rpi-imager-1.7.2-5] in git repo [[email protected]:bryce-carson/rpi-imager.git]
Building package [rpi-imager-1.7.2-5]
Wrote: /tmp/tito/rpi-imager-1.7.2.tar.gz

error: Bad source: /tmp/tito/rpmbuild-rpi-imagerl4byjj29/SOURCES/rpi-imager-1.7.2-5.tar.gz: No such file or directory

The problem is actually that there is a mismatch between the tag/release name that Tito is building for, 1.7.2-5 in my case, and the targz it writes, which is 1.7.2 only.

The release is not downloaded from the SCM forge as it normally might, so this complicates things.

🤷🏼

[bryce-carson]

Here is a debug log:

[cyber@fedora rpi-imager]$ tito build --rpm --debug
Creating output directory: /tmp/tito
---- Builder class is None
Importing tito.builder.Builder
Using builder class: <class 'tito.builder.main.Builder'>
Building in temp dir: /tmp/tito/rpmbuild-rpi-imager5frfwi7a
Getting latest package info from: /home/cyber/code/rpi-imager/.tito/packages/rpi-imager
Command: awk '{ print $1 ; exit }' /home/cyber/code/rpi-imager/.tito/packages/rpi-imager
Status code: 0
Command output: 1.7.2-9

Command: git ls-remote ./. --tag rpi-imager-1.7.2-9 | awk '{ print $1 ; exit }'
Status code: 0
Command output: a46973af49586d0aa473d805b1628c794e3c1b1d

Local tag SHA1: a46973af49586d0aa473d805b1628c794e3c1b1d
Command: git config remote.origin.url
Status code: 0
Command output: [email protected]:bryce-carson/rpi-imager.git

Command: git config remote.origin.url
Status code: 0
Command output: [email protected]:bryce-carson/rpi-imager.git

Checking for tag [rpi-imager-1.7.2-9] in git repo [[email protected]:bryce-carson/rpi-imager.git]
Command: git ls-remote [email protected]:bryce-carson/rpi-imager.git --tag rpi-imager-1.7.2-9 | awk '{ print $1 ; exit }'
Status code: 0
Command output: a46973af49586d0aa473d805b1628c794e3c1b1d

Remote tag SHA1: a46973af49586d0aa473d805b1628c794e3c1b1d
Command: git ls-remote ./. --tag rpi-imager-1.7.2-9 | awk '{ print $1 ; exit }'
Status code: 0
Command output: a46973af49586d0aa473d805b1628c794e3c1b1d

Command: git rev-list --max-count=1 a46973af49586d0aa473d805b1628c794e3c1b1d
Status code: 0
Command output: 80d8ad105835e37d80643c010429fa16717c5747

Got package metadata: ['1.7.2-9', './']
Building package [rpi-imager-1.7.2-9]
Creating rpi-imager-1.7.2.tar.gz from git tag: 80d8ad105835e37d80643c010429fa16717c5747...
Command: git rev-list --timestamp --max-count=1 80d8ad105835e37d80643c010429fa16717c5747 | awk '{print $1}'
Status code: 0
Command output: 1660636760

Command: git archive --format=tar --prefix=rpi-imager-1.7.2/ 80d8ad105835e37d80643c010429fa16717c5747: --output=/tmp/tito/rpmbuild-rpi-imager5frfwi7a/SOURCES/rpi-imager-1.7.2.tar.initial
Status code: 0
Command output: 

git-archive fails if relative dir is not in git tree
Command: git archive --format=tar --prefix=rpi-imager-1.7.2/ 80d8ad105835e37d80643c010429fa16717c5747: --output=/tmp/tito/rpmbuild-rpi-imager5frfwi7a/SOURCES/rpi-imager-1.7.2.tar.initial > /dev/null

Status code: 0

Command output: 

Command: gzip -n -c < /tmp/tito/rpmbuild-rpi-imager5frfwi7a/SOURCES/rpi-imager-1.7.2.tar.tar > /tmp/tito/rpmbuild-rpi-imager5frfwi7a/SOURCES/rpi-imager-1.7.2.tar.gz
Status code: 0
Command output: 

Copying git source to: /tmp/tito/rpmbuild-rpi-imager5frfwi7a/SOURCES/rpi-imager-1.7.2
Command: cd /tmp/tito/rpmbuild-rpi-imager5frfwi7a/SOURCES/ && tar xzf rpi-imager-1.7.2.tar.gz
Status code: 0
Command output: 

Show contents of the directory structure we just extracted:
['src', 'rpi-imager.spec', 'license.txt', 'embedded', 'doc', 'debian', 'README.md', '.tito']
Looking for .spec in /tmp/tito/rpmbuild-rpi-imager5frfwi7a/SOURCES/rpi-imager-1.7.2
Using file: rpi-imager.spec
Command: cp /tmp/tito/rpmbuild-rpi-imager5frfwi7a/SOURCES/rpi-imager-1.7.2.tar.gz /tmp/tito/
Status code: 0
Command output: 

Wrote: /tmp/tito/rpi-imager-1.7.2.tar.gz
Command: rpm --eval '%scl'
Status code: 0
Command output: %scl


error: Bad source: /tmp/tito/rpmbuild-rpi-imager5frfwi7a/SOURCES/rpi-imager-1.7.2-9.tar.gz: No such file or directory


ERROR: Error running command: rpmbuild   --eval '%undefine scl' --define "_topdir /tmp/tito/rpmbuild-rpi-imager5frfwi7a" --define "_sourcedir /tmp/tito/rpmbuild-rpi-imager5frfwi7a/SOURCES" --define "_builddir /tmp/tito/rpmbuild-rpi-imager5frfwi7a/BUILD" --define "_srcrpmdir /tmp/tito" --define "_rpmdir /tmp/tito"   --clean  -ba /tmp/tito/rpmbuild-rpi-imager5frfwi7a/SOURCES/rpi-imager-1.7.2/rpi-imager.spec

Status code: 1

Command output: ['', 'error: Bad source: /tmp/tito/rpmbuild-rpi-imager5frfwi7a/SOURCES/rpi-imager-1.7.2-9.tar.gz: No such file or directory']

ERROR: Error running command: rpmbuild   --eval '%undefine scl' --define "_topdir /tmp/tito/rpmbuild-rpi-imager5frfwi7a" --define "_sourcedir /tmp/tito/rpmbuild-rpi-imager5frfwi7a/SOURCES" --define "_builddir /tmp/tito/rpmbuild-rpi-imager5frfwi7a/BUILD" --define "_srcrpmdir /tmp/tito" --define "_rpmdir /tmp/tito"   --clean  -ba /tmp/tito/rpmbuild-rpi-imager5frfwi7a/SOURCES/rpi-imager-1.7.2/rpi-imager.spec
Cleaning up /tmp/tito/rpmbuild-rpi-imager5frfwi7a

[bryce-carson]

The problem reduces to the definition of the _get_display_version() method of the Builder class.

In the instance that .tito/packages/ has the format: {version}-{release}, such as 1.7.2-9, the build_version is not used and the display_version is used.

The display version function definition comment implies that it should be whatever .tito/packages/ evaluates to, which when there is a build_tag it will be {version}-{release}, but the creation of the targz 🐗 uses the display name.

I don't know enough about the use of Tito, as this is my first build, to bother with submitting a pull request to fix this because it may break something else.

[bryce-carson]

However, the problem created by this definition is real and the code is likely incorrect, as self.spec_version should be what is used if the intention was to get the upstream project version only.

tito/src/tito/builder/main.py

Lines 481 to 482 in bb3e661

	self.spec_version = build_version.split('-')[0]
	self.spec_release = build_version.split('-')[-1]

When version is defined in _get_display_version it does the text object splitting a second time, when self.spec_version is already initialized with the same value.

tito/src/tito/builder/main.py

Lines 660 to 675 in bb3e661

	def _get_display_version(self):
	"""
	Get the package display version to build.
	Normally this is whatever is .tito/packages/. In the case of a --test
	build it will be the SHA1 for the HEAD commit of the current git
	branch.
	"""
	if self.test:
	# should get latest commit for given directory *NOT* HEAD
	latest_commit = get_latest_commit(".")
	self.commit_count = get_commit_count(self.build_tag, latest_commit)
	version = "git-%s.%s" % (self.commit_count, latest_commit[:7])
	else:
	version = self.build_version.split("-")[0]
	return version

Overall, the concept an intent of "build version" and "display version" should be better documented. If the intent is to build the build version, and merely display the display version, like in the below example, then the implementation is incorrect.

Creating output directory: /tmp/tito
Checking for tag [rpi-imager-1.7.2-9] in git repo [[email protected]:bryce-carson/rpi- 
imager.git]
Building package [rpi-imager-1.7.2-9] for upstream [rpi-imager-1.7.2]
Wrote: /tmp/tito/rpi-imager-1.7.2-9.tar.gz # -9 appended, as per this issue ya know

[FrostyX]

Thank you for the detailed feedback @bryce-carson, and I am sorry that I didn't pay better attention to this earlier @ibotty.

I think the main problem is .tar.bz2. I cloned the winexe project, and did tito build --rpm --no-clean. It failed as expected but then I could examine the rpmbuild directory

[jkadlcik@zeratul winexe]$ ls -1 /tmp/tito/rpmbuild-winexekq1ozc75/SOURCES/
winexe-1.5
winexe-1.5.tar.gz
winexe-1.5.tar.initial
winexe-1.5.tar.tar

The error message was

error: Bad source: /tmp/tito/rpmbuild-winexekq1ozc75/SOURCES/winexe-1.5.tar.bz2: No such file or directory

which doesn't exist, but the same file exists but as .tar.gz.

So I think a workaround should be changing the Source0 to

Source0:        %{name}-%{version}.tar.gz

and tagging a new version. But for sure we need to fix the issue for .tar.bz2.

[bryce-carson]

I think the problem I encountered has a different cause. I expected Tito to behave differently without understanding the various taggers and workflows.

When I invoked Tito, this blob was the spec file I was using, and I invoked Tito in an unpacked Git directory with the spec file I wrote. I wanted to contribute it upstream, but I didn't realize that Fedora already packages rpi-imager! Silly me.

https://github.com/bryce-carson/rpi-imager/blob/80d8ad105835e37d80643c010429fa16717c5747/rpi-imager.spec#L1-L35

The Source field in the spec file linked above is problematic (it should be Source0).

I'm not sure what the correct behaviour or warning should be in this situation, but that is the rest of the information that I can provide to you, @FrostyX.

[FrostyX]

and I invoked Tito in an unpacked Git directory with the spec file I wrote

Meaning a directory where nobody did tito init? I am trying that but that fails with the following error instead of this #398 issue.

[jkadlcik@zeratul tito]$ tito build --rpm --test
Creating output directory: /tmp/tito
ERROR: Unable to locate branch configuration: /home/jkadlcik/git/tito/rel-eng/tito.props
Please run 'tito init'

I also changed Source0 to Source in my spec file and the build continued working correctly.

If you've got a chance, can you please write a step-by-step reproducer that leads to the issue you mentioned in your last comment @bryce-carson? I will try to either fix the tito behavior or add some proper error messages.

[bryce-carson]

and I invoked Tito in an unpacked Git directory with the spec file I wrote

Meaning a directory where nobody did tito init? I am trying that but that fails with the following error instead of this #398 issue.

No, tito init had been run. I meant that the directory is managed with Git, and the sources are unpacked. Plain sources/Git repository + spec file is what I was working with.

[jkadlcik@zeratul tito]$ tito build --rpm --test
Creating output directory: /tmp/tito
ERROR: Unable to locate branch configuration: /home/jkadlcik/git/tito/rel-eng/tito.props
Please run 'tito init'

I also changed Source0 to Source in my spec file and the build continued working correctly.

If you've got a chance, can you please write a step-by-step reproducer that leads to the issue you mentioned in your last comment @bryce-carson? I will try to either fix the tito behavior or add some proper error messages.

When I have some extra time I'll come back to this with detailed reproduction steps.