Add basic distcc and ccache support to haikuporter

[SneakyWhoami]

• These modifications inject cmd:ccache and cmd:distcc into the build workflow as dependencies. These altered requirements do not flow through to completed packages.
• Haikuporter knows to activate either or both when passed --use-ccache or --use-distcc.
• Configuration for each command is copied from the default location in $HOME and symlinks are injected into the path before /bin to make build tools pick the commands up correctly.
• Distcc's pump mode is explicitly not supported for now, because it requires finnicky configuration of volunteer filesystems (and my goal was to have Linux volunteers).
• One thing I considered, but haven't tested or implemented, is extending the support to anything other than the host architecture. That would take me a little longer as it's beyond my current understanding.
• I wonder if the symlinking itself is the best approach, or if I can/should rearrange $PATH, AND I wonder if my approach to writing and injecting the scritplets (from python end and in shell) could be improved.

So, I don't know if you want this, but it works for me and if you have any feedback I am happy to clean it up further :-)

[waddlesplash]

(and my goal was to have Linux volunteers).

Yeah, that won't be possible, we need to run Haiku-native binaries inside the chroot.

[waddlesplash]

We really can't guarantee that sources/headers don't change during the build like that apparently requires, as there are lots of packages which do that.

[SneakyWhoami]

Yes. cmd:pump is the thing that really needs that to be true, so I didn't hook that up (and isn't functional without python3). There is still a chance that an overly-clever build script might try to invoke pump on its own volition, though, which is the scary part of that. I don't have an example of this happening, but compare to WebKit, which will detect and automatically use ld.gold if it is available (which would make linking fail on haikuwebkit, but doesn't exist right now so isn't specifically an issue)

[waddlesplash]

I don't see how the distcc stuff won't interfere with the chroot, i.e, the entire reason we use one is to mount packages into it that may not be on the host system.

The ccache stuff will almost certainly break under GCC2, and probably under more esoteric configurations with Clang or the like; it appears to be pretty fragile. So, this definitely can't be merged without substantial changes at the very least.

[pulkomandy]

distcc preprocesses sources locally and sends the preprocessed header to the remote. So that's fine for header files, however you have to be really sure the compiler on the linux machines is the one you expect.

I definitely would not use this in production (for our buildslaves) and clearly tag it as unsupported/"here be dragons"/"don't blame us if it breaks", but it seems ok for local builds, possibly

[waddlesplash]

I seem to recall that the Haiku target in GCC has some special sauce for object creation. So indeed this may produce subtly broken binaries...

[SneakyWhoami]

reason we use one is to mount packages into it that may not be on the host system.

Right. BuildPlatform.py injects either or both commands as requirements for building, so it should suck them into the chroot and run them from there along with their dependencies. It (well, distcc) does require having an existing configuration, no way around that, although it gets copied into the chroot.

The ccache stuff will almost certainly break under GCC2, and probably under more esoteric configurations with Clang or the like; it appears to be pretty fragile.

This bit's tricky. I haven't done any testing with gcc2 at all. In theory ccache likes gcc2 just fine, (it basically hashes input files and data about the environment, and stores the completed compilation for subsequent runs) but being ignorant of how it all fits together on 32bit systems I've just handed over what works on my box to see how it fares.
CLANG! Yes... I did not add any clang support at all, so clang would (except in overzealous build scripts) not receive any benefit or detriment at all; they'd be oblivious in its current form. So I guess that's either a plus or a big minus depending on one's perspective.

you have to be really sure the compiler on the linux machines is the one you expect

True. In the basic case, for example, if distcc is invoked as "gcc" and targets haiku, the volunteer system would call "gcc" and potentially target linux, and the build would fail. That bullet I believe I've dodged by specfiying the triplets, but there is the more horrible case of breaking differences in compiler versions, eg some potential incompatibility between gcc4 and gcc8, which I haven't accounted for. And on that note yeah that "being really sure" part makes it fragile like waddlespash was thinking, right.

GCC has some special sauce for object creation

And this is the thing that keeps me awake at night. That's magic to me and if you said that once a month it would require some kind of sacrifice or tribute I would take your word for it.

Well, I hope that all makes sense. I really don't expect it to make it anywhere near big important production build systems or anything. It's scratched my itch for stuff at home and done something slightly unsual, so even if you hate it I hope you like it :-). I could make "Haikuporter - Roulette Edition" (this is a joke). Thanks for having a look, guys! I didn't think anyone had, and just checked on a whim. But, cheers!

[pulkomandy]

In any case, let us know what happens and if you encounter any issues with it!