Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Move from Wren to Flamingo #50

Merged
merged 136 commits into from
Nov 6, 2024
Merged

Move from Wren to Flamingo #50

merged 136 commits into from
Nov 6, 2024

Conversation

obiwac
Copy link
Member

@obiwac obiwac commented Oct 4, 2024

This is really going to overhaul much more than just the language used. These are the things I wanna do:

  • Actually move over from Wren to Flamingo, obviously. Resolves wren: Fork of the language (?) #31
  • Start with dependency management and (not yet) overhauling the output directory.
  • Already support building simple setups with other build systems, I'm notably thinking of Meson/Ninja as is quite popular nowadays, CMake, bmake/gmake (how can we automatically detect this? default to gmake and only use bmake if e.g. a FreeBSD port makefile is detected?), ./configure, Cargo (which will completely replace the RustC class), and Go. This is in build: Support many other buildsystems #28
  • Figure out how I want to do asynchronous build operations. I think I'm going to ignore this for now and see how far I can get relying on the dependency system of asynchronous builds.
  • Make v0.1.0 out of Wren Bob.
  • Update CI (including for macOS and FreeBSD!) and tests.
  • Update README.
  • Prune files that weren't ever touched by the build pass. Will do this later, Prune files that weren't ever touched by the build pass #51
  • Remove useless holdovers (thinking notably of the skeletons).
  • Figure out the bob.fl import path situation.
  • Rename build.sh and sh-bin to bootstrap.sh and .bootstrap respectively to be a little clearer.
  • BIG ISSUE!! Don't chown the installed stuff of course. We're not actually doing this, but I do want to use install instead of cp eventually install: Use install command instead of cp #52
  • Fix the whole fork()ing in multithreaded code thing -> use spawn instead?

Concretely, this is what I still need to build a project like IAR or Umber:

  • Make CC.compile and Linker.link use the flags passed at instantiation.
  • Rename CC to Cc, I think it looks nicer.
  • Cache the warnings emitted for a compilation if nothing changed (so they don't just magically disappear).
  • Logging for compilation and linking (need a project with more files to properly test the aesthetics here).
  • Get the number of CPU's available correctly.
  • Re-compile/link when flags change.
  • Re-compile when included files change. The new way of doing this resolves CC: Cache header dependencies #18
  • Installing to .bob/prefix with install map. Maybe do this before a bob run instead of after a bob build, because practically I think bob run is the only thing that needs the prefix. OTOH, running it before bob run will introduce unnecessary delay. OTOOH, bob run anyway has to run bob build before, and we can speed up installation by checking modification times.
  • Res class for resource installation (or maybe we could just accept a path as-is as a key to the install map like in previous versions?). Indeed, Res is obsolete.
  • bob run itself, obviously.
  • Default runners with the run vector.
  • bob install.
  • Simple testing framework? Or should that just not be the concern of Bob? For now not our concern, IAR will just have its tests run with test.sh.
  • Frugal install.

Dependencies

See #54

Asynchronous build operations

I don't know how necessary this is really because CC can already do this and the dependencies are also entirely handled by Bob.

Cargo and Go will also be able to handle this themselves.

The only usecase for this is going to be for the AQUA devices, but there's really multiple ways we could do without this:

  • ✅ Have each device be a dependency. Depending on the DEVSET environment variable (depending on if AQUA 4.X even goes with device sets or drops them), the deps vector can be set to different local directories.
  • Keep the File.bob idea, but be able to pass a vector of projects to build in parallel. Fundamentally this is the same as adding all the stuff we wanna build to the deps vector.
  • Just ignore this as each device will already be parallel, though it probably won't be able to use all threads available as a lot of devices are pretty small. Crappy solution.
  • Do away with each device being buildable on its own and have everything in a giant build.fl file. Not too in favour of this solution. This is going to be unmanagable and requires a proper system for these async jobs.

If we go with the dependency idea, there has to be a way to divvy up thread counts between dependencies on the same BFS layer. Say we have 16 threads available and we're building 4 dependencies which use make, we could pass -j$(16 / 4) to each one. We do need a strategy for when maybe 3 of the dependencies finish building in 5 seconds but the last one takes like 60 seconds; then, we may need a heuristic to decide if it's worth it to stop building so that we can re-run make with -j16.

Architectural revamp

For the different build systems Bob supports, I'm thinking of having a handler architecture where each build system handler would register functions for the different operations Bob supports (i.e. build, install, &c). Bob itself would also be a handler.

I thought about making these pluggable, but I don't think there ever really will be a situation where you'd need support for a build system which can't be upstreamed into Bob. Besides, the "workaround" for these kinds of buildsystems would just be using them as originally intended.

Each buildsystem would have its own subdirectory in .bob, and after each build (each? How slow is installing?) they'd install into a .bob/prefix directory, which has a FHS-like filestructure (.bob/prefix/bin, .bob/prefix/lib, .bob/prefix/include). When building a C program for instance, -L.bob/prefix/lib would be passed to the compiler so that it can search for libraries in its dependencies. When prefixing a command with bob run, this would simply append .bob/prefix/bin to $PATH and set $LD_PRELOAD to .bob/prefix/lib (similar to how poetry run activates the relevant venv for you). (Maybe we can find a better name than prefix? install-prefix? Just install?)

The only unknown is how installing to the system is going to work, since we can't just overwrite existing dependencies. Maybe just install the program itself and provide an explicitly "dangerous" option for also installing all the dependencies.

The .bob/bob subdirectory would comprise of all the artifacts generated by Bob, which would have the name of the original source plus a randomly generated cookie UUID.

Finally, the .bob/deps subdirectory would comprise of a directory for each dependency, and each dependency would have its own .bob directory for its own stuff. The only difference would be that they'd share the same .bob/prefix as their parent.

Issues identified

Here are a few issues I've already identified which need fixing:

  • When checking for the output's last modification time, I'm using the cookie as the path to stat. gen_cookie should really automatically prepend the $out_path/bob stuff and probably take in an ext argument to add the .o for compiling e.g.

obiwac added 30 commits October 4, 2024 13:14
@obiwac obiwac mentioned this pull request Nov 5, 2024
10 tasks
@obiwac obiwac marked this pull request as ready for review November 6, 2024 19:10
@obiwac obiwac merged commit 9f5ce0c into main Nov 6, 2024
8 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

wren: Fork of the language (?) CC: Cache header dependencies
1 participant