Eliminating the need for Dub: Explanation and (for demo purposes) implementation

[bachmeil]

I've been saying for a very long time that D should find better ways to use external libraries from a program. This weekend I wrote up a lengthy blog post showing how it can be done, including a basic, non-production implementation (I had been using a Linux shell script earlier but it was way easier writing this implementation in D). Here's the post:

https://bachmeil.github.io/the-blog/2024/02/14/no-dub-build.html

The brief version is that Dub does a lot more than just build programs with dependencies on the Dub registry. By adding a line like

dub dxml;

to your program, the compiler can use Dub to figure out which source files and shared libraries to add to the compilation command. I didn't waste my time proposing this upstream because I know the reception it would get. I did talk a bit about this here earlier, so I'm providing more details in case there's interest in it.

[adamdruppe]

You don't even need that much really - dub's silly insistence on having a million import paths can be bypassed if you extract the files and add them to a single location. This is what the opend extractor does to build the packaged libraries here. Then dmd -i just works, and if it has pragma(lib) in the source (I should extract them from the dub.json too, but currently I actually completely ignore dub.json as being totally redundant and it has been fine so far), libs just work too.

I still need to clean it up so you can use it locally to add more stuff to it, but there's some of the result in the CI build right now - do a ldc2 -i with the opend build and quite a few dub libs already just work.

But yeah, I'm sure there's stuff missing right now and we'll eventually need to work toward something more like what you have there, so thanks for the link!

[bachmeil]

I wasn't aware that you were working on that, but I'll wait for the release. The appeal of doing it this way was having something that didn't affect someone that likes using Dub (it's 100% compatible with existing Dub packages) but you may well have that case solved too.

[adamdruppe]

If you do use dub, it will download its own copy of the code anyway and that will take precedence over the default so it should still work.

[whitebyte]

Sorry to necropost but it's better late than never.
I just tried the approach suggested in the article. Something like this:

$ opend app.d $(dub describe observable --data=import-paths | tr -d "'")

It kinda works, but without local dub.json I have no immediate visiblity of what deps of what versions a project has. The general idea of package manager is not bad, and declarative dependencies are not bad. Be it dub, npm, cargo or whatever. I know dub gets a lot of flak and has done some debatable choices, but please don't throw the baby out with the bathwater here.

[adamdruppe]

Dependencies are already declared in the D code, and the compiler can extract them automatically (dmd -v or dmd -deps, neither output is great but it is there) and accurately for actual usage, including with optional dependencies and metaprogramming introduced dependencies.

There's no need to repeat it over and over again.

[whitebyte]

opend app.d -deps on an empty app.d (void main() {}) gives like two screens of unintelligible text. One can try to make sense of it, but this is not the ergonomics to aim for

[adamdruppe]

yeah it is meant to be part of a makefile process so it is hella ugly, but the functionality is there. we could surely process it nicer and do a lot of things with it - easy reading, file watching, etc.