Enabling java community to have easier access to jbang "friendly" java projects

[maxandersen]

For the last few years I've been proposing various projects I saw could benefit from having a jbang-catalog.
Often the project is inside an org and I then propose to make a org wide jbang-catalog to shorten the name.

Sometime this succeeds http://github.com/microsoft/jbang-catalog (thanks @brunoborges), other times, it gets discouraged and sometimes even having any jbang-catalog gets rejected.

Most of the discouragement or rejection falls into one of two categories:

A) having a jbang-catalog in org or repo is felt like an overhead/unclear who maintain it
B) "no one asked for it"

Its a classical chicken-egg situation - if jbang was well-known and fully understood this would not be a problem but to become well-known and fully understood need to have it more used.

I fully get the resistance and sometime it just wont happen...but even if others don't use jbang-catalog in their project I really would like to make it easy for anyone to use other peoples great software.

I've resisted adding various commands to jbangdev central catalog as I much rather prefer each project to own their tooling experience; but I also see that for some projects its just not going to happen...so can we make a community driven approach that does not hurt or offend project owners and encourage them to adopt owning their jbang catalogs?

In a conversation with @parttimenerd he had some ideas and suggestions that made think if we could find a way to improve this.

jbangapp/jbang-catalog

One way of doing this would be to add aliases to a central jbang catalog. It can't be jbangdev/jbang-catalog as it is automatically blessed and I don't want to accidentally expose jbang users to a security issue; but it could be i.e. jbangapp/jbang-catalog making jbangapp the key name. (I'm not married to jbangapp but so far all my other ideas (jtool, javaapp, jbox, ..) was taken or had trademark issues)

Idea is that in that org we or anyone else can submit PR's to add their favorite set of tools.

i.e. sqlline who don't want a catalog would then be made available to anyone by using jbang sqlline@jbangapp or ap-profiler could shorten their jbang jfr2flame@jvm-profiling-tools/ap-profiler to jbang jfr2flame@jbangapp.

The advantages of this are that:

• its short and concise
• easy to maintain (renovate can be used to keep versions uptodate)
• its testable (could add small testsuite for each alias to check it still works between updates)
• anyone can contribute
• will enable a lot of tools to be instantly available to anyone with java/jbang installed, enables us to bootstrap a way to easily do what other language ecosystems has (npm install, pip install etc.)

The downsides are:

• its one big catalog thus over time it will be updated often and might slow things down for users; even for commands that have not updated
• stable versioning is problematic (if we constantly update the commands users will get a potential update without doing anything and no easy way to "lock" the version because its a global catalog rather than a smaller catalog - one workaround is that users do `jbang sqlline@jbangapp/)
• adding commands to this might discourage/offend project owners of those commands to have their own catalog

...those downsides and in particular the latter two is what has kept me away from doing it until now - but I'm also feeling we've tried for a few years now to get project owners to adopt it so something has to give.

Thus I'm inclined to say - lets go make jbangapp a thing. Yes, it won't be most efficient, it wont give stable versioning but it will make every awesome tool available more or less instantly.

For the latter part of possibly offending/discouraging @parttimenerd gave an idea...and i think maybe @quintesse proposed similar in the past:

global aliases for catalogs

(sorry - need a better name)

This idea is that on top of jbang's ability to resolve catalogs available on github/gitlab/bitbucket (jbang playwright@microsoft) and DNS names (i.e. jbang [email protected] refers to https://xam.dk/jbang-catalog.json) we would have a new mechanism that jbang to have aliases looked up in another catalog.

ie. jbang sqlline@@sqlline@ or jbang jfr2flame@@ap-loader@ would be "expanded" according to a file bundled in jbang or even from github.com/jbangdev/jbang-catalog somewhere that defines whatever @ap-loader@ shuold get expanded to.

It could be jvm-profiling-tools/ap-loader or it could be jbangapp/jbang-catalog ...and it could change over time.

This keeps the association to the project name (i.e. ap-loader) and makes it possible to make even shorter.

This global alias would be truly global - if users want custom aliases just use the existing alias mechanism; but with this we have the ability to offer an approach to spread the awesomeness of jbang while still recognizing project owners names/repo.

It could also backfire and many will be upset it does not make it clear jbangapp is the "owner" of it.

@parttimenerd suggested this could be alleviated by having jbang print out when it resolves @ap-line@ so its visible it comes from jbangapp or jvm-profiling-tools/ap-profiler.

WDYT ? Do one or the other or both?

p.s. I grabbed https://github.com/jbangapp already :)

[quintesse]

So the first part of this is clear to me and seems like a good idea.

It's the second proposal that I'm somewhat leery about. Messing about with (even more) magic symbols just seems very unintuitive. Personally I'm already not extremely happy with what we currently have (and I designed it) and this just seems to complicate it even more.

I understand it would be very easy to just be able to do jbang jfr2flame@ap-loader (or jbang jfr2flame@@ap-loader@ and the like in case of expansion) but isn't that all a bit too much black magic? Of lookups upon lookups? We might get lost in a forest of lookups (we lookup a global alias to look up a catalog to look up an alias).

We can already name catalogs, you can for example say jbang catalog add --name ap-loader http://url/to/catalog. Wouldn't it be better to have people explicitly add a catalog they are interested in? A one-time command that's slightly longer but afterwards the aliases to run can use the shorter format.

We can already add catalogs to catalogs (I'm actually not really sure how we're using that or if we're even using that as of today, so perhaps that's an opportunity) so imagine something like this:

$ jbang catalog list jbangapp
Catalogs:
----------
ap-loader = Commands for dealing with ap-loader
                    (https://github.com/jvm-profiling-tools/ap-loader/blob/main/jbang-catalog.json)
sqlline =     Some useful sqlline scripts
                   (https://raw.githubusercontent.com/jbangdev/jbangapp/master/sqlline/jbang-catalog.json)
.
.
.
$ jbang catalog add sqlline@jbangapp
$ jbang run@sqlline

(Btw, right now you can't add catalogs this way, that would need to be implemented)

This would allow for short aliases in daily use and still leave open the possibility that at some point the sqlline devs decide to support JBang directly, and a simple jbang catalog add sqlline would switch to the official catalog.

[aalmiray]

not a super fan of "central" ...is that because of maven ccentral you suggest that?

Bingo!

[maxandersen]

@aalmiray your idea if i grok it would be possible even without a "special" index - if we make jbang sqlline@@sqlline be short hand forjbang sqlline@jbangapp/sqllineandjbangapp` is the "default" alias for @.

Maybe even make it sqlline@/sqlline ...i.e. empty implicit git org lookup...

@quintesse wdyt about that ? I don't think that "conflicts" what we have today.

[maxandersen]

Now if we did that (i.e. sqlline@/sqlline) I would still prefer there was a way to have jbangapp/sqlline/jbang-catalog go point to sqlline own catalog if that exists some day. having it just import the aliases, no "catalog aliasing" as what is happening today.

wdyt @quintesse ?

[maxandersen]

turned out it was easy to do...we already fail on "empty catalog" so turned that into mean "jbangapp" #1738

wdyt?

[maxandersen]

not a super fan of "central" ...is that because of maven ccentral you suggest that?

Bingo!

Going with jbanghub. Not specific to app; and hub is shorter than central :)

[maxandersen]

As an experiment I started to add content to https://github.com/jbanghub.

Thoughts made from chatting with @quintesse was that we would default import this catalog so all its catalogs would be available to users. Kinda felt like how we would have done appstore from the start if we knew then what we know today...

Idea was to make it so sqlline@sqlline would end up resoling to sqlline@jbanghub/sqlline and if sqlline one day made their own we could just point to it.

But after adding content to jbanghub I've kinda pulled back from it - because its hard to name things :)

i.e. sqlline is actually not in github.com/sqlline but in github.com/julianhyde/sqlline - so sqlline catalog name is kinda globally namesquatting sqlline....but also it removes julianhyde...and maybe julianhyde likes that is how to refer to...so we could add julianhyde/sqlline alias - but then it starts looking like its him who manages it ... of course julianhyde here is just used as an example, it could be same with orgs/corporations like apple, microsoft, eclipse etc.

Thus for now I've gone with NOT having jbanghub use its power of default imported and instead added things that are in jbanghub/* org as jbanghub/... names.

This means there is actual value for people to add their own jbang-catalog even if jbanghub has something that maps to it, if they do that there shorthand name wins automatically.

For example, I added jbanghub/eclipse - it has ecj and transformer cli in it at this stage.
It will be listed as jbanghub/eclipse in catalog (because default imported) and command will be ecj@jbanghub/eclipse.
If eclipse adds their own - jbang ecj@eclipse will work without interference.

...and if we do this - we could still add the notion of no catalog means jbanghub...so jbang ecj@/eclipse will be available for shorthand form.

This gives consistency and control to the original authors but jbang community can also easily add commands and have them visible.

wdyt - am I overthinking it ? and should we do the shorthand form or just stay with enabling jbanghub as a "thing"?

[maxandersen]

related bug I found with alias naming of catalogs not handled consistently: #1746