Augment a registry to catalog existing rulesets

[alexeagle]

There are existing resources for Bazel users to find rulesets, but none are good enough:

• https://bazel.build/rules is too hard for community to update, outdated but "looks official"
• https://awesomebazel.com has manual curation, but no data pipeline keeping it updated, not comprehensive
• https://registry.bazel.build is restricted in what data it can use, must be exclusively backed by bazel-central-registry
• https://registry.build shows the README for the module and copy-paste install instructions
• https://slack.bazel.build has a bunch of threads, you might find someone's opinion on what ruleset to use
• https://bazel-contrib.github.io/bazel-catalog/ is an MVP developed for this issue, but incomplete WIP

The goal of this issue is to improve one of these to satisfy user needs, which are:

1. Not have six different places to look
2. Automation to stay updated with respect to canonical datasources
3. Full-text search, i.e. a search for "typescript" should yield aspect_rules_ts. Implies that README content appears.
4. Can figure out which rulesets are "recommended": active, maintained, released recently, some automated or manual curation of a "quality signal"
5. Can see how widely adopted the ruleset is, based on stargazers, download count, etc
6. Can see/filter on important attributes like "is tested on Windows", "works with Bzlmod", "tested on Bazel 8"
7. Can see/filter on "is this a ruleset" vs. "it's a C++ library with some BUILD files"
8. Include rules on BCR, but maybe also those which aren't
9. [Stretch] includes a stardoc pipeline, similar to readthedocs.com so that Starlark API docs appear, are searchable

[jsharpe]

On thing we should probably suggest is a standard repository tagging structure; this is how buildkite discovers plugins; we could do something similar with rules? e.g. bazel-rules github repository tag?

[alexeagle]

Maybe we also want to take over a page like https://docs.bazel.build/versions/main/rules.html#recommended-rules as part of this issue.

[alexeagle]

I think an "MVP" to resolve this issue should have:

• a web view like https://docs.aspect.build/stats.html (I'm happy to donate what little code exists to run that: https://gist.github.com/alexeagle/8d40bd35debc89f0450ebc5f3b610470)
• automation to refresh the data periodically
• sortable columns [stretch]
• usage stats [if @aiuto can get us the data]
• publish the result on our existing GitHub Pages under https://bazel-contrib.github.io/SIG-rules-authors/

[hobofan]

So from what I've gathered from today's SIG meeting, it seems like having the catalog is still something that has use for the SIG, and will only be partially supplemented/replaced by the BCR.

On top of that, it seems like having an up-to-date list of all rulesets and their existing (pre-bzlmod) dependencies would also help in bootstrapping the BCR by giving insight into which are the "most blocking" rulesets for wider bzlmod adoption (= rulesets that many other rulesets depend on).

So as the first step toward that, my plan would be to:

• Try to extend the existing catalog to be as comprehensive as possible regarding the number of rulesets
• Extend the information about the existing rulesets with:
  • Dependencies (via manual inspection of WORKSPACE/README)
  • Name in the BCR (if already published there)
  • Whether automatic publishing to the BCR is set up (to differentiate from sporadic one-off publications that you can't depend on yet as a rules author; I've seen that a GH Action for that is in the works)
  • Whether it's bzlmod module has feature parity (which I understand isn't a given for everything that exposes repository rules?)

@alexeagle WDYT?

[aiuto]

I'll once again repeat my assertion that we need more than a catalog. There should be a repository that does nothing but cross rule-set CI. - it requires a specific bazel track (4.x, 5.x, head) - has a WORKSPACE or MODULE.bazel file that depends on the top N (let's say 10-15) common rules at specific versions - bazel build //... should just work without flags - bazel test //... should pass without flags - it includes both android and ios applications that depends on native C++ code. - It has a web UI that interoperates with the same back end as the mobile apps. The goal is to go beyond saying that rules_go passes its tests with Bazel 5.x. It is to say that rules_go works with rules_proto + rules_apple to build a complex product. In the past I had presumed that Google should own this, but perhaps it is a worthy community project. Or, perhaps Google pays for the CI machines, but does not own the content of the test projects.

…

On Tue, May 31, 2022 at 4:22 PM Maximilian Goisser ***@***.***> wrote: So from what I've gathered from today's SIG meeting, it seems like having the catalog is still something that has use for the SIG, and will only be partially supplemented/replaced by the BCR. On top of that, it seems like having an up-to-date list of all rulesets and their existing (pre-bzlmod) dependencies would also help in bootstrapping the BCR by giving insight into which are the "most blocking" rulesets for wider bzlmod adoption (= rulesets that many other rulesets depend on). So as the first step toward that, my plan would be to: - Try to extend the existing catalog to be as comprehensive as possible regarding the number of rulesets - Extend the information about the existing rulesets with: - Dependencies (via manual inspection of WORKSPACE/README) - Name in the BCR (if already published there) - Whether automatic publishing to the BCR is set up (to differentiate from sporadic one-off publications that you can't depend on yet as a rules author; I've seen that a GH Action for that is in the works) @alexeagle <https://github.com/alexeagle> WDYT? — Reply to this email directly, view it on GitHub <#2 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAXHHHCN6ECV2OOO64KW6Z3VMZYH3ANCNFSM5FQFP5RA> . You are receiving this because you were mentioned.Message ID: ***@***.***>

[alexeagle]

That sounds great, but running that CI sounds expensive and hard, and the SIG has barely done any meaningful engineering so far, so I don't think we should try to expand the scope of this one.
Also, the BCR has CI for interoperability as an explicit goal and has corporate funding to operate it, so maybe this one should be on the Google side of the "fence"?

[alexeagle]

Update: @ashi009 has made some progress and we now have https://github.com/bazel-contrib/bazel-catalog as a home for it.

[alexeagle]

Design doc: https://docs.google.com/document/d/1SVYOrKRJ6jZaK43-wSJfk16ghwx5q_8GpeQpSzrhFa4/edit#heading=h.lqnathh99lzv

[alexeagle]

A "minimal viable product" for this is now at https://bazel-contrib.github.io/bazel-catalog/

[alexeagle]

Another update 15mo later - I have not found volunteer time to make progress on that MVP.

Discussed at today's Rules Authors SIG meeting that we would fund a developer to come make progress on this.

Note that the volunteer will have to be mostly self-managing since the SIG only meets bi-weekly and no one has much time to mentor/direct this work.