What can be done about supporting dynamic references?

[gnidan]

Months ago, I told myself I wouldn't fall into an old habit of making everything generic... yes, $dynamicRef and $dynamicAnchor are quite interesting, esp. how the JSON Schema blog suggests a very nice way to specify a List<T> schema, but it's madness, right? Certainly it's no problem that the excellent docusaurus-json-schema-plugin doesn't support it... no problem at all that @stoplight/json-ref-resolver (used by this plugin) is now unmaintained, so even if I wanted to use $dynamicRef, it'd be quite unfeasible to get support...

Anyway, hi, nice to see you again. Turns out that dynamic references are exactly the tool I need for one of these schemas I'm writing.

I might be opening this discussion a bit early, since I've only just started playing around with the new UnsolvedRefs customization (which I think I'll need in conjunction with swizzling), so I don't yet have a lot of findings. I plan to add to this discussion as I see what the current capabilities are (and how much of a pain in the neck they are ;).

In the meantime, do you have any near-term goals for addressing this (beyond the unsolved refs stuff)? I apologize if I am forgetting some discussion in this repo somewhere, but I'm wondering what your thoughts are regarding the stoplight library, since I imagine you're probably not too happy about its end of life... will this plugin need to find another alternative, or maybe add custom support itself for all these latest JSON Schema features? I ask in case there's a way for me to implement my customizations that'd be appropriate for eventual merging upstream

[jy95]

Welcome back @gnidan ,

My motivation for contributing is like many open-source projects : to have new contributions / contributors and users.
Supporting dynamic references like I mentioned in the docs would be quite a huge achievement in this project which started, it must be said, as many do: out of frustration at not having a solution or something not efficient ^^

That said, in the past I'd started a plan (not perfect and still open to discussion) :

1. Reflexions about the resolving library & usages here

Many libraries are out :

• @stoplight/json-ref-resolver
• @apidevtools/json-schema-ref-parser
• json-schema-library
• @hyperjump/browser
• ...

First thing would be to compare the libraries : what they offer, their maintenance, strenghs and weakness, ...
I don't mind having a breaking 2.x release in the future If balance is beneficial

2. Design the proposal (how to handle it, display it, ...) extensively

This is the point with the most functional, technical and visual questions.
This must take into account many scenarios that will need to be tested and visualized in the documentation.
(Not to mention recursion problems, like infinite Russian matryoshka dolls ^^)

3. Getting users feedbacks about the proposal

Many actors (like ajv ) acknowledge that draft-07 is most widely used version of JSON Schema specification. Remember on the roadmap page, I mentioned which version of the specification a keyword was linked to ? 2019-09 and 2020-12. I saw that many other projects (like monacco I also used) started considering these versions thanks to user requests like in microsoft/monaco-editor#3160 or microsoft/monaco-editor#4114

If we manage to find feedback of any kind on it ( for example, https://json-schema.org/#community have a Slack but not only : other teams / companies could be involved from my opinion ), it would make the product / solution better

Side Note

In the future, the UnsolvedRefs component will somehow likely still remain useful in case users ask "Why my references weren't resolved" (as people can make mistake like everyone).

If the project had more contributors, we'd be able to make faster progress on the subject.

[gnidan]

Thanks for these initial thoughts, @jy95! I certainly agree that your project here would benefit from more contributors ;)

I'm hopeful these days (read: "years") that we're about to see a growth in tooling support for the latest JSON Schema drafts. It does seem like the JSON Schema folks are looking to use draft 2020-12 as the structural basis for a first "stable" version of JSON Schema, which I imagine should cause a bit of a rebirth around tooling improvements.

Unfortunately it does seem like we are still a bit early - yes, draft-07 is the most widely-used, probably because draft-07 is the most widely supported. I took a quick look at the $ref resolution libraries you linked; they all seem to have significant downsides that make them not fit for use here (like how two out of the three are at end-of-maintenance).

You didn't mention it, but I haven't yet reviewed hyperjump's stuff, either... they seem to be very good at implementing the JSON Schema spec precisely, so maybe their logic for evaluating $refs is self-contained and usable directly. I'm not entirely hopeful, though, since their API expects you to addSchema()s to some singleton module scope... I imagine this plugin would much prefer an approach like schemas.add(), since you don't know that a particular <JSONSchemaViewer /> makes sense within an unknown global context.

Regarding good design for presenting dynamic references... this also is not trivial.

See this screenshot of how I present a "concrete" schema. (I haven't done any presentation work on the "generic" schema yet)

My plan with this approach is to rely on my application-specific mapping of "schema IDs to page URLs", which seems essential for this presentation to not become overwhelming. I'm also using the word "generic" as if it has some meaning... all around, not ideal.

The code is even worse... I use swizzling to override default component rendering, to look for conventions I'm using in my schema that indicate a semantic meaning for specific display strategy, then based on the results of this detection, I replace objects containing $dynamicRef: ... with the detected $dynamicAnchors, to mimic how stoplight's json-ref-resolver does it.

My key takeaway right now with implementation is that this plugin (thanks to json-ref-resolver) loses all information on schemas that define a $ref field, since json-ref-resolver clobbers the object-containing-$ref completely. So I have to do { allOf: [{ "$ref": { /* ... */ }], ...otherSchemaProperties } if I want to be able to reference otherSchemaProperties at all.

That's all I have for now... I'll spend some more time researching $ref resolution (and hope in the meantime that I come up with a better, more-general-purpose strategy for display)

[gnidan]

Update: I was able to find a way to avoid the use of dynamic references (with only a small loss of expressiveness). For now, I won't be digging into this too much more, although we'll see... other parts of my schema might end up needing those later.

[jy95]

Update for the first point : new libraries are out like @json-schema-tools/dereferencer - let's see how they are doing in the future

• https://github.com/json-schema-tools/dereferencer
• https://github.com/fastify/json-schema-ref-resolver
• https://github.com/json-schema-tools/reference-resolver