AsyncAPI document bundler

[magicmatatjahu]

I do not plan to implement the idea during the hackathon. If no one comes forward, then sooner or later we have to implement something similar, so the idea won't be lost. However, feel free to pick this one up.

Some people in the community have asked if we know of a tool that would bundle all references from an single AsyncAPI document into a single file - similar to https://github.com/APIDevTools/json-schema-ref-parser/blob/main/docs/ref-parser.md#bundleschema-options-callback

In short, if you have an AsyncAPI file with references to local files/external sources like web pages etc, you need to bundle these references (or more precisely, their values) into a single AsyncAPI file under components/* paths. Also, the resulting AsyncAPI document should be small, since it should still contain internal JSON references rather than being fully-dereferenced.

Some example:

# messages.yaml
messages:
  UserSignedUp:
    payload:
      type: object
      properties:
        displayName:
          type: string
          description: Name of the user
        email:
          type: string
          format: email
          description: Email of the user
          
# extensions.yaml
extensions:
  some-extension:
    foobar: barfoo

# asyncapi.yaml
asyncapi: '2.2.0'
info:
  title: Account Service
  version: 1.0.0
  description: This service is in charge of processing user signups
channels:
  user/signedup:
    subscribe:
      message:
        $ref: './messages.yaml#/messages/UserSignedUp'
x-extensions:
  some-extension:
    $ref: './extensions.yaml#/extensions/some-extension'

and then after bundling we should have single file like:

asyncapi: '2.2.0'
info:
  title: Account Service
  version: 1.0.0
  description: This service is in charge of processing user signups
channels:
  user/signedup:
    subscribe:
      message:
        $ref: '#/components/messages/UserSignedUp'
components:
  messages:
    UserSignedUp:
      payload:
        type: object
        properties:
          displayName:
            type: string
            description: Name of the user
          email:
            type: string
            format: email
            description: Email of the user
x-extensions:
  some-extension:
    $ref: '#/x-components/some-extension'
x-components:
  some-extension:
    foobar: barfoo

Note that things that are not part of the specification but are referenced, such as extensions, should be referenced from x-components or some other section.

If possible, use the source code from https://github.com/APIDevTools/json-schema-ref-parser package

The idea can be implemented as separate package and in future we can introduce it in parser-js under bundle function.

[siathalysedI]

Actually this is related to the Single Schema Dictionary Definitions Management Tool for Multiple AsyncAPI Definitions and this could relate to a remote data dictionary definition

[magicmatatjahu]

Depending on how you understand Scheme. If as JSON Scheme for payloads etc then not really, because the idea of the tool is to support also other parts of the spec like message (also extensions) etc. However, if you identify the Scheme as part of the specification then I agree :)

[siathalysedI]

I don't get why exactly it would be part to or being related to because what you want if I'm understanding right is to being able to reference to external references in a more comprehensible way. So, in your bundle you could have somehing like an url to an external component (And here is where the data dictionary management tool comes in)

[magicmatatjahu]

It's hard for me to answer, but if I understand this dictionary as a set of reusable schemas/models/spec parts, then still the bundler should "copy" for you only those parts to them you use in the AsyncAPI document.

[boyney123]

Love this idea @magicmatatjahu, think it is very powerful.

I guess $ref can also point to external urls, I assume this tool would/could also pull down from these areas too, and not just limited to local file system?

[magicmatatjahu]

Yep, it should also bundle all external urls :)

[patrickp-at-work]

this proposal is really useful. For example if you want to paste your spec into browser-based tools to validate them or experiment (such as async api playground). Or any other tool without access to the $referenced files. Tools from the REST ecosystem feature such dereferencing, e.g. SwaggerHub. Stoplight has also implemented it using json-schema-ref-parser, as explained here, so that's probably a good choice.

[magicmatatjahu]

Yes, but as far as Stoplight or SwaggerHub have a server where they can "dereference" in the browser, we want to go for pure browser applications with the possibility of running locally via cli :) So the idea of bundler is a bit different than in case of Stoplight or SwaggerHub.

[patrickp-at-work]

Yes, understood. I wanted to emphasize that the idea has indeed very high practical value for API designers and that the proposed library is battle proven. Basically a thumbs up with a justification why I like it.

[derberg]

https://github.com/asyncapi/bundler is there