Support loading external files in OpenAPI

[mwaterous]

Regarding the updates made in 0.74.0: https://gohugo.io/news/0.74.0-relnotes/

The current implementation doesn't appear to allow isExternalRefsAllowed to be configured anywhere and it is false by default. This causes any attempt to unmarshal an openapi spec using external references to fail: https://github.com/getkin/kin-openapi/blob/9b9280d707773bb764ec07ac7363248d8240343d/openapi3/swagger_loader.go#L67

This could be a really fantastic feature for some of us building documentation sites in Hugo based on the OpenAPI spec, but due to the above, it's at present only useful for smaller and non-complex openapi specifications.

Could this either be set to true by default when initializing the kin-openapi library, or could this be designed so that we can set it to true in config.toml or somewhere else that makes sense?

[illeatmyhat]

My OpenAPI YAMLs are getting pretty big, would be appreciated

[bep]

Question: I assume that you by external (mostly) mean files on the same file system (and not remote files)?

[illeatmyhat]

As for me, yes on the same file system.
Specifically, I organized all of my YAMLs under the assets/ directory since that's what works with resources.Get

[mwaterous]

Same for me, all on the same file system.

[bep]

OK,

I hadded the initial OpenAPi support in Hugo, and the project I used it in has the API source mounted as a Hugo Module. So, this is what we need to do:

1. Set SwaggerLoader.IsExternalRefsAllowed to true
2. Implement a SwaggerLoader.ReadFromURIFunc that resolves the files in the Assets (only) filesystem (which allows them to live in modules).

[illeatmyhat]

I was able to hack something together that works for my specific use-case, but isn't production ready.

There are a number of issues

1. Relative $refs (e.g. $ref: ./definitions/foobar.yaml#/components/schema/Foobar need to be resolved relative to the current YAML file being parsed, and it isn't clear how we're supposed to figure out the location of the current YAML file with SwaggerLoader.ReadFromURIFunc(). Without this, we will not be able to recursively resolve $refs.
   Turns out ReadFromURIFunc will automatically provide the correct relative URL from the root file.

2. It isn't clear how we're supposed to determine whether a file is in the Assets filesystem, and then receive a filepath in that filesystem. (e.g. go from ./definitions/foobar.yaml to /workspaces/my-docs/assets/api/definitions/foobar.yaml)
   I attempted to use SourceFilesystem.MakePathRelative() and SourceFilesystem.Path(), and neither worked for me.

loader := kopenapi3.NewSwaggerLoader()
loader.IsExternalRefsAllowed = true
loader.ReadFromURIFunc = func(loader *kopenapi3.SwaggerLoader, url *url.URL) ([]byte, error) {
	if url.Scheme != "" || url.Host != "" || url.RawQuery != "" {
		return nil, errors.Errorf("unsupported URI: %q", url.String())
	}

	fullpath := filepath.Join(filepath.Dir(key), url.Path)
	assetpath := ns.deps.SourceFilesystems.Assets.RealDirs(fullpath)[0]
	return ioutil.ReadFile(assetpath)
}
err = loader.ResolveRefsIn(s, nil)

[bep]

@illeatmyhat It's hard for me to debug your example/code without additional context, but I'll try:

• I think the closest existing example would be somthing like this:

  hugo/resources/resource_transformers/js/options.go

  Line 184 in 8a30894

  resolveImport := func(args api.OnResolveArgs) (api.OnResolveResult, error) {

• You by definition needs to mount all your API files inside /assets, so if it's not found there, it does not exist.
• Then, in my head, there are two cases:
  • Either the file includes are root relative (e.g. definitions/foobar.yaml)
  • Or they are relative to the current file (e.g. ./baz.yaml)

There may be discussions about the details above, but the first is simple (use fs.Stat directly), for the second you need to determine the relative directory via MakePathRelative.

[EPoY74]

Hello! Is this issues still relevant?