Add support for gwit sites

[ivilata]

(This comes from a few messages in #agregore:mauve.moe around the beginning of March.)

gwit is a specification (based on an idea from Gemini's Solderpunk) for publishing lightweight sites over Git. A PGP key identifies the site & signs commits, URIs are a PGP fingerprint + optional version (commit) + path in repo. It doesn't dictate a transport, a site may be hosted at several untrusted Git remotes which one uses to clone/update the whole site, then URI retrieval (or content search) is a local op. Remotes are either provided manually (which may feel odd to a web-like browser on first access) or recommended by already visited sites.

gwit strives to cover some use cases with maximum simplicity (just Git+PGP+URI format): personal site hosting on forges or dumb HTTP servers for lightweight static content (to adapt existing Web/Gemini sites), offline private browsing of news, blogs, or reference sites à la Kiwix+Wikipedia, trivial distributed site preservation… Because of its intended use and its distributed nature (inherited from Git), I think it'd be a great match for Agregore.

Please note that currently there is no implementation yet (I plan to start one soon), and that the spec, though kinda complete, is still a work in progress (reviews and comments welcome in its mailing list!). I'll post updates about the spec and implementation here whenever they are relevant for the potential inclusion in Agregore. Thanks!

[ivilata]

I'm planning to create an initial/reference implementation (which may evolve along the spec while the latter is WIP) of a client for the command line. It would go along the lines of Wget or cURL, i.e. it would get a gwit URI as an argument, it would retrieve it and produce the resource contents (i.e. Git repo file) to stdout. It wouldn't produce any headers or other metadata for the moment (for the moment, the spec recommends to use file extensions as a hint on what to do with the file, though some guessing could be done on the content of the file itself).

Also note that the client would be read-only: content creation would be done via plain Git for the moment. I'll eventually publish some usage examples for the hypothetical client, but first I need to create some publication channels for that. I'll ping here when they're ready.

Do you think that such minimal client interface may play well with Agregore?

[RangerMauve]

Yeah! Read only sounds like a good target to start with.

Generally, the easiest path is to make a module that builds off of make-fetch.

If your implementation isn't easy to load into Node.js I could see about running it as a sidecar of sorts (the way we do with go-ipfs). For that having some sort of CI with cross platform builds would make it easier.

For the headers, I usually add some for making life easier for devs like guessing the content-type form the file extension and giving the content-length for sizes. Also I usually add a basic way to get a directory listing in either a JSON array of files/subfolders, or a basic HTML page if the Accept flag contains text/html. I also find that supporting Range headers can make a huge difference but that's more optional.

[ivilata]

Thanks @RangerMauve for the hints! Yeah, I guess that the sidecar approach may do the trick. I may add an URI retrieval mode or option to the client that produces HTTP-like output with those headers, that shouldn't be a big deal (and the client would support some dir listing as well, so all the required functionality would be already in place quite early, except maybe ranges).

I'll come back with more info as the implementation progresses! 🙂

[RangerMauve]

In my experience, sidecars that provide an HTTP API (via RPC or REST) has been the easiest to integrate with other apps. Either that or FFI bindings for languages (but that's usually more flakey and harder to set up)