Skip to content

Commit

Permalink
Merge branch 'master' into spt
Browse files Browse the repository at this point in the history
  • Loading branch information
blake-mealey committed May 21, 2022
2 parents 70532bc + 2612a9f commit 05470db
Show file tree
Hide file tree
Showing 19 changed files with 461 additions and 1,269 deletions.
3 changes: 3 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
@@ -1,5 +1,8 @@
# See https://help.github.com/articles/ignoring-files/ for more about ignoring files.

# secrets
.env

# dependencies
/node_modules
/.pnp
Expand Down
10 changes: 10 additions & 0 deletions _pages/fun.mdx
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
---
title: Fun
---

## Why not?

I play a lot of Rocket League... this widget updates regularly with my latest time played according
to Steam.

<RocketLeague />
4 changes: 2 additions & 2 deletions _pages/home.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -8,9 +8,9 @@ Hi! I'm Blake Mealey, and this is my little corner of the internet.

What's on this site:

- [~/work](/work): A list of projects I've built in my free
time
- [~/projects](/projects): A list of projects I've built in my free time
- [~/posts](/posts): A collection of my thoughts
- [~/fun](/fun): Random fun things

You can find me elsewhere on the internet on [GitHub](https://github.com/blake-mealey) and
[Twitter](https://twitter.com/blakemdev).
Expand Down
4 changes: 2 additions & 2 deletions _pages/work.mdx → _pages/projects.mdx
Original file line number Diff line number Diff line change
@@ -1,8 +1,8 @@
---
title: Work
title: Projects
---

## Work
## Projects

A list of projects I've built in my free time.

Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,124 @@
---
title: Documenting a Rust Project with JSON Schemas
date: 2022-02-04
status: draft
---

My side project [Mantle](https://mantle-docs.vercel.app) is an infra-as-code tool built with Rust
which is configured with YAML files. The key thing my docs need to communicate past the getting
started phase to my users is the format for these config files.

The [first
iteration](https://github.com/blake-mealey/mantle-docs/blob/80b64fdf06485f1efc78395ba5664a08c60699bd/docs/configuration.md)
of my configuration docs were written by hand in a markdown file in the docs site repo. They weren't
terrible, but I had a few issues with them.

## Choose the format for the job

When I started writing these docs, I organized them by complex types. At the top of the file, I
listed each of the top-level properties in the config file along with their type. If the type was
complex, I created a new heading in the document and linked to it.

This worked, but it's a format that's better suited to documenting the classes in a library than the
properties in a config file.

I once watched someone read the document for the first time, and it was unclear to them how the
information they were seeing related to the config file itself, even with the smattering of example
blocks I included.

## Manual changes are tedious

Because I was writing the document by hand, if I ever wanted to make a style change to the page, I
had to make it manually for every object and property. This was not ideal.

## Separate from code, separate from tooling

Documentation in a standalone markdown file is fine for humans to read, but not so good for
computers. If I ever wanted to surface this documentation in other mediums (e.g. inline in VSCode or
with an interactive CLI), I would need to rethink things.

## Enter, JSON Schemas

JSON Schemas are a format for specifying what makes JSON documents valid. Schemas themselves are
JSON documents which makes them easy to read and write. Since YAML is basically just JSON with
cleaner syntax, you can use a JSON schema to validate YAMl files too!

Now I _could_ have replaced my manually written markdown file with a manually written JSON schema
file, but this would have been very unpleasant. I still want to format my docs with markdown, and
considering JSON has no multiline strings that would have been very painful. It also would have
meant manually documenting all of my type information.

Fortunately, many languages have tooling for generating JSON schemas from their type systems, and
Rust is no exception. I discovered [schemars](https://graham.cool/schemars/) which can be easily
used to generate JSON schemas from my Rust structs and enums just by adding a
`#[derive(JsonSchema)]` attribute to them. And the great thing is, schemars integrates with serde so
that it understands serde attributes and produces a schema which matches serde's validation!

I did experience a couple of challenges using schemars, however. schemars uses Rust's doc comments
to capture the JSON schema `description` property, but it does some mangling to the text first which
means that it breaks markdown formatting. To get around this I am working off of a fork of the
library, but I hope to merge in an option to address this properly in the future.

Another challenge I ran into was wanting to add extension properties to my JSON schema. schemars
does support this through its Visitor API, but there is currently no attribute to add an extension
property to a Rust type directly. My solution was to add a special line at the top of my
descriptions when I wanted to add a custom property and then I parsed these out in a Visitor. I also
hope to add this as an attribute to the core library when I get a chance.

## Generating some docs

Now I just needed to generate some docs. Coming back to the format question from earlier, I decided
to try and emulate some examples of other projects documenting large config files like [GitHub
Actions](https://docs.github.com/en/actions/using-workflows/workflow-syntax-for-github-actions),
[GitLab CI](https://docs.gitlab.com/ee/ci/yaml/index.html), and
[Rollup.js](https://rollupjs.org/guide/en/#big-list-of-options). Basically, instead of organizing by
classes, these docs are just a nested list of all the properties in the schema. Each property
includes the full path name (e.g. `jobs.<job-id>.name`). This makes it very clear to the reader how
the documentation can be applied to the config file itself!

I took a look at some existing JSON schema docs generators but none of them really looked the way I
wanted them to. Most of them used a similar organization to what I started with, so I decided to
roll my own.

I wrote a TS script to load a JSON schema and flatten the properties into a big list, along with
some additional metadata like the full property name, and whether it is a required property.

Then, I wrote a simple Handlebars template which iterated over the properties and printed their docs
in a consistent format.

## Autocomplete and in-editor docs

To take advantage of my schemas even more, I tried loading it into VSCode. With the YAML extension
installed, it was as easy as adding `"yaml.schemas": { "mantle.yml": "schema.json" }` to add
autocomplete and in-editor docs to my config files.

But, they looked... bad. The formatting was all off.

It turns out VSCode interprets the `description` property of schemas as plaintext, and if you want
it to look correct, you need to use a `markdownDescription` property. I was also using some
non-standard markdown syntax which my docs platform ([Docusaurus](https://docusaurus.io)) supports
(like admonitions and code block titles).

To resolve this, I wrote another TS script to transform a schema into something VSCode can properly
interpret. I used remark to parse the markdown into an AST, then modified the AST.

My biggest challenge here was related to Node.js's ESM support. Recently, the remark ecosystem has
"upgraded" to ESM-only, but currently TS has very bad interop with these modules. In the end, I
decided to just use older versions of these packages which use CommonJS modules.

## Tying it all together

Once I was happy with the schema I was generating from my types, I updated my `deploy` GitHub Action
to generate the schema and upload it with my GitHub release assets. Now whenever I bump my version,
a GitHub Action builds my project, creates a Release, and uploads the binaries and schema.

Then I updated my docs generation script to start by downloading all schemas from past releases. I
use the latest one to generate my docs page, then I run all of them through my transformer to make
them VSCode-ready and save them to my site's static directory. Now when I deploy the site, these
schemas are hosted with it (e.g. my
[v0.11.0](https://mantle-docs.vercel.app/schemas/v0.11.0/schema.json) schema).

My users can now just add `"yaml.schemas": { "mantle.yml": "https://mantle-docs.vercel.app/schemas/v0.11.0/schema.json" }`
to add autocomplete to their editor!

TODO: Conclusion
18 changes: 17 additions & 1 deletion components/Layout.tsx
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,7 @@ import clsx from 'clsx';
import Link from 'next/link';
import Badge from '../components/Badge';
import formatDate from '../util/formatDate';
import Script from 'next/script';

type LayoutProps = {
slug: string;
Expand All @@ -25,6 +26,18 @@ const Layout: NextPage<LayoutProps> = ({ children, slug, meta }) => {

return (
<div className={styles.container}>
<Script
strategy="afterInteractive"
src="https://www.googletagmanager.com/gtag/js?id=UA-144998331-5"
/>
<Script id="google-analytics" strategy="afterInteractive">
{`window.dataLayer = window.dataLayer || [];
function gtag(){dataLayer.push(arguments);}
gtag('js', new Date());
gtag('config', 'UA-144998331-5');`}
</Script>

<Head>
<title>{title}</title>
<meta name="description" content={description} />
Expand All @@ -43,11 +56,14 @@ const Layout: NextPage<LayoutProps> = ({ children, slug, meta }) => {
<Link href="/home">home</Link>
</li>
<li>
<Link href="/work">work</Link>
<Link href="/projects">projects</Link>
</li>
<li>
<Link href="/posts">posts</Link>
</li>
<li>
<Link href="/fun">fun</Link>
</li>
</ul>
</nav>

Expand Down
2 changes: 2 additions & 0 deletions components/MdxRenderer.tsx
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,7 @@ import createHeadingComponent from './createHeadingComponent';
import Shortcut from './Shortcut';
import Image from 'next/image';
import SubstantialPresenceTest from './SubstantialPresenceTest';
import RocketLeague from './RocketLeague';

const shortcodes = {
Shortcut,
Expand All @@ -21,6 +22,7 @@ const shortcodes = {
h5: createHeadingComponent('h5'),
h6: createHeadingComponent('h6'),
SubstantialPresenceTest, // TODO: Should be able to import from just the article that uses this
RocketLeague,
};

const MdxRenderer = function ({ source }: { source: any }) {
Expand Down
72 changes: 72 additions & 0 deletions components/RocketLeague.tsx
Original file line number Diff line number Diff line change
@@ -0,0 +1,72 @@
import { FC, useEffect, useState } from 'react';
import styles from './rocket-league.module.css';

interface ResponseData {
minutes: number;
}

type State =
| {
status: 'loading';
}
| { status: 'loaded'; data: ResponseData }
| { status: 'error'; error: any };

const numberFormatter = new Intl.NumberFormat('en-US', {
style: 'unit',
unit: 'hour',
unitDisplay: 'long',
});

interface ContentProps {
state: State;
}

const Content = ({ state }: ContentProps) => {
if (state.status === 'loading') {
return <div>...</div>;
}

if (state.status === 'error') {
console.error(state.error);
return <div>Something went wrong :(</div>;
}

if (state.status === 'loaded') {
return (
<div>
<div className={styles.label}>Time played</div>
<div>{numberFormatter.format(Math.floor(state.data.minutes / 60))}</div>
</div>
);
}

return null;
};

export default function RocketLeague() {
const [state, setState] = useState<State>({
status: 'loading',
});
useEffect(() => {
fetch('/api/rocket-league')
.then((res) => {
res
.json()
.then((data) => setState({ status: 'loaded', data }))
.catch((error) => {
setState({ status: 'error', error });
});
})
.catch((error) => {
setState({ status: 'error', error });
});
}, []);

return (
<div className={styles.container}>
<img className={styles.logo} src="/images/rocket-league.svg" />
<Content state={state} />
</div>
);
}
28 changes: 28 additions & 0 deletions components/rocket-league.module.css
Original file line number Diff line number Diff line change
@@ -0,0 +1,28 @@
.container {
font-size: 40px;
padding: var(--theme-spacing-2) var(--theme-spacing-2);
border: 2px solid var(--theme-primary);
border-radius: var(--theme-roundness);
width: fit-content;
display: flex;
align-items: center;
gap: 1em;
}

.logo {
width: 5em;
}

.label {
font-size: 0.5em;
font-weight: bold;
opacity: 0.75;
margin-bottom: -0.5em;
}

@media only screen and (max-width: 600px) {
.container {
font-size: 20px;
flex-wrap: wrap;
}
}
6 changes: 6 additions & 0 deletions next.config.js
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,12 @@ module.exports = {
destination: '/home',
permanent: true,
},
{
source: '/work',
destination: '/projects',
// Not totally sure that we want this to be permanent - maybe we want to have something else at /work in the future?
permanent: false,
},
];
},
};
2 changes: 1 addition & 1 deletion package.json
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@
"@types/remark-prism": "^1.3.0",
"clsx": "^1.1.1",
"gray-matter": "^4.0.3",
"next": "12.0.7",
"next": "12.0.9",
"next-mdx-remote": "^3.0.8",
"prism-theme-night-owl": "^1.4.0",
"react": "17.0.2",
Expand Down
2 changes: 1 addition & 1 deletion pages/[slug].tsx
Original file line number Diff line number Diff line change
Expand Up @@ -49,7 +49,7 @@ export const getStaticProps: GetStaticProps<PageProps> = async ({ params }) => {

export const getStaticPaths: GetStaticPaths = async () => {
return {
paths: ['/home', '/work'],
paths: ['/home', '/projects', '/fun'],
fallback: false,
};
};
Expand Down
2 changes: 2 additions & 0 deletions pages/_document.tsx
Original file line number Diff line number Diff line change
Expand Up @@ -4,10 +4,12 @@ export default function Document() {
return (
<Html>
<Head>
<link rel="icon" type="image/svg+xml" href="favicon.svg" />
<link
rel="stylesheet"
href="https://fonts.googleapis.com/css2?family=Fira+Code:wght@400;500;600;700&display=swap"
/>
<script src="https://cdn.splitbee.io/sb.js" async></script>
</Head>
<body>
<Main />
Expand Down
Loading

0 comments on commit 05470db

Please sign in to comment.