reStructuredText Documentation Proposal

[morrone]

I propose that we adopt reStructuredText (rst) for all LDMS's documentation needs. OVIS-HPC in general could also adopt rst.

I have also mentioned asciidoc as a possibility, but reStructuredText has two main advantages:

• reStructuredText has a much simpler tools ecosystem (sphinx, vs a mess of various tools in the asciidoc/docbook world)
• reStructuredText has integration with Read the Docs. And Read the Docs has good integration with Github.

But I think we need to go about using rst in a method that is different from what we see in PR #1423.

I propose that the documentation in the LDMS source tree be specific to the products of said source tree. In other words, it should not contain slides or other materials from conferences, nor should it have detailed information about other OVIS-HPC subprojects like sos (except insomuch as we document things like a sos store which is part of the ldms code repository).

I propose that documentation specific to other ovis-hpc subprojects (sos, maestro, etc) have documentation specific to those projects in each of those respective repositories.

We should set up automated linkage between Read the Docs and github such that every release of a subproject (ldms, sos, etc.) results in a new build of the documentation on Read the Docs, and users of the code can always pull up either the "latest" documentation, or documentation specific to a

We should also set up the linkage with Read the Docs so that every PR results in a documenation build in Read the Docs to enable the ability to verify that the documentation is correctness before landing a PR. (This is an ability build in to Read the Docs.)

I propose that we create a separate top-level meta documentation project that can be used to document those things that are not specific to a single subproject (pointers to the individual subprojects' docuemtation, conference materials, tutorials that span multiple subprojects, etc.)

As a tangible example, we should consider the flux-framework project. Flux-framework does everything I suggested here, and more.

You can see the meta-documentation page, Flux-framework. The source for this meta-documentation live in the subproject flux-framework/flux-docs. Note that there is a Sub-Projects tab at the top of that project that links off to, for instance, Flux Core's documenation, which live in the flux-framework/flux-core repository with flux-core's source code.

To complete the analogy, the mapping of flux code repos to the equivalents in ovis-hpc would be:

• fllux-docs: ovis-docs (top-level documentation)
• flux-core: ldms (a sub-project)
• flux-sched: sos (a sub-project)

Note too that even man pages are written in rst in the flux project. The roff versions of the man pages are generated at build time. I think (eventually) we should do that as well. rst is much easier to write than roff, and we of course do not want two copies of the same documentation in the source tree. One or the other needs to be generated automatically.

[tom95858]

@morrone, where do want these .RST files to actually live? Alongside the code or in a separate directory or directories?

[morrone]

I think most of the documentation would go in the top level doc/ subdirectory in each repository. Although if we want to have the man page rst files more distributed like we currently have with our man pages, I imaging that would be possible.

[tom95858]

@morrone, my thinking is only that when we're evaluating pull requests for new features or fixes that might require a documentation update, this is not possible to do trivially if the doc files are in a different repo. If they reside in the same repo, the 'missing' RST updates will not appear in the pull request and the reviewer can complain.

[tom95858]

@morrone, I did review the Flux docs and documentation source and it is much more powerful than what I had read. So I think your proposal is a good one.