chore: Miscellaneous website improvements

[bradenhilton]

Fixes #3180

• Move website dependencies to a requirements.txt file, remove unnecessary or deprecated dependencies.
• Update website contribution steps.
• Prefer POSIX paths in hooks to prevent potential execute-template errors.
• Replace symlinks with real files for each Link type.
• Rename Doppler template function docs directory.
• Add gitHubReleases and gitHubTags template function docs to nav.

[halostatue]

LGTM, but I haven’t tried anything yet.

[halostatue]

Still works well on macOS.

[halostatue]

This looks good.

[halostatue]

I also ran black over the python scripts, and aside from needing to use black -S (to prevent changing all ' to ", which should be done in its own commit), this looks good to me.

As a side note for @twpayne (who I think might be on vacation), I don’t see a way for maintainers to trigger a website update (there’s no workflow file for the pages-build-deployment workflow, and the website build/update in the main workflow is only executed on release). Maybe I’m missing something?

[bradenhilton]

My latest snapshot renames the Doppler template function docs directory and also adds gitHubReleases and gitHubTags to the nav. These changes remove this output:

INFO    -  The following pages exist in the docs directory, but are not included in the "nav" configuration:
             - reference\templates\doppler\index.md
             - reference\templates\doppler\doppler.md
             - reference\templates\doppler\dopplerProjectJson.md
             - reference\templates\github-functions\gitHubReleases.md
             - reference\templates\github-functions\gitHubTags.md

We should be able to configure dependabot to manage the requirements.txt file, which would mean we could move the mkdocs version pin in the main workflow to requirements.txt (along with pinning the other dependencies), e.g.

# requirements.txt

mkdocs==1.5.2
mkdocs-material==9.1.21
mkdocs-mermaid2-plugin==1.0.8

then just use pip3 install -r requirements.txt in the workflow jobs, and the same package versions would be used everywhere.

@halostatue I didn't go out of my way to format the file(s), but I wouldn't mind setting up a linter and/or formatter if we can come to some sort of consensus on which one to use and how to configure it.

[halostatue]

My latest snapshot renames the Doppler template function docs directory and also adds gitHubReleases and gitHubTags to the nav. These changes remove this output:

Ooops. Sorry about that; I forgot to add the index entries for the new gitHub* functions when I was documenting everything else (shame on me for only doing fd instead of fd and ag).

We should be able to configure dependabot to manage the requirements.txt file, which would mean we could move the mkdocs version pin in the main workflow to requirements.txt (along with pinning the other dependencies), e.g.

# requirements.txt

mkdocs==1.5.2
mkdocs-material==9.1.21
mkdocs-mermaid2-plugin==1.0.8

then just use pip3 install -r requirements.txt in the workflow jobs, and the same package versions would be used everywhere.

Yeah. Python package management is a mess. I’ve been playing with different solutions from Pipenv to Poetry to (most recently) PDM. The workflows can use pip3 install -r requirements.txt, but the instructions for developers should be pushing the venv approach unless we wanted to add something like PDM — and our needs are simple enough that I believe the answer should be "nah".

That could change in the future if we need to do more Python scripting, and ultimately it does not matter which one we use because we aren’t shipping a library, but are instead dealing with an application, and all of the options push very strongly toward venv configurations.

@halostatue I didn't go out of my way to format the file(s), but I wouldn't mind setting up a linter and/or formatter if we can come to some sort of consensus on which one to use and how to configure it.

I think that there’s more than a few linters, but the least controversial choice right now would be ruff, I think, as a Python outsider. I would add it to requirements.txt as ruff==0.0.285 and then you can run it with ruff check --show-source . in the workflow. This will currently pass on your branch, if added.

For formatters, the answer is black, so black==23.7.0 and run as black --check .. This will currently fail if added, because black prefers " over ' (there is a slightly less compromising formatter called blue which has the opposite preference), and the current Python code has mixed single quotes and double quotes. So I would have a single commit that is just the output of black .

Dependabot configuration is easy:

version: 2
updates:
  - package-ecosystem: pip
    directory: /assets
    schedule:
      interval: <whatever>

I would move assets/chezmoi.io/requirements.txt to assets/requirements.txt so that it’s clear that the requirements apply to both assets/chezmoi.io/docs/hooks.py and assets/scripts/format-yaml.py, or I would have two different requirements.txt files, separating the use of ruamel.yaml==0.17.32 with assets/scripts/format-yaml.py.

Simplicity to me says one file used for both scripts.

[twpayne]

Thank you all, this looks excellent!

As a side note for @twpayne (who I think might be on vacation), I don’t see a way for maintainers to trigger a website update (there’s no workflow file for the pages-build-deployment workflow, and the website build/update in the main workflow is only executed on release). Maybe I’m missing something?

This statement is correct (including the vacation part!). So far, I've been using the main / deploy-website GitHub job (eventually triggered automatically when you push a tag like v2.3.4) to deploy the website as I want the website to document the most recent stable version of chezmoi. For emergency fixes I've been using mkdocs gh-deploy in the assets/chezmoi.io which should work for everyone with write access to the github.com/twpayne/chezmoi repo, i.e. all maintainers.

The reason for not having a separate GitHub action to push the website is that, as chezmoi uses a single branch for all changes (master), there is not a way to push the website without including website changes related to changes since the last release. I don't think this is worth fixing, as it's rare that we have to do this, and it's quicker to do it manually when needed rather than set up a more complex branching system and backporting/cherry-picking changes to previous versions.

Please do merge this as-is. We have accumulated enough changes for a new release after this PR is merged.

I agree that at formatter like black would be fantastic, and we should use a single requirements.txt file for simplicity, but these can be done in a follow-up PR, and are not user-visible changes and so can be done at any time in the future.

[bradenhilton]

Thanks all for the discussion and tips!

I'll squash the commits shortly and merge if/when the checks pass.