treewide: Improve docker container build, CI caching and python dependencies

[fischeti]

Docker container

• Use the official Verilator container as final base container, which has verilator prebuilt
• Improve docker build stages for better caching
• Clean up apt installs and getting rid of apt-requirements.txt
• Bump python to 3.11
• Bump Verilator to 5.020

CI

• Build Docker container on every branch as part of the ci.yml workflow. Before, the docker container had to be manually changed and triggered on the working branch if the Docker container needed modification.
• Cache docker build (~3min if cached, ~15min for clean build)
• Cache Verilator simulation build if file dependencies (generated from Makefiles) remain unchanged (saves ~20min)
• Disable redundant runs for pull_request if the pull request is from an internal contributor.
• Temporarily disable Banshee in CI, as installation fails with a dependency error.

Python dependencies

• Consolidate all dependencies into the pyproject.toml file, replaces all requirements.txt files.

[colluca]

Thanks @fischeti! LGTM apart for a few comments which follow :)

[fischeti]

Undrafting this again after quite a lot of changes. The PR now touches a lot more than just consolidating python dependencies (see updated description).

[colluca]

Thanks for the extensive efforts @fischeti!

It looks very good, just have a few questions/comments.

[colluca]

I pushed commit 4f89a6f to automate generation of the dependency list which determines the Verilator model build.

It is based on the List Make Prerequisites Action, which parses Make's database output (from make -pq), to generate the dependency tree for any top-level target (in our case bin/snitch_cluster.vsim). It then lists all leaf nodes of the tree, i.e. all file dependencies the target ultimately depends on, and hashes their contents.

The method is still not 100% correct, e.g. if FESVR_VERSION is changed it will not trigger a cache miss. A solution would be to include the Makefiles themselves in the list of dependencies. The List Make Prerequisites Action could be extended to this end, but probably also the Makefiles would have to be split up to mix the least amount of logic, in order not to trigger unnecessary builds on every Makefile edit.

Nonetheless, the method is at least as precise as the previously proposed method, so I think we can continue with this solution, and eventually refine it in the future, when needed.