Skip to content

Commit

Permalink
floogen: Add NoC generation framework (#24)
Browse files Browse the repository at this point in the history
* refactor: Rename `src` folder to `hw`

* refactor: Move testbenches into `hw/tb`

* refactor: Move test modules into `hw/test` folder

* refactor: Create new `hw/tb/wave` directory for all wave scripts

* refactor: Move `typedef.svh` in `hw/include` folder

* refactor: Modify `Bender.yml` accordingly

* refactor: Change paths in Makefile

* refactor: Update `fusesoc.core`

* refactor: Update links in README

* refactor: fix directory of `test/include` folder in `Bender.yml`

* refactor: Fix more include folders

* refactor: Fix more paths

* floogen: Copy in repository

* license: Update license headers in python and YAML files

* ci: Add pylint

* ci: Install floogen as package before linting

* py: Build only `floogen` package

* ci: Install pylint before linting

* py: Add click as python dependency

* py: Update Homepage in `pyproject.toml`

* py: Lint python files

* py: add `jsonref` as python dependency

* py: Ignore similar python code lint warning for now

* jobs: Minor fix to pass CI

* ci: Remove Whitespace

* ci: Add NoC generation to CI

* ci: Try to execute floogen as executable

* ci: Write output to SV files

* ci: Format and upload the specific file

* ci: Lint generated SV files

* ci: Update action versions

* ci: Minor Change

* routing: Check for mismatched start, end and size of `AddrRange`

* tests: Add first unit tests with pytest

* py: Add `pylint` to dependencies

* ci: Run pytests

* test: Add more unit tests for routing tables

* test: Add graph creation unit tests

* py: Remove `requirements.txt` file since this is now integrated into the `pyproject.toml`

* ci: Cache `pip` + use python version 3.10

* LICENSE: Use different licenses for HW and SW

* LICENSE: Change wording sligthly

* LICENSE: Change a couple of license headers

* floogen(org): Move utils to `floogen` folder

* floogen(util): Add functions for rendering system verilog declarations and typedefs

* floogen(utils): Add function for formatting code with verible

* floogen(routing): Implement rendering functions as class methods instead of templates

* floogen(link): Render flits with a class method instead of templating

* floogen(protocol): Render parameters and typdefs as class methods instead of templating

* floogen: Add support for rendering package-only configurations

* floogen(examples): Add package-only example configurations for narrow-wide and axi

* floogen(examples): Fix ID widths of AXI protocols

The id widths were switch at some point when refactoring the direction from `input`/`output` to `manager`/`subordinate`. In a Snitch cluster, the manager ports usually have a larger ID width than subordinate ports

* floogen(pkg): Remove hjson dumping

Everything is now handled directly by floogen. The flit configuration in `hjson` is not needed anymore

* floogen(protocol): Don't render `axi` prefix twice

If the protocol is already named `axi`, there is no need to prefix it again

* floogen(pkg): Move pkg template to floogen template folder and clean up

Most of the declarations and typedefs are now rendered with class methods instead of templating. This cleans up the template quite a bit. Helper functions were also moved to `utils.py`

* floogen: Source formatting

* hw(pkg): Regenerate the SV packages

* hw(chimney): Align with the slightly new naming conventions of the parameters

Every AXI parameter does now have a `Axi` prefix instead of starting with `Narrow` or `Wide`

* floogen: Lint sources

* make: Add `floogen` targets

`floogen` can now be installed with `install-floogen`.
The Makefile will automatically do that if it does not find a `floogen` executable in the path.
The previous targets for generating the flit packages have been moved to `floogen` targets

* py: Bump the `floogen` version to `0.1.0` for release

* ci: Install `floogen` as non-editable

* ci: Fix argument usage

Before the argument was called `no_formatting` which was refactored to `--no-format`

* py: Support non-editable installation with pip

Before the specific modules of `floogen` where not included which is why it failed when it was installed non-editable i.e. `pip install .` instead of `pip install -e .`

* floogen(lint): Refactor nested f-string

Nested f-strings are only supported in python 3.12, but `floogen` currently should support 3.10

* ci: Disable formatting of `floogen` and use github action instead

The problem is that `floogen` needs `verible-verilog-format` as an executable in the path which is a bit annyoing.
Instead `floogen` will not format the output which is taken care of by the verible github action

* ci(verible): Specify required `config_file`

* make: Make it possible to pass additional arguments to the `source` target

This is primarily needed in the CI where the output of `floogen` should not be formatted, since the CI does not have a formatter installed

* Revert "ci(verible): Specify required `config_file`"

This reverts commit a0179c1.

* ci(lint): Update hw folder

* ci(verible): Add `config_file` parameter

The `config_file` parameter is required apparently, and it triggers a warning if it is not set.

* hw(lint): Whole bunch of linting fixes and aligning with the new naming conventions

* ci: Ignore reviewdog folder which causes unclean state after linting

* hw(lint): More fixes

* hw(lint): Fix directory in waiver file

* ci(gitlab): Install `floogen` before running script

* ci(gitlab): Try using installed `floogen`

* ci(gitlab): Create conda environment to install python 3.10

* ci(gitlab): Fix small typo

* ci(gitlab): Try again with other python installation

* ci(gitlab): Remove old python venv creation

* bender: Remove deprecated `axi_channel_compare`

* tb(axi_chimney): Add missing ports

* hw(pkg): Use previous configuration that is actually supported

* util(jobs): Re-add removed python shabang

* test(dma_mesh): Fix a couple of stuff

* test(hbm_model): Remove unused `TCK` parameter

* tb(nw_chimney): Set `IgnoreId` parameter

The `axi_chan_compare` module from the `axi` has a `IgnoreId` parameter for remapped Ids. The test module used previously implemented this functionality as well, but didn't defined a parameter. Since this feature was upstreamed, the previous version can be deleted now.

* tb(nw_chimney): Add a cut between the chimneys

Without the cut, the chimneys have zero latency, which means AXI requests on the manager side are not yet registered by `axi_chan_compare`, when the arrive on the subbordinate side.
I am not sure though, why it worked before with the old module `axi_channel_compare`

* make(vsim): Disable accessibility, when running in batch/console mode

This should speed up simulation in theory. In GUI mode the accessibility is still enabled, since it is needed to check the waves

* sim(nw_chimney): Update the wave script

* hw(pkg): Increase the number of X and Y bits, as well as the address offset

* tb(dma_mesh): Use types from the generated package

* floogen(pkg): Fix names for address offset bits

* tb(dma_mesh): Use parameters from package

* tb(dma_mesh): Add a couple of assertions

Check that enough bits were specified in the configuration file to generate the packages

* ci: Disable the reviewdog when formatting

* Revert "ci: Disable the reviewdog when formatting"

This reverts commit 1198e22.

* test(hbm_model): Remove `HBMSize` parameter and use only `AddrWidth` instead

Size is anyway not used in the model, it was only used before to derive the `AddrWidth`

* test(dma_model): Remove unused `TCK` parameter

* hw(pkg): Use 3 bits for X and Y ID, and fix address offset

This matches a 4x4 mesh with HBM on the boundaries, which is the case for the `dma_mesh`

* tb(dma_mesh): Fix coordinates of east HBMs

* util(gen_jobs): Fix base address calculation

* ci(gitlab): Make the virtual environment an artifact

* floogen(routing): Declare `UseIdTable` parameteer in package

* floogen(routing): Render the address tables as parameter instead of signals

* floogen(routing): Provide default values for all routing parameters

The SystemVerilog modules need defined parameters even if they are not used for a specific routing algorithm

* floogen(render): Fix whitespaces when rendering address tables

* hw(chimney): Move routing related parameters to the package

* floogen(pkg): Render routing related information in the package

* util(flitgen)!: Remove previous flitgen script and config files

This is now integrated into floogen

* hw(tb): Remove parameters from chimney instances that were removed

* hw(chimney): Define Address Map as parameter instead of input

* hw(tb): Fix missing signal in BW monitor

* hw(test): Add void casts to get rid of warnings

* hw(tb): Remove `xy_id_t` typedefs

* hw(pkg): Align coordinate dimensions in rob testbench

* doc: Update README with information about `floogen`

* doc: Move around sections in README

* doc: Try to fix the generation link in the README

* floogen(vis): Fix visualization feature

* doc: Update required python version in README

* doc: Update the CHANGELOG

* doc: Add `floogen` documentation

* doc: Small fix in CHANGELOG

* doc: Add `route_comp` module in README

* doc: Move both licenses to the root directory

* make: Bump QuestaSim default version
  • Loading branch information
fischeti authored Jan 9, 2024
1 parent 0531e95 commit e0b7a5e
Show file tree
Hide file tree
Showing 100 changed files with 4,935 additions and 1,412 deletions.
84 changes: 84 additions & 0 deletions .github/workflows/floogen.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,84 @@
# Copyright 2024 ETH Zurich and University of Bologna.
# Solderpad Hardware License, Version 0.51, see LICENSE for details.
# SPDX-License-Identifier: SHL-0.51

# Author: Tim Fischer <[email protected]>

name: floogen

on: [push, pull_request]

jobs:

unit-test:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- name: Set up Python
uses: actions/setup-python@v5
with:
python-version: '3.10'
- name: Install dependencies
run: |
python -m pip install --upgrade pip
python -m pip install .
- name: Run unit tests
run: |
python -m pytest -v
#####################
# Generate SV files #
#####################
gen-nocs:
runs-on: ubuntu-latest
strategy:
matrix:
examples: ["single_cluster", "occamy_mesh", "occamy_tree"]
steps:
- uses: actions/checkout@v4
- name: Set up Python
uses: actions/setup-python@v5
with:
python-version: '3.10'
- name: Install dependencies
run: |
python -m pip install --upgrade pip
- name: Install floogen
run: |
python -m pip install .
- name: Generate NoCs
run: |
floogen -c floogen/examples/${{ matrix.examples }}.yml -o generated --no-format
- name: Format generated NoCs
uses: chipsalliance/verible-formatter-action@main
with:
files:
./generated/${{ matrix.examples }}_floo_noc.sv
github_token: ${{ secrets.GITHUB_TOKEN }}
- name: Upload artifact
uses: actions/upload-artifact@v4
with:
name: ${{ matrix.examples }}
path: generated/${{ matrix.examples }}_floo_noc.sv
if-no-files-found: error
retention-days: 1

#################
# Lint SV files #
#################
lint-nocs:
runs-on: ubuntu-latest
needs: gen-nocs
steps:
- uses: actions/checkout@v4
- name: Download artifact
uses: actions/download-artifact@v4
with:
path: generated
- name: Lint SV files
uses: chipsalliance/verible-linter-action@main
with:
config_file: ''
paths:
./generated
github_token: ${{ secrets.GITHUB_TOKEN }}
79 changes: 57 additions & 22 deletions .github/workflows/lint.yml
Original file line number Diff line number Diff line change
Expand Up @@ -16,21 +16,28 @@ jobs:
check-clean:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v3
- uses: actions/setup-python@v4
- uses: actions/checkout@v4
- uses: actions/setup-python@v5
with:
python-version: '3.9'
python-version: '3.10'
cache: 'pip'
- run: pip install -r requirements.txt
- name: Install dependencies and package
run: |
python -m pip install --upgrade pip
pip install .
- name: Install bender
uses: pulp-platform/pulp-actions/bender-install@v2
- name: Install verible
- name: Make clean sources
run: |
curl -Ls -o verible.tar.gz https://github.com/chipsalliance/verible/releases/download/v0.0-3313-gddcea377/verible-v0.0-3313-gddcea377-Ubuntu-22.04-jammy-x86_64.tar.gz
mkdir verible && tar -xzf verible.tar.gz -C verible --strip-components 1
- name: Check clean make targets
make clean sources FLOOGEN_ARGS="--no-format"
- name: Format with verible
uses: chipsalliance/verible-formatter-action@main
with:
files:
./hw/**/*.sv
github_token: ${{ secrets.GITHUB_TOKEN }}
- name: Check clean
run: |
make clean sources VERIBLE_FMT=verible/bin/verible-verilog-format
git status && test -z "$(git status --porcelain)"
###############
Expand All @@ -39,22 +46,29 @@ jobs:
check-stale:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v3
- uses: actions/setup-python@v4
- uses: actions/checkout@v4
- uses: actions/setup-python@v5
with:
python-version: '3.9'
python-version: '3.10'
cache: 'pip'
- run: pip install -r requirements.txt
- name: Install dependencies and package
run: |
python -m pip install --upgrade pip
pip install .
- name: Install bender
uses: pulp-platform/pulp-actions/bender-install@v2
- name: Install verible
- name: Make clean sources
run: |
curl -Ls -o verible.tar.gz https://github.com/chipsalliance/verible/releases/download/v0.0-3313-gddcea377/verible-v0.0-3313-gddcea377-Ubuntu-22.04-jammy-x86_64.tar.gz
mkdir verible && tar -xzf verible.tar.gz -C verible --strip-components 1
- name: Check clean makefile
make -B sources FLOOGEN_ARGS="--no-format"
- name: Format with verible
uses: chipsalliance/verible-formatter-action@main
with:
files:
./hw/**/*.sv
github_token: ${{ secrets.GITHUB_TOKEN }}
- name: Check clean
run: |
make -B sources VERIBLE_FMT=verible/bin/verible-verilog-format
git status && test -z "$(git status --porcelain)"
git status && test -z "$(git status --porcelain)"
#####################
# Bender up-to-date #
Expand All @@ -68,19 +82,40 @@ jobs:
################
# Verible Lint #
################
lint:
verible-lint:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v3
- uses: actions/checkout@v4
- uses: chipsalliance/verible-linter-action@main
with:
config_file: ''
paths: |
./src
./hw
extra_args: "--waiver_files util/verible.waiver"
github_token: ${{ secrets.GITHUB_TOKEN }}
fail_on_error: true
reviewdog_reporter: github-check

##########
# PyLint #
##########
pylint:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- name: Set up Python
uses: actions/setup-python@v5
with:
python-version: '3.10'
cache: 'pip'
- name: Install dependencies and package
run: |
python -m pip install --upgrade pip
pip install .
- name: Analysing the code with pylint
run: |
pylint $(git ls-files '*.py')
#################
# Check License #
#################
Expand Down
85 changes: 75 additions & 10 deletions .gitignore
Original file line number Diff line number Diff line change
@@ -1,20 +1,85 @@
# Temporary and binary files
*~
*.py[cod]
*.so
*.cfg
!.isort.cfg
!setup.cfg
*.orig
*.log
*.pot
__pycache__/*
.cache/*
.*.swp
*/.ipynb_checkpoints/*
.DS_Store

# Project files
.ropeproject
.project
.pydevproject
.settings
.idea
.vscode
.dvt
tags

# Package files
*.egg
*.eggs/
.installed.cfg
*.egg-info

# Unittest and coverage
htmlcov/*
.coverage
.coverage.*
.tox
junit*.xml
coverage.xml
.pytest_cache/

# Build and docs folder/files
build/*
dist/*
sdist/*
docs/api/*
docs/_rst/*
docs/_build/*
cover/*
MANIFEST

# Per-project virtualenvs
.venv*/
.conda*/
.python-version

# bender
.bender
*modelsim.ini
*transcript
*work
Bender.lock
Bender.local
deps

# Auto generated sources
generated
*.png
hw/test/jobs

# QuestaSim
work
work-vsim
modelsim.ini
transcript
vsim.wlf
scripts/compile_vsim.tcl
working_dir/*
compile.log
Bender.lock
.dvt
dvt_build.log
.vscode

# Verible
verible*

# spyglass
scripts/spyglass/*

test/jobs
# Misc
*.csv
**/*.log
reviewdog
17 changes: 15 additions & 2 deletions .gitlab-ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -7,27 +7,40 @@
variables:
VSIM: questa-2023.4 vsim
BENDER: bender
PYTHON: /home/fischeti/micromamba/envs/floo/bin/python

stages:
- sources
- init
- build
- run

collect-bender-sources:
stage: sources
stage: init
script:
- $BENDER checkout
artifacts:
paths:
- .bender/
- Bender.lock

python-virtualenv:
stage: init
script:
- $PYTHON -m venv .venv
- source .venv/bin/activate
- pip install .
artifacts:
paths:
- .venv/

compile-vsim:
stage: build
script:
- source .venv/bin/activate
- make compile-sim | tee compile.log 2>&1
- '! grep "Error: " compile.log'
needs:
- python-virtualenv
- collect-bender-sources
artifacts:
paths:
Expand Down
Loading

0 comments on commit e0b7a5e

Please sign in to comment.