diff --git a/README.md b/README.md index 43658082b..943dbc423 100644 --- a/README.md +++ b/README.md @@ -111,22 +111,23 @@ T1 includes a hardware design written in Chisel and an emulator powered by a ver Users can add their own pokemon to `configgen/src/Main.scala` to add configurations with different variations. You can build its components with the following commands: + ```shell $ nix build .#t1.elaborator # the wrapped jar file of the Chisel elaborator $ nix build .#t1..ip.rtl # the elaborated IP core .sv files $ nix build .#t1..ip.emu-rtl # the elaborated IP core .sv files with emulation support -$ nix build .#t1..ip.emu # build the IP core emulator -$ nix build .#t1..ip.emu-trace # build the IP core emulator with trace support +$ nix build .#t1..ip.verilator-emu # build the IP core emulator using verilator +$ nix build .#t1..ip.vcs-emu # build the IP core emulator using VCS +$ nix build .#t1..ip.vcs-emu-trace # build the IP core emulator using VCS w/ trace support $ nix build .#t1..subsystem.rtl # the elaborated soc .sv files $ nix build .#t1..subsystem.emu-rtl # the elaborated soc .sv files with emulation support $ nix build .#t1..subsystem.emu # build the soc emulator $ nix build .#t1..subsystem.emu-trace # build the soc emulator with trace support - -$ nix build .#t1..cases.all # the testcases ``` -where `` should be replaced with a configuration name, e.g. `bulbasaur`. The build output will be put in `./result` directory by default. + +where `` should be replaced with a configuration name, e.g. `blastoise`. The build output will be put in `./result` directory by default. Currently under tested configs: @@ -137,34 +138,60 @@ Currently under tested configs: | **Sandslash** | `DLEN1K VLEN4K ; NOFP; VRF p0rw bank4; LSU bank16 beatbyte 16` | | **Alakazam** | `DLEN2K VLEN16K; NOFP; VRF p0rw bank8; LSU bank8 beatbyte 64` | +The `` could also be `t1rocket`, +this is special configuration name that enable rocket-chip support for scalar instruction. + #### Run Testcases To run testcase on IP emulator, use the following script: ```shell -$ nix develop -c t1-helper ipemu -c -C +$ nix develop -c t1-helper run -c -e ``` wheres - `` is the configuration name; -- `` is the name of a testcase, you can resolve runnable test cases by command: `make list-testcases`; +- `` is one of the `verilator-emu`, `verilator-emu-trace`, `vcs-emu`, `vcs-emu-trace` +- `` is the name of a testcase, you can resolve runnable test cases by command: `t1-helper listCases -c `; For example: + ```shell -$ nix develop -c t1-helper ipemu --config blastoise -C intrinsic.linear_normalization +$ nix develop -c t1-helper run -c blastoise -e vcs-emu intrinsic.linear_normalization +``` + +To get waveform, use the trace emulator + +```console +$ nix develop -c t1-helper run -c blastoise -e vcs-emu-trace intrinsic.linear_normalization +``` + +The `` and `` option will be cached under `$XDG_CONFIG_HOME`, +so if you want to test multiple test case with the same emulator, +you don't need to add `-c` and `-e` option every time. + +For example: + +```console +$ nix develop -c t1-helper run -c blastoise -e vcs-emu-trace intrinsic.linear_normalization +$ nix develop -c t1-helper run pytorch.llama ``` -To get waveform, add `--trace` at the end of the above command: +To get verbose logging, add the `-v` option ```console -$ nix develop -c t1-helper ipemu --config blastoise -C intrinsic.linear_normalization --trace +$ nix develop -c t1-helper run -v pytorch.lenet ``` -The `t1-helper` is a emulator wrapper that provides various command-line options for different use cases. -To get all available options, run: +The `t1-helper run` subcommand only run the driver without validating internal status. +To run design verification, use the `t1-helper check` subcommand: ```console -$ nix develop -c t1-helper --help +$ nix develop -c t1-helper run -c blastoise -e vcs-emu mlir.hello +$ nix develop -c t1-helper check ``` +The `t1-helper check` subcommand will read RTL event produced in `run` stage, +so make sure you `run` a test before `check`. + #### Export RTL Properties ```shell @@ -218,127 +245,74 @@ $ nix develop .#t1.elaborator.editable # or if you want submodules editable $ mill -i elaborator # build and run elaborator ``` -#### Developing Emulator +#### Developing DPI ```shell -$ nix develop .#t1..ip.emu # replace with your configuration name +$ nix develop .#t1..ip.vcs-dpi-lib # replace with your configuration name $ cd ipemu/csrc -$ cmake -B build -GNinja -DCMAKE_BUILD_TYPE=Debug -$ cmake --build build -$ cd ..; ./scripts/run-test.py verilate --emulator-path=ipemu/csrc/build/emulator conv-mlir -``` - -If using clion, -```shell -$ nix develop .#t1..ip.emu -c clion ipemu/csrc -``` - -#### Rocket emulator - -Rocket emulator contains multiple build phrase: RTL -> MLIR Bytecode -> -system verilog -> verilated C sources -> Rust emulator. - -Most of the developer doesn't need to care about MLIR, system verilog and verilate detail. -To develop the Rocket-chip RTL, run: - -```bash -# This command provide a environment that contains mill, circt, espresso... development tools. -nix develop '.#t1.elaborator' -``` - -> Metals LSP users are recommended to switch to mill-bsp mode instead of the default bloop mode. - -To elaborate the RTLs, run mill or use the nix chroot: - -```bash -# for development -mill -i elaborator.runMain org.chipsalliance.t1.elaborator.Main -# for clean build -nix build .#t1.rocketv-mlirbc -``` - -To develop the emulator, use the below nix environment: - -```bash -nix develop .#t1.rocketv-emu.driver.devShell -``` - -This will setup the verilated C src in environment, download rust-analyzer. - -```bash -cd rocketemu/driver -cargo build --release +$ cargo build --feature dpicommon/vcs ``` #### Developing Testcases -The `tests/` contains the testcases. There are four types of testcases: +The `tests/` directory contains all the testcases. - asm +- codegen - intrinsic - mlir -- codegen - perf +- pytorch +- rvv_bench -To add new testcases for asm/intrinsic/mlir, create a new directory with `default.nix` and source files. -Refer to the existing code for more information on how to write the nix file. - -To add new testcases for codegen type cases, add new entry in `codegen/*.txt`, then our nix macro will automatically populate new testcases to build. - -To view what is available to ran, use the `nix search` sub command: +To view what is available to run, use the `t1-helper listCases` sub command: ```console -$ nix search .#t1 +$ t1-helper listCases -c ``` For example, ```console -$ nix search .#t1 asm -* legacyPackages.x86_64-linux.t1..cases.asm.fpsmoke - Test case 'fpsmoke', written in assembly. - -* legacyPackages.x86_64-linux.t1..cases.asm.memcpy - Test case 'memcpy', written in assembly. - -* legacyPackages.x86_64-linux.t1..cases.asm.mmm - Test case 'mmm', written in assembly. - -* legacyPackages.x86_64-linux.t1..cases.asm.smoke - Test case 'smoke', written in assembly. - -* legacyPackages.x86_64-linux.t1..cases.asm.strlen - Test case 'strlen', written in assembly. - -* legacyPackages.x86_64-linux.t1..cases.asm.utf8-count - Test case 'utf8-count', written in assembly. - -# Then ignore the `legacyPackage.x86_64-linux` attribute, build the testcase like below: -$ nix build .#t1..cases.asm.smoke +$ t1-helper listCases -c blastoise mlir +[INFO] Fetching current test cases + +* mlir.axpy_masked +* mlir.conv +* mlir.hello +* mlir.matmul +* mlir.maxvl_tail_setvl_front +* mlir.rvv_vp_intrinsic_add +* mlir.rvv_vp_intrinsic_add_scalable +* mlir.stripmining +* mlir.vectoradd + +$ t1-helper listCases -c blastoise '.*vslid.*' +[INFO] Fetching current test cases + +* codegen.vslide1down_vx +* codegen.vslide1up_vx +* codegen.vslidedown_vi +* codegen.vslidedown_vx +* codegen.vslideup_vi +* codegen.vslideup_vx ``` To develop a specific testcases, enter the development shell: ```shell -# nix develop .#t1..cases.. +# nix develop .#t1..ip.cases.. # # For example: -$ nix develop .#t1..cases.asm.smoke +$ nix develop .#t1.blastoise.ip.cases.pytorch.llama ``` Build tests: ```shell # build a single test -$ nix build .#t1..cases.intrinsic.matmul -L -$ ls -al ./result - -# build all tests -$ nix build .#t1..cases.all --max-jobs $(nproc) +$ nix build .#t1..cases.ip.intrinsic.matmul -L $ ls -al ./result ``` -> [!TIP] -> All the `mk*Case` expression are defined in `./nix/t1/default.nix`. - ### Bump Dependencies Bump nixpkgs: ```shell @@ -347,7 +321,7 @@ $ nix flake update Bump chisel submodule versions: ```shell -$ cd nix/t1 +$ cd nix/t1/dependencies $ nix run '.#nvfetcher' ``` diff --git a/difftest/readme.md b/difftest/readme.md deleted file mode 100644 index dfd1c0380..000000000 --- a/difftest/readme.md +++ /dev/null @@ -1,11 +0,0 @@ -## Build - -```bash -nix build ".#t1..ip.difftest" -``` - -## Develop - -```bash -nix develop ".#t1..ip.difftest.devShell" -``` diff --git a/nix/overlay.nix b/nix/overlay.nix index dfb2f4b50..5d14c6221 100644 --- a/nix/overlay.nix +++ b/nix/overlay.nix @@ -48,7 +48,6 @@ rec { t1-script = final.callPackage ../script { }; inherit (t1-script) t1-helper ci-helper; - dev-mode = final.callPackage ./pkgs/dev-mode { }; # stdenv for compiling rvv programs, with ilp32f newlib and clang rv32-stdenv = rv32_pkgs.stdenv.override { diff --git a/nix/pkgs/dev-mode/default.nix b/nix/pkgs/dev-mode/default.nix deleted file mode 100644 index e76256bc2..000000000 --- a/nix/pkgs/dev-mode/default.nix +++ /dev/null @@ -1,8 +0,0 @@ -{ mkShellNoCC, jq, ncurses, gettext }: -mkShellNoCC { - name = "t1-dev-mode"; - nativeBuildInputs = [ jq ncurses gettext ]; - shellHook = '' - source ${./dev-mode.sh} - ''; -} diff --git a/nix/pkgs/dev-mode/dev-mode.sh b/nix/pkgs/dev-mode/dev-mode.sh deleted file mode 100644 index 20f791403..000000000 --- a/nix/pkgs/dev-mode/dev-mode.sh +++ /dev/null @@ -1,263 +0,0 @@ -export TMPDIR=/tmp - -# prefer terminal safe colored and bold text when tput is supported -if tput setaf 0 &>/dev/null; then - ALL_OFF="$(tput sgr0)" - BOLD="$(tput bold)" - BLUE="${BOLD}$(tput setaf 4)" - GREEN="${BOLD}$(tput setaf 2)" - RED="${BOLD}$(tput setaf 1)" - YELLOW="${BOLD}$(tput setaf 3)" -else - ALL_OFF="\e[0m" - BOLD="\e[1m" - BLUE="${BOLD}\e[34m" - GREEN="${BOLD}\e[32m" - RED="${BOLD}\e[31m" - YELLOW="${BOLD}\e[33m" -fi -readonly ALL_OFF BOLD BLUE GREEN RED YELLOW - -info() { - local mesg=$1; shift - printf "${GREEN}[T1]${ALL_OFF}${BOLD} ${mesg}${ALL_OFF}\n" "$@" -} - -info2() { - local mesg=$1; shift - printf "${BLUE} ->${ALL_OFF}${BOLD} ${mesg}${ALL_OFF}\n" "$@" -} - -error() { - local mesg=$1; shift - printf "${RED}[T1] $(gettext "ERROR:")${ALL_OFF}${BOLD} ${mesg}${ALL_OFF}\n" "$@" >&2 -} - -_ROOT_PATTERN="flake.nix" -if [ ! -r "$_ROOT_PATTERN" ]; then - error "Not in T1 root directory" "Please run nix develop at the root of the T1 project directory" - exit 1 -fi - -mkdir -p .cache/dev-mode -_CTX=.cache/dev-mode/state.json -if [ ! -r "$_CTX" ]; then - printf "{}" > "$_CTX" -fi - -_CFG="" -get_config() { - local cfg=$(jq -r ".config" "$_CTX") - _CFG="$cfg" - echo "$_CFG" -} - -_config_is_set() { - _CFG=$(get_config) - if [[ -z "$_CFG" || "$_CFG" = "null" ]]; then - info "Config not set yet, use 'set_config ' to set config" - info2 "Available configs: ${BLUE}$(list_config | xargs)${ALL_OFF}" - return 1 - else - return 0 - fi -} - -list_config() { - nix --no-warn-dirty eval ".#t1.allConfigs" --json | jq -r 'keys[]' -} - -set_config() { - if [[ -z "$@" ]]; then - error "No argument was given, use 'set_config ' to set config" - return 1 - fi - - local config=$1; shift - local configArray=( $(list_config) ) - # TODO: this is not an elegant way - configArray+=("t1rocket") - - local verifiedConfig="" - for c in ${configArray[@]}; do - if [[ "$c" == "$config" ]]; then - verifiedConfig="$c" - fi - done - if [[ -z "$verifiedConfig" ]]; then - error "Config '$config' is not supported, available configs: ${configArray[*]}" - return 1 - fi - - local newCtx=$(jq --arg config "$config" '.config = $config' $_CTX) - printf "$newCtx" > "$_CTX" - - _CFG=$(get_config) - - info "Config is set to ${BLUE}${_CFG}${ALL_OFF}" -} - -_get_ip_attrs() { - nix --no-warn-dirty eval --json ".#legacyPackages.x86_64-linux" \ - --apply "pkgs: pkgs.lib.filterAttrs (_: v: pkgs.lib.isDerivation v) pkgs.t1.$_CFG.ip" \ - | jq -r 'keys[]' -} - -list_emulator() { - _get_ip_attrs | grep -E '(-emu$|-emu-trace$)' -} - -_EMU="" -get_emulator() { - local emu=$(jq -r ".emulator" "$_CTX") - if [[ -z "$emu" || "$emu" = "null" ]]; then - error "Emulator not set yet, please use 'set_emulator '" - return 1 - fi - - _EMU="$emu" - echo "$_EMU" -} - -_emulator_is_set() { - _EMU=$(get_emulator) - if [[ -z "$_EMU" || "$_EMU" = "null" ]]; then - return 1 - else - return 0 - fi -} - -set_emulator() { - if [[ -z "$@" ]]; then - error "No argument was given, use 'set_emulator ' to set main testing emulator" - return 1 - fi - - local emulator=$1; shift - local allEmuArray=( $(list_emulator) ) - - local verifiedEmulator="" - for e in ${allEmuArray[@]}; do - if [[ "$e" == "$emulator" ]]; then - verifiedEmulator="$e" - fi - done - if [[ -z "$verifiedEmulator" ]]; then - error "Emulator '$emulator' is not supported, available emulators: ${allEmuArray[*]}" - return 1 - fi - - local newCtx=$(jq --arg emulator "$verifiedEmulator" '.emulator = $emulator' $_CTX) - printf "$newCtx" > "$_CTX" - - _EMU=$(get_emulator) - - info "Emulator is set to ${BLUE}${_EMU}${ALL_OFF}" -} - -build() { - if [[ -z "$@" || "$@" = "help" ]]; then - error "Use ${BLUE}'build '${ALL_OFF} to build" - info2 "Available attrs" - _get_ip_attrs - return 1 - fi - - local attr=$1; shift - local availableAttrsArray=( $(_get_ip_attrs) ) - local verifiedAttr="" - for a in ${availableAttrsArray[@]}; do - if [ "$a" = "$attr" ]; then - verifiedAttr="$a" - fi - done - if [ -z "$verifiedAttr" ]; then - error "Invalid attr '$attr', available attrs:" - info2 "${availableAttrsArray[*]}" - return 1 - fi - - if ! _config_is_set; then - return 1 - fi - - local fullAttr=".#t1.$_CFG.ip.$attr" - info "Building $fullAttr to result dir ./result with flag '$@'" - nix build --no-warn-dirty --print-build-logs "$fullAttr" $@ -} - -run_test() { - if [[ -z "$@" || "$@" = "help" ]]; then - error "Use 'test ' to run a test" - return 1 - fi - - local case=$1; shift - - if [[ -z "$_EMU" ]]; then - error "Emulator is not set yet" - info2 "Use 'set_emulator ' to set the main testing emulator" - return 1 - fi - - if ! printf "$case" | grep -Eq '\S\.\S'; then - error "Invalid case form '$case', expect '.'" - return 1 - fi - - local hasAttr=$(nix eval --json --no-warn-dirty ".#t1.$_CFG.ip.$_EMU.cases" --apply "cases: cases ? $case") - if [[ "x$hasAttr" = "xfalse" ]]; then - error "Unknown cases $case" - return 1 - fi - - DEBUG=${DEBUG:-0} - caseAttr=".#t1.$_CFG.ip.$_EMU.cases.$case.emu-result" - if (( DEBUG )); then - caseAttr="$caseAttr.debug" - fi - - local args="$@" - if [[ "$_EMU" =~ "vcs-" ]]; then - args="$args --impure" - - if [[ -z "$SNPSLMD_LICENSE_FILE" ]]; then - error "SNPSLMD_LICENSE_FILE not set" - return 1 - fi - - if [[ -z "$VC_STATIC_HOME" ]]; then - error "VC_STATIC_HOME not set" - return 1 - fi - fi - - info "Running test case $case with emulator $_EMU, output result to ./result" - nix build --no-warn-dirty "$caseAttr" $args - ls -l ./result/ -} - -main() { - info "Welcome to T1 dev mode, here are some convenient commands" - info2 "set_config: set current configs" - info2 "list_config: get all available configs" - info2 "get_config: print current config" - info2 "set_emulator: set current emulators" - info2 "list_emulator: get all available emulators" - info2 "get_emulator: print the main testing emulator" - info2 "build: build T1 artifacts with config $_CFG" - info2 "run_test: run test case with specific emulator" - - echo - - if _config_is_set; then - info "Config is set to ${BLUE}$_CFG${ALL_OFF}, use ${GREEN}'set_config '${ALL_OFF} to modify config" - fi - - if _emulator_is_set; then - info "Emulator is set to ${BLUE}${_EMU}${ALL_OFF}, use ${GREEN}'set_emulator '${ALL_OFF} to modify emulator" - fi -} - -main diff --git a/script/emu/src/Main.scala b/script/emu/src/Main.scala index d8b05c1a7..0d54632b9 100644 --- a/script/emu/src/Main.scala +++ b/script/emu/src/Main.scala @@ -258,6 +258,10 @@ object Main: if driverProc.exitCode() != 0 then Logger.fatal("online driver run failed") Logger.info("Driver finished") + + if os.exists(os.pwd / "perf.json") then + os.move(os.pwd / "perf.json", outputPath / "perf.json", replaceExisting = true) + Logger.info(s"Output saved under ${outputPath}") end run diff --git a/t1rocketemu/readme.md b/t1rocketemu/readme.md deleted file mode 100644 index dfd1c0380..000000000 --- a/t1rocketemu/readme.md +++ /dev/null @@ -1,11 +0,0 @@ -## Build - -```bash -nix build ".#t1..ip.difftest" -``` - -## Develop - -```bash -nix develop ".#t1..ip.difftest.devShell" -``` diff --git a/tests/pytorch/README.md b/tests/pytorch/README.md index f6510cc95..4171f9ab1 100644 --- a/tests/pytorch/README.md +++ b/tests/pytorch/README.md @@ -88,6 +88,18 @@ buildBuddyE2ETest { Here you can think `optPhase` as a bash function. Developers can write their own pass in this function. Each `optPhase` should modify the `optArtifacts` array, to indicate our build system about the final output. +Some cases might require `memrefCopy` symbol, you can add this into optPhase too: + +```bash +# ... + echo "Compiling memrefCopy library" + $CXX -nostdlib -c ${../lib/MemrefCopy.cc} -o memrefCopy.o + llcArtifacts+=( + memrefCopy.o + ) +# ... +``` + The `caseName` and `optPhase` attribute is always required. We also offer the below attribute for you to override: