Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

a Pure LLVM Lit test framework #3596

Merged
merged 3 commits into from
Jan 20, 2024
Merged

a Pure LLVM Lit test framework #3596

merged 3 commits into from
Jan 20, 2024

Conversation

sequencer
Copy link
Member

@sequencer sequencer commented Oct 21, 2023

I wanna use #3596 to subsume #3410, see this simple test:

//> RUN: scala-cli $THIS | firtool -format=fir
import chisel3._
import circt.stage.ChiselStage
class EmptyModule extends Module
println(ChiselStage.emitCHIRRTL(new EmptyModule))

In the build system, it will be processed into:

//>using scala "2.13.12"
//> using jars /home/sequencer/.cache/coursier/v1/https/repo1.maven.org/maven2/org/scala-lang/scala-library/2.13.12/scala-library-2.13.12.jar /home/sequencer/.cache/coursier/v1/https/repo1.maven.org/maven2/org/scala-lang/scala-reflect/2.13.12/scala-reflect-2.13.12.jar /home/sequencer/.cache/coursier/v1/https/repo1.maven.org/maven2/com/lihaoyi/os-lib_2.13/0.9.1/os-lib_2.13-0.9.1.jar /home/sequencer/.cache/coursier/v1/https/repo1.maven.org/maven2/com/lihaoyi/upickle_2.13/3.1.0/upickle_2.13-3.1.0.jar /home/sequencer/.cache/coursier/v1/https/repo1.maven.org/maven2/org/json4s/json4s-native_2.13/4.0.6/json4s-native_2.13-4.0.6.jar /home/sequencer/.cache/coursier/v1/https/repo1.maven.org/maven2/io/github/alexarchambault/data-class_2.13/0.2.5/data-class_2.13-0.2.5.jar /home/sequencer/.cache/coursier/v1/https/repo1.maven.org/maven2/org/apache/commons/commons-text/1.10.0/commons-text-1.10.0.jar /home/sequencer/.cache/coursier/v1/https/repo1.maven.org/maven2/com/github/scopt/scopt_2.13/3.7.1/scopt_2.13-3.7.1.jar /home/sequencer/.cache/coursier/v1/https/repo1.maven.org/maven2/com/lihaoyi/geny_2.13/1.0.0/geny_2.13-1.0.0.jar /home/sequencer/.cache/coursier/v1/https/repo1.maven.org/maven2/com/lihaoyi/ujson_2.13/3.1.0/ujson_2.13-3.1.0.jar /home/sequencer/.cache/coursier/v1/https/repo1.maven.org/maven2/com/lihaoyi/upack_2.13/3.1.0/upack_2.13-3.1.0.jar /home/sequencer/.cache/coursier/v1/https/repo1.maven.org/maven2/com/lihaoyi/upickle-implicits_2.13/3.1.0/upickle-implicits_2.13-3.1.0.jar /home/sequencer/.cache/coursier/v1/https/repo1.maven.org/maven2/org/json4s/json4s-core_2.13/4.0.6/json4s-core_2.13-4.0.6.jar /home/sequencer/.cache/coursier/v1/https/repo1.maven.org/maven2/org/json4s/json4s-native-core_2.13/4.0.6/json4s-native-core_2.13-4.0.6.jar /home/sequencer/.cache/coursier/v1/https/repo1.maven.org/maven2/org/apache/commons/commons-lang3/3.12.0/commons-lang3-3.12.0.jar /home/sequencer/.cache/coursier/v1/https/repo1.maven.org/maven2/com/lihaoyi/upickle-core_2.13/3.1.0/upickle-core_2.13-3.1.0.jar /home/sequencer/.cache/coursier/v1/https/repo1.maven.org/maven2/org/json4s/json4s-ast_2.13/4.0.6/json4s-ast_2.13-4.0.6.jar /home/sequencer/.cache/coursier/v1/https/repo1.maven.org/maven2/org/json4s/json4s-scalap_2.13/4.0.6/json4s-scalap_2.13-4.0.6.jar /home/sequencer/.cache/coursier/v1/https/repo1.maven.org/maven2/com/thoughtworks/paranamer/paranamer/2.8/paranamer-2.8.jar /home/sequencer/projects/chisel/compile-resources /home/sequencer/projects/chisel/src/main/resources /home/sequencer/projects/chisel/out/chisel/2.13.12/compile.dest/classes /home/sequencer/projects/chisel/svsim/compile-resources /home/sequencer/projects/chisel/svsim/src/main/resources /home/sequencer/projects/chisel/out/svsim/2.13.12/compile.dest/classes /home/sequencer/projects/chisel/firrtl/compile-resources /home/sequencer/projects/chisel/firrtl/src/main/resources /home/sequencer/projects/chisel/out/firrtl/2.13.12/compile.dest/classes /home/sequencer/projects/chisel/macros/compile-resources /home/sequencer/projects/chisel/macros/src/main/resources /home/sequencer/projects/chisel/out/macros/2.13.12/compile.dest/classes /home/sequencer/projects/chisel/core/compile-resources /home/sequencer/projects/chisel/core/src/main/resources /home/sequencer/projects/chisel/out/core/2.13.12/compile.dest/classes /home/sequencer/projects/chisel/binder/compile-resources /home/sequencer/projects/chisel/binder/src/main/resources /home/sequencer/projects/chisel/out/circtpanamabinder/2.13.12/compile.dest/classes /home/sequencer/projects/chisel/filecheck/utility/compile-resources /home/sequencer/projects/chisel/filecheck/utility/resources /home/sequencer/projects/chisel/out/filecheckutility/2.13.12/compile.dest/classes

import chisel3._
import circt.stage.ChiselStage
class EmptyModule extends Module
println(ChiselStage.emitCHIRRTL(new EmptyModule))

I know this is magic...
But I think this maybe the simplest way to add test and being able to migrate most of tests to this framework to share test for firtool and binding frameworks.

Farewell ScalaTest.

One possible test speed downgrade is for each RUN, it will spwan and JVM, but we have no other way to avoid this. but one of the benefit is we can control each javac and scalac option in a fine-grand way, providing a better way to increase the coverage and flexibility.

I'm thinking using LLVM LIT to replace mill execution, but not quite sure if this is a good approach, since mill can provide a prefect run cache(we can cache out folder in the future)

Contributor Checklist

  • Did you add Scaladoc to every public function/method?
  • Did you add at least one test demonstrating the PR?
  • Did you delete any extraneous printlns/debugging code?
  • Did you specify the type of improvement?
  • Did you add appropriate documentation in docs/src?
  • Did you request a desired merge strategy?
  • Did you add text to be included in the Release Notes for this change?

Type of Improvement

  • Feature (or new API)
  • API modification
  • API deprecation
  • Backend code generation
  • Performance improvement
  • Bugfix
  • Documentation or website-related
  • Dependency update
  • Internal or build-related (includes code refactoring/cleanup)

Desired Merge Strategy

  • Squash: The PR will be squashed and merged (choose this if you have no preference).
  • Rebase: You will rebase the PR onto master and it will be merged with a merge commit.

Release Notes

Reviewer Checklist (only modified by reviewer)

  • Did you add the appropriate labels? (Select the most appropriate one based on the "Type of Improvement")
  • Did you mark the proper milestone (Bug fix: 3.5.x, 3.6.x, or 5.x depending on impact, API modification or big change: 6.0)?
  • Did you review?
  • Did you check whether all relevant Contributor checkboxes have been checked?
  • Did you do one of the following when ready to merge:
    • Squash: You/ the contributor Enable auto-merge (squash), clean up the commit message, and label with Please Merge.
    • Merge: Ensure that contributor has cleaned up their commit history, then merge with Create a merge commit.

@sequencer sequencer added the Internal Internal change, does not affect users, will be included in release notes label Oct 21, 2023
@sequencer sequencer force-pushed the scala-cli_filecheck branch 3 times, most recently from 71c560d to a7ddc41 Compare October 22, 2023 06:14
@sequencer sequencer changed the title a Pure FileCheck test framework a Pure LLVM Lit test framework Oct 22, 2023
@sequencer sequencer force-pushed the scala-cli_filecheck branch from 27fba4b to 8ead846 Compare October 24, 2023 09:18
@sequencer sequencer mentioned this pull request Oct 24, 2023
14 tasks
@sequencer sequencer force-pushed the scala-cli_filecheck branch 3 times, most recently from 48898b8 to e677b52 Compare October 26, 2023 04:12
@sequencer sequencer force-pushed the scala-cli_filecheck branch from e677b52 to b6a7128 Compare November 9, 2023 13:02
@sequencer sequencer force-pushed the scala-cli_filecheck branch 2 times, most recently from 399523f to 88322c8 Compare December 6, 2023 08:19
@sequencer
Copy link
Member Author

This PR is blocked by the scala-cli bug: VirtusLab/scala-cli#2621

@sequencer
Copy link
Member Author

sequencer commented Jan 20, 2024

After NixOS/nixpkgs#281556 merged, I can remove the vended nix/scala-cli.nix.
This PR finally rocks!!!

❯ mill "lit[2.13.12].run"
[345/345] lit[2.13.12].run
[lit] -- Testing: 2 tests, 2 workers --
[lit] PASS: CHISEL :: CompilerPlugin/WithCompilerPlugin.sc (1 of 2)
[lit] PASS: CHISEL :: CompilerPlugin/WithoutCompilerPlugin.sc (2 of 2)
[lit]
[lit] Testing Time: 6.99s
[lit]   Passed: 2

I'm going to slip this PR into the main branch and skip review, since it only affect the test code. The panama binding framework will use this as its test infrastructure and CI can use https://github.com/jackkoenig/FileCheck in the future.

@sequencer sequencer force-pushed the scala-cli_filecheck branch 4 times, most recently from 5752893 to 31a346f Compare January 20, 2024 12:47
@sequencer sequencer force-pushed the scala-cli_filecheck branch from 31a346f to 63a8013 Compare January 20, 2024 12:53
@sequencer sequencer enabled auto-merge January 20, 2024 12:56
@sequencer sequencer disabled auto-merge January 20, 2024 12:56
@sequencer sequencer merged commit acb3066 into main Jan 20, 2024
14 checks passed
@sequencer sequencer deleted the scala-cli_filecheck branch January 20, 2024 13:18
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Internal Internal change, does not affect users, will be included in release notes
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant