-
-
Notifications
You must be signed in to change notification settings - Fork 3.4k
Unit Tests
In ILSpy we use four different kinds of unit tests:
- Correctness Tests
- Pretty Tests
- IL-Pretty Tests
- Roundtrip Tests
It is important for each language feature to have at least correctness tests. Pretty tests are preferred, because they do test the code output of the decompiler as well, but sometimes it is not possible to create a working pretty test. If that's the case, try to create correctness tests instead. See below for more details.
Each test is run in four different configurations:
- Debug + Legacy CSC
- Optimize + Legacy CSC
- Debug + Roslyn CSC
- Optimize + Roslyn CSC
Of course, language features that are not supported by the legacy C# compiler should only be tested using a Roslyn configuration.
The test runner is /ICSharpCode.Decompiler.Tests/CorrectnessTestRunner.cs. The test cases are located in /ICSharpCode.Decompiler.Tests/TestCases/Correctness/. Each test case consists of either a cs or il file. One test consists of the following steps:
- compile/assemble a test case (call the result "executable 1")
- decompile "executable 1" to C# ("decompiled.cs")
- compile decompiled.cs, resulting in "executable 2"
- run both executable and compare their output (exit code, stdout, stderr)
We repeat the steps above with each test configuration.
The tests pass if the code compiles without error and produces the same output. The tests do not care at all if the resulting code is pretty, or if any high-level constructs like closures were detected.
When adding new tests, please make sure that you only check in the test case in CorrectnessTestRunner.cs and the test case files (i.e. source files) in the TestCases/Correctness directory. Any executables that are generated in the test run must not be added. Please note further that the executables are compiled to the TestCases directory, so you can open them in ILSpy for debugging and testing purposes more easily.
The test runner is /ICSharpCode.Decompiler.Tests/PrettyTestRunner.cs. The test cases are located in /ICSharpCode.Decompiler.Tests/TestCases/Pretty/. Each test case consists of a cs file and a set of IL files, one for each test configuration. The cs file and the generated IL files are to be checked into the repository.
One test consists of the following steps:
- If no IL file is present for the given configuration, generate it.
- Assemble the generated IL file for the configuration.
- Decompile the generated assembly.
- Compare the source code generated by the decompiler with the original source code and output a diff, if there are any differences.
These steps are repeated with each test configuration.
The test passes, if the output generatd by the decompiler matches the code in the cs file. Comments, preprocessor directives and whitespace lines are ignored by the diffing algorithm.
TODO
TODO