Skip to content

Latest commit

 

History

History
164 lines (110 loc) · 6.13 KB

unix-test-instructions.md

File metadata and controls

164 lines (110 loc) · 6.13 KB
Raw

Building and running tests on Linux, macOS, and FreeBSD

CoreCLR tests

Building

Build CoreCLR on Unix.

Building the Tests

Dotnet CLI is required to build the tests. This can be done on any platform then copied over if the architecture or OS does not support Dotnet.

To build the tests on Unix:

./src/tests/build.sh

By default, the test build uses Release as the libraries configuration. To use a different configuration, set the LibrariesConfiguration property to the desired configuration. For example:

./src/tests/build.sh /p:LibrariesConfiguration=Debug

Please note that this builds the Priority 0 tests. To build priority 1:

./src/tests/build.sh -priority1

Generating Core_Root

The src/tests/build.sh script generates the Core_Root folder, which contains the test host (corerun), libraries, and coreclr product binaries necessary to run a test. To generate Core_Root without building the tests:

./src/tests/build.sh generatelayoutonly

The output will be at <repo_root>/artifacts/tests/coreclr/<os>.<arch>.<configuration>/Tests/Core_Root.

Building Test Subsets

The src/tests/build.sh script supports three options that let you limit the set of tests to build; when none of these is specified, the entire test tree (under src/tests) gets built but that can be lengthy (especially in -priority1 mode) and unnecessary when working on a particular test.

  1. -test:<test-project> - build a particular test project specified by its project file path, either absolute or relative to src/tests. The option can be specified multiple times on the command line to request building several individual projects; alternatively, a single occurrence of the option can represent several project paths separated by semicolons.

Example: src/tests/build.sh -test:JIT/Methodical/divrem/div/i4div_cs_do.csproj;JIT/Methodical/divrem/div/i8div_cs_do.csproj

  1. -dir:<test-folder> - build all test projects within a given directory path, either absolute or relative to src/tests. The option can be specified multiple times on the command line to request building projects in several folders; alternatively, a single occurrence of this option can represent several project folders separated by semicolons.

Example: src/tests/build.sh -dir:JIT/Methodical/Arrays/huge;JIT/Methodical/divrem/div

  1. -tree:<root-folder> - build all test projects within the subtree specified by its root path, either absolute or relative to src/tests. The option can be specified multiple times on the command line to request building projects in several subtrees; alternatively, a single instance of the option can represent several project subtree root folder paths separated by semicolons.

Example: src/tests/build.sh -tree:baseservices/exceptions;JIT/Methodical

Please note that priority filtering is orthogonal to specifying test subsets; even when you request building a particular test and the test is Pri1, you need to specify -priority1 in the command line, otherwise the test build will get skipped. While somewhat unintuitive, 'fixing' this ad hoc would likely create more harm than good and we're hoping to ultimately get rid of the test priorities in the long term anyway.

Building Individual Tests

During development there are many instances where building an individual test is fast and necessary. All of the necessary tools to build are under coreclr. It is possible to use ~/runtime/dotnet.sh msbuild as you would normally use MSBuild with a few caveats.

!! Note !! -- Passing /p:TargetOS=[osx|linux] is required.

Building an Individual Test

./dotnet.sh msbuild src/tests/path-to-proj-file /p:TargetOS=<TargetOS> /p:Configuration=<BuildType>

In addition to the test assembly, this will generate a .sh script next to the test assembly in the test's output folder. The test's output folder will be under <repo_root>/artifacts/tests/coreclr/<os>.<arch>.<configuration> at a subpath based on the test's location in source.

Running Tests

The following instructions assume that on the Unix machine:

  • The CoreCLR repo is cloned at /mnt/coreclr

src/tests/build.sh will have set up the Core_Root directory correctly after the test build.

./src/tests/run.sh x64 checked

Please use the following command for help.

./src/tests/run.sh -h

Unsupported and temporarily disabled tests

To support building all tests for all targets on single target, we use the conditional property

<CLRTestTargetUnsupported Condition="...">true</CLRTestTargetUnsupported>

This property disables building of a test in a default build. It also disables running a test in the bash/batch wrapper scripts. It allows the test to be built on any target in CI when the allTargets option is passed to the build.* scripts.

Tests which never should be built or run are marked

<DisableProjectBuild>true</DisableProjectBuild>

This propoerty should not be conditioned on Target properties to allow all tests to be built for allTargets.

Running Individual Tests

After building an individual test, to run the test:

  1. Set the CORE_ROOT environment variable to the Core_Root folder.

  2. Run the test using the .sh generated for the test.

PAL tests

Build CoreCLR with PAL tests on the Unix machine:

./build.sh clr.paltests

Run tests:

To run all tests including disabled tests

./src/coreclr/pal/tests/palsuite/runpaltests.sh $(pwd)/artifacts/bin/coreclr/$(uname).x64.Debug/paltests
# on macOS, replace $(uname) with osx

To only run enabled tests for the platform the tests were built for:

artifacts/bin/coreclr/$(uname).x64.Debug/paltests/runpaltests.sh $(pwd)/artifacts/bin/coreclr/$(uname).x64.Debug/paltests
# on macOS, replace $(uname) with osx

To run only specific tests, edit paltestlist.txt locally to delete the ones you don't want to run.

Test results will go into: /tmp/PalTestOutput/default/pal_tests.xml

To disable tests in the CI edit src/coreclr/pal/tests/palsuite/issues.targets