Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Enabling Surefire to run test classes and test methods in any order specified by a runOrder #7

Open
wants to merge 1 commit into
base: umaster
Choose a base branch
from

Conversation

winglam
Copy link

@winglam winglam commented Apr 15, 2021

The changes in this pull request enable Surefire to run test classes and test methods in any order specified by a newly added <runOrder> (testorder). Namely, the changes include

  • A new <runOrder> called testorder that would make Surefire run test classes and test methods in the order specified by <test>. Regex is supported as is the case for other runOrders
  • A new feature so that Surefire will attempt to parse the content of a file path for a list of tests to include or exclude if the argument specified by <test> is a file path (each line is assumed to be a test to include or exclude)
  • Tests to TestListResolverTest and RunOrderCalculatorTest for the newly added features

The newly added testorder <runOrder> is supported for all JUnit 3 and JUnit 4 tests. I will plan on supporting JUnit 5 tests after these changes are accepted.

Simple (no regex) example using Dubbo:

mvn test -Dsurefire.runOrder=testorder \
-Dtest=org.apache.dubbo.rpc.protocol.dubbo.DubboLazyConnectTest#testSticky2,\
org.apache.dubbo.rpc.protocol.dubbo.DubboLazyConnectTest#testSticky1,\
org.apache.dubbo.rpc.protocol.dubbo.DubboLazyConnectTest#testSticky3,\
org.apache.dubbo.rpc.protocol.dubbo.DubboProtocolTest#testNonSerializedParameter\
 -pl dubbo-rpc/dubbo-rpc-dubbo

Output from Maven should say that the test class DubboLazyConnectTest ran before DubboProtocolTest

The file DubboLazyConnectTest.xml (dubbo-rpc/dubbo-rpc-dubbo/target/surefire-reports/TEST-org.apache.dubbo.rpc.protocol.dubbo.DubboLazyConnectTest.xml) should say

<testcase name="testSticky2"...>
<testcase name="testSticky1"...>
<testcase name="testSticky3...>

Regex example using Dubbo:

mvn test -Dsurefire.runOrder=testorder \
-Dtest=org.apache.dubbo.rpc.protocol.dubbo.DubboLazyConnectTest*,\
org.apache.dubbo.rpc.protocol.dubbo.DubboProtocolTest#testDubboProtocol*\
 -pl dubbo-rpc/dubbo-rpc-dubbo

Output from Maven should say

Tests run: 4 … in org.apache.dubbo.rpc.protocol.dubbo.DubboLazyConnectTest
Tests run: 3 … in org.apache.dubbo.rpc.protocol.dubbo.DubboProtocolTest

The file DubboProtocolTest.xml (dubbo-rpc/dubbo-rpc-dubbo/target/surefire-reports/TEST-org.apache.dubbo.rpc.protocol.dubbo.DubboProtocolTest.xml) should say

<testcase name=”testDubboProtocol”...>
<testcase name="testDubboProtocolWithMina"...>
<testcase name="testDubboProtocolMultiService”...>

Include/Exclude example using Dubbo:

mvn test -Dsurefire.runOrder=testorder -Dtest=org.apache.dubbo.rpc.protocol.dubbo.DubboLazyConnectTest#test*,\!org.apache.dubbo.rpc.protocol.dubbo.DubboLazyConnectTest#testSticky2  -pl dubbo-rpc/dubbo-rpc-dubbo

Output from Maven should say
Tests run: 3 … in org.apache.dubbo.rpc.protocol.dubbo.DubboLazyConnectTest

The file DubboLazyConnectTest.xml (dubbo-rpc/dubbo-rpc-dubbo/target/surefire-reports/TEST-org.apache.dubbo.rpc.protocol.dubbo.DubboLazyConnectTest.xml) should say

<testcase name="testSticky1"...>
<testcase name="testSticky3"...>
<testcase name="testSticky4...>

@waterlooyh
Copy link

Look Good

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants