Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[PHEE-534] Parallel execution of feature file #258

Open
wants to merge 48 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
48 commits
Select commit Hold shift + click to select a range
c5640da
[PHEE-534] enable parallel execution
dhruvsonagara Mar 3, 2024
be2b3eb
Updated the test runner class to run with multi threads
dhruvsonagara Mar 4, 2024
1be9614
Reverted the workflow
dhruvsonagara Mar 4, 2024
330f7f3
Resolved checkstyle errors
dhruvsonagara Mar 4, 2024
2168042
updated the docker image
dhruvsonagara Mar 4, 2024
f9d959a
removed the unwanted paralellism from the circle CI config
dhruvsonagara Mar 4, 2024
fb2524f
Updating the result path for multi threads
dhruvsonagara Mar 5, 2024
50f1934
Testing the new path and updated the readme with new changes
dhruvsonagara Mar 6, 2024
3f12650
Testing with new path for results
dhruvsonagara Mar 6, 2024
49c84f9
updated the path with courgette
dhruvsonagara Mar 6, 2024
86ec308
instead of running feature as run level running scenarios
dhruvsonagara Mar 6, 2024
a714de5
adding the new plugins for the run
dhruvsonagara Mar 6, 2024
a8e5608
updated the test results
dhruvsonagara Mar 6, 2024
d3bc990
Updated the path and fetching report with info
dhruvsonagara Mar 6, 2024
8c0be04
Trying to host html for test results
dhruvsonagara Mar 6, 2024
bd85c97
Trying to host html for test results
dhruvsonagara Mar 6, 2024
8637586
Updated the results path and artifacts
dhruvsonagara Mar 6, 2024
4b73b6f
Removed the unnescessary sleep for the AMS test cases
dhruvsonagara Mar 6, 2024
4ba683a
Reverted the unnescessary changes
dhruvsonagara Mar 6, 2024
3242921
Updated the run leavel from runner
dhruvsonagara Mar 8, 2024
4fd7b62
Updated the threads and run
dhruvsonagara Mar 8, 2024
6e6fca9
Testing with 3 threads
dhruvsonagara Mar 8, 2024
d7bc3ae
Testing with 4 threads
dhruvsonagara Mar 8, 2024
73cf1d8
Updated the thread to 3
dhruvsonagara Mar 25, 2024
2404a16
dynamic port allocation
apurbraj Apr 1, 2024
f07af57
checkstyle
apurbraj Apr 1, 2024
a295bcf
log port
apurbraj Apr 1, 2024
4e34403
fix
apurbraj Apr 1, 2024
c2a1ff1
annotate
apurbraj Apr 3, 2024
cd19383
fix
apurbraj Apr 3, 2024
79c9ad4
test
apurbraj Apr 3, 2024
36a6bf3
puv
apurbraj Apr 3, 2024
772632a
test
apurbraj Apr 3, 2024
0797984
ps
apurbraj Apr 3, 2024
9e59c41
comment
apurbraj Apr 3, 2024
924dfa0
Test
apurbraj Apr 4, 2024
1b6f1ce
fix
apurbraj Apr 4, 2024
13bdeec
cleanup
apurbraj Apr 4, 2024
a7fe8a3
revert
apurbraj Apr 5, 2024
1f30ce1
comment
apurbraj Apr 5, 2024
6918135
singleton
apurbraj Apr 14, 2024
28920d5
style
apurbraj Apr 14, 2024
d9b33e3
chexk
apurbraj Apr 14, 2024
4e25f99
chexk
apurbraj Apr 14, 2024
5b7b080
change
apurbraj Apr 15, 2024
81ff7bd
changes
apurbraj Apr 15, 2024
aa6734c
changes
apurbraj Apr 15, 2024
4042474
tag
apurbraj Apr 15, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 6 additions & 6 deletions .circleci/config.yml
Original file line number Diff line number Diff line change
Expand Up @@ -156,7 +156,7 @@ jobs:
NAMESPACE: paymenthub
steps:
- setup_remote_docker:
version: 20.10.14
version: 20.10.24
- kubernetes/install-kubectl
- run: git clone https://github.com/openmf/ph-ee-env-labs
- aws-eks/update-kubeconfig-with-authenticator:
Expand Down Expand Up @@ -231,9 +231,9 @@ jobs:
echo -n "Test execution is completed, kill ngrok"
pkill ngrok
- store_test_results:
path: build/test-results/test/TEST-org.mifos.integrationtest.TestRunner.xml
path: build/test-results/test/TEST-courgette.api.junit.Courgette.xml
- store_artifacts:
path: build/test-results
path: build/courgette-report
test-chart-ams:
docker:
- image: cimg/openjdk:17.0.0
Expand Down Expand Up @@ -264,13 +264,13 @@ jobs:
export CALLBACK_URL="https://$NGROK_PUBLIC_URL"
echo -n "Public url ."
echo $CALLBACK_URL
./gradlew test -Dcucumber.filter.tags="@amsIntegration"
./gradlew test -Dcucumber.tags="@amsIntegration"
echo -n "Test execution is completed, kill ngrok"
pkill ngrok
- store_test_results:
path: build/test-results/test/TEST-org.mifos.integrationtest.TestRunner.xml
path: build/test-results/test/TEST-courgette.api.junit.Courgette.xml
- store_artifacts:
path: build/test-results
path: build/courgette-report
workflows:
deploy:
jobs:
Expand Down
64 changes: 26 additions & 38 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -73,54 +73,42 @@ Now we can run respective feature file directly form the intellij.

## Adding runner configuration
Below java class will make sure to run cucumber test using JUnit test command.
Where the `glue` property is for defining the package which contains the step definitions, `feature` refers to the path where feature file is located and `plugin` is for providing different plugin configuration supported by cucumber.
Where the `glue` property is for defining the package which contains the step definitions and `plugin` is for providing different plugin configuration supported by cucumber.
```java
@RunWith(Cucumber.class)
@CucumberOptions(
features = {"src/test/java/resources"},
glue = {"org.mifos.integrationtest.cucumber"},
plugin = {
"html:cucumber-report",
"json:cucumber.json",
"pretty",
"html:build/cucumber-report.html",
"json:build/cucumber-report.json"

@RunWith(Courgette.class)
@CourgetteOptions(threads = 3, runLevel = CourgetteRunLevel.FEATURE, rerunFailedScenarios = false,
testOutput = CourgetteTestOutput.CONSOLE,

reportTitle = "Courgette-JVM Example",
reportTargetDir = "build",
environmentInfo = "browser=chrome; git_branch=master",
cucumberOptions = @CucumberOptions(features = "src/test/java/resources",
glue = "org.mifos.integrationtest.cucumber",
tags = "@gov",
publish = true,
plugin = {
"pretty", "json:build/cucumber-report/cucumber.json",
"html:build/cucumber-report/cucumber.html",
"junit:build/cucumber-report/cucumber.xml"
}))

## Adding gradle configuration
Adding gradle configuration will allow us to run all the cucumber feature file at using using a CLI.
Adding gradle configuration will allow us to run all the cucumber feature file at using a CLI.

```gradle
configurations {
cucumberRuntime {
extendsFrom testImplementation
}
tasks.withType(Test) {
systemProperties = System.getProperties()
}

task cucumberCli() {
dependsOn assemble, testClasses
doLast {
javaexec {
main = "io.cucumber.core.cli.Main"
classpath = configurations.cucumberRuntime + sourceSets.main.output + sourceSets.test.output
args = [
'--plugin', 'pretty',
'--plugin', 'html:target/cucumber-report.html',
'--glue', 'org.mifos.connector.slcb.cucumber',
'src/test/java/resources']
}
)
public class TestRunner {
}
```
Adding below configuration will allow us to wire the CLI arguments be passed in the actual runner configuration while running the cucumber test using JUnit.
```groovy
test {
systemProperty "cucumber.filter.tags", System.getProperty("cucumber.filter.tags")
task parallelRun(type: Test) {
include '**/TestRunner.class'
outputs.upToDateWhen { false }
}
```
## Running an integration test
Use below command to execute the integration test.
```shell
./gradlew test -Dcucumber.filter.tags="<cucumber tag>"
./gradlew test -Dcucumber.tags="<cucumber tag>"
```
Where `<cucumber tag>` has to be replaced with valid tag, for example if you are willing to run test cases related to g2p scenario then pass the tag `@gov`. If `-Dcucumber.filter.tags` flag is omitted then all the test cases would be triggered independent of the tag.
```shell
Expand Down
50 changes: 10 additions & 40 deletions build.gradle
Original file line number Diff line number Diff line change
Expand Up @@ -171,50 +171,20 @@ dependencies {
implementation 'com.opencsv:opencsv:5.5.2'
testImplementation 'org.apache.commons:commons-csv:1.5'
testImplementation 'org.awaitility:awaitility:4.2.0'
implementation 'io.github.prashant-ramcharan:courgette-jvm:6.12.0'
}

tasks.named('test') {
useJUnitPlatform()
}
//tasks.named('test') {
// useJUnitPlatform()
//}


configurations {
cucumberRuntime {
extendsFrom testImplementation
}
testlogger {
// pick a theme - mocha, standard, plain, mocha-parallel, standard-parallel or plain-parallel
theme 'mocha'
showSkipped false
showStackTraces true
}
}

test {
systemProperty "cucumber.filter.tags", System.getProperty("cucumber.filter.tags")
systemProperty "cucumber.filter.name", System.getProperty("cucumber.filter.name")
tasks.withType(Test) {
systemProperties = System.getProperties()
}
task cucumberCli() {
dependsOn assemble, testClasses
doLast {
javaexec {
main = "io.cucumber.core.cli.Main"
classpath = configurations.cucumberRuntime + sourceSets.main.output + sourceSets.test.output
args = [
'--plugin',
'html:cucumber-report',
'--plugin',
'json:cucumber.json',
'--plugin',
'pretty',
'--plugin',
'html:build/cucumber-report.html',
'--plugin',
'json:build/cucumber-report.json',
'--glue',
'org.mifos.integrationtest.cucumber',
'src/test/java/resources'
]
}
}

task parallelRun(type: Test) {
include '**/TestRunner.class'
outputs.upToDateWhen { false }
}
39 changes: 32 additions & 7 deletions src/test/java/org/mifos/integrationtest/TestRunner.java
Original file line number Diff line number Diff line change
@@ -1,11 +1,36 @@
package org.mifos.integrationtest;

import io.cucumber.junit.Cucumber;
import io.cucumber.junit.CucumberOptions;
import courgette.api.CourgetteAfterAll;
import courgette.api.CourgetteBeforeAll;
import courgette.api.CourgetteOptions;
import courgette.api.CourgetteRunLevel;
import courgette.api.CourgetteTestOutput;
import courgette.api.CucumberOptions;
import courgette.api.junit.Courgette;
import org.junit.runner.RunWith;
import org.mifos.integrationtest.config.WireMockServerSingleton;

@RunWith(Cucumber.class)
@CucumberOptions(features = { "src/test/java/resources" }, glue = { "org.mifos.integrationtest.cucumber" }, plugin = {
"html:cucumber-report", "json:cucumber.json", "pretty", "html:build/cucumber-report.html", "json:build/cucumber-report.json",
"junit:build/cucumber.xml" })
public class TestRunner {}
//@RunWith(Cucumber.class)
//@CucumberOptions(features = { "src/test/java/resources" }, glue = { "org.mifos.integrationtest.cucumber" }, plugin = {
// "html:cucumber-report", "json:cucumber.json", "pretty", "html:build/cucumber-report.html", "json:build/cucumber-report.json",
// "junit:build/cucumber.xml" })

@RunWith(Courgette.class)
@CourgetteOptions(threads = 3, runLevel = CourgetteRunLevel.FEATURE, rerunFailedScenarios = false, testOutput = CourgetteTestOutput.CONSOLE, generateCourgetteRunLog = true,

reportTitle = "Paymenthub Test results", reportTargetDir = "build", cucumberOptions = @CucumberOptions(features = "src/test/java/resources", glue = "org.mifos.integrationtest.cucumber", tags = "@gov", publish = true, plugin = {
"html:cucumber-report", "json:cucumber.json", "pretty", "html:build/cucumber-report.html",
"json:build/cucumber-report.json", "junit:build/cucumber.xml" }))
@SuppressWarnings({ "FinalClass", "HideUtilityClassConstructor" })
public class TestRunner {

@CourgetteBeforeAll
public static void setupWireMockServer() {
WireMockServerSingleton.getInstance(); // Start WireMock server
}

@CourgetteAfterAll
public static void stopWireMockServer() {
WireMockServerSingleton.getInstance().stop(); // Stop WireMock server
}
}
Original file line number Diff line number Diff line change
Expand Up @@ -11,11 +11,11 @@
@Slf4j
public class MockServerConfig implements MockServer {

private static WireMockServer singleInstance = null;

@Value("${mock-server.port}")
private int port;

private static WireMockServer singleInstance = null;

public WireMockServer getMockServer() {
if (MockServerConfig.singleInstance == null) {
MockServerConfig.singleInstance = new WireMockServer(wireMockConfig().port(this.port));
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,60 @@
package org.mifos.integrationtest.config;

import com.github.tomakehurst.wiremock.WireMockServer;
import com.github.tomakehurst.wiremock.common.FatalStartupException;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.stereotype.Component;

@Component
@SuppressWarnings({ "FinalClass", "HideUtilityClassConstructor" })
public class WireMockServerSingleton {

static Logger logger = LoggerFactory.getLogger(WireMockServerSingleton.class);
private static final ThreadLocal<WireMockServer> threadLocalInstance = new ThreadLocal<>();

public static WireMockServer getInstance() {
WireMockServer instance = threadLocalInstance.get();
if (instance == null || !instance.isRunning()) {
synchronized (WireMockServerSingleton.class) {
instance = threadLocalInstance.get(); // Double-check idiom
if (instance == null || !instance.isRunning()) {
instance = startWireMockServerWithRetry(3); // Retry 3 times
threadLocalInstance.set(instance);
}
}
}
return instance;
}

private static WireMockServer startWireMockServerWithRetry(int maxRetries) {
WireMockServer server = null;
for (int attempt = 1; attempt <= maxRetries; attempt++) {
int port = getRandomPort();
server = new WireMockServer(port);
try {
server.start();
logger.info("WireMock started on port {}", server.port());
return server;
} catch (FatalStartupException e) {
logger.error("Failed to start WireMock on port {} (Attempt {}/{}). Retrying...", port, attempt, maxRetries, e);
// Optionally, add a short delay here if needed
}
}
throw new IllegalStateException("Failed to start WireMock server after " + maxRetries + " attempts.");
}

private static int getRandomPort() {
// This returns a port number in the range 1024-65535
return 1024 + (int) (Math.random() * ((65535 - 1024) + 1));
}

public static int getPort() {
WireMockServer instance = threadLocalInstance.get();
if (instance != null && instance.isRunning()) {
return instance.port();
} else {
throw new IllegalStateException("WireMock server is not running.");
}
}
}
Original file line number Diff line number Diff line change
Expand Up @@ -22,10 +22,10 @@
import org.mifos.integrationtest.config.ChannelConnectorConfig;
import org.mifos.integrationtest.config.FspConfig;
import org.mifos.integrationtest.config.IdentityMapperConfig;
import org.mifos.integrationtest.config.MockServer;
import org.mifos.integrationtest.config.OperationsAppConfig;
import org.mifos.integrationtest.config.TenantConfig;
import org.mifos.integrationtest.config.VoucherManagementConfig;
import org.mifos.integrationtest.config.WireMockServerSingleton;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.beans.factory.annotation.Autowired;
Expand All @@ -46,9 +46,6 @@ public class BaseStepDef {
@Autowired
ChannelConnectorConfig channelConnectorConfig;

@Autowired
MockServer mockServer;

@Autowired
IdentityMapperConfig identityMapperConfig;

Expand Down Expand Up @@ -77,6 +74,7 @@ public class BaseStepDef {

@Autowired
ScenarioScopeState scenarioScopeState;
int port = WireMockServerSingleton.getPort();

Logger logger = LoggerFactory.getLogger(this.getClass());
protected static BatchRequestDTO batchRequestDTO;
Expand Down
Original file line number Diff line number Diff line change
@@ -1,6 +1,5 @@
package org.mifos.integrationtest.cucumber.stepdef;

import static com.github.tomakehurst.wiremock.client.WireMock.getAllServeEvents;
import static com.google.common.truth.Truth.assertThat;
import static java.util.concurrent.TimeUnit.SECONDS;
import static org.awaitility.Awaitility.await;
Expand Down Expand Up @@ -60,6 +59,7 @@
import org.mifos.integrationtest.config.BulkProcessorConfig;
import org.mifos.integrationtest.config.ChannelConnectorConfig;
import org.mifos.integrationtest.config.MockPaymentSchemaConfig;
import org.mifos.integrationtest.config.WireMockServerSingleton;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.beans.factory.annotation.Value;
import org.springframework.core.env.Environment;
Expand Down Expand Up @@ -667,7 +667,7 @@ public void iShouldAssertTotalTxnCountAndSuccessfulTxnCountInPaymentBatchDetailR
@Then("I should be able to extract response body from callback for batch")
public void iShouldBeAbleToExtractResponseBodyFromCallbackForBatch() {
boolean flag = false;
List<ServeEvent> allServeEvents = getAllServeEvents();
List<ServeEvent> allServeEvents = WireMockServerSingleton.getInstance().getAllServeEvents();
for (int i = allServeEvents.size() - 1; i >= 0; i--) {
ServeEvent request = allServeEvents.get(i);
if (!(request.getRequest().getBodyAsString()).isEmpty()) {
Expand Down
Loading