TestNG Practitest integration implementation #4

vdemkiv · 2018-01-22T14:41:07Z

No description provided.

stask · 2018-01-31T09:28:20Z

api.v2/java.v2/src/main/java/com/practitest/api/example/PractiTestAPI.java

    public static Response sendGetSteps(String id)
    {
-        return RequestFactory.doGet("com/v2/projects/4650/steps.json?test-ids=" +id);
+        return RequestFactory.doGet("com/v2/projects/"+projectID+"/steps.json?test-ids=" +id);


Why the URL starts with 'com/'?

stask · 2018-01-31T09:32:16Z

api.v2/java.v2/src/main/java/com/listeners/TestNGListenerForPractiTest.java

+                Integer setID = PractiTestWriter.createNewSet(testIDs);
+                //Store SetID for further usage
+                System.setProperty("currentSetId", setID.toString());
+                PractiTestWriter.createNewSet(testIDs);


Not sure i understand the logic here. Are you going to create a new TestSet every time 'mvn test' runs?

As I understand TestSet this is collection of tests which should be executed.
For example if you specify certain test for execution you'll need new TestSet in Practitest to store results for it. Correct me if I'm wrong

You right, TestSet is a collection of Tests. But subsequent executions are supposed to reuse existing TestSet.
Meaning that when you run your project first time, we should create the Tests and TestSet. When you run the project next time, we should use the TestSet created during first run.

Most of the time automated and manual tests are grouped to Smoke and Regression test suites or by some other project specific definition.
This means we should have multiple TestSets for them. Including test automation process into consideration we will have only some part automated at the start and number of automated != manual test cases. From this perspective it seam to me logical to create new TestSet each time as collection of tests most of the time will be different.

I can add validation to reuse test set if collection of tests already exist for this group. Please let me know if this will work for you.

Yes, I think we should reuse testsets. If tests are added or removed, we should change the existing testset.

removed com from end point path.

vdemkiv · 2018-02-25T10:53:14Z

Please check latest changes.

stask · 2018-03-13T22:04:18Z

api.v2/java.v2/src/main/java/com/practitest/api/example/PractiTestWriter.java

+     */
+    public static Integer getSetID()
+    {
+        Response response = PractiTestAPI.sendGetTestSet();


The PractiTestAPI.sendGetTestSet() returns all testsets in the given project, right? You need to use name_exact parameter to return specific testset (see https://www.practitest.com/api-v2/#get-all-testsets-in-your-project)

stask · 2018-03-13T22:04:56Z

api.v2/java.v2/src/main/java/com/practitest/api/example/PractiTestWriter.java

    public static Integer createNewInstance(Integer setID, Integer testID)
    {
        return PractiTestAPI.sendCreateInstance(setID, testID).getBody().jsonPath().get("data.id");
    }

+    public static Integer createNewInstance(Integer setID, List<Integer> testID)
+    {
+        return PractiTestAPI.sendCreateInstance(setID, testID.get(0)).getBody().jsonPath().get("data.id");


Aren't you creating instance just for first test here?

stask · 2018-03-13T22:05:54Z

api.v2/java.v2/src/main/java/com/listeners/TestNGListenerForPractiTest.java

@@ -10,17 +11,21 @@

 public class TestNGListenerForPractiTest implements ITestListener {

+        protected Integer instanceID = null;


Why just one instanceID? There should be one instance for every test in the test set

stask · 2018-03-13T22:07:40Z

api.v2/java.v2/src/main/java/com/listeners/TestNGListenerForPractiTest.java

+                }
+                else
+                {
+                        Log.info("Using existing TestSEtID: "+existingTestID.toString());


If the testset exists (and going to be reused), you still have to synchronize the test in that testset. For example, a new test might be added, or some test might be removed. You should retrieve the current list of instances for the existing testset, add/remove instances as needed, and then create the runs and upload results

stask · 2018-03-21T13:31:57Z

Trying to run it (mvn test) and got few issues. Did you commit everything?

The issues so far:

seems like some properties are taken from src/main/resources/project.properties while other should be specified via -D (-DPROJECT_ID=2 for instance)
NullPointerExceptions

Can you please tell me how to run it properly?

vdemkiv · 2018-03-21T13:35:12Z

it's populated with fake data and can not be executed at this stage. Integration with test project will be done as part of second assignment.

stask · 2018-03-21T13:58:44Z

I will wait till it's runnable then

stask · 2018-05-17T12:33:24Z

api.v2/java.v2/src/main/java/com/Config/GeneralConfig.java

+
+
+
+    public static String getConfigurationValue(final String configurationName) {


Why does it need to load the properties file every time you want to read one property?

stask · 2018-05-17T12:34:15Z

api.v2/java.v2/src/main/java/com/Config/GeneralConfig.java

+    }
+
+    public static boolean setConfigurationValue(final String configurationName, final String value) {
+        return m_props.setProperty(configurationName, value) != null;


What's the purpose of this method? It doesnt actually change anything because every time you call getConfigurationValue, you re-read the properties file from disk

might be used in the future if we need to modify some config parameters during test execution

stask · 2018-05-17T12:38:55Z

api.v2/java.v2/src/main/java/com/listeners/TestNGListenerForPractiTest.java

+                        //Remove existing test IDs from List to get removed
+                        currentTestSetTestIDs.removeAll(testIDs);
+                        //Remove Instance IDs
+                        for (Integer currentTestSetTestID : currentTestSetTestIDs) {


This is not good because when instance is removed, the information about executions of these tests in this testset is removed too. Which affects statistics and reports.

You should add/remove tests that are actually added or removed.

added comments with proper description how it's implemented.

stask · 2018-05-17T12:43:41Z

api.v2/java.v2/src/main/java/com/listeners/TestNGListenerForPractiTest.java

+        @Override
+        public void onTestFailure(ITestResult result) {
+                List<String> instanceID = PractiTestWriter.getInstancesByTestIDAndTestSetID(result.getMethod().getDescription(), this.setID);
+                PractiTestWriter.submitResults(instanceID.get(0), 1);


We need to send more information (how it failed, what was the exception) than just status code to PractiTest.

Added throwable message to test result

Added description for add/remove tests from Test Set Failed test will contain throwable message.

TestNG Practitest integration implementation

344af08

stask requested changes Jan 31, 2018

View reviewed changes

vdemkiv and others added 2 commits February 7, 2018 18:56

Update PractiTestAPI.java

71ab180

removed com from end point path.

Existing Test Set iD usage implementation

13cd1c6

stask reviewed Mar 19, 2018

View reviewed changes

vdemkiv added 2 commits March 20, 2018 04:34

implementation based on code review comments from Stas.

baebba4

missed file

acf1de9

vdemkiv added 3 commits March 27, 2018 16:49

fix all dependencies for project to be executable

10ccf9a

fixes for set creating. execution by groups

dafa0fc

validation fix

789ce45

stask requested changes May 17, 2018

View reviewed changes

vdemkiv added 2 commits May 23, 2018 00:33

Implemented proper usage for config load

39f2a7a

Added description for add/remove tests from Test Set Failed test will contain throwable message.

updated readme.

25642e2

stask approved these changes Jul 15, 2018

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

TestNG Practitest integration implementation #4

TestNG Practitest integration implementation #4

vdemkiv commented Jan 22, 2018

stask Jan 31, 2018

stask Jan 31, 2018

vdemkiv Feb 7, 2018

stask Feb 7, 2018

vdemkiv Feb 8, 2018

vdemkiv Feb 12, 2018

stask Feb 12, 2018

vdemkiv commented Feb 25, 2018

stask Mar 13, 2018

stask Mar 13, 2018

stask Mar 13, 2018

stask Mar 13, 2018

stask commented Mar 21, 2018

vdemkiv commented Mar 21, 2018

stask commented Mar 21, 2018

stask May 17, 2018

stask May 17, 2018

vdemkiv May 22, 2018

stask May 17, 2018

vdemkiv May 22, 2018

stask May 17, 2018

vdemkiv May 22, 2018

		@@ -10,17 +11,21 @@

		public class TestNGListenerForPractiTest implements ITestListener {

		protected Integer instanceID = null;




		public static String getConfigurationValue(final String configurationName) {

TestNG Practitest integration implementation #4

Are you sure you want to change the base?

TestNG Practitest integration implementation #4

Conversation

vdemkiv commented Jan 22, 2018

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

vdemkiv commented Feb 25, 2018

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

stask commented Mar 21, 2018

vdemkiv commented Mar 21, 2018

stask commented Mar 21, 2018

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment