From aa3f2fda31d8ec58e3b64187de0b60ad4903c4de Mon Sep 17 00:00:00 2001 From: Shilpa Padgaonkar <77152136+shilpa-padgaonkar@users.noreply.github.com> Date: Mon, 8 Jan 2024 20:01:09 +0100 Subject: [PATCH 1/6] Create API-Testing-Guidelines.md --- documentation/API-Testing-Guidelines.md | 40 +++++++++++++++++++++++++ 1 file changed, 40 insertions(+) create mode 100644 documentation/API-Testing-Guidelines.md diff --git a/documentation/API-Testing-Guidelines.md b/documentation/API-Testing-Guidelines.md new file mode 100644 index 00000000..a168990b --- /dev/null +++ b/documentation/API-Testing-Guidelines.md @@ -0,0 +1,40 @@ +# API Testing Guidelines + +## Table of Contents +1. [Introduction](#introduction) +2. [Test Cases contributions for APIs](#contribution) +3. [Location of feature file](#location) +4. [Best practises and recommendations](#recommendations) +7. [References](#references) + +## Introduction +This document captures guidelines for the API testing in CAMARA project. These guidelines are applicable to every API to be worked out under the CAMARA initiative. + +## Test Cases contribution for APIs +Based on the decision taken in the commonalities working group and as documented in [API-Readiness-Checklist](https://github.com/camaraproject/Commonalities/blob/main/documentation/API-Readiness-Checklist.md), a Gherkin feature file will be added to the main subproject repo and this will fulfil the minimum criteria of readiness w.r.t API test cases and documentation. If subprojects also intend to add test implementations, an aligned single implementation that is agreed amongst all provider implementors could be added to the main subproject repo. If no alignment is possible, each provider implementor will add the test implementation to their own repos. + +## Location of feature file +The feature file will reside under +``` +Subproject_Repository/code/Test_definitions/file-name.feature +``` + +## Best practises and recommendations + +* One feature file per API is advisable so that all scenarios can be covered corresponding to the API & the corresponding resource. +* Third Person pronoun usage in feature file is advisable as using the third person, conveys information in a more official manner. +* It is recommended to only have one When/Then in a feature file as per cucumber official documentation's recommendations. However, in case of complex scenarios, several when/then can be allowed. +* The recommended format for scenario identifier is shown below + ``` + '@'<(mandatory)name of the resource>_<(mandatory)number XX>_<(optional)short detail in lower case and using underscore “_” as the separator>) + for e.g. (@check_simswap_01_verify_swap_true_default_maxage). + ``` + +* Recommendation is to have API literal request value (with example) but in case of exceptions, it is fine to use ready payload as well. + + +## References + +* [One feature file per API](https://www.testquality.com/blog/tpost/v79acjttj1-cucumber-and-gherkin-language-best-pract) +* [Scenario Identifer](https://support.smartbear.com/cucumberstudio/docs/tests/best-practices.html#scenario-content-set-up-writing-standards) +* [One when/then](https://cucumber.io/docs/gherkin/reference/) From 57ef1a2cfb695c78c684e4e7c18226e0972e14f0 Mon Sep 17 00:00:00 2001 From: Shilpa Padgaonkar <77152136+shilpa-padgaonkar@users.noreply.github.com> Date: Tue, 9 Jan 2024 08:02:48 +0100 Subject: [PATCH 2/6] Update API-Testing-Guidelines.md spellcheck --- documentation/API-Testing-Guidelines.md | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/documentation/API-Testing-Guidelines.md b/documentation/API-Testing-Guidelines.md index a168990b..1d739a4b 100644 --- a/documentation/API-Testing-Guidelines.md +++ b/documentation/API-Testing-Guidelines.md @@ -4,14 +4,14 @@ 1. [Introduction](#introduction) 2. [Test Cases contributions for APIs](#contribution) 3. [Location of feature file](#location) -4. [Best practises and recommendations](#recommendations) +4. [Best practices and recommendations](#recommendations) 7. [References](#references) ## Introduction This document captures guidelines for the API testing in CAMARA project. These guidelines are applicable to every API to be worked out under the CAMARA initiative. ## Test Cases contribution for APIs -Based on the decision taken in the commonalities working group and as documented in [API-Readiness-Checklist](https://github.com/camaraproject/Commonalities/blob/main/documentation/API-Readiness-Checklist.md), a Gherkin feature file will be added to the main subproject repo and this will fulfil the minimum criteria of readiness w.r.t API test cases and documentation. If subprojects also intend to add test implementations, an aligned single implementation that is agreed amongst all provider implementors could be added to the main subproject repo. If no alignment is possible, each provider implementor will add the test implementation to their own repos. +Based on the decision taken in the commonalities working group and as documented in [API-Readiness-Checklist](https://github.com/camaraproject/Commonalities/blob/main/documentation/API-Readiness-Checklist.md), a Gherkin feature file will be added to the main subproject repo and this will fullfil the minimum criteria of readiness w.r.t API test cases and documentation. If subprojects also intend to add test implementations, an aligned single implementation that is agreed amongst all provider implementers could be added to the main subproject repo. If no alignment is possible, each provider implementer will add the test implementation to their own repos. ## Location of feature file The feature file will reside under @@ -19,7 +19,7 @@ The feature file will reside under Subproject_Repository/code/Test_definitions/file-name.feature ``` -## Best practises and recommendations +## Best practices and recommendations * One feature file per API is advisable so that all scenarios can be covered corresponding to the API & the corresponding resource. * Third Person pronoun usage in feature file is advisable as using the third person, conveys information in a more official manner. @@ -36,5 +36,5 @@ Subproject_Repository/code/Test_definitions/file-name.feature ## References * [One feature file per API](https://www.testquality.com/blog/tpost/v79acjttj1-cucumber-and-gherkin-language-best-pract) -* [Scenario Identifer](https://support.smartbear.com/cucumberstudio/docs/tests/best-practices.html#scenario-content-set-up-writing-standards) +* [Scenario Identifier](https://support.smartbear.com/cucumberstudio/docs/tests/best-practices.html#scenario-content-set-up-writing-standards) * [One when/then](https://cucumber.io/docs/gherkin/reference/) From 2cbf0da257a8413a1cb870f2bebfb56769cba68b Mon Sep 17 00:00:00 2001 From: Shilpa Padgaonkar <77152136+shilpa-padgaonkar@users.noreply.github.com> Date: Fri, 12 Jan 2024 13:11:36 +0100 Subject: [PATCH 3/6] Update API-Testing-Guidelines.md To fix https://github.com/camaraproject/Commonalities/pull/117#pullrequestreview-1815660576 --- documentation/API-Testing-Guidelines.md | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/documentation/API-Testing-Guidelines.md b/documentation/API-Testing-Guidelines.md index 1d739a4b..950ea294 100644 --- a/documentation/API-Testing-Guidelines.md +++ b/documentation/API-Testing-Guidelines.md @@ -16,7 +16,8 @@ Based on the decision taken in the commonalities working group and as documented ## Location of feature file The feature file will reside under ``` -Subproject_Repository/code/Test_definitions/file-name.feature +Subproject_Repository/code/Test_definitions/file_name.feature +for e.g. https://github.com/camaraproject/QualityOnDemand/blob/main/code/Test_definitions/QoD_API_Test.feature ``` ## Best practices and recommendations From cbfb886beb4fefa924de1cfdbc1253a16c08bcfe Mon Sep 17 00:00:00 2001 From: Shilpa Padgaonkar <77152136+shilpa-padgaonkar@users.noreply.github.com> Date: Fri, 19 Jan 2024 13:26:42 +0100 Subject: [PATCH 4/6] Update documentation/API-Testing-Guidelines.md Co-authored-by: Jose Luis Urien --- documentation/API-Testing-Guidelines.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/documentation/API-Testing-Guidelines.md b/documentation/API-Testing-Guidelines.md index 950ea294..b2b4f1dd 100644 --- a/documentation/API-Testing-Guidelines.md +++ b/documentation/API-Testing-Guidelines.md @@ -11,7 +11,7 @@ This document captures guidelines for the API testing in CAMARA project. These guidelines are applicable to every API to be worked out under the CAMARA initiative. ## Test Cases contribution for APIs -Based on the decision taken in the commonalities working group and as documented in [API-Readiness-Checklist](https://github.com/camaraproject/Commonalities/blob/main/documentation/API-Readiness-Checklist.md), a Gherkin feature file will be added to the main subproject repo and this will fullfil the minimum criteria of readiness w.r.t API test cases and documentation. If subprojects also intend to add test implementations, an aligned single implementation that is agreed amongst all provider implementers could be added to the main subproject repo. If no alignment is possible, each provider implementer will add the test implementation to their own repos. +Based on the decision taken in the Commonalities working group and as documented in [API-Readiness-Checklist](https://github.com/camaraproject/Commonalities/blob/main/documentation/API-Readiness-Checklist.md), at least one Gherkin feature file will be added to the main subproject repo and this will fulfill the minimum criteria of readiness with respect to API test cases and documentation. If subprojects also intend to add test implementations, an aligned single implementation that is agreed among all provider implementors could be added to the main subproject repo. If no alignment is possible, each provider implementer will add the test implementation to their own repos. ## Location of feature file The feature file will reside under From b02627055d3e7e7089f8526f6725ebfcfa3111b4 Mon Sep 17 00:00:00 2001 From: Shilpa Padgaonkar <77152136+shilpa-padgaonkar@users.noreply.github.com> Date: Mon, 22 Jan 2024 17:32:39 +0100 Subject: [PATCH 5/6] Update API-Testing-Guidelines.md To apply https://github.com/camaraproject/Commonalities/pull/117#issuecomment-1904354932 --- documentation/API-Testing-Guidelines.md | 7 +++++-- 1 file changed, 5 insertions(+), 2 deletions(-) diff --git a/documentation/API-Testing-Guidelines.md b/documentation/API-Testing-Guidelines.md index b2b4f1dd..9517b65c 100644 --- a/documentation/API-Testing-Guidelines.md +++ b/documentation/API-Testing-Guidelines.md @@ -22,7 +22,10 @@ for e.g. https://github.com/camaraproject/QualityOnDemand/blob/main/code/Test_de ## Best practices and recommendations -* One feature file per API is advisable so that all scenarios can be covered corresponding to the API & the corresponding resource. +* Granularity of the feature file must be decided at the project level but it is recommended to: + - group in one file all scenarios for one given API capability (that can cover several endpoints). + - provide several files when one CAMARA API (yaml) covers several independent functions that can be provided independently + * Third Person pronoun usage in feature file is advisable as using the third person, conveys information in a more official manner. * It is recommended to only have one When/Then in a feature file as per cucumber official documentation's recommendations. However, in case of complex scenarios, several when/then can be allowed. * The recommended format for scenario identifier is shown below @@ -36,6 +39,6 @@ for e.g. https://github.com/camaraproject/QualityOnDemand/blob/main/code/Test_de ## References -* [One feature file per API](https://www.testquality.com/blog/tpost/v79acjttj1-cucumber-and-gherkin-language-best-pract) +* [Feature files]( https://copyprogramming.com/howto/multiple-feature-inside-single-feature-file#multiple-feature-inside-single-feature-file) * [Scenario Identifier](https://support.smartbear.com/cucumberstudio/docs/tests/best-practices.html#scenario-content-set-up-writing-standards) * [One when/then](https://cucumber.io/docs/gherkin/reference/) From 0e98b460c3db10b972edada35ef94c93aed63501 Mon Sep 17 00:00:00 2001 From: Shilpa Padgaonkar <77152136+shilpa-padgaonkar@users.noreply.github.com> Date: Mon, 22 Jan 2024 19:13:45 +0100 Subject: [PATCH 6/6] Update documentation/API-Testing-Guidelines.md Co-authored-by: Jose Luis Urien --- documentation/API-Testing-Guidelines.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/documentation/API-Testing-Guidelines.md b/documentation/API-Testing-Guidelines.md index 9517b65c..1b032de0 100644 --- a/documentation/API-Testing-Guidelines.md +++ b/documentation/API-Testing-Guidelines.md @@ -23,7 +23,7 @@ for e.g. https://github.com/camaraproject/QualityOnDemand/blob/main/code/Test_de ## Best practices and recommendations * Granularity of the feature file must be decided at the project level but it is recommended to: - - group in one file all scenarios for one given API capability (that can cover several endpoints). + - group in one file all scenarios testing one closely related API capability (that can cover one or several endpoints). - provide several files when one CAMARA API (yaml) covers several independent functions that can be provided independently * Third Person pronoun usage in feature file is advisable as using the third person, conveys information in a more official manner.