From 37d6f4973c685ab65921a83a6118b379e843bbf8 Mon Sep 17 00:00:00 2001 From: Madalin Ilie Date: Mon, 6 Nov 2023 18:54:56 +0200 Subject: [PATCH] Correct some details which were outdated --- docs/getting-started/installation.md | 6 +----- docs/getting-started/interpreting-results.md | 3 ++- docs/getting-started/slicing-strategies.md | 4 ++-- 3 files changed, 5 insertions(+), 8 deletions(-) diff --git a/docs/getting-started/installation.md b/docs/getting-started/installation.md index c4b51aa1c..de8c660c6 100644 --- a/docs/getting-started/installation.md +++ b/docs/getting-started/installation.md @@ -52,7 +52,7 @@ You can build CATS from sources on you local box. You need [Java 17+](https://sd :::caution Before running the first build Before running the first build, please make sure you do a `./mvnw clean`. CATS uses a fork ok [OKHttp](https://square.github.io/okhttp/) which will install locally -under the `4.10.0-CATS` version, so don't worry about overriding the official versions. +under the `4.11.0-CATS` version, so don't worry about overriding the official versions. ::: You can use the following Maven command to build the project as an uberjar: @@ -66,10 +66,6 @@ You can also build native images using a [GraalVM Java version](https://www.graa `./mvnw package -Pnative` -:::info -You will need to configure Maven with a [Github PAT](https://docs.github.com/en/free-pro-team@latest/packages/guides/configuring-apache-maven-for-use-with-github-packages) with `read-packages` scope to get some dependencies for the build. -::: - ### Notes on Unit Tests You may see some `error` log messages while running the Unit Tests. Those are expected behaviour for testing the negative scenarios of the Fuzzers. diff --git a/docs/getting-started/interpreting-results.md b/docs/getting-started/interpreting-results.md index 29a7a7242..02d5c8a3a 100644 --- a/docs/getting-started/interpreting-results.md +++ b/docs/getting-started/interpreting-results.md @@ -41,7 +41,8 @@ Understanding the `Result Reason` values: - `Unexpected Response Code` - reported as an `error`; this indicates a possible bug in the service - the response code is documented, but is not expected for this scenario - `Unexpected Behaviour` - reported as an `error`; this indicates a possible bug in the service - the response code is neither documented nor expected for this scenario - `Not Found` - reported as an `error` in order to force providing more context; this indicates that CATS needs additional business context in order to run successfully - you can do this using the `--refData` and/or `--urlParams` arguments - +- `Response time exceeds max` - reported as an `error` if the `--maxResponseTimeInMs` is supplied and the response time exceeds this number +- `Not Implemented` - reported as `warn` if response code is `501` This is what you get when you click on a specific test: diff --git a/docs/getting-started/slicing-strategies.md b/docs/getting-started/slicing-strategies.md index 629b10c90..7d94cd400 100644 --- a/docs/getting-started/slicing-strategies.md +++ b/docs/getting-started/slicing-strategies.md @@ -5,7 +5,7 @@ description: How to get meaningful results in a timely manner # Slicing Strategies -CATS has a significant number of Fuzzers. Currently, **100+** and growing. Some Fuzzers are executing multiple tests for every given field within the request. +CATS has more than **110** fuzzers at the moment. Some Fuzzers are executing multiple tests for every given field within the request. For example the `ControlCharsOnlyInFieldsFuzzer` has **63** control chars values that will be tried for each request field. If a request has 15 fields, this will result in **945 tests**. Considering that there are additional Fuzzers with the same magnitude, you can easily get to 20k tests being executed on a typical run. This will result in huge reports and long-running times (i.e. minutes, rather than seconds). @@ -13,7 +13,7 @@ This will result in huge reports and long-running times (i.e. minutes, rather th Below are some recommended strategies on how you can separate the tests in chunks which can be executed as stages in a deployment pipeline, one after the other. :::caution -Running CATS with **all** Fuzzers will produce a significant amount of logging. +Running CATS with **all** Fuzzers and `--verbosity=detailed` (or without verbosity, before CATS 10.x) will produce a significant amount of logging. Please make sure you have a purging strategy in place, especially when choosing to store the output in files. Additionally, you can control the logging level using the `--log` argument. :::