Working with generated Allure report for further processing via API (Java)

[mcahornsirup]

Having JUnit5 and Allure2 in place, we generate reports, giving us all the details we need.

Afterwards, we read the generated allure report and sync certain aspects to Xray.

With Allure 2.17.0 the io.qameta.allure.AllureResultsReader was removed.
With Allure 2.17.0 the io.qameta.allure.FileSystemResultsReader was removed.

Any suggestions on how to work with a generated Allure report afterwards?
(We try to reinvent the wheel by implementing our own mapping/pojos)

[baev]

The main reason is that we don't want to have Allure Results Reader as a part of public API as it rarely used by anyone. Besides that, Allure itself supports not only Allure JSON results (known as Allure 2 model) but many others, like Allure XML (from Allure 1), CucumberJSON, XCtest results etc.

I can suggest you a few options:

1. Allure Report has XRay plugin, so you can use that https://github.com/allure-framework/allure2/tree/master/plugins/xray-plugin
2. You can create a custom Allure Report Plugin that will do required API exports during report generation
3. Allure TestOps is a good option that not only provide you realtime reports but also can sync results to other systems like XRAY on the fly

[mcahornsirup]

Thanks for the fast reply. Of course it would be great to define a specific output format for the results like Allure Json (default) and others in a configurable way (like Open Test Reporting). This would make it easy to pick it up by using an existing library if needed.

The problem with JUnit are its limitations e.g. the concept of steps are not supported. This is why Allure adds additional value by providing all of these extra informations to the Allure Results. Especially, when you want to sync not only the overall result, but the steps too in order to see e.g. in Xray which steps failed.

1. The Xray plugin is limited when we look at the e.g the sync options - steps are not synced.
2. From what I have seen, it would be necessarry to develop the plugin, install the plugin, configure the plugin. Which adds extra complexity - especially when trying to do this with maven. It seems not to be solvable by just providing a plugin url (nexus).
3. Another tool ... just for syncing

4.? Looking at the FileSystemResultReader, I am wondering, how stable it is to be used with the latest Allure version. Do you know if it should still work? Maybe I could extract it. But than, the question is if we have to adopt it with every new release of Allure because of a modell change or something like that.

[baev]

The Xray plugin is limited when we look at the e.g the sync options - steps are not synced.

you can send us a PR with the change, I'll be happy to review & merge it

From what I have seen, it would be necessarry to develop the plugin, install the plugin, configure the plugin. Which adds extra complexity - especially when trying to do this with maven. It seems not to be solvable by just providing a plugin url (nexus).

yes, that one of the points highlighted for future Allure versions. To create a more simple way to work with extensions

Another tool ... just for syncing

Nah, you'll like it if you try it. Many of our customers completely migrate from Xray to Allure TestOps after giving it a try

Looking at the FileSystemResultReader, I am wondering, how stable it is to be used

it should be. Just use the following https://github.com/allure-framework/allure2/blob/master/allure-generator/src/main/java/io/qameta/allure/allure2/Allure2Plugin.java#L92-L100 Jackson configuration and that's it

[mcahornsirup]

I looked into the implementation and we extracted the removed classes for the moment. With the usage of jackson, the deserialization is in fact reliable and straight forward.

I am going to ask our TM and maybe we can provide a PR with the missing functionallity. Having everything within the official allure xray plugin would be great. Of course, if this is going to be available to the public it needs to be polished and documented 😃 Therefore, the PR will take some time.

Never the less, thanks a lot for your very helpfull support. I'll keep you updated on the progress!