Add out of the box Allure support

[a-simeshin]

Introduction

Hi there!

I propose to add an improvement in the form of out-of-the-box support for allure-java functionality and allure-report generation.
At the moment, any project that contains a large number of unit and integration tests needs a reporting system for a clear display of execution results and troubleshooting.
Allure is one of the most popular systems for this purpose.
However, connecting Allure reports requires a fairly large number of configurations at the project builder level, which takes time and requires a bit of knowledge how surefire and javaagent works.
By the way Allure integrates perfectly with Spring-boot application projects.

Motivation

Spring-boot is objectively the best framework project for building integration tests projects that are separated from the application code, for example for rust/с++ projects. For my region this is the most common practice, because it is much easier to find a QA engineer in java rather than in the original application stack.

Also gaining popularity in the fintech sector is the practice of shift-left for Spring-boot applications, when high-level integration tests aimed at testing builded application with full Application Context and infrastructure in TestContainers. These tests are also usually writes as a module in the Spring-boot application project.

Allure is currently the nobrainer for integrate with test management systems and displaying reports on the tests you've run, so why not integrate both projects?

Allure-java in the nutshell

Allure-java is an java ecosystem for preparing data in Allure-report data model that displays all the interesting information about the tests:

• Test name
• Test description
• Test execution time with support of concurrent test execution
• Test status: passed, failed, broken, etc
• Test exception with full stack trace
• Steps hierarchy
• Steps attachments (requests, responses, images, screenshots, json, xml, etc)
• Any additional test information, that dev described in the test code with annotations or allure-java sdk

Allure-java is regularly updated, and is primarily maintained by @baev and community.
The project contains excellent documentation - https://allurereport.org/docs/

Finally Allure-java integrates well by modular structure with components that are already in the spring-boot project:

• Junit5
• TestNG
• Hamcrest
• AssertJ
• Awaitility
• Aspectj
• HttpClient, RestClient and RestTemplate
  etc

Allure-report in the nutshell

First of all, Allure-report is a data model describing the life cycle of a test.
Docs about data model for tests - https://allurereport.org/docs/how-it-works-test-result-file/
Also, the documentation of other sections contains descriptions of data models for other lifecycle objects.

A separate maven/gradle plugin is used to generate rich html from Allure-report data and start a local server.
https://github.com/allure-framework/allure-maven
https://github.com/allure-framework/allure-gradle

What I propose to add in the details

Add allure-bom to spring-boot-dependencies in dependencyManagement node

        <dependency>
            <groupId>io.qameta.allure</groupId>
            <artifactId>allure-bom</artifactId>
            <version>${allure.version}</version>
            <type>pom</type>
            <scope>import</scope>
        </dependency>

Add allure.version property version to spring-boot-dependencies

<properties>
    <allure.version>2.29.0</allure.version>
</properties>

Add allure-junit5 as dependency to the spring-boot-starter-test

<dependency>
    <groupId>io.qameta.allure</groupId>
    <artifactId>allure-junit5</artifactId>
</dependency>

Add maven-surefire-plugin configuration to the spring-boot-starter-parent, because allure-java using aspectj for working with annotations.

<plugin>
    <groupId>org.apache.maven.plugins</groupId>
    <artifactId>maven-surefire-plugin</artifactId>
    <configuration>
        <argLine>
            -javaagent:"${settings.localRepository}/org/aspectj/aspectjweaver/${aspectj.version}/aspectjweaver-${aspectj.version}.jar"
        </argLine>
	<systemPropertyVariables>
		<allure.results.directory>target/allure-results</allure.results.directory>
	</systemPropertyVariables>
    </configuration>
    <dependencies>
        <dependency>
            <groupId>org.aspectj</groupId>
            <artifactId>aspectjweaver</artifactId>
            <version>${aspectj.version}</version>
        </dependency>
    </dependencies>
</plugin>

Add allure-maven plugin to spring-boot-dependencies in pluginManagement node

<plugin>
	<groupId>io.qameta.allure</groupId>
	<artifactId>allure-maven</artifactId>
	<version>${allure.version}</version>
</plugin>

Add allure-maven plugin configuration to the spring-boot-starter-parent, because sometimes you want to generate reports locally for sure in case of troubleshooting

<plugin>
	<groupId>io.qameta.allure</groupId>
	<artifactId>allure-maven</artifactId>
	<configuration>
		<reportVersion>${allure.version}</reportVersion>
	</configuration>
</plugin>

Profit

A lot of configurations are provided out of the box, which saves time on project setup. Spring-boot projects get detailed and beautiful Allure-reports, which help to understand what the hell wrong with unit and integration tests.

[wilkinsona]

Thanks for the suggestion, but I consider this to be out of scope for Spring Boot. I also don't think we can safely add AspectJ's weaver to Surefire's classpath for every project. For these reasons, I think it's better that using Allure remains opt-in by users making the necessary configuration changes themselves.

[a-simeshin]

Thanks for the suggestion, but I consider this to be out of scope for Spring Boot.

Can you please tell me in more detail why Allure is outside the scope of the Spring-boot project? In my opinion Spring-boot provides a convenient configurable ecosystem for building applications, part of which is the testing stage and analysis of problems as a result of the testing. If I'm totally wrong, why not to start to provide this feature?

I also don't think we can safely add AspectJ's weaver to Surefire's classpath for every project.

True, but it can be done by providing an additional maven profile with Surefire configuration disabled by default or as a part of spring-boot-initializer template.

[wilkinsona]

Reporting of test results isn't something that Spring Boot deals with. We consider that role to be fulfilled by the test framework (typically JUnit) and the build tool (typically Maven or Gradle). There are also other tools in this space such as Develocity that can analyse tests and their trends for which we also don't provide any out-of-the-box support.