[GRADLE-2649] Test report uses suite class name instead of implementation class name for tests run via @RunWith(Suite.class). Created: 21/Jan/13  Updated: 24/Feb/14  Resolved: 29/Jan/13

Status: Resolved
Project: Gradle
Affects Version/s: None
Fix Version/s: 1.5-rc-1

Type: Bug
Reporter: Adam Murdoch Assignee: Unassigned
Resolution: Fixed Votes: 0

Comment by Peter Niederwieser [ 01/Feb/13 ]

I think we should show both the suite and its constituents (i.e. add one additional level). This is how suites are modelled in JUnit, how they are commonly thought of, how they are displayed by IDEs, and AFAIK also how we report them via the test listener.

Comment by Adam Murdoch [ 02/Feb/13 ]

Absolutely. That's the next story.

Comment by Remy de Wolf [ 24/Feb/14 ]

we recently upgraded from gradle 1.4 to 1.9 and along the way found this issue related to the generation of the junit XML reports.

We have organized our tests in multiple sub projects.
Our tests are defined in some Excel documents and are executed with junit using a custom Junit runner (@RunWith).
The custom runner reads excel files and creates instances of junit test classes such as CustomJunitTestCase extends TestCase.

With gradle 1.4, junit XML report file names use suite class names which would create such an output:

  • /subproject1/build/test-results/TEST-Project1Suite.xml: testsuite name="Project1Suite" tests="123"
  • /subproject2/build/test-results/TEST-Project2Suite.xml: testsuite name="Project2Suite" tests="123"

When updating to gradle 1.9 the xml report file name have changed to use implementation class and are not unique anymore:

  • /subproject1/build/test-results/TEST-CustomJunitTestCase.xml: testsuite name="CustomJunitTestCase" tests="123"
  • /subproject2/build/test-results/TEST-CustomJunitTestCase.xml: testsuite name="CustomJunitTestCase" tests="123"

The issue with gradle 1.9 is that the test reports are not unique and it creates some incorrect test reporting on CI servers.

As Peter commented in the Jira, test file names can be made unique by including both the suite and its constituents.
In our case, this is preventing us to upgrade from 1.4 at the moment.

Generated at Wed Jun 30 12:27:40 CDT 2021 using Jira 8.4.2#804003-sha1:d21414fc212e3af190e92c2d2ac41299b89402cf.