[GRADLE-2454] Interpret tests that throw org.junit.internal.AssumptionViolatedException as "skipped" Created: 29/Aug/12  Updated: 21/Mar/13  Resolved: 24/Jan/13

Status: Resolved
Project: Gradle
Affects Version/s: 1.0, 1.1
Fix Version/s: 1.5-rc-1

Type: Improvement
Reporter: Peter Niederwieser Assignee: Unassigned
Resolution: Fixed Votes: 7


 Description   

Tools like Maven and IntelliJ interpret tests that throw org.junit.internal.AssumptionViolatedException as "skipped", which seems sensible. Gradle currently interprets them as "passed". We should probably change this.



 Comments   
Comment by AleWir [ 16/Jan/13 ]

Hi, Will assumption violation provide a log in report the same way assertion error does?
I think that it would be very useful...

Just for additional info:
We are using assumptions in tests when they are affected by a "known issues" (will not be fixed in a near future because of some business/capacity reasons). We put such assumptions before failing assertions. It would be really great if Gradle Test task treated such cases as ignored tests. We already know those issues and don't want them to break our tests (tests should not fail) but still, we would like to know how many tests are affected by those (tests should not pass either). Yet, the information what values were not matching and/or stacktraces may still be useful in test reports (so such tests should not be treated as class or method level @Ignore which causes them excluded from execution and because of that not having any output).

Comment by AleWir [ 21/Mar/13 ]

It is still not working correctly in gradle-1.5-rc-2.

In XML test report that is created by Gradle, it should tot up the number of skipped tests and place it in 'skipped' attribute of the 'testsuite' tag, according to schema:
<xs:element name="testsuite">
<xs:complexType>
<xs:sequence>
...
</xs:sequence>
<xs:attribute name="tests" type="xs:string" use="required"/>
<xs:attribute name="failures" type="xs:string" use="optional"/>
<xs:attribute name="errors" type="xs:string" use="optional"/>
<xs:attribute name="skipped" type="xs:string" use="optional"/>
...

Otherwise tools like Hudson don't see any skipped tests during reports/charts generation. Therefore the change that was done is unfortunately not yet very useful. See: http://stackoverflow.com/questions/4922867/junit-xml-format-specification-that-hudson-supports

Comment by AleWir [ 21/Mar/13 ]

Moreover, currently ignored tests are marked with weird tag 'ignored-testcase' and again, according to schema, it should be 'skipped' tag inside 'testcase' tag:
<xs:element name="testcase">
<xs:complexType>
<xs:sequence>
<xs:element ref="skipped" minOccurs="0" maxOccurs="1"/>
...
And this is what is actually taken into account by Hudson.

Generated at Wed Jun 30 12:22:21 CDT 2021 using Jira 8.4.2#804003-sha1:d21414fc212e3af190e92c2d2ac41299b89402cf.