[GRADLE-1554] Allow build to continue on test failure, yet still end with a failed status Created: 16/May/11 Updated: 10/Feb/17 Resolved: 10/Feb/17 |
|
Status: | Resolved |
Project: | Gradle |
Affects Version/s: | 1.0-milestone-3 |
Fix Version/s: | None |
Type: | Improvement | ||
Reporter: | Luke Daley | Assignee: | Unassigned |
Resolution: | Won't Fix | Votes: | 13 |
Description |
The "ignoreFailures" flag on the test task produces the desired behaviour by having the build continue on test failure, but the build finished with a success status which is confusing. This feature is needed (at least) for long builds in a CI environment. Typically, developers want the CI build to run all the tests to get the most accurate feedback, but don't want the build to be considered successful if any of the tests failed. |
Comments |
Comment by Steve Saliman [ 24/Jul/12 ] |
While it is not as clean as deferring failure for later, There is a workaround for this issue. The details can be found at http://forums.gradle.org/gradle/topics/can_you_set_a_property_to_defer_test_failure |
Comment by Per Lambæk [ 06/Jan/14 ] |
Any plan on prioritizing this? To me it seems like a pretty basic feature. |
Comment by Luke Daley [ 06/Jan/14 ] |
Does --continue not do what you want? |
Comment by Per Lambæk [ 06/Jan/14 ] |
Ah, that looks exactly right, my bad, but maybe being able to configure this, especially on test tasks, would make sense, instead of using a switch that enables it for all tasks. But as a workaround it should work, thanks. |
Comment by Luke Daley [ 06/Jan/14 ] |
Per Lambæk I agree, there's still room for improvement here. |
Comment by Mathias Birkkjær Nielsen [ 09/Jan/14 ] |
We have the same problem on our pipeline. We have set ignoreFailures and made jenkins detect failures. This works fine, but then when developers run the tests locally it doesn't fail. The --continue solution seems like a bit of a hack for a pretty common situation. If all task failures are ignored, then it is harder to detect what is actually wrong (because failures ripples down to the next tasks). Multiple failed tests are perfectly fine when tests are independent (one test failure doesn't result in another test failure). |
Comment by Luke Daley [ 09/Jan/14 ] |
Can you elaborate on how --continue is not what you want? If you want to make this implicitly on always, add this to your build script: gradle.startParameter.continueOnFailure = true |
Comment by Mathias Birkkjær Nielsen [ 10/Jan/14 ] |
Well i'm afraid I did not read the docs careful enough. Still it could be nice as a test task option continueOnFailure, making it more of a feature specifically for tests, not your entire build. |
Comment by Benjamin Muschko [ 15/Nov/16 ] |
As announced on the Gradle blog we are planning to completely migrate issues from JIRA to GitHub. We intend to prioritize issues that are actionable and impactful while working more closely with the community. Many of our JIRA issues are inactionable or irrelevant. We would like to request your help to ensure we can appropriately prioritize JIRA issues you’ve contributed to. Please confirm that you still advocate for your JIRA issue before December 10th, 2016 by:
We look forward to collaborating with you more closely on GitHub. Thank you for your contribution to Gradle! |
Comment by Benjamin Muschko [ 10/Feb/17 ] |
Thanks again for reporting this issue. We haven't heard back from you after our inquiry from November 15th. We are closing this issue now. Please create an issue on GitHub if you still feel passionate about getting it resolved. |