[GRADLE-2579] SourceTask is always up-to-date after all source files are deleted Created: 30/Nov/12  Updated: 05/Oct/16  Resolved: 05/Oct/16

Status: Resolved
Project: Gradle
Affects Version/s: 1.3
Fix Version/s: 3.2-rc-1

Type: Bug
Reporter: Gradle Forums Assignee: Lóránt Pintér
Resolution: Fixed Votes: 21

Issue Links:
Duplicate
Duplicated by GRADLE-2531 Changes to buildSrc directory are not... Resolved
Duplicated by GRADLE-2440 JavaCompile task does not remove stal... Resolved

 Description   
    1. Problem

Gradle up-to-date check does not take into account deleted files. This has been confirmed for Gradle 1.3 with 'java' and 'groovy' plugins.

    1. Steps to reproduce:

1. Create a build.gradle containing apply plugin: 'java'
2. Create the directories src/main/java
3. Create a file src/main/java/Foo.java containing public class Foo {}
4. Execute gradle build - compileJava will be executed.
5. Execute gradle build - compileJava will be UP-TO-DATE.
6. Delete the file src/main/java/Foo.java.
7. Execute gradle build - compileJava will still be UP-TO-DATE.

    1. Result
      Because of this, the deleted class will still be available in both build/classes/main and the created jar in build/lib.


 Comments   
Comment by Gradle Forums [ 30/Nov/12 ]

In your scenario, you delete all source files. The problem is that gradle marks the compile task as up-to-date as soon as sources is empty. If this is a problem in your build this workaround might help:

if(sourceSets.main.allJava.empty){
compileJava.dependsOn cleanCompileJava
}

Comment by Gradle Forums [ 30/Nov/12 ]

But isn't this a quite evil bug nevertheless?

If compileJava was previously involved in the creation of classes and/or jar then those tasks need to be considered changed if compileJava won't be executed anymore.

We had the situation that we renamed the whole src/main/java folder to src/main/groovy. We also did some refactoring that resulted in the deletion of some classes. But all previously compiled classes were still kept in both the classes and lib resulting in utter chaos. The local build was still working (until performing a mavenesque manual 'clean') but our CI server that always performs a clean build was puking because one of the deleted classes was still used in a test..

I think that an empty directory shouldn't be considered UP-TO-DATE but something else, like e.g. SKIPPED, thus resulting in a refresh of dependent tasks.

Comment by Gradle Forums [ 30/Nov/12 ]

Hey Huxi,

It's defnitely a bug you're running into. The workaround is just ... well a workaround . We'll export this thread here to our jira.

Comment by Joern Huxhorn [ 26/Feb/13 ]

Any news on this issue?

We get bitten by this issue quite regularly, for example while switching tests from JUnit (/src/test/java) to Spockframework (/src/test/groovy)

Comment by Sebastien Tardif [ 17/Apr/14 ]

My team lost couple of hours this week due to this. I have flashback of the documentations describing how cool and flexible is Gradle, you can define your own logic for incremental build...

Comment by Joern Huxhorn [ 17/Apr/14 ]

Yep, it's quite annoying.
It undermines the trust in the built artifacts and is the reason why we always perform a gradle clean before a release (vs. snapshot) build. Very mavenesque.

Comment by Joern Huxhorn [ 21/Jul/14 ]

Wouldn't this be fixable by simply removing the SkipEmptySourceFilesTaskExecuter from the executer chain returned by org.gradle.internal.service.scopes.TaskExecutionServices.createTaskExecuter? Either that or SkipEmptySourceFilesTaskExecuter would need to check if the task has been previously executed at all. Would it make sense to add a org.gradle.cache.PersistentIndexedCache.contains method to find out if a TaskHistory exists at all without actually loading it, i.e. cheaper than get?

I understand that skipping a task without inputs early is a nice optimization but right now it does so at the expense of correctness.

Comment by Joern Huxhorn [ 05/May/15 ]

This will be a showstopper for Gradle 3.0 since it sabotages a 100% reliable cache, right?
(According to http://www.infoq.com/presentations/future-gradle-javascript )

Comment by Joern Huxhorn [ 06/Jul/16 ]

Every few month, I decide to take a dive into the Gradle source code to see if I can find a way to fix this elegantly.

This time around, I realized that the current behavior is actually documented in the public API so fixing the problem isn't just a patch but rather a political decision.

So instead of trying to fix the problem itself I did the next best thing and wrote an integration test for the issue:

StaleOutputTest.groovy
package org.gradle.integtests

import org.gradle.integtests.fixtures.AbstractIntegrationSpec
import spock.lang.Issue

class StaleOutputTest extends AbstractIntegrationSpec {
    @Issue(['GRADLE-2440', 'GRADLE-2579'])
    def 'Stale output is removed after input source directory is emptied.'() {
        setup: 'a minimal java build'
        buildScript("apply plugin: 'java'")
        def fooJavaFile = file('src/main/java/Foo.java') << 'public class Foo {}'
        def fooClassFile = file('build/classes/main/Foo.class')

        when: 'a build is executed'
        succeeds('clean', 'build')

        then: 'class file exists as expected'
        fooClassFile.exists()

        when: 'only java source file is deleted, leaving empty input source directory'
        fooJavaFile.delete()

        and: 'another build is executed'
        succeeds('build')

        then: 'source file was actually deleted'
        !fooClassFile.exists()
    }
}

This test confirmed my suspicion that simply removing SkipEmptySourceFilesTaskExecuter in the TaskExecutionServices.createTaskExecuter method wouldn't magically fix this problem.

Doing so causes a java.lang.IllegalStateException: no source files in com.sun.tools.javac.main.Main - and this is just the special case of java compilation. Who knows which other tools might strictly require actual input files instead of defaulting to NOP in that case?

Skipping tasks without input source files is indeed the correct behavior and getting rid of previously created output simply hasn't been adressed so far for this special use case.

Tasks usually remove stale files on their own if they realize it's necessary (e.g. StaleClassCleaner) but I'm not aware of a general way to tell a task that it should just get rid of any previous output instead of executing its originally intended functionality.

Talking about skipping...

I'd suggest to change the line state.upToDate(); in SkipEmptySourceFilesTaskExecuter to state.skipped("SKIPPED"); regardless of any other fixes. Because that's what's happening.

Well... not what I hoped for... but I hope this helps at least a little bit in getting this fixed before the Gradle 3.0 release.

Comment by Joern Huxhorn [ 07/Jul/16 ]

I think I fixed this issue. Pull request

Only change in behavior beside fixing the original issue is that a task without source files is now reporting SKIPPED instead of UP-TO-DATE.

Please also take a look at this comment in SkipEmptySourceFilesTaskExecuter.

I was unsure if this type of cleanup would justify a different skipped message, e.g. CLEANED, but decided to leave it out for now, using SKIPPED as usual.

Comment by Joern Huxhorn [ 20/Jul/16 ]

Please consider taking a look at the pull request before going 3.0-rc.

Comment by Lóránt Pintér [ 05/Oct/16 ]

Now tasks clean up their previous output after all their sources are gone.

Generated at Wed Jun 30 12:25:46 CDT 2021 using Jira 8.4.2#804003-sha1:d21414fc212e3af190e92c2d2ac41299b89402cf.