[GRADLE-1843] Crash on dangling symlink: "Could not list contents of X" Created: 19/Oct/11  Updated: 08/Feb/17  Resolved: 08/Feb/17

Status: Resolved
Project: Gradle
Affects Version/s: 1.0-milestone-3
Fix Version/s: None

Type: Bug
Reporter: Greg Ward Assignee: Unassigned
Resolution: Duplicate Votes: 21

Attachments: File dangling-symlink.tar.gz    

 Description   

If there is a dangling symlink (symlink that points to a non-existent filename) in the source tree, Gradle crashes.

Example: here is my source tree:

$ find . ( -type f -o -type l ) -ls
542043 0 lrwxrwxrwx 1 gward gward 10 Oct 19 10:36 ./src/main/java/Goodbye.java -> Hello.java
542042 0 lrwxrwxrwx 1 gward gward 4 Oct 19 10:35 ./src/main/java/oops.java -> oops
542044 4 rw-rw--- 1 gward gward 74 Oct 19 10:34 ./src/main/java/Hello.java
542045 4 rw-rw--- 1 gward gward 21 Oct 19 10:34 ./build.gradle

and my build script:

$ cat build.gradle
apply plugin: 'java'

When I try to build, here's what happens:

$ gradle --stacktrace build
:compileJava

FAILURE: Build failed with an exception.

  • What went wrong:
    Could not list contents of '/tmp/dangling-symlink/src/main/java/oops.java'.
  • Try:
    Run with --info or --debug option to get more log output.
  • Exception is:
    org.gradle.api.GradleException: Could not list contents of '/tmp/dangling-symlink/src/main/java/oops.java'.
    at org.gradle.api.internal.file.collections.DirectoryFileTree.walkDir(DirectoryFileTree.java:141)
    at org.gradle.api.internal.file.collections.DirectoryFileTree.walkDir(DirectoryFileTree.java:166)
    at org.gradle.api.internal.file.collections.DirectoryFileTree.visit(DirectoryFileTree.java:119)
    at org.gradle.api.internal.file.collections.FileTreeAdapter.visit(FileTreeAdapter.java:96)
    at org.gradle.api.internal.file.AbstractFileTree.getFiles(AbstractFileTree.java:37)
    at org.gradle.api.internal.file.CompositeFileCollection.getFiles(CompositeFileCollection.java:39)
    at org.gradle.api.internal.file.AbstractFileCollection.iterator(AbstractFileCollection.java:59)
    at org.gradle.api.internal.changedetection.DefaultFileSnapshotter.snapshot(DefaultFileSnapshotter.java:42)
    at org.gradle.api.internal.changedetection.InputFilesChangedUpToDateRule.create(InputFilesChangedUpToDateRule.java:35)
    at org.gradle.api.internal.changedetection.CompositeUpToDateRule.create(CompositeUpToDateRule.java:35)
    at org.gradle.api.internal.changedetection.DefaultTaskArtifactStateRepository$HistoricExecution.calcCurrentState(DefaultTaskArtifactStateRepository.java:106)
    at org.gradle.api.internal.changedetection.DefaultTaskArtifactStateRepository$HistoricExecution.isUpToDate(DefaultTaskArtifactStateRepository.java:114)
    at org.gradle.api.internal.changedetection.DefaultTaskArtifactStateRepository$TaskArtifactStateImpl.isUpToDate(DefaultTaskArtifactStateRepository.java:154)
    at org.gradle.api.internal.changedetection.ShortCircuitTaskArtifactStateRepository$1.isUpToDate(ShortCircuitTaskArtifactStateRepository.java:35)
    at org.gradle.api.internal.changedetection.FileCacheBroadcastTaskArtifactStateRepository$1.isUpToDate(FileCacheBroadcastTaskArtifactStateRepository.java:37)
    at org.gradle.api.internal.tasks.execution.SkipUpToDateTaskExecuter.execute(SkipUpToDateTaskExecuter.java:44)
    at org.gradle.api.internal.tasks.execution.ValidatingTaskExecuter.execute(ValidatingTaskExecuter.java:57)
    at org.gradle.api.internal.tasks.execution.SkipEmptySourceFilesTaskExecuter.execute(SkipEmptySourceFilesTaskExecuter.java:41)
    at org.gradle.api.internal.tasks.execution.SkipTaskWithNoActionsExecuter.execute(SkipTaskWithNoActionsExecuter.java:51)
    at org.gradle.api.internal.tasks.execution.SkipOnlyIfTaskExecuter.execute(SkipOnlyIfTaskExecuter.java:52)
    at org.gradle.api.internal.tasks.execution.ExecuteAtMostOnceTaskExecuter.execute(ExecuteAtMostOnceTaskExecuter.java:42)
    at org.gradle.api.internal.AbstractTask.execute(AbstractTask.java:237)
    at org.gradle.execution.DefaultTaskGraphExecuter.executeTask(DefaultTaskGraphExecuter.java:167)
    at org.gradle.execution.DefaultTaskGraphExecuter.doExecute(DefaultTaskGraphExecuter.java:160)
    at org.gradle.execution.DefaultTaskGraphExecuter.execute(DefaultTaskGraphExecuter.java:78)
    at org.gradle.execution.TaskNameResolvingBuildExecuter.execute(TaskNameResolvingBuildExecuter.java:113)
    at org.gradle.execution.DelegatingBuildExecuter.execute(DelegatingBuildExecuter.java:54)
    at org.gradle.initialization.DefaultGradleLauncher.doBuildStages(DefaultGradleLauncher.java:158)
    at org.gradle.initialization.DefaultGradleLauncher.doBuild(DefaultGradleLauncher.java:112)
    at org.gradle.initialization.DefaultGradleLauncher.run(DefaultGradleLauncher.java:80)
    at org.gradle.launcher.RunBuildAction.execute(RunBuildAction.java:41)
    at org.gradle.launcher.RunBuildAction.execute(RunBuildAction.java:27)
    at org.gradle.launcher.ExceptionReportingAction.execute(ExceptionReportingAction.java:32)
    at org.gradle.launcher.ExceptionReportingAction.execute(ExceptionReportingAction.java:21)
    at org.gradle.launcher.CommandLineActionFactory$WithLoggingAction.execute(CommandLineActionFactory.java:219)
    at org.gradle.launcher.CommandLineActionFactory$WithLoggingAction.execute(CommandLineActionFactory.java:203)
    at org.gradle.launcher.Main.execute(Main.java:55)
    at org.gradle.launcher.Main.main(Main.java:40)
    at org.gradle.launcher.ProcessBootstrap.runNoExit(ProcessBootstrap.java:46)
    at org.gradle.launcher.ProcessBootstrap.run(ProcessBootstrap.java:28)
    at org.gradle.launcher.GradleMain.main(GradleMain.java:24)

BUILD FAILED

Total time: 1.982 secs

Expected result: Gradle should ignore the dangling symlink. Or maaaaaybe print a warning, but I don't think that's necessary.

OTOH, if a build script explicitly references a dangling symlink, it should fail.



 Comments   
Comment by Greg Ward [ 20/Oct/11 ]

I just discovered that this does not depend on weird legacy build systems that leave dangling links behind (like mine). All it requires is for someone to use Emacs and have unsaved changes in a file. E.g. I was just editing build/javac.py, which caused Emacs to create a dangling symlink as a lock file:

build/.#javac.py -> $user@$hostname.16010:1318868569

which in turn caused Gradle to crash. Sigh. I should be able to run a build when I have unsaved files open in my editor!

Comment by Evgeny Goldin [ 06/Sep/12 ]

I have the same failure with Gradle 1.1 and

copy{ from ..; into .. }

if one of the files isn't recognised as a file and then treated as directory by DirectoryFileTree:153 (Gradle 1.1). The reason file isn't recognised by java as a file is one of documents uploaded to WordPress has non-UTF characters in its name. I run

new File('wp-content/uploads/2012/08').listFiles().each{ println "[$it][$it.file]" }

and had

..
[wp-content/uploads/2012/08/b-w??chst.pdf][false]
..
Comment by Evgeny Goldin [ 06/Sep/12 ]

Btw, same directory can't be deleted later:

Caused by: org.gradle.api.file.UnableToDeleteFileException: Unable to delete directory: .../wp-content/uploads/2012/08
	at org.gradle.api.internal.file.copy.DeleteActionImpl.handleFailedDelete(DeleteActionImpl.java:93)
	at org.gradle.api.internal.file.copy.DeleteActionImpl.doDelete(DeleteActionImpl.java:69)
	at org.gradle.api.internal.file.copy.DeleteActionImpl.doDelete(DeleteActionImpl.java:64)
	at org.gradle.api.internal.file.copy.DeleteActionImpl.doDelete(DeleteActionImpl.java:64)
	at org.gradle.api.internal.file.copy.DeleteActionImpl.doDelete(DeleteActionImpl.java:64)
	at org.gradle.api.internal.file.copy.DeleteActionImpl.doDelete(DeleteActionImpl.java:64)
	at org.gradle.api.internal.file.copy.DeleteActionImpl.delete(DeleteActionImpl.java:49)
	at org.gradle.api.internal.file.DefaultFileOperations.delete(DefaultFileOperations.java:128)
	at org.gradle.api.internal.project.AbstractProject.delete(AbstractProject.java:708)

Seems to me once there is a file not recognised by Java as a file - Gradle copy/delete/etc operations start breaking.

Comment by Evgeny Goldin [ 02/Dec/12 ]

Opened a separate GRADLE-2581 issue about failing copy/delete operations.

Comment by Sean D Gillespie [ 27/Jan/13 ]

I've been looking at this for a few days. I can't find a real good solution to the problem. The latest version commons-io does not detect broken symlinks (it thinks they are regular files). There are 2 possible solutions that I know of:

1) Write a Native Interface
2) Use the new Java 7 Files API

I'm guessing that (1) is no good, since it would require a C compiler to build gradle (or platform-specific binaries). (2) isn't an ideal solution either, since this would require a JDK7, for building and running gradle.

The only acceptable solution, in my mind, is to try to use (2), but only if those classes exist in the classpath. This would allow us to detect symlinks if Java 7 is used, but wouldn't require ALL users to run Java 7.

Does this make sense? Is there a better way?

Comment by Tim Slatcher [ 27/Mar/15 ]

Just wondered if there was any update on this? We've recently integrated node/npm into our builds and we're seeing this quite often with all the symlinks node_modules creates.

Comment by Joshua Warner [ 05/Jun/15 ]

We've run into this recently, in the context of building .app bundles on mac, where there are typically symlinks in the Frameworks directory, which can actually have relative paths (and thus look broken).

We're working around the problem by building, packaging, and then deleting the intermediate .app directory (which ends up in a .dmg). This rules out incremental builds of any sort, and is thus a total pain in the ass.

I'd be willing to help implement a fix for this, if someone can give me some pointers on where to start.

Comment by Artur Gajowy [ 24/Aug/15 ]

We've also encountered this issue when copying some files to a jenkins workspace (which can have dangling symlinks quite often). Is it still the case that compatibility with the non supported since Feb 2013 JDK 6 has to be maintained? Would a pull request requiring JDK 7 be accepted?

Comment by Daz DeBoer [ 25/Aug/15 ]

There are no current plans to remove Gradle support for JDK 6. Gradle will maintain support for JDK 6, at least until Gradle 3.0 is released.

Gradle already makes use of native filesystem APIs via the 'native-platform' library (which is built and maintained by our lead engineer). I would recommend:
1) A pull request for Gradle, including an integration test that demonstrates the issue.
1) A pull request for https://github.com/adammurdoch/native-platform that adds any required support for testing & fixing this issue.

Comment by chris warth [ 27/Jan/16 ]

Can someone describe the complexities of fixing this?

Like Greg Ward in 2011 [ comment-14167 ], I'm running into this problem because I use Emacs. It is not even possible to workaround this bug by excluding the offending source files as the error occurs before the exclude is considered.

Today the stack trace looks different than it did four and a half years ago, but the effect is the same.

org.gradle.api.GradleException: Could not list contents of '/Users/Development/mcmc-binomial/mcb/.#View.java'. Couldn't follow symbolic link.
        at org.gradle.api.internal.file.collections.jdk7.Jdk7DirectoryWalker$1.visitFile(Jdk7DirectoryWalker.java:71)
        at org.gradle.api.internal.file.collections.jdk7.Jdk7DirectoryWalker$1.visitFile(Jdk7DirectoryWalker.java:48)
        at org.gradle.api.internal.file.collections.jdk7.Jdk7DirectoryWalker.walkDir(Jdk7DirectoryWalker.java:48)
        at org.gradle.api.internal.file.collections.DirectoryFileTree.walkDir(DirectoryFileTree.java:143)
        at org.gradle.api.internal.file.collections.DirectoryFileTree.visitFrom(DirectoryFileTree.java:127)

Why is it not a simple fix to just catch and suppress the exception thrown in org.gradle.api.internal.file.collections.jdk7.Jdk7DirectoryWalker$1.visitFile?

Comment by Ronald Brindl [ 15/Mar/16 ]

@GRADLE-3400, which has been fixed in 2.12.

For this issue (GRADLE-1843) a pull request has just been created https://github.com/gradle/gradle/pull/593

Comment by Dzmitry Shylovich [ 18/Apr/16 ]

Any updates? I still receive this error in my Jenkins on Linux 2.6.32-504.30.3.el6.x86_64. On Windows 10 everything is fine.

Comment by Alpar Torok [ 12/Oct/16 ]

quick way to reproduce:

gradle init --type java-library
ln -s ../foo src/main/java/bar.java
./gradlew -s clean build

Note that the error message has changed, and is now easier to understand and more actionable than in the initial report.

* Exception is:
org.gradle.api.GradleException: Could not list contents of '/tmp/alpar/test/src/main/java/bar.java'. Couldn't follow symbolic link.
	at org.gradle.api.internal.file.collections.jdk7.Jdk7DirectoryWalker$1.visitFile(Jdk7DirectoryWalker.java:79)
	at org.gradle.api.internal.file.collections.jdk7.Jdk7DirectoryWalker$1.visitFile(Jdk7DirectoryWalker.java:54)
	at org.gradle.api.internal.file.collections.jdk7.Jdk7DirectoryWalker.walkDir(Jdk7DirectoryWalker.java:54)
	at org.gradle.api.internal.file.collections.DirectoryFileTree.walkDir(DirectoryFileTree.java:146)
	at org.gradle.api.internal.file.collections.DirectoryFileTree.visitFrom(DirectoryFileTree.java:130)
	at org.gradle.api.internal.file.collections.DirectoryFileTree.visit(DirectoryFileTree.java:115)
	at org.gradle.api.internal.file.collections.FileTreeAdapter.visit(FileTreeAdapter.java:109)
	at org.gradle.api.internal.file.AbstractFileTree.isEmpty(AbstractFileTree.java:49)
	at org.gradle.api.internal.file.CompositeFileCollection.isEmpty(CompositeFileCollection.java:70)
	at org.gradle.api.internal.tasks.execution.SkipEmptySourceFilesTaskExecuter.execute(SkipEmptySourceFilesTaskExecuter.java:44)
	at org.gradle.api.internal.tasks.execution.SkipTaskWithNoActionsExecuter.execute(SkipTaskWithNoActionsExecuter.java:52)
	at org.gradle.api.internal.tasks.execution.SkipOnlyIfTaskExecuter.execute(SkipOnlyIfTaskExecuter.java:53)
	at org.gradle.api.internal.tasks.execution.ExecuteAtMostOnceTaskExecuter.execute(ExecuteAtMostOnceTaskExecuter.java:43)
	at org.gradle.execution.taskgraph.DefaultTaskGraphExecuter$EventFiringTaskWorker.execute(DefaultTaskGraphExecuter.java:233)
	at org.gradle.execution.taskgraph.DefaultTaskGraphExecuter$EventFiringTaskWorker.execute(DefaultTaskGraphExecuter.java:215)
	at org.gradle.execution.taskgraph.AbstractTaskPlanExecutor$TaskExecutorWorker.processTask(AbstractTaskPlanExecutor.java:74)
	at org.gradle.execution.taskgraph.AbstractTaskPlanExecutor$TaskExecutorWorker.run(AbstractTaskPlanExecutor.java:55)
	at org.gradle.execution.taskgraph.ParallelTaskPlanExecutor.process(ParallelTaskPlanExecutor.java:50)
	at org.gradle.execution.taskgraph.DefaultTaskGraphExecuter.execute(DefaultTaskGraphExecuter.java:113)
	at org.gradle.execution.SelectedTaskExecutionAction.execute(SelectedTaskExecutionAction.java:37)
	at org.gradle.execution.DefaultBuildExecuter.execute(DefaultBuildExecuter.java:37)
	at org.gradle.execution.DefaultBuildExecuter.access$000(DefaultBuildExecuter.java:23)
	at org.gradle.execution.DefaultBuildExecuter$1.proceed(DefaultBuildExecuter.java:43)
	at org.gradle.execution.DryRunBuildExecutionAction.execute(DryRunBuildExecutionAction.java:32)
	at org.gradle.execution.DefaultBuildExecuter.execute(DefaultBuildExecuter.java:37)
	at org.gradle.execution.DefaultBuildExecuter.execute(DefaultBuildExecuter.java:30)
	at org.gradle.initialization.DefaultGradleLauncher$4.run(DefaultGradleLauncher.java:186)
	at org.gradle.internal.Factories$1.create(Factories.java:22)
	at org.gradle.internal.progress.DefaultBuildOperationExecutor.run(DefaultBuildOperationExecutor.java:91)
	at org.gradle.internal.progress.DefaultBuildOperationExecutor.run(DefaultBuildOperationExecutor.java:53)
	at org.gradle.initialization.DefaultGradleLauncher.doBuildStages(DefaultGradleLauncher.java:183)
	at org.gradle.initialization.DefaultGradleLauncher.access$200(DefaultGradleLauncher.java:33)
	at org.gradle.initialization.DefaultGradleLauncher$1.create(DefaultGradleLauncher.java:112)
	at org.gradle.initialization.DefaultGradleLauncher$1.create(DefaultGradleLauncher.java:106)
	at org.gradle.internal.progress.DefaultBuildOperationExecutor.run(DefaultBuildOperationExecutor.java:91)
	at org.gradle.internal.progress.DefaultBuildOperationExecutor.run(DefaultBuildOperationExecutor.java:63)
	at org.gradle.initialization.DefaultGradleLauncher.doBuild(DefaultGradleLauncher.java:106)
	at org.gradle.initialization.DefaultGradleLauncher.run(DefaultGradleLauncher.java:92)
	at org.gradle.launcher.exec.GradleBuildController.run(GradleBuildController.java:66)
	at org.gradle.tooling.internal.provider.ExecuteBuildActionRunner.run(ExecuteBuildActionRunner.java:28)
	at org.gradle.launcher.exec.ChainingBuildActionRunner.run(ChainingBuildActionRunner.java:35)
	at org.gradle.launcher.exec.InProcessBuildActionExecuter.execute(InProcessBuildActionExecuter.java:41)
	at org.gradle.launcher.exec.InProcessBuildActionExecuter.execute(InProcessBuildActionExecuter.java:26)
	at org.gradle.tooling.internal.provider.ContinuousBuildActionExecuter.execute(ContinuousBuildActionExecuter.java:79)
	at org.gradle.tooling.internal.provider.ContinuousBuildActionExecuter.execute(ContinuousBuildActionExecuter.java:51)
	at org.gradle.launcher.daemon.server.exec.ExecuteBuild.doBuild(ExecuteBuild.java:59)
	at org.gradle.launcher.daemon.server.exec.BuildCommandOnly.execute(BuildCommandOnly.java:36)
	at org.gradle.launcher.daemon.server.api.DaemonCommandExecution.proceed(DaemonCommandExecution.java:120)
	at org.gradle.launcher.daemon.server.exec.WatchForDisconnection.execute(WatchForDisconnection.java:47)
	at org.gradle.launcher.daemon.server.api.DaemonCommandExecution.proceed(DaemonCommandExecution.java:120)
	at org.gradle.launcher.daemon.server.exec.ResetDeprecationLogger.execute(ResetDeprecationLogger.java:26)
	at org.gradle.launcher.daemon.server.api.DaemonCommandExecution.proceed(DaemonCommandExecution.java:120)
	at org.gradle.launcher.daemon.server.exec.RequestStopIfSingleUsedDaemon.execute(RequestStopIfSingleUsedDaemon.java:34)
	at org.gradle.launcher.daemon.server.api.DaemonCommandExecution.proceed(DaemonCommandExecution.java:120)
	at org.gradle.launcher.daemon.server.exec.ForwardClientInput$2.call(ForwardClientInput.java:74)
	at org.gradle.launcher.daemon.server.exec.ForwardClientInput$2.call(ForwardClientInput.java:72)
	at org.gradle.util.Swapper.swap(Swapper.java:38)
	at org.gradle.launcher.daemon.server.exec.ForwardClientInput.execute(ForwardClientInput.java:72)
	at org.gradle.launcher.daemon.server.api.DaemonCommandExecution.proceed(DaemonCommandExecution.java:120)
	at org.gradle.launcher.daemon.server.exec.LogAndCheckHealth.execute(LogAndCheckHealth.java:55)
	at org.gradle.launcher.daemon.server.api.DaemonCommandExecution.proceed(DaemonCommandExecution.java:120)
	at org.gradle.launcher.daemon.server.exec.LogToClient.doBuild(LogToClient.java:60)
	at org.gradle.launcher.daemon.server.exec.BuildCommandOnly.execute(BuildCommandOnly.java:36)
	at org.gradle.launcher.daemon.server.api.DaemonCommandExecution.proceed(DaemonCommandExecution.java:120)
	at org.gradle.launcher.daemon.server.exec.EstablishBuildEnvironment.doBuild(EstablishBuildEnvironment.java:72)
	at org.gradle.launcher.daemon.server.exec.BuildCommandOnly.execute(BuildCommandOnly.java:36)
	at org.gradle.launcher.daemon.server.api.DaemonCommandExecution.proceed(DaemonCommandExecution.java:120)
	at org.gradle.launcher.daemon.server.exec.HintGCAfterBuild.execute(HintGCAfterBuild.java:44)
	at org.gradle.launcher.daemon.server.api.DaemonCommandExecution.proceed(DaemonCommandExecution.java:120)
	at org.gradle.launcher.daemon.server.exec.StartBuildOrRespondWithBusy$1.run(StartBuildOrRespondWithBusy.java:50)
	at org.gradle.launcher.daemon.server.DaemonStateCoordinator$1.run(DaemonStateCoordinator.java:293)
	at org.gradle.internal.concurrent.ExecutorPolicy$CatchAndRecordFailures.onExecute(ExecutorPolicy.java:54)
	at org.gradle.internal.concurrent.StoppableExecutorImpl$1.run(StoppableExecutorImpl.java:40)
------------------------------------------------------------
Gradle 3.1
------------------------------------------------------------

Build time:   2016-09-19 10:53:53 UTC
Revision:     13f38ba699afd86d7cdc4ed8fd7dd3960c0b1f97

Groovy:       2.4.7
Ant:          Apache Ant(TM) version 1.9.6 compiled on June 29 2015
JVM:          1.8.0_102 (Oracle Corporation 25.102-b14)
OS:           Linux 4.7.6-1-ARCH amd64

The same error is raised if one tries to copy or sync a folder that has a broken symlink:

[:~/tmp/test] 1 % ls -l test                                                                                                                                                                                                                                                                                                                            ⏎
total 0
lrwxrwxrwx 1 alpar users 6 Oct 12 09:18 bar -> ../foo
-rw-r--r-- 1 alpar users 0 Oct 12 09:18 file.txt
task testCopy(type: Copy) {
    from "test"
    into "$buildDir/test"
}

task testSync(type: Sync) {
    from "test"
    into "$buildDir/test"
}
 
Could not list contents of '/tmp/alpar/test/test/bar'. Couldn't follow symbolic link

I'm not convinced this behavior can be considered incorrect.
I guess it depends on why the symlink is broken in the first place, but I would rather have this error, than be in the situation where
I look at why something is missing from the artifacts only to find that it's because Gradle decided to ignore the broken symlinks.
In any case the behavior between Copy, Sync and Compile should be kept consistent and it seems to make even
less sense for Copy and Sync to silently ignore.

Further discussion can be found here: https://discuss.gradle.org/t/how-can-i-make-gradle-ignore-dangling-symlinks/7152/9.
There's a workaround included too there, but it uses exec and shell command to accomplish this, and will not work on windows
(symlinks are supported there: https://msdn.microsoft.com/en-us/library/windows/desktop/aa365680(v=vs.85).aspx)
A gradle only workaround is not achievable, because a simple fileTree reproduces the issue as well.
( it uses the same implementation )

task log << {
    fileTree(dir: "$buildDir/test").each { println it }
}

Java 7 makes the work-around obvious and Java 8 makes it brief:

task cleanBrokenSymlinks << {
    Files.find(
        file("$buildDir/test").toPath(),
        5, /* max depth to search in */ 
        { path, attr -> Files.isSymbolicLink(path) && !Files.exists(path) }
    ).each {
        Files.delete(it)
    }
}

Groovy makes for a brief implementation that will also work on java 6,
and thus should be the preferred workaround

task cleanBrokenSymlinks << {
    file("$buildDir/test").eachFileRecurse {
        if (! it.exists()) { it.delete() }
    }
}

summary I think it's ok to consider the problem descried in this report as "works as designed' and propose
the workaround (included below) from above for anyone who might need it, with the mention that it should be avoided to run it on anything not in buildDir.

Comment by Alpar Torok [ 12/Oct/16 ]

Not directly related, but discussing here because this ticket is referenced in various places for related issues.

Consider the testCopy and testSync tasks from my previous comments.

mkdir -p build/test
[:~/tmp/test] % ln -s ../foo build/test
[:~/tmp/test] % touch build/test/valid
[:~/tmp/test] % ln -s valid build/test/valid_link
[:~/tmp/test] % ls -l build/test test
build/test:
total 0
-rw-r--r-- 1 alpar users 0 Oct 12 11:44 file.txt
lrwxrwxrwx 1 alpar users 6 Oct 12 11:46 foo -> ../foo
-rw-r--r-- 1 alpar users 0 Oct 12 11:46 valid
lrwxrwxrwx 1 alpar users 5 Oct 12 11:46 valid_link -> valid

test:
total 0
-rw-r--r-- 1 alpar users 0 Oct 12 09:18 file.txt
[:~/tmp/test] 1 % ./gradlew -s testCopy
* Exception is:
org.gradle.api.GradleException: Could not list contents of '/tmp/alpar/test/build/test/foo'. Couldn't follow symbolic link.
	at org.gradle.api.internal.file.collections.jdk7.Jdk7DirectoryWalker$1.visitFile(Jdk7DirectoryWalker.java:79)
	at org.gradle.api.internal.file.collections.jdk7.Jdk7DirectoryWalker$1.visitFile(Jdk7DirectoryWalker.java:54)
	at org.gradle.api.internal.file.collections.jdk7.Jdk7DirectoryWalker.walkDir(Jdk7DirectoryWalker.java:54)
	at org.gradle.api.internal.file.collections.DirectoryFileTree.walkDir(DirectoryFileTree.java:146)
	at org.gradle.api.internal.file.collections.DirectoryFileTree.visitFrom(DirectoryFileTree.java:130)
	at org.gradle.api.internal.file.collections.DirectoryFileTree.visit(DirectoryFileTree.java:115)
	at org.gradle.api.internal.file.collections.DirectoryFileTree.visitTreeOrBackingFile(DirectoryFileTree.java:111)
	at org.gradle.api.internal.file.collections.FileTreeAdapter.visitTreeOrBackingFile(FileTreeAdapter.java:114)
	at org.gradle.api.internal.changedetection.state.DefaultFileCollectionSnapshotter.visitFiles(DefaultFileCollectionSnapshotter.java:42)
	at org.gradle.api.internal.changedetection.state.AbstractFileCollectionSnapshotter.snapshot(AbstractFileCollectionSnapshotter.java:62)
	at org.gradle.api.internal.changedetection.state.DefaultFileCollectionSnapshotter.snapshot(DefaultFileCollectionSnapshotter.java:31)
	at org.gradle.api.internal.changedetection.state.AbstractFileCollectionSnapshotter.snapshot(AbstractFileCollectionSnapshotter.java:100)
	at org.gradle.api.internal.changedetection.state.DefaultFileCollectionSnapshotter.snapshot(DefaultFileCollectionSnapshotter.java:31)
	at org.gradle.api.internal.changedetection.state.OutputFilesCollectionSnapshotter.snapshot(OutputFilesCollectionSnapshotter.java:60)
	at org.gradle.api.internal.changedetection.rules.AbstractNamedFileSnapshotTaskStateChanges.buildSnapshots(AbstractNamedFileSnapshotTaskStateChanges.java:85)
	at org.gradle.api.internal.changedetection.rules.AbstractNamedFileSnapshotTaskStateChanges.<init>(AbstractNamedFileSnapshotTaskStateChanges.java:53)
	at org.gradle.api.internal.changedetection.rules.OutputFilesTaskStateChanges.<init>(OutputFilesTaskStateChanges.java:31)
	at org.gradle.api.internal.changedetection.rules.TaskUpToDateState.<init>(TaskUpToDateState.java:54)
	at org.gradle.api.internal.changedetection.changes.DefaultTaskArtifactStateRepository$TaskArtifactStateImpl.getStates(DefaultTaskArtifactStateRepository.java:162)
	at org.gradle.api.internal.changedetection.changes.DefaultTaskArtifactStateRepository$TaskArtifactStateImpl.isUpToDate(DefaultTaskArtifactStateRepository.java:82)
	at org.gradle.api.internal.tasks.execution.SkipUpToDateTaskExecuter.execute(SkipUpToDateTaskExecuter.java:52)
	at org.gradle.api.internal.tasks.execution.ValidatingTaskExecuter.execute(ValidatingTaskExecuter.java:58)
	at org.gradle.api.internal.tasks.execution.SkipEmptySourceFilesTaskExecuter.execute(SkipEmptySourceFilesTaskExecuter.java:52)
	at org.gradle.api.internal.tasks.execution.SkipTaskWithNoActionsExecuter.execute(SkipTaskWithNoActionsExecuter.java:52)
	at org.gradle.api.internal.tasks.execution.SkipOnlyIfTaskExecuter.execute(SkipOnlyIfTaskExecuter.java:53)
	at org.gradle.api.internal.tasks.execution.ExecuteAtMostOnceTaskExecuter.execute(ExecuteAtMostOnceTaskExecuter.java:43)
	at org.gradle.execution.taskgraph.DefaultTaskGraphExecuter$EventFiringTaskWorker.execute(DefaultTaskGraphExecuter.java:233)
	at org.gradle.execution.taskgraph.DefaultTaskGraphExecuter$EventFiringTaskWorker.execute(DefaultTaskGraphExecuter.java:215)
	at org.gradle.execution.taskgraph.AbstractTaskPlanExecutor$TaskExecutorWorker.processTask(AbstractTaskPlanExecutor.java:74)
	at org.gradle.execution.taskgraph.AbstractTaskPlanExecutor$TaskExecutorWorker.run(AbstractTaskPlanExecutor.java:55)
	at org.gradle.execution.taskgraph.ParallelTaskPlanExecutor.process(ParallelTaskPlanExecutor.java:50)
	at org.gradle.execution.taskgraph.DefaultTaskGraphExecuter.execute(DefaultTaskGraphExecuter.java:113)
	at org.gradle.execution.SelectedTaskExecutionAction.execute(SelectedTaskExecutionAction.java:37)
	at org.gradle.execution.DefaultBuildExecuter.execute(DefaultBuildExecuter.java:37)
	at org.gradle.execution.DefaultBuildExecuter.access$000(DefaultBuildExecuter.java:23)
	at org.gradle.execution.DefaultBuildExecuter$1.proceed(DefaultBuildExecuter.java:43)
	at org.gradle.execution.DryRunBuildExecutionAction.execute(DryRunBuildExecutionAction.java:32)
	at org.gradle.execution.DefaultBuildExecuter.execute(DefaultBuildExecuter.java:37)
	at org.gradle.execution.DefaultBuildExecuter.execute(DefaultBuildExecuter.java:30)
	at org.gradle.initialization.DefaultGradleLauncher$4.run(DefaultGradleLauncher.java:186)
	at org.gradle.internal.Factories$1.create(Factories.java:22)
	at org.gradle.internal.progress.DefaultBuildOperationExecutor.run(DefaultBuildOperationExecutor.java:91)
	at org.gradle.internal.progress.DefaultBuildOperationExecutor.run(DefaultBuildOperationExecutor.java:53)
	at org.gradle.initialization.DefaultGradleLauncher.doBuildStages(DefaultGradleLauncher.java:183)
	at org.gradle.initialization.DefaultGradleLauncher.access$200(DefaultGradleLauncher.java:33)
	at org.gradle.initialization.DefaultGradleLauncher$1.create(DefaultGradleLauncher.java:112)
	at org.gradle.initialization.DefaultGradleLauncher$1.create(DefaultGradleLauncher.java:106)
	at org.gradle.internal.progress.DefaultBuildOperationExecutor.run(DefaultBuildOperationExecutor.java:91)
	at org.gradle.internal.progress.DefaultBuildOperationExecutor.run(DefaultBuildOperationExecutor.java:63)
	at org.gradle.initialization.DefaultGradleLauncher.doBuild(DefaultGradleLauncher.java:106)
	at org.gradle.initialization.DefaultGradleLauncher.run(DefaultGradleLauncher.java:92)
	at org.gradle.launcher.exec.GradleBuildController.run(GradleBuildController.java:66)
	at org.gradle.tooling.internal.provider.ExecuteBuildActionRunner.run(ExecuteBuildActionRunner.java:28)
	at org.gradle.launcher.exec.ChainingBuildActionRunner.run(ChainingBuildActionRunner.java:35)
	at org.gradle.launcher.exec.InProcessBuildActionExecuter.execute(InProcessBuildActionExecuter.java:41)
	at org.gradle.launcher.exec.InProcessBuildActionExecuter.execute(InProcessBuildActionExecuter.java:26)
	at org.gradle.tooling.internal.provider.ContinuousBuildActionExecuter.execute(ContinuousBuildActionExecuter.java:79)
	at org.gradle.tooling.internal.provider.ContinuousBuildActionExecuter.execute(ContinuousBuildActionExecuter.java:51)
	at org.gradle.launcher.daemon.server.exec.ExecuteBuild.doBuild(ExecuteBuild.java:59)
	at org.gradle.launcher.daemon.server.exec.BuildCommandOnly.execute(BuildCommandOnly.java:36)
	at org.gradle.launcher.daemon.server.api.DaemonCommandExecution.proceed(DaemonCommandExecution.java:120)
	at org.gradle.launcher.daemon.server.exec.WatchForDisconnection.execute(WatchForDisconnection.java:47)
	at org.gradle.launcher.daemon.server.api.DaemonCommandExecution.proceed(DaemonCommandExecution.java:120)
	at org.gradle.launcher.daemon.server.exec.ResetDeprecationLogger.execute(ResetDeprecationLogger.java:26)
	at org.gradle.launcher.daemon.server.api.DaemonCommandExecution.proceed(DaemonCommandExecution.java:120)
	at org.gradle.launcher.daemon.server.exec.RequestStopIfSingleUsedDaemon.execute(RequestStopIfSingleUsedDaemon.java:34)
	at org.gradle.launcher.daemon.server.api.DaemonCommandExecution.proceed(DaemonCommandExecution.java:120)
	at org.gradle.launcher.daemon.server.exec.ForwardClientInput$2.call(ForwardClientInput.java:74)
	at org.gradle.launcher.daemon.server.exec.ForwardClientInput$2.call(ForwardClientInput.java:72)
	at org.gradle.util.Swapper.swap(Swapper.java:38)
	at org.gradle.launcher.daemon.server.exec.ForwardClientInput.execute(ForwardClientInput.java:72)
	at org.gradle.launcher.daemon.server.api.DaemonCommandExecution.proceed(DaemonCommandExecution.java:120)
	at org.gradle.launcher.daemon.server.exec.LogAndCheckHealth.execute(LogAndCheckHealth.java:55)
	at org.gradle.launcher.daemon.server.api.DaemonCommandExecution.proceed(DaemonCommandExecution.java:120)
	at org.gradle.launcher.daemon.server.exec.LogToClient.doBuild(LogToClient.java:60)
	at org.gradle.launcher.daemon.server.exec.BuildCommandOnly.execute(BuildCommandOnly.java:36)
	at org.gradle.launcher.daemon.server.api.DaemonCommandExecution.proceed(DaemonCommandExecution.java:120)
	at org.gradle.launcher.daemon.server.exec.EstablishBuildEnvironment.doBuild(EstablishBuildEnvironment.java:72)
	at org.gradle.launcher.daemon.server.exec.BuildCommandOnly.execute(BuildCommandOnly.java:36)
	at org.gradle.launcher.daemon.server.api.DaemonCommandExecution.proceed(DaemonCommandExecution.java:120)
	at org.gradle.launcher.daemon.server.exec.HintGCAfterBuild.execute(HintGCAfterBuild.java:44)
	at org.gradle.launcher.daemon.server.api.DaemonCommandExecution.proceed(DaemonCommandExecution.java:120)
	at org.gradle.launcher.daemon.server.exec.StartBuildOrRespondWithBusy$1.run(StartBuildOrRespondWithBusy.java:50)
	at org.gradle.launcher.daemon.server.DaemonStateCoordinator$1.run(DaemonStateCoordinator.java:293)
	at org.gradle.internal.concurrent.ExecutorPolicy$CatchAndRecordFailures.onExecute(ExecutorPolicy.java:54)
	at org.gradle.internal.concurrent.StoppableExecutorImpl$1.run(StoppableExecutorImpl.java:40)

Both tasks fail in this way.
Looks like Gradle is listing the contents of the target as well, which would make sense for Sync but seems odd for Copy. The stack trace seems to suggest it's due to the up-to-date check logic, that would indeed need to know what's in the destination directory, but would probably not have to care about symlinks and would not want to follow them. If the symlink points outside, the output files collection, it's not really an out ( so the output is the symlink itself ), if it point's within than the target will be considered for the up to date check anyhow.

The Copy task should just work, and Sync should remove the broken symlink.

GRADLE-3400 is also related, probably the one that improved the error message.

The mailing list has a discussion which led to a design-doc highly relevant for this context.

GRADLE-2892 and the discussion have some context on the thinking around symlinks (spec).

Another related discussion brings up the point that the problem might go beyond just symlinks
and can extend to file permissions issues to in a similar way. It does seem less likely for this to happen however.

Comment by Benjamin Muschko [ 15/Nov/16 ]

As announced on the Gradle blog we are planning to completely migrate issues from JIRA to GitHub.

We intend to prioritize issues that are actionable and impactful while working more closely with the community. Many of our JIRA issues are inactionable or irrelevant. We would like to request your help to ensure we can appropriately prioritize JIRA issues you’ve contributed to.

Please confirm that you still advocate for your JIRA issue before December 10th, 2016 by:

  • Checking that your issues contain requisite context, impact, behaviors, and examples as described in our published guidelines.
  • Leave a comment on the JIRA issue or open a new GitHub issue confirming that the above is complete.

We look forward to collaborating with you more closely on GitHub. Thank you for your contribution to Gradle!

Comment by Martin Schaaf [ 06/Dec/16 ]

This is still an issue for me.

Comment by Benjamin Muschko [ 07/Feb/17 ]

I'd have to agree with Alpar's argument. Why is there a dangling symlink in the first place? From my perspective it should be removed by the owner of the file system as it does not serve any purpose. Gradle correctly indicates that the symlink doesn't lead anywhere. It's an important message to have as a user. The copy operation isn't done under the false preconception that resources were copied that in fact were not copied at runtime. Swallowing the information would be more confusing than helpful.

I am not going the close the issue as "Won't fix". In case someone is interested in discussing the topic any further, I'd suggest creating a new post on the gradle-dev list.

Comment by chris warth [ 07/Feb/17 ]

If Gradle were named "Check Symlinks And Fall Over Dead With A Useless Stack Trace" then I would agree with you, this would be a reasonable behavior. But it is not called that, and I didn't ask it to find broken symlinks at all. I'm trying to get a build done, and instead it dies mysteriously without any workaround. I don't understand the attitude that a stacktrace is a ever a reasonable response. Thankfully I don't have to worry about it because I moved on to other tools once I saw this bug was not going to get fixed.

Comment by Lee Nave [ 07/Feb/17 ]

Couldn't have said it any better myself.

Comment by Alpar Torok [ 07/Feb/17 ]

Benjamin you are referring to my first comment only but I think the scenario in the second one is what people run into more often and it is something that does need fixing

Comment by Benjamin Muschko [ 08/Feb/17 ]

Thanks for pointing toward the different issues mentioned in this issue. I wasn't fully aware of the full breadth of use cases here. I created https://github.com/gradle/gradle/issues/1365 on GitHub for it. Please open another issue on GitHub if the existing issue doesn't reflect your use case so we don't convolute the problem space.

Generated at Wed Jun 30 12:05:51 CDT 2021 using Jira 8.4.2#804003-sha1:d21414fc212e3af190e92c2d2ac41299b89402cf.