Interesting. The Gradle cache is tolerant to files being removed from /tmp/testcache/caches/artifacts-8/filestore/. The problem is that the "module-metadata" directory is linked from the binary file module-metadata.bin, which does not tolerate missing files.

The new --refresh dependencies switch should reduce your need to tinker with the cache. But making the cache tolerant to files being user-purged is certainly a good idea.

But making the cache tolerant to files being user-purged is certainly a good idea.

Frankly, I would consider this more like a "critical requirement" than a good idea. The ability to reason about what's in the cache and selectively and intuitively purge things from it is something that Maven and Ivy users do all the time. I am very impressed with the dependency resolution and cache improvements that have come along in recent milestones, but in my opinion these cannot come at the cost of turning the cache into a black box.

I don't think it's a far-fetched prediction that this issue, if unaddressed, will become a major source of complaints from users as Gradle becomes more and more popular. As it is right now, if the cache ends up with a dependency I do not want there, I have to wipe it out the cache completely, forcing an otherwise totally unnecessary re-downloading of all dependencies. I'm not sure what the semantics of `--refresh dependencies` are, and I hope it helps, but I can't encourage strongly enough that Gradle either (a) make it straightforward to search for and delete cached artifacts (i.e. `find ~/.gradle -name some-dep | xargs rm -rf`) or (b) provide a very friendly mechanism (preferably interactive) for cache management via `gradle` itself. For simplicity's sake, and in order to take advantage of all the goodness of the usual unix tools, option (a) is strongly preferred.

Thanks for the follow-up in any case.

Thanks for you encouragement and feedback.

One thing that sets the Gradle dependency cache apart from that in Maven or Ivy is that we utilise binary storage for much of our cache meta-data. This choice has made it easy for us to be much more sophisticated in our caching logic, and will allow us to easily extend this in the future. However, by using a binary format the cache has become necessarily opaque in some regards.

As an example, we currently cache the resolution of a dynamic version (1.+) to a static version (1.3) on a per-repository basis. This is stored in dynamic-versions.bin, and so cannot be manipulated by simple filesystem operations. In a similar way we cache the absence of a module in a repository, in module-metadata.bin.

I believe that the new '--refresh dependencies' switch will cater for most of the use cases that have traditionally required manual cache hackery. This switch functions almost the same as deleting the entire cache, except that artifacts previously downloaded are used as candidates to prevent re-download based on SHA1 key. So any binary cache data is ignored for resolution, but will be updated based on what's actually resolved.

Milestone 8 includes an initial chapter describing the dependency cache: http://gradle.org/docs/release-candidate/userguide/dependency_management.html#sec:dependency_cache. Hopefully that will clarify the way the cache currently works, even if it doesn't do anything to alleviate your concerns .

To the problem at hand: I consider it a bug that Gradle doesn't tolerate the remove of *.ivy files under the /module-metadata directory; this is an easy fix. However this won't provide a full cache management capability via the filesystem. It won't be possible to manually remove the cached absense of a module, or the cached resolution of a dynamic version. The fact that '--refresh dependencies' provides a better alternative will hopefully make that less of an issue.

Just to be clear - I see that this has been resolved for milestone-9. Does that mean that `--refresh dependencies` is being considered as the fix?

No. We've fixed the cache to be tolerant of manually removed artifacts. The following will no longer break Gradle:

find ~/.gradle -name bundlor-plugin | xargs rm -rf

However, there are some cache operations that you cannot easily do via manipulation of the filesystem. Examples include:

• forcing Gradle to look for a module in a repository where it was previously found to be missing
• forcing Gradle to re-evaluate a dynamic version like '1.+'

The only way to perform these via the filesystem would be to remove *.bin in the artifacts-N directory. So you'd still have any downloaded artifacts, but the calculated metadata would be discarded.

Great!

Just to check, the fixes mentioned above are milestone-9 only? I just tried the following against m8a with no luck as you can see:

cbeams@anakata:/Work/model-project-gradle[master]
$ find ~/.gradle -name docbook-reference-plugin | grep 8
/Users/cbeams/.gradle/caches/artifacts-8/filestore/org.springframework.build.gradle/docbook-reference-plugin
/Users/cbeams/.gradle/caches/artifacts-8/module-metadata/org.springframework.build.gradle/docbook-reference-plugin
cbeams@anakata:/Work/model-project-gradle[master]
$ find ~/.gradle -name docbook-reference-plugin | grep 8 | xargs rm -rf
cbeams@anakata:/Work/model-project-gradle[master]
$ gradle clean reference 
Note: the Gradle build daemon is an experimental feature.
As such, you may experience unexpected build failures. You may need to occasionally stop the daemon.

FAILURE: Build failed with an exception.

* What went wrong:
Could not resolve all dependencies for configuration ':classpath'.
> Could not resolve group:org.springframework.build.gradle, module:docbook-reference-plugin, version:0.1.2.
  Required by:
      :model-project:1.0.0.BUILD-SNAPSHOT
   > java.text.ParseException: /Users/cbeams/.gradle/caches/artifacts-8/module-metadata/org.springframework.build.gradle/docbook-reference-plugin/0.1.2/efdc0d20916eefdddb3920ec1099ea8.ivy.xml (No such file or directory) in file:/Users/cbeams/.gradle/caches/artifacts-8/module-metadata/org.springframework.build.gradle/docbook-reference-plugin/0.1.2/efdc0d20916eefdddb3920ec1099ea8.ivy.xml

@Chris, the fix is in m9.