Excessive dependency download of SNAPSHOT

It turned out that this is an ivy bug. You would have the same behavior with an ant build script. I have filed an Ivy Jira: https://issues.apache.org/jira/browse/IVY-1117

I still leave this bug open (as our user's don't care where this bug comes from) but I postpone it to 0.9.

I've just hit the same issue. The spock-example build downloads groovy-all-1.8.0-beta-1-SNAPSHOT five times instead of once (see below). I understand this is not Gradle's fault, but for people coming from Maven, this is a fairly critical issue. What's the perspective of getting this fixed? Do we have to wait until the forthcoming new Maven 3 dependency resolver has been integrated?

:compileJava
http://snapshots.repository.codehaus.org/org/codehaus/groovy/groovy-all/1.8.0-beta-1-SNAPSHOT/groovy-all-1.8.0-beta-1-SNAPSHOT.pom downloaded (26 KB)
http://snapshots.repository.codehaus.org/org/codehaus/groovy/groovy-all/1.8.0-beta-1-SNAPSHOT/groovy-all-1.8.0-beta-1-SNAPSHOT.jar downloaded (5.08 MB)
:compileGroovy
http://snapshots.repository.codehaus.org/org/codehaus/groovy/groovy-all/1.8.0-beta-1-SNAPSHOT/groovy-all-1.8.0-beta-1-SNAPSHOT.pom downloaded (26 KB)
http://snapshots.repository.codehaus.org/org/codehaus/groovy/groovy-all/1.8.0-beta-1-SNAPSHOT/groovy-all-1.8.0-beta-1-SNAPSHOT.jar downloaded (5.08 MB)
:processResources UP-TO-DATE
:classes
:compileTestJava
http://snapshots.repository.codehaus.org/org/codehaus/groovy/groovy-all/1.8.0-beta-1-SNAPSHOT/groovy-all-1.8.0-beta-1-SNAPSHOT.pom downloaded (26 KB)
http://snapshots.repository.codehaus.org/org/codehaus/groovy/groovy-all/1.8.0-beta-1-SNAPSHOT/groovy-all-1.8.0-beta-1-SNAPSHOT.jar downloaded (5.08 MB)
:compileTestGroovy
http://snapshots.repository.codehaus.org/org/codehaus/groovy/groovy-all/1.8.0-beta-1-SNAPSHOT/groovy-all-1.8.0-beta-1-SNAPSHOT.pom downloaded (26 KB)
http://snapshots.repository.codehaus.org/org/codehaus/groovy/groovy-all/1.8.0-beta-1-SNAPSHOT/groovy-all-1.8.0-beta-1-SNAPSHOT.jar downloaded (5.08 MB)
:processTestResources UP-TO-DATE
:testClasses
:test
http://snapshots.repository.codehaus.org/org/codehaus/groovy/groovy-all/1.8.0-beta-1-SNAPSHOT/groovy-all-1.8.0-beta-1-SNAPSHOT.pom downloaded (26 KB)
http://snapshots.repository.codehaus.org/org/codehaus/groovy/groovy-all/1.8.0-beta-1-SNAPSHOT/groovy-all-1.8.0-beta-1-SNAPSHOT.jar downloaded (5.08 MB)

I agree that this is serious. I have moved it to 0.9 so we not forget about it. But I'm not sure yet whether we will be able to provide a fix for it. We might come up with some work around. In any case, the way we want to evolve our dependency management for 1.0 will fix this. Independent of a native Maven resolver.

Our new snapshot handling should solve this issue for all practical purposes. The default snapshot update policy is now daily (before it was always, which is the only thing Ivy can do out of the box for Maven snapshots). Yet we should also solve this more fundamentally that there will be only one tried resolve for each dependency per multiproject build run. I postpone this to 1.0.

The snapshot policy doesn't work for our purposes, since in some scenarios I want the latest snapshot. We've organized some build jobs on our CI server in more than one multiproject build, so we use the snapshotTimeout 0. I've digged a bit into ivy and found a (dirty) workarround for this problem described in the comments of GRADLE-1049.

For cases where snapshot modules can be cached, this problem is now fixed. However, in the cases where the most up-to-date version of the snapshot is required (cacheChangingModulesFor 0, 'seconds'), we still perform unnecessary downloads. This will be fixed when we:
1) Only resolve a module once per build: this will fix the problem of multiple downloads within a single build
2) Where hash values or etags can assist us, only download an artifact if it has changed since the previous download

This issue is now resolved (Milestone 8). For snapshots and other changing modules, we:
1) Only resolve once per build: artifacts cached during the build are reused later in the build, even with cacheChangingModulesFor 0, 'seconds'
2) Use the sha1 checksum of the existing snapshot to determine if the new one should be downloaded or not