[GRADLE-600] Excessive dependency download of SNAPSHOT Created: 20/Aug/09  Updated: 04/Jan/13  Resolved: 07/Jan/12

Status: Resolved
Project: Gradle
Affects Version/s: 0.8
Fix Version/s: 1.0-milestone-8

Type: Bug
Reporter: Jeppe Nejsum Madsen Assignee: Daz DeBoer
Resolution: Fixed Votes: 5


 Description   

If I do "gradle test" and some SNAPSHOT dependencies have been updated
since last build, the same dependencies will be downloaded by the
compile, compileTests and test tasks.



 Comments   
Comment by Hans Dockter [ 02/Sep/09 ]

It turned out that this is an ivy bug. You would have the same behavior with an ant build script. I have filed an Ivy Jira: https://issues.apache.org/jira/browse/IVY-1117

I still leave this bug open (as our user's don't care where this bug comes from) but I postpone it to 0.9.

Comment by Peter Niederwieser [ 30/May/10 ]

I've just hit the same issue. The spock-example build downloads groovy-all-1.8.0-beta-1-SNAPSHOT five times instead of once (see below). I understand this is not Gradle's fault, but for people coming from Maven, this is a fairly critical issue. What's the perspective of getting this fixed? Do we have to wait until the forthcoming new Maven 3 dependency resolver has been integrated?

:compileJava
http://snapshots.repository.codehaus.org/org/codehaus/groovy/groovy-all/1.8.0-beta-1-SNAPSHOT/groovy-all-1.8.0-beta-1-SNAPSHOT.pom downloaded (26 KB)
http://snapshots.repository.codehaus.org/org/codehaus/groovy/groovy-all/1.8.0-beta-1-SNAPSHOT/groovy-all-1.8.0-beta-1-SNAPSHOT.jar downloaded (5.08 MB)
:compileGroovy
http://snapshots.repository.codehaus.org/org/codehaus/groovy/groovy-all/1.8.0-beta-1-SNAPSHOT/groovy-all-1.8.0-beta-1-SNAPSHOT.pom downloaded (26 KB)
http://snapshots.repository.codehaus.org/org/codehaus/groovy/groovy-all/1.8.0-beta-1-SNAPSHOT/groovy-all-1.8.0-beta-1-SNAPSHOT.jar downloaded (5.08 MB)
:processResources UP-TO-DATE
:classes
:compileTestJava
http://snapshots.repository.codehaus.org/org/codehaus/groovy/groovy-all/1.8.0-beta-1-SNAPSHOT/groovy-all-1.8.0-beta-1-SNAPSHOT.pom downloaded (26 KB)
http://snapshots.repository.codehaus.org/org/codehaus/groovy/groovy-all/1.8.0-beta-1-SNAPSHOT/groovy-all-1.8.0-beta-1-SNAPSHOT.jar downloaded (5.08 MB)
:compileTestGroovy
http://snapshots.repository.codehaus.org/org/codehaus/groovy/groovy-all/1.8.0-beta-1-SNAPSHOT/groovy-all-1.8.0-beta-1-SNAPSHOT.pom downloaded (26 KB)
http://snapshots.repository.codehaus.org/org/codehaus/groovy/groovy-all/1.8.0-beta-1-SNAPSHOT/groovy-all-1.8.0-beta-1-SNAPSHOT.jar downloaded (5.08 MB)
:processTestResources UP-TO-DATE
:testClasses
:test
http://snapshots.repository.codehaus.org/org/codehaus/groovy/groovy-all/1.8.0-beta-1-SNAPSHOT/groovy-all-1.8.0-beta-1-SNAPSHOT.pom downloaded (26 KB)
http://snapshots.repository.codehaus.org/org/codehaus/groovy/groovy-all/1.8.0-beta-1-SNAPSHOT/groovy-all-1.8.0-beta-1-SNAPSHOT.jar downloaded (5.08 MB)
Comment by Hans Dockter [ 08/Jun/10 ]

I agree that this is serious. I have moved it to 0.9 so we not forget about it. But I'm not sure yet whether we will be able to provide a fix for it. We might come up with some work around. In any case, the way we want to evolve our dependency management for 1.0 will fix this. Independent of a native Maven resolver.

Comment by Hans Dockter [ 27/Jul/10 ]

Our new snapshot handling should solve this issue for all practical purposes. The default snapshot update policy is now daily (before it was always, which is the only thing Ivy can do out of the box for Maven snapshots). Yet we should also solve this more fundamentally that there will be only one tried resolve for each dependency per multiproject build run. I postpone this to 1.0.

Comment by René Gröschke (Inactive) [ 02/Mar/11 ]

The snapshot policy doesn't work for our purposes, since in some scenarios I want the latest snapshot. We've organized some build jobs on our CI server in more than one multiproject build, so we use the snapshotTimeout 0. I've digged a bit into ivy and found a (dirty) workarround for this problem described in the comments of GRADLE-1049.

Comment by Daz DeBoer [ 14/Nov/11 ]

For cases where snapshot modules can be cached, this problem is now fixed. However, in the cases where the most up-to-date version of the snapshot is required (cacheChangingModulesFor 0, 'seconds'), we still perform unnecessary downloads. This will be fixed when we:
1) Only resolve a module once per build: this will fix the problem of multiple downloads within a single build
2) Where hash values or etags can assist us, only download an artifact if it has changed since the previous download

Comment by Daz DeBoer [ 07/Jan/12 ]

This issue is now resolved (Milestone 8). For snapshots and other changing modules, we:
1) Only resolve once per build: artifacts cached during the build are reused later in the build, even with cacheChangingModulesFor 0, 'seconds'
2) Use the sha1 checksum of the existing snapshot to determine if the new one should be downloaded or not

Generated at Wed Jun 30 11:34:04 CDT 2021 using Jira 8.4.2#804003-sha1:d21414fc212e3af190e92c2d2ac41299b89402cf.