Gradle
  1. Gradle
  2. GRADLE-600

Excessive dependency download of SNAPSHOT

    Details

    • Type: Bug Bug
    • Status: Resolved Resolved
    • Resolution: Fixed
    • Affects Version/s: 0.8
    • Fix Version/s: 1.0-milestone-8

      Description

      If I do "gradle test" and some SNAPSHOT dependencies have been updated
      since last build, the same dependencies will be downloaded by the
      compile, compileTests and test tasks.

        Activity

        Hide
        Hans Dockter
        added a comment -

        It turned out that this is an ivy bug. You would have the same behavior with an ant build script. I have filed an Ivy Jira: https://issues.apache.org/jira/browse/IVY-1117

        I still leave this bug open (as our user's don't care where this bug comes from) but I postpone it to 0.9.

        Show
        Hans Dockter
        added a comment - It turned out that this is an ivy bug. You would have the same behavior with an ant build script. I have filed an Ivy Jira: https://issues.apache.org/jira/browse/IVY-1117 I still leave this bug open (as our user's don't care where this bug comes from) but I postpone it to 0.9.
        Hide
        Peter Niederwieser
        added a comment -

        I've just hit the same issue. The spock-example build downloads groovy-all-1.8.0-beta-1-SNAPSHOT five times instead of once (see below). I understand this is not Gradle's fault, but for people coming from Maven, this is a fairly critical issue. What's the perspective of getting this fixed? Do we have to wait until the forthcoming new Maven 3 dependency resolver has been integrated?

        :compileJava
        http://snapshots.repository.codehaus.org/org/codehaus/groovy/groovy-all/1.8.0-beta-1-SNAPSHOT/groovy-all-1.8.0-beta-1-SNAPSHOT.pom downloaded (26 KB)
        http://snapshots.repository.codehaus.org/org/codehaus/groovy/groovy-all/1.8.0-beta-1-SNAPSHOT/groovy-all-1.8.0-beta-1-SNAPSHOT.jar downloaded (5.08 MB)
        :compileGroovy
        http://snapshots.repository.codehaus.org/org/codehaus/groovy/groovy-all/1.8.0-beta-1-SNAPSHOT/groovy-all-1.8.0-beta-1-SNAPSHOT.pom downloaded (26 KB)
        http://snapshots.repository.codehaus.org/org/codehaus/groovy/groovy-all/1.8.0-beta-1-SNAPSHOT/groovy-all-1.8.0-beta-1-SNAPSHOT.jar downloaded (5.08 MB)
        :processResources UP-TO-DATE
        :classes
        :compileTestJava
        http://snapshots.repository.codehaus.org/org/codehaus/groovy/groovy-all/1.8.0-beta-1-SNAPSHOT/groovy-all-1.8.0-beta-1-SNAPSHOT.pom downloaded (26 KB)
        http://snapshots.repository.codehaus.org/org/codehaus/groovy/groovy-all/1.8.0-beta-1-SNAPSHOT/groovy-all-1.8.0-beta-1-SNAPSHOT.jar downloaded (5.08 MB)
        :compileTestGroovy
        http://snapshots.repository.codehaus.org/org/codehaus/groovy/groovy-all/1.8.0-beta-1-SNAPSHOT/groovy-all-1.8.0-beta-1-SNAPSHOT.pom downloaded (26 KB)
        http://snapshots.repository.codehaus.org/org/codehaus/groovy/groovy-all/1.8.0-beta-1-SNAPSHOT/groovy-all-1.8.0-beta-1-SNAPSHOT.jar downloaded (5.08 MB)
        :processTestResources UP-TO-DATE
        :testClasses
        :test
        http://snapshots.repository.codehaus.org/org/codehaus/groovy/groovy-all/1.8.0-beta-1-SNAPSHOT/groovy-all-1.8.0-beta-1-SNAPSHOT.pom downloaded (26 KB)
        http://snapshots.repository.codehaus.org/org/codehaus/groovy/groovy-all/1.8.0-beta-1-SNAPSHOT/groovy-all-1.8.0-beta-1-SNAPSHOT.jar downloaded (5.08 MB)
        
        Show
        Peter Niederwieser
        added a comment - I've just hit the same issue. The spock-example build downloads groovy-all-1.8.0-beta-1-SNAPSHOT five times instead of once (see below). I understand this is not Gradle's fault, but for people coming from Maven, this is a fairly critical issue. What's the perspective of getting this fixed? Do we have to wait until the forthcoming new Maven 3 dependency resolver has been integrated? :compileJava http://snapshots.repository.codehaus.org/org/codehaus/groovy/groovy-all/1.8.0-beta-1-SNAPSHOT/groovy-all-1.8.0-beta-1-SNAPSHOT.pom downloaded (26 KB) http://snapshots.repository.codehaus.org/org/codehaus/groovy/groovy-all/1.8.0-beta-1-SNAPSHOT/groovy-all-1.8.0-beta-1-SNAPSHOT.jar downloaded (5.08 MB) :compileGroovy http://snapshots.repository.codehaus.org/org/codehaus/groovy/groovy-all/1.8.0-beta-1-SNAPSHOT/groovy-all-1.8.0-beta-1-SNAPSHOT.pom downloaded (26 KB) http://snapshots.repository.codehaus.org/org/codehaus/groovy/groovy-all/1.8.0-beta-1-SNAPSHOT/groovy-all-1.8.0-beta-1-SNAPSHOT.jar downloaded (5.08 MB) :processResources UP-TO-DATE :classes :compileTestJava http://snapshots.repository.codehaus.org/org/codehaus/groovy/groovy-all/1.8.0-beta-1-SNAPSHOT/groovy-all-1.8.0-beta-1-SNAPSHOT.pom downloaded (26 KB) http://snapshots.repository.codehaus.org/org/codehaus/groovy/groovy-all/1.8.0-beta-1-SNAPSHOT/groovy-all-1.8.0-beta-1-SNAPSHOT.jar downloaded (5.08 MB) :compileTestGroovy http://snapshots.repository.codehaus.org/org/codehaus/groovy/groovy-all/1.8.0-beta-1-SNAPSHOT/groovy-all-1.8.0-beta-1-SNAPSHOT.pom downloaded (26 KB) http://snapshots.repository.codehaus.org/org/codehaus/groovy/groovy-all/1.8.0-beta-1-SNAPSHOT/groovy-all-1.8.0-beta-1-SNAPSHOT.jar downloaded (5.08 MB) :processTestResources UP-TO-DATE :testClasses :test http://snapshots.repository.codehaus.org/org/codehaus/groovy/groovy-all/1.8.0-beta-1-SNAPSHOT/groovy-all-1.8.0-beta-1-SNAPSHOT.pom downloaded (26 KB) http://snapshots.repository.codehaus.org/org/codehaus/groovy/groovy-all/1.8.0-beta-1-SNAPSHOT/groovy-all-1.8.0-beta-1-SNAPSHOT.jar downloaded (5.08 MB)
        Hide
        Hans Dockter
        added a comment -

        I agree that this is serious. I have moved it to 0.9 so we not forget about it. But I'm not sure yet whether we will be able to provide a fix for it. We might come up with some work around. In any case, the way we want to evolve our dependency management for 1.0 will fix this. Independent of a native Maven resolver.

        Show
        Hans Dockter
        added a comment - I agree that this is serious. I have moved it to 0.9 so we not forget about it. But I'm not sure yet whether we will be able to provide a fix for it. We might come up with some work around. In any case, the way we want to evolve our dependency management for 1.0 will fix this. Independent of a native Maven resolver.
        Hide
        Hans Dockter
        added a comment -

        Our new snapshot handling should solve this issue for all practical purposes. The default snapshot update policy is now daily (before it was always, which is the only thing Ivy can do out of the box for Maven snapshots). Yet we should also solve this more fundamentally that there will be only one tried resolve for each dependency per multiproject build run. I postpone this to 1.0.

        Show
        Hans Dockter
        added a comment - Our new snapshot handling should solve this issue for all practical purposes. The default snapshot update policy is now daily (before it was always, which is the only thing Ivy can do out of the box for Maven snapshots). Yet we should also solve this more fundamentally that there will be only one tried resolve for each dependency per multiproject build run. I postpone this to 1.0.
        Hide
        René Gröschke
        added a comment -

        The snapshot policy doesn't work for our purposes, since in some scenarios I want the latest snapshot. We've organized some build jobs on our CI server in more than one multiproject build, so we use the snapshotTimeout 0. I've digged a bit into ivy and found a (dirty) workarround for this problem described in the comments of GRADLE-1049.

        Show
        René Gröschke
        added a comment - The snapshot policy doesn't work for our purposes, since in some scenarios I want the latest snapshot. We've organized some build jobs on our CI server in more than one multiproject build, so we use the snapshotTimeout 0. I've digged a bit into ivy and found a (dirty) workarround for this problem described in the comments of GRADLE-1049 .
        Hide
        Daz DeBoer
        added a comment - - edited

        For cases where snapshot modules can be cached, this problem is now fixed. However, in the cases where the most up-to-date version of the snapshot is required (cacheChangingModulesFor 0, 'seconds'), we still perform unnecessary downloads. This will be fixed when we:
        1) Only resolve a module once per build: this will fix the problem of multiple downloads within a single build
        2) Where hash values or etags can assist us, only download an artifact if it has changed since the previous download

        Show
        Daz DeBoer
        added a comment - - edited For cases where snapshot modules can be cached, this problem is now fixed. However, in the cases where the most up-to-date version of the snapshot is required (cacheChangingModulesFor 0, 'seconds'), we still perform unnecessary downloads. This will be fixed when we: 1) Only resolve a module once per build: this will fix the problem of multiple downloads within a single build 2) Where hash values or etags can assist us, only download an artifact if it has changed since the previous download
        Hide
        Daz DeBoer
        added a comment -

        This issue is now resolved (Milestone 8). For snapshots and other changing modules, we:
        1) Only resolve once per build: artifacts cached during the build are reused later in the build, even with cacheChangingModulesFor 0, 'seconds'
        2) Use the sha1 checksum of the existing snapshot to determine if the new one should be downloaded or not

        Show
        Daz DeBoer
        added a comment - This issue is now resolved (Milestone 8). For snapshots and other changing modules, we: 1) Only resolve once per build: artifacts cached during the build are reused later in the build, even with cacheChangingModulesFor 0, 'seconds' 2) Use the sha1 checksum of the existing snapshot to determine if the new one should be downloaded or not

          People

          • Assignee:
            Daz DeBoer
            Reporter:
            Jeppe Nejsum Madsen
          • Votes:
            5 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved: