[GRADLE-2472] Maven snapshots are not being updated Created: 12/Sep/12  Updated: 10/Feb/17  Resolved: 10/Feb/17

Status: Resolved
Project: Gradle
Affects Version/s: None
Fix Version/s: None

Type: Bug
Reporter: Gradle Forums Assignee: Unassigned
Resolution: Won't Fix Votes: 11

Issue Links:
Related
Related to GRADLE-3512 Gradle daemon corrupts resources in s... Resolved

 Description   

Hi folks,
I use maven-style SNAPSHOT version numbering in my projects and deploy them to a Maven repository hosted in Artifactory. According to what I've read in a couple of other threads (listed below) Gradle should be treating these dependencies as changing modules and download from the remote repository whenever the SHA1 checksum in the remote repo is different from the checksum of the locally cached jar.

For me that's not happening. Its only when I add the --refresh-dependencies switch that the remote jar will be downloaded.

This was a reported bug quite some time ago but was marked as resolved, seemingly without a specific fix being made. GRADLE-629

It gets worse if I'm using a local Maven repo. If another member of the team updates a dependency and it gets deployed to the remote repo my local build will not pull it down. Gradle sees the local Maven repo at the beginning of the repository chain, finds a version that matches what it has in its local cache and thinks that there is nothing to download. Not even the --refresh-dependencies switch works in this case.

I need the local Maven repo to workaround the fact that Gradle doesn't publish to its own local cache. See GRADLE-1619. If mavenLocal() is not the first repository listed then I have a similar issue as before. The new jar in the local Maven repo will be ignored because Gradle will see that the jar in the remote repo is the same as it has in its local cache and it will stop checking at that point.

So I'm in a bit of a catch 22. Does anyone have any suggestions on to how to work effectively with snapshots? Is there a more Gradle-centric way of doing things?

I'm using Gradle version 1.1

[1]http://forums.gradle.org/gradle/topic...

[2]http://forums.gradle.org/gradle/topic...
----------------------------------------------------------------------------------------
[1] http://forums.gradle.org/gradle/topics/dependency_resolution_improvements
[2] http://forums.gradle.org/gradle/topics/dependency_resolution_in_gradle_1_0_milestone_8



 Comments   
Comment by Gradle Forums [ 12/Sep/12 ]

Changing modules are cached for 24 hours by default (same as in Maven). See [ResolutionStrategy]([1]http://gradle.org/docs/current/dsl/or...) for how to configure the TTL.

Gradle isn't currently capable of searching for the latest snapshot across repositories (e.g. remote and local). If you often need to update dependencies locally, one potential solution is to make them part of the same multi-project build.
----------------------------------------------------------------------------------------
[1] http://gradle.org/docs/current/dsl/org.gradle.api.artifacts.ResolutionStrategy.html

Comment by Gradle Forums [ 12/Sep/12 ]

Hi Peter,
Thanks for the response.

My understanding of multi-module builds is that all the projects are on the same release cycle. In other words, they all share the same version number, which they inherit from the master module, and therefore they all get released and tagged at the same time.

Most of my projects have independent release cycles and I want to be able to treat them as standalone modules for the purpose of checking out, branching, merging, tagging etc. I also don't want them to rely on a filesystem reference to any other project or build script. From the point of view of one project that is dependent on another, it just sees the other project as a classpath dependency.

So with that in mind, I don't think I can use multi-module features. Please correct me if I'm wrong.

Regarding ResolutionStrategy, I think I understand your point. It sounds like you're saying that newer snapshots in the remote repository are not supposed to be pulled down within the TTL period. That sounds fine to me and its definitely what I'm seeing. It just wasn't clear to me from Daz's posting that that was the intended behaviour. Having to use the --refresh-dependencies switch in order to force updates of cached dependencies is not a problem. However, when combined with a local Maven repo it won't work.

So I think I'm still stuck. Is there any way I can work on multiple projects locally and still get correct behaviour with regards to updating snapshots?

Is there any plan to be able to order snapshots by timestamp? I don't see how we can manage snapshots reliably without knowing which one is newest.

Comment by Gradle Forums [ 12/Sep/12 ]

I think the best you can do as of now is to conditionally include either the remote or the local Maven repo based on, say, the value of a system property. I hope we'll soon have a better solution but can't make any promises.

Comment by Kevin Stembridge [ 15/Sep/12 ]

I got the Jira number wrong in the original posting when referring to the local cache. Instead of GRADLE-1619 it should have been GRADLE-1615. Can someone with edit permissions please fix this? Thanks.

Comment by Sebastien Tardif [ 04/Oct/13 ]

If we could have the concept of be able to get the "more" recent artifact from a set of repos that would handle this. It's not clear that even a custom http://www.gradle.org/docs/current/dsl/org.gradle.api.artifacts.ResolutionStrategy.html could handle that.

Comment by Kevin Stembridge [ 24/Nov/13 ]

I'm happy to see new releases of Gradle coming out but I'm a little disappointed that this issue is being ignored. Improving the performance of Gradle is nice but this issue is a blocker for adoption on projects that have many independent multi-module projects. After spending a couple of years on projects using Gradle (which was great) I'm now back on a Maven project (not so great). I'd love to be able to recommend that we move to Gradle but due to this issue I can't honestly say that we would be better off.

I didn't get a response to my last posting on the forum. I've reposted it here because the original description in this jira doesn't adequately explain the problem.

I've followed your suggestion to conditionally include the local Maven repo. It gets me part of the way there but there is still a problem. First, this is what I've done:

repositories {
if (gradle.startParameter.refreshDependencies == false)

Unknown macro: { mavenLocal() }

maven

Unknown macro: { name 'Releases Repository' url 'http}

mavenCentral()
}
Using this setup, I can work on one project, install it locally and the other project will pick up those changes. Good so far. If another developer deploys a newer instance of the snapshot to the remote repository I can specify the --refresh-dependencies switch and it will download the jar and build against it. Good so far. If i then do another build without specifying --refresh-dependencies, my project will build against the obsolete version of the snapshot in my local Maven repo.

If I understand correctly, Gradle has no notion of chronological sequence with regard to snapshots in the way that Maven does. But that wouldn't necessarily be a problem if Gradle was able to publish to its own local cache.

I'm not a big fan of Maven (really dislike it actually) but their support for working with snapshots is worth emulating. Being able to work in that way has a big impact on the productivity of a development team. I'm interested in knowing how the Gradle team envisage this tool being used by developers who work across multiple projects that are not part of a multi-module build. (That's a genuine interest, not having a dig. )

Comment by Youri Bonnaffé [ 24/Nov/13 ]

This is for me one of the main drawback using Gradle. Especially when using Git you tend to have several repositories with several independent multi module projects instead of one big trunk with SVN.

Comment by Prasanna P [ 17/Mar/14 ]

Our company moved to gradle but was hit by this snag. Multiproject is not an option for us. It would be nice if there would be some solution to this, since we'd like to continue using Gradle.

Our workaround is to use timestamped versions (which is not a snapshot in maven concept, but similar), and it forces every developer to check for the latest timestamp before the build. This is not idea, but at least seems to be working.

Comment by Prasanna P [ 18/Mar/14 ]

Suggestion:

Can we have:
repositories {
mavenLocal(){
snapshot-remote

{ url "http://a.b.c" url "http://d.e.f" }

}
}

This is basically treating local maven as a special repo, where for changing versions (snapshots), it checks the timestamps for a given checksum with multiple urls configured

Comment by Mauro Molinari [ 01/Oct/14 ]

I think at least part of this is a duplicate of GRADLE-2257: I mean, the part regarding the resolution of dependencies over two different repositories (a remote one and a local one).

However, there's also a foundamental problem when you deal with just a Maven local repository: a SNAPSHOT module JAR does not get refreshed by Gradle even if it has changed in the local .m2 repo meanwhile. I tried the following:

  • adding resolutionStrategy.cacheDynamicVersionsFor and resolutionStrategy.cacheChangingModulesFor to my build script to reduce the cache exspiration time to just 60 seconds, but it doesn't work
  • add --refresh-dependencies Gradle command line argument, but it doesn't work either

The only workaround I found is to remove the dependency from the build script, perform an action, then re-add the dependency to the build script... after this, the updated JAR is finally retrieved from the local Maven repo.

This is extremely annoying when using Gradle inside an IDE (STS) to manage your project dependencies: refreshing dependencies becomes problem when dealing with "temporary" or "under development" modules coming from different (multi) projects.

Could this be finally fixed please? I'm currently using Gradle 2.1.

Comment by Steve Ash [ 21/Jul/16 ]

2016 and we're hitting this as well. How are other people working around this? The best that we have done is just not ever committing the mavenLocal() repo and if you as an individual developer need to work on multiple projects at the same time, then the dev uncomments the mavenLocal() and refreshes to pick up the local SNAPSHOT dependencies. Then before pushing, just re-comment it out.
This isn't perfect because if you have multiple projects in SNAPSHOT then you have to make sure that your local maven repo is up to date with installed dependencies.

Our use case is similar to the original poster: we have a few "core" libraries (company java commons, team commons, shared module across multiple products, etc.). All of these are versioned and released in different lifecycles and thus are separate projects. Each dev that works on these will have clones or forks of the git repos and will work locally then push their changes once complete to the CI server which will publish to our company artifactory snapshot repo. Eventually (before release) we will coordinate releases of all of the shared modules.

The maven behavior of pulling the "latest" SNAPSHOT (by timestamp) across repos (at least across local vs our company artifactory repo) is perfectly acceptable. Not perfect, but certainly better than the fragile solution of un-/re-commenting build config files. I thought we could get this behavior by doing something like declaring the dependency 3.6.0-SNAPSHOT+ or 3.6.-SNAPSHOT to make it a dynamic dependency (in addition to a changing dependency). But that doesn't work. I've also seen Peter N suggest using "latest.integration" but many of these core company libraries have multiple concurrent versions: 3.6.0-SNAPSHOT for the 3.6 line (against java7); 4.0.0-SNAPSHOT for the 4.x line (against java8). So we can't just always use the absolute latest we have to constrain that within a major/minor version: i.e. 3.6.-SNAPSHOT would be fine.

Comment by Benjamin Muschko [ 15/Nov/16 ]

As announced on the Gradle blog we are planning to completely migrate issues from JIRA to GitHub.

We intend to prioritize issues that are actionable and impactful while working more closely with the community. Many of our JIRA issues are inactionable or irrelevant. We would like to request your help to ensure we can appropriately prioritize JIRA issues you’ve contributed to.

Please confirm that you still advocate for your JIRA issue before December 10th, 2016 by:

  • Checking that your issues contain requisite context, impact, behaviors, and examples as described in our published guidelines.
  • Leave a comment on the JIRA issue or open a new GitHub issue confirming that the above is complete.

We look forward to collaborating with you more closely on GitHub. Thank you for your contribution to Gradle!

Comment by Benjamin Muschko [ 10/Feb/17 ]

Thanks again for reporting this issue. We haven't heard back from you after our inquiry from November 15th. We are closing this issue now. Please create an issue on GitHub if you still feel passionate about getting it resolved.

Generated at Wed Jun 30 12:22:51 CDT 2021 using Jira 8.4.2#804003-sha1:d21414fc212e3af190e92c2d2ac41299b89402cf.