[GRADLE-2034] Existence of pom file requires that declared artifacts can be found in the same repository Created: 06/Jan/12 Updated: 15/Sep/13 Resolved: 09/Sep/13 |
|
Status: | Resolved |
Project: | Gradle |
Affects Version/s: | None |
Fix Version/s: | 1.9-rc-1 |
Type: | Bug | ||
Reporter: | Luke Daley | Assignee: | Daz DeBoer |
Resolution: | Fixed | Votes: | 20 |
Issue Links: |
|
Description |
This is easy to reproduce with m7. Create a java project that pulls from mavenLocal(). Make sure hamcrest-core-1.1 is in your mavenLocal, rhen remove .m2/repository/org/hamcrest/hamcrest-core/1.1/hamcrest-core-1.1.jar. If you run the testClasses task you'll get:
12:07:12.343 [DEBUG] [org.gradle.api.internal.artifacts.ivyservice.ivyresolve.IvyResolverBackedArtifactToFileResolver] Attempting to download org.hamcrest#hamcrest-core;1.1!hamcrest-core.jar using resolvers [org.gradle.api.internal.artifacts.ivyservice.ivyresolve.DependencyResolverAdapter@1586b489]
12:07:12.343 [DEBUG] [org.gradle.api.internal.artifacts.ivyservice.IvyLoggingAdaper] tried /Users/ld/.m2/repository/org/hamcrest/hamcrest-core/1.1/hamcrest-core-1.1.jar
12:07:12.344 [DEBUG] [org.gradle.api.internal.artifacts.repositories.ResourceCollectionResolver] Loading /Users/ld/.m2/repository/org/hamcrest/hamcrest-core/1.1/hamcrest-core-1.1.jar
12:07:12.344 [DEBUG] [org.gradle.api.internal.artifacts.repositories.ResourceCollectionResolver] Resource not reachable for org/hamcrest#hamcrest-core;1.1: res=/Users/ld/.m2/repository/org/hamcrest/hamcrest-core/1.1/hamcrest-core-1.1.jar
If you remove ~/.m2/repository/org/hamcrest/hamcrest-core/1.1/ then everything will work (if you are also using mavenCentral()). Wild guess: when seeing if mavenLocal has the dependency, we are testing for /Users/ld/.m2/repository/org/hamcrest/hamcrest-core/1.1/ instead of the actual jar file. |
Comments |
Comment by Daz DeBoer [ 07/Jan/12 ] |
This is the expected behaviour: we treat repositories as self-contained, and only search for artifact files in the same repository that provided the metadata. I guess the issue is that we could make a better effort to try other repositories when one repository is found to be incomplete. |
Comment by Luke Daley [ 09/Jan/12 ] |
Hmm, I see. Tricky situation. It kinda seems like we should, but perhaps we should treat this as low priority. When I first encountered it I thought it might be a serious regression, but I guess this scenario is kind of edge. |
Comment by Adam Murdoch [ 09/Jan/12 ] |
It'll be hard to distinguish between a meta-data only module and a module that is missing the jar. Perhaps this should be part of our 'best' strategy, where a module with some artifacts is 'better' than one with the same version and no artifacts. With perhaps a warning that we're not using the first one we found. |
Comment by Mauro Molinari [ 10/Jan/12 ] |
This problem seems to be in some way related to what I've described here: I think Gradle should look at other repositories when it can't find something in the repository that provided the metadata, as a fallback option. Otherwise I really don't know how I could solve the problem I outlined in the forum topic above. |
Comment by Ryan Stewart [ 03/May/12 ] |
This definitely isn't an edge case, and it's still there in rc-2. It's fairly normal for a local Maven repo to have directories like this in it with only a POM and no associated artifact. I don't yet know a definite series of steps you can take to reproduce it, but if Maven doesn't have to download a jar, it won't download it. That doesn't mean that it never downloaded the POM in order to discover the full dependency tree, though. This issue is currently a problem for my team, where we have Maven and Gradle projects mixed together. At present, the only workaround we've found is to remove mavenLocal from our repository list so that it can always resolve artifacts from a remote repo, where the jar is never missing. More importantly, it means when we need to install a new SNAPSHOT locally, we have to either 1) locally modify the build script to add mavenLocal to the top of the list, or 2) deploy the new snapshot to the remote repo. Bottom line: this is just normal behavior for Maven, and it makes mavenLocal useless. |
Comment by Daz DeBoer [ 11/Jun/12 ] |
Comment by Ryan Stewart [ 13/Aug/12 ] |
I commented over in the thread mentioned above. A workaround is suggested, but it doesn't work for local maven repos. Is anybody paying attention to this issue, or is it effectively ignored? It's a really annoying problem and is one of the primary things holding back our adoption of gradle on a wider scale. |
Comment by Daz DeBoer [ 13/Aug/12 ] |
This issue is on the list to get fixed. It is not being ignored, but it is also not our highest priority. |
Comment by Ryan Stewart [ 13/Aug/12 ] |
Cool. Thanks for the reply. |
Comment by Kurt Harriger [ 03/Nov/12 ] |
Ran into this as well when working with mixed projects. I found that deleting .m2/repository seemed to resolve the issue but then it would return again later, I then discovered I could just comment out mavenLocal and it would build providing all the artifacts where uploaded to remote repo. Now that I know the issue and how to work around it not such a bit deal, but perhaps a short term improvement would be to add additional error messaging when using multiple repositories so that other users like me don't spend hours digging through dependency trees trying to isolate the problem only to have it start working then break again later next time you update a dependency. |
Comment by Andrey Myatlyuk [ 04/Mar/13 ] |
We have this issue and I still looking for the reason why artifacts are missing in .m2, do you build with Maven on the same node? |
Comment by Eric Pantera [ 03/May/13 ] |
Hi there. We were considering moving a 80 developers teams from Maven to Gradle, but this issue is a blocker point. It is just not acceptable to ask to everybody to remove artifacts from local repo or to do comment if necessary the mavenLocal(). |
Comment by Mauro Molinari [ 06/May/13 ] |
I agree. I follow Gradle development with much interest, but this is one of the most important fixes that is still missing from it |
Comment by Fedor Belov [ 06/May/13 ] |
It's so annoying to remove it from local maven repo again and again. Pls fix it! ps |
Comment by Ryan Stewart [ 06/May/13 ] |
@Fedor: To my knowledge, it's because Maven might need to read a pom in order to discover correct transitive dependencies, but then that specific artifact doesn't end up in the final dependency graph, so it doesn't need to download it. This still affects my team daily, and we've been dealing with it for a year now. It's even worse when you consider that Gradle still has no "local install" option besides borrowing the Maven plugin's capability, which forces you to use Maven local. <rant>How can this critical function (local artifact installation/resolution) still be missing from such an "evolved" build tool after more than a year?!?</rant> |
Comment by Andrey Myatlyuk [ 06/May/13 ] |
@Ryan @Fedor |
Comment by Fedor Belov [ 06/May/13 ] |
@Andrey I still think that maven is my default repository. I think there are a lot of people like me. You should fix it, because: |
Comment by Andrey Myatlyuk [ 06/May/13 ] |
@Fedor |
Comment by Danno Ferrin [ 06/May/13 ] |
I can say that this is one of the issues blocking my day job from switching any of our builds to Gradle. Peaceful coexistence with Maven is required. |
Comment by Fedor Belov [ 06/May/13 ] |
@Andrey as you can see this may be important for someone. I can't fix it because I din't have enough time and knowledge. But I can vote for this issue and wait it to be fixed |
Comment by Davide Cavestro [ 07/May/13 ] |
It's too easy to end up with a maven cache containing only poms for some GAVs: if you have mavenLocal as an additional repo, then there's no hope for you Seriously, I don't blame Gradle guys - they said it's on the list to get fixed - but simply give supporting evidence for this issue intricacy and importance. Please note IDE integration is sometimes affected by this issue in an underhand manner: i.e. I have several subprojects using mavenLocal in order to support independent installs of internal libraries snapshots. This morning I spent more than an hour with GGTS simply ignoring all dependencies on jdom: there was a pom with no jars for jdom in my .m2 cache, just because some days ago I used maven to build a plugin. |
Comment by Dave Syer [ 21/Aug/13 ] |
This looks like an Ivy problem. See https://issues.apache.org/jira/browse/IVY-1432 for a workaround. |
Comment by John Engelman [ 21/Aug/13 ] |
So this particular issue is causing some issues with our organization because we have some Gradle and some Maven projects building on the same CI server...also locally for developers, it's a serious nightmare. I've dug through the code and I understand where the issue is, but I'm just looking for some direction from the Gradle guys on how they think it should get fixed. Basically the issue is that in MavenResolver.java the isUsepoms() is used to check if the POM is present in the repository. If so, then it selects that repository to resolve the file. If MavenLocal() is your first repository, then it is selected and there is no mechanism for falling out of using that repository. I see 2 possible solutions: Option 1 would be the simplest solution, though it wouldn't be very clean since most people would start adding a URL to Maven Central to the MavenLocal repository. It seems like we could handle more transparently to the user. I'm hoping to have a pull request together this week, so any thoughts would be awesome. |
Comment by Mauro Molinari [ 22/Aug/13 ] |
Please note the problem is not only with MavenLocal. In our use case, we have dependencies on libraries whose POM is on Maven Central, but the JAR is missing because of licensing issues. We want to provide that JAR through a local flat repository. However, this currently does not work, because if Gradle finds the POM on the Maven Central, it searches for the JAR on there and, since it doesn't find it, it makes the build fail. Other similar use case: having the POM+JAR files on a repository, but the sources and/or Javadoc archives on a flat dir repository (because they are missing on the first one). |
Comment by John Engelman [ 01/Sep/13 ] |
@Mauro, I'm not quite sure where I'm understanding your issue. You mention a dependency where the POM is in Maven Central but the Jar isn't...You can resolve that issue by configuring an additionalUrl for the mavenCentral repository which accepts file URIs. |
Comment by Mauro Molinari [ 02/Sep/13 ] |
@John: I'm not sure I understood what you mean. Maybe you mean to use the artifactlUrls parameter of the mavenCentral DSL element. However, this would solve only part of my problem and would require us to set up a full Maven repository just to host JARs (all of them?). There are many cases:
If I understand the Gradle documentation properly, the artifaclUrls parameter of maven/mavenCentrl would help to solve just the first case. |
Comment by Adam Murdoch [ 09/Sep/13 ] |
This is fixed, but only for mavenLocal(). For all other repositories it is still considered an error for the artifact to be missing when the pom is present (unless packaging == 'pom') |
Comment by Mauro Molinari [ 10/Sep/13 ] |
So should we better open new issues for the other use cases? It is not possible anymore to open them from here, while I already opened a thread in the forums at http://forums.gradle.org/gradle/topics/making_gradle_find_a_jar_in_a_flatdir_repo_when_it_is_missing_from_a_maven_repo for my use case. |
Comment by Adam Murdoch [ 10/Sep/13 ] |
@Mauro, yes please. |
Comment by Osian Hughes [ 11/Sep/13 ] |
I see this was fixed yesterday, did it make it into the RC build released yesterday? if not, would it be possible to get it into RC2? Because another 3rd party framework we are using is now starting to use the local maven repo, and it is leaving this "pom only" structure all over the place in the local repo, so this fix would be very much appreciated. |
Comment by Mauro Molinari [ 12/Sep/13 ] |
@Adam: but how to do? I added a comment at http://forums.gradle.org/gradle/topics/making_gradle_find_a_jar_in_a_flatdir_repo_when_it_is_missing_from_a_maven_repo requesting the opening of a new issue. Or should I do in a different way? |
Comment by Adam Murdoch [ 15/Sep/13 ] |
@Mauro, that's fine. |