[GRADLE-1663] Gradle should provide an option to perform an Ivy deliver when producing the uploaded Ivy file for a module Created: 05/Jul/11  Updated: 10/Feb/17  Resolved: 10/Feb/17

Status: Resolved
Project: Gradle
Affects Version/s: 1.0-milestone-3
Fix Version/s: None

Type: Bug
Reporter: Carl Quinn Assignee: Unassigned
Resolution: Won't Fix Votes: 16

Attachments: Zip Archive GradleIvyPlugin.zip    

 Description   

I am migrating an Ant+Ivy build system to Gradle, and in my experiments, my uploads are uploading ivy files that are not resolved. That is, the orignal dynamic rev values are still in place for the dependencies. What I would expect (and what Ivy does during deliver) is that the rev value moves to revConstraint, and then rev gets the static revision value.

I'm currently seeing this with my little experiments, and using the Ivy URLResolver and the GradleIBiblioResolver with the same results.

Gradle should provide an option for producing resolved Ivy files with static rev's during regular upload tasks.



 Comments   
Comment by Martin Kacer [ 31/Oct/11 ]

Yes, please! This currently seems as a stopper for us.

Any workaround? E.g., a hook that could modify the content of the ivy.xml before it gets published?

Comment by Carl Quinn [ 07/Mar/12 ]

Can we bump the priority on this one up to Critical? I don't think Gradle should go to 1.0 without it being able to perform this basic Ivy function.

Comment by Stephane Gallès [ 07/Mar/12 ]

Yes, please, that would be great. In our migration from Ant+Ivy to Gradle we had to ask our teams to stop using the dynamic revs because each time someone accidentally pushed a dynamic dependency to our repos, it was the chaos in the transitive deps, and the build results on the CI server were not reproducible anymore.

Comment by Luk Morbee [ 08/Mar/12 ]

We wrote an Ivy-plugin for Gradle 0.5.2 which does resolve dynamic revisions. We would like to upgrade from Gradle 0.5.2 to 1.0, but if the 1.0 release doesn't resolve dynamic revisions, this would definitely be a showstopper. Therefore, yes please, can we bump the priority on this one to critical?

Comment by Stephane Gallès [ 23/Mar/12 ]

I'd like to create a plugin to implement a workaround for this, but before I go any further I'd like to know if what I want to try is theorically feasible with the current API because it is not a slam dunk AFAICT.

My plugin would :

  • detect when an Upload task is in the DAG (else stop)
  • Traverse all configurations and dependencies to detect any dynamic dependency (else stop)
  • copy all configurations with dynamic dependencies to temporary configurations (no need to add to 'configurations'), and resolve them.
  • Traverse these resolved configurations and find the real revisions
  • Mutate the original dynamic revision with the real revision

Would this work ?

Comment by Vinod Sajja [ 23/Mar/12 ]

Yes it would, we are doing something similar.

Comment by Adam Murdoch [ 26/Mar/12 ]

@Stephane, a better approach would be to change the ivy.xml generation to use the actual resolved value from the configuration. Each Configuration has an associated ResolvedConfiguration object, which will tell you which version was actually used.

Comment by Stephane Gallès [ 27/Mar/12 ]

@Vinod, thank you for this information.

@Adam, do you mean that there is an API that allows me to modify the ivy.xml before publication (to create a plugin)? Where this API would be located ? Or do you mean that a better approach would be to modify the code of Gradle itself ? Thank you.

Comment by Adam Murdoch [ 27/Mar/12 ]

@Stephane, there's no such API yet, but we want to add one. There is currently an API for modifying the generated pom, but there's no equivalent for modifying the ivy.xml.

Our plan for supporting an ivy deliver equivalent is (roughly):

  • Split out ivy.xml and pom.xml generation into stand-alone tasks.
  • Add hooks to these tasks to allow you to influence the xml generation.
  • Add some option (possibly as a plugin, or maybe as a dsl element) which will tweak these tasks to use the actual resolved dependencies instead of the requested dependencies. It will do this for both ivy.xml and pom.xml generation.

I think we'll end up with a better result from this approach, than we will with duplicating the configurations. If you feel like helping out with this, then that would be excellent (we should move this discussion to the dev mailing list, if this is the case). If you want to go ahead with an external plugin, that is also cool. Either way, at some point soon-ish, we'll start with the above plan.

Comment by Chris Hilton [ 11/Jul/13 ]

Is there any update here? This is pretty critical to having usable published Ivy files.

Comment by Daz DeBoer [ 15/Jul/13 ]

@Chris : it's now possible to perform an 'ivy deliver' when using the new 'ivy-publish' plugin. You'll need to code a little logic into `descriptor.withXml`, but it's pretty trivial. There may be plugins out there that provide this functionality directly on top of 'ivy-publish', but I'm not aware of any public ones.

It's unlikely that we'll be retrofitting this functionality into the Gradle 1.0 publishing API.

Comment by Adam Murdoch [ 16/Jul/13 ]

@Chris, we'd welcome a contribution to add this, if you (or any one else) is interested in helping out.

Comment by Chris Hilton [ 17/Jul/13 ]

Interested, but it looks like it would take a lot of digging for us to understand the code and we have a relatively easy, if unsatisfactory, workaround. We'll be running the dependencies task to get the resolved versions and then, after uploading artifacts, hack the versions into the ivy.xml file and re-upload it.

Comment by Adam Murdoch [ 17/Jul/13 ]

@Chris, the first step would be to package up your work-around into an `ivy-deliver` plugin and add an integration test or two and some words in the user guide. It doesn't need to be deeply integrated at all. This would already be useful for a bunch of people (including yourself).

Once that is done, we can look at subsequent steps to integrate this more deeply, and also make it available for Maven publications. If you want to be involved in later steps, great. If not, that's also completely fine.

So, in theory at least, you don't need any more knowledge of the details of Gradle than you already have.

A disclaimer: I'm assuming you're using the new Ivy-publish plugin and the withXml hook in your workaround.

Comment by Chris Hilton [ 18/Jul/13 ]

Unfortunately not as it looks to be beyond our limited Gradle expertise. If there's an easy way to access the resolved versions of dependencies within Gradle, that might change our minds.

Comment by Justin Ryan [ 18/Jul/13 ]

This is sorta what I do in my resolved ivy plugin, there more which I had to cut out since it's company specific, but this should be a starting place:

        project.getPlugins().apply(PublishingPlugin)
        project.getExtensions().configure(PublishingExtension, new Action<PublishingExtension>() {
            @Override
            void execute(PublishingExtension pubExt) {
                pubExt.publications.withType(IvyPublication) {
                    descriptor {
                        withXml {
                            def perConfigResolutionMap = project.configurations.collectEntries { conf ->
                                ResolutionResult resolution = conf.incoming.resolutionResult // Forces resolve of configuration
                                def resolutionMap = resolution.getAllModuleVersions().collectEntries { ResolvedModuleVersionResult versionResult ->
                                    [versionResult.id.module, versionResult]
                                }
                                [conf.name, resolutionMap]
                            }
                            md?.dependencies?.dependency.each { dep ->
                                def id = new DefaultModuleIdentifier(dep.@org, dep.@name)
                                def confs = extractConfs(dep.@conf)
                                def results = confs.collect { perConfigResolutionMap[it][id] }.findAll { it != null }
                                def versionMap = results.collectEntries { [VersionFactory.create(it.id.version), it] }
                                def versionMax = versionMap.keySet().max()
                                def version = versionMap[versionMax]
                                def oldVersion = dep.@rev

                                dep.@revConstraint = oldVersion
                                if (version != null) {
                                    dep.@rev = version.id.version
                                }
                            }
                        }
                   }
               }
          }
      }
Comment by Adam Murdoch [ 18/Jul/13 ]

@Justin, this is exactly the sort of thing we need. We can just copy this into a plugin, add an int test or two and a quick mention in the user guide, and we have Ivy deliver. We can make it better (eg don't mess with the xml directly) later.

Comment by Luk Morbee [ 19/Jul/13 ]

I attached our Ivy-plugin (compatible with Gradle 1.1).

example.gradle
buildscript {
  repositories {
    ivy {
      url     'http://ivy.example.com/'
      layout  'pattern', {
        ivy      "releases/[organisation]/[module]/[revision]/ivy.xml"
        artifact "releases/[organisation]/[module]/[revision]/[type]s/[artifact].[ext]"
      }
    }
  }
  dependencies {
    classpath group: 'com.compumark', name: 'gradle-ivy-plugin', version: '2.0.0'
  }
}

apply plugin: com.compumark.gradle.plugin.ivy.IvyPlugin

ext.ivySettingsURL = 'http://ivy.example.com/ivysettings_http.xml'

Comment by Justin Ryan [ 19/Jul/13 ]

@Adam, I'd do it myself but my laptop is out of commission and my new one isn't coming for a day or two, hence I'm web-browser based right now. It already exists as a plugin, and I can look at contributing it back next week. It probably should be a stand-alone plugin, but as you it's a lot of work to get a single stand-alone plugin out the door.

@Luk Can you put that in a gist or github, and attach a license to it?

Comment by Luk Morbee [ 22/Jul/13 ]

Hi Justin,

Our company policy is a bit stiff about licensing. Therefore please use this code as sample code to create a new Ivy-plugin. Being an early adopter of GRADLE, this is my way to give something back but my hands are pretty much tied.

Comment by Benjamin Muschko [ 15/Nov/16 ]

As announced on the Gradle blog we are planning to completely migrate issues from JIRA to GitHub.

We intend to prioritize issues that are actionable and impactful while working more closely with the community. Many of our JIRA issues are inactionable or irrelevant. We would like to request your help to ensure we can appropriately prioritize JIRA issues you’ve contributed to.

Please confirm that you still advocate for your JIRA issue before December 10th, 2016 by:

  • Checking that your issues contain requisite context, impact, behaviors, and examples as described in our published guidelines.
  • Leave a comment on the JIRA issue or open a new GitHub issue confirming that the above is complete.

We look forward to collaborating with you more closely on GitHub. Thank you for your contribution to Gradle!

Comment by Benjamin Muschko [ 10/Feb/17 ]

Thanks again for reporting this issue. We haven't heard back from you after our inquiry from November 15th. We are closing this issue now. Please create an issue on GitHub if you still feel passionate about getting it resolved.

Generated at Wed Jun 30 12:01:16 CDT 2021 using Jira 8.4.2#804003-sha1:d21414fc212e3af190e92c2d2ac41299b89402cf.