[GRADLE-197] Allow the usage of ivy.xml and ivysettings.xml with Gradle Created: 29/Aug/08  Updated: 10/Feb/17  Resolved: 10/Feb/17

Status: Resolved
Project: Gradle
Affects Version/s: None
Fix Version/s: None

Type: Improvement
Reporter: Hans Dockter Assignee: Unassigned
Resolution: Won't Fix Votes: 45

Attachments: File import-ivy.gradle     XML File ivy.xml     XML File ivysettings.xml    

 Description   

It should be rather easy to make this possible. We should also offer to merge declaration from ivy files with dependency declarations in the gradle build file.



 Comments   
Comment by Helmut Denk [ 29/Aug/08 ]

implementation if 'config by ivysettings.xml' should have priority IMO.

Comment by Willem Verstraeten [ 02/Jul/09 ]

Is there a target version for this ? Being able to have Ivy files would allow me to reuse the Ivy dependency information in the build process as well as in the IDE's of our developers, which is quite an important requirement for us.

Comment by Willem Verstraeten [ 02/Jul/09 ]

Alternatively having gradle support in our IDE's (in the sense that the IDE's can get their module configurations from the gradle build scripts) would also be acceptable, but then IntelliJ as well as Eclipse should be supported. I think it's easier if you can parse the ivy.xml files in gradle and let the existing IDE-Ivy plugins (ie. IvyIDEA and IvyDE) handle the IDE configuration.

Comment by Hans Dockter [ 02/Jul/09 ]

We might do it step-by-step. What would be very easy to implement is an Ivy-plugin that allows you to generate an ivy.xml and ivysetting.xml file from the Gradle dependency information. Gradle does this anyhow internally. The other way round is more difficult but has also a pretty high priority to use. I hope we can ship a first version of the Ivy plugin in 0.8.

Comment by Uldis Karlovs-Karlovskis [ 23/Oct/09 ]

For me highest priority is to have ivy.xml per module of multi-project build.

Comment by Barry MacMahon [ 05/Nov/09 ]

This feature would speed up or enable the adoption of gradle by companies who have an ivy based build in and also by allowing integration with IDEs via ivy plugins as mentioned above.

Comment by Richard Vowles [ 17/Aug/10 ]

What else is critical is to have the option to override the dependencies that are in the Gradle build file from the ivy.xml settings (and hopefully ivysettings.xml as well). This is important for a reproduceability of build reason - you need to know exactly what versions of what artifacts were resolved when the module was built and tagged and released. If you cannot rebuild the exact same release with the exact same versions of your dependencies, you don't have the same build.

I say the ivysettings.xml as well, as some of us have different version range resolvers for compile scope vs run scope. That said, the resolvers themselves won't change in the Gradle build file, so it is less important.

Comment by Carl Quinn [ 17/Nov/10 ]

For me and my company, I don't think we can adopt Gradle until it can read ivy.xml files for module dependencies, for two reasons:

  • for migration sanity: I will need to support our old Ant-based system along side Gradle. We have 300+ little projects that would be migrated, and they can't all go perfectly in one shot.
  • for IDE compatibility: The IntelliJ and Eclipse Ivy plugins are functional enough for us and part of our regular workflow. I wouldn't want to foce developers to take a step back.

The ivysettings.xml is less of an issue for us as that is a single file that can be manually maintained in parallel with a Gradle file.

Comment by Qamar mahmood [ 17/Nov/10 ]

This feature would aid adoption of gradle.

Comment by Hans Dockter [ 18/Nov/10 ]

@Carl I absolutely see your point. But usually there are reasonable, pretty easy, work arounds. This issue here is for a generic import of ivy.xml's including the corner cases. We plan to solve this as part of a 1.0 milestone when we add further functionality to our dependency DSL. But for most enterprise use cases, where the structure of the ivy.xml is not using every degree of freedom Ivy allows, it is easy to write your own adapter which imports the ivy.xml into Gradle at runtime. We did this for a number of builds and it worked fine. We are talking about something like 30 lines of code. We did this for example as a POC for the Ant build of the spring core framework.

BTW: I'm confident that IntelliJ and Eclipse will soon be able to understand build.gradle files. With the adapter approach, you then have the choice. Another thing we should and will provide is a script that coverts ivy.xml into build.gradle files.

Comment by Qamar mahmood [ 18/Nov/10 ]

Thanks Hans.
Please can you post some sample code for the adapter to allow import of ivy.xml into Gradle at runtime.

Comment by Martin Kacer [ 03/Jan/11 ]

I absolutely agree with Carl. We would like to adopt Gradle but need to solve this issue for the same two reasons that Carl described.

Hans, would it be possible to see some example of those "30 lines of code"?
I believe I would be able to tailor the adapter for our needs but it is difficult to write one from scratch.

Comment by Davide Cavestro [ 19/Apr/11 ]

At my company we would migrate our old ant style build scripts (with no deps management) to gradle, but we need real IDE support to reflect dynamically resolved dependencies.
Ivy integration is good on IDEs, while I haven't found yet a way to share gradle dependencies to the IDE (generating i.e. eclipse project artifacts is really not the same thing).
So, the optimum for us would be having the IDE understand gradle dependencies (something like IvyDE).
That said, we could fall back to use ivy descriptors into gradle scripts, but because of this issue, we should duplicate our dependencies declaration writing them both in gradle and also in Ivy descriptors: that makes the migration not acceptable.
The alternative would be to write the adapter Hans mentioned above, but the fact that those "30 lines of code" example is not yet available sounds somewhat strange...

PS: Please beware that the old copy of this issue has more votes than this one. I think it is due to the fact that people can still vote there and don't know about the new issue tracker (or at least I didn't know till some minutes ago and started voting there).

Comment by Hans Dockter [ 01/May/11 ]

Finally I have attached an import snippet which I used as a proof of concept for the spring-core build.

Comment by Davide Cavestro [ 02/May/11 ]

Hi Hans,
I've just easily adapted your snippet for my GWT project: it works like a charm!
For other people interested on these details, this is the adapted snippet:

build.gradle
import org.apache.ivy.core.settings.IvySettings
import org.apache.ivy.core.module.descriptor.DefaultModuleDescriptor
import org.apache.ivy.plugins.parser.xml.XmlModuleDescriptorParser
import org.apache.ivy.core.module.descriptor.DependencyDescriptor
import org.gradle.api.internal.artifacts.dependencies.DefaultExternalModuleDependency
import org.apache.ivy.core.module.id.ModuleRevisionId

//...

configurations {
    compile
    runtime.extendsFrom compile
    testCompile.extendsFrom runtime
}

def rootProjectConfigurations = [
    compile: configurations.compile,
    runtime: configurations.runtime,
    test: configurations.testCompile]

IvySettings ivySettings = new IvySettings();
ivySettings.load(new File("ivysettings.xml"));

DefaultModuleDescriptor moduleDescriptor = (DefaultModuleDescriptor) XmlModuleDescriptorParser.getInstance().parseDescriptor(ivySettings, new File("ivy.xml").toURL(), false);
moduleDescriptor.getDependencies().each { DependencyDescriptor descriptor ->
    def mappableConfiguration = descriptor.moduleConfigurations.find { moduleConf ->
        rootProjectConfigurations.containsKey(moduleConf)
    }
    
    if (mappableConfiguration) {
        ModuleRevisionId id = descriptor.dependencyRevisionId
        rootProjectConfigurations[mappableConfiguration].addDependency(
            new DefaultExternalModuleDependency(id.organisation, id.name, id.revision, descriptor.getDependencyConfigurations(mappableConfiguration)[0])
        )
    }
}

repositories {
    mavenCentral()
    ivySettings.getResolvers ().each {
        add(it)
    }
}
//...

Thanks a lot to Hans.

Cheers
Davide

UPDATE 20110519: I noticed that directly calling

rootProjectConfigurations[mappableConfiguration].addDependency(
            new DefaultExternalModuleDependency(id.organisation, id.name, id.revision, descriptor.getDependencyConfigurations(mappableConfiguration)[0])

looses the changing attribute value, thus missing to get new version for snapshot deps marked as changing=true into ivy.xml, so it should be converted to

DefaultExternalModuleDependency dep = new DefaultExternalModuleDependency(id.organisation, id.name, id.revision, descriptor.getDependencyConfigurations(mappableConfiguration)[0])
dep.setChanging (descriptor.changing)
rootProjectConfigurations[mappableConfiguration].addDependency(dep)
Comment by Martin Kacer [ 10/May/11 ]

Thanks, Hans.
I am among those who did not know about the Jira transfer to a new server.
Not getting an answer on codehaus, I somehow managed to write my own script yesterday, similar to the one above.
Today, I found this new thread. And I will be able to use it to make my script better and shorter.

Comment by Hans Dockter [ 10/May/11 ]

@Martin So you did not get any update emails for this issue from issues.gradle.org? Haven you been watching this issue on codehaus? Or just voted?

Comment by Leonard Brünings [ 19/May/11 ]

@Davide how would you get it to work with the default configuration?

Comment by Davide Cavestro [ 19/May/11 ]

@Leonard I'm not sure I understand your question: what do you mean with default configuration?
As per my previous post I modified my gradle script to get dependencies from ivy resolvers and then merged them into the gradle standard compile classpath, and that's all:

sourceSets {
   main {
      compileClasspath = configurations.compile + configurations.sdk //the latter is only for gwt
   }
   test {
      compileClasspath = compileClasspath + configurations.sdk
   }
}

But maybe you would mean something else...

Comment by Leonard Brünings [ 20/May/11 ]

@Davide Yes I mean something else. In ivy you don't have to define a configuration like 'compile' or 'runtime' and if you don't the dependencies will get added to the 'default' configuration. My question is how can you adapt this script to work with the 'default' configuration.

And could you please post the minimal ivy.xml and ivysettings.xml to get it to work with this script. I keep getting the error
"Could not resolve all dependencies for configuration ':compile'.
Cause: Unexpected cache manager default-cache for repository xyz"
but using IvyDE in eclipse it works fine.

Comment by Davide Cavestro [ 20/May/11 ]

@Leonard I declared ad-hoc named configurations on my ivy.xml and ivysettings.xml. I followed this example cause I have to produce gwt libraries.

I've never used ivy default configuration directly from gradle, but I'd try adding to the compiler classpath that configuration as any other of interest with code similar to

sourceSets {
   main {
       compileClasspath = configurations.default
   }

PS: You'll notice some variables into xml files: I put them there when I had to duplicate deps between ivy descriptors (for IDE support) and the gradle script (the matter this issue solved). Now I could directly write most of them directly into ivy descriptors instead of reading them from gradle.properties.

Comment by Carl Quinn [ 15/Jun/11 ]

Just starting to try this out now using snippets from Hans and Davide (thanks guys). I think this is going to force me to learn the inner workings of Gradle on Ivy

First thing I hit is that my resolvers are all instances of: org.apache.ivy.plugins.resolver.URLResolver, ChainResolver, and FileSystemResolver, and after they are added to the resolvers collection, their cache property is null. I think Ivy would then just use the default cache, but Gradle is having trouble with this:

Cause: Unexpected cache manager default-cache for repository nfrepo-everything (nfrepo-everything)

If anyone has some hints that would be great. I'm thinking that I'll need to pass more settings down from the local ivySettings to Gradle.

Comment by Carl Quinn [ 15/Jun/11 ]

So then the next big question is: how can I configure Gradle's underlying Ivy settings to be exactly like my Ant's Ivy settings?

I don't mind writing DSL or Groovy to mimic what I do in ivysettings.xml, but I am having a hard time finding how to get at the general bits other than the resolvers. And even those seem to be specialized in Gradle and may not match what I have in my Ivy.

Comment by Szczepan Faber [ 21/Jun/11 ]

Hey Carl,

>Cause: Unexpected cache manager default-cache for repository nfrepo-everything (nfrepo-everything)

For consistency reasons, gradle requires all remote resolvers to share the same instance of the repository cache manager. Hence this exception. My understanding is that you're initiating resolvers from the ivy xml files? I'd suggest that your script should make sure that the for each remote resolver the repository cache manager is null (that means it will use the 'global' default repository cache manager.

>So then the next big question is: how can I configure Gradle's underlying Ivy settings to be exactly like my Ant's Ivy settings?

It may depend on the complexity of your ivy configuration. At this moment I can think of 2 ivy features not supported by gradle: remote resolvers with separate repository cache managers (see above) and this GRADLE-1583. However, it does not mean that you cannot migrate your ivy project setup into gradle! I means that certain complex ivy configuration will have a different, possibly simpler implementation in Gradle.

Let me know how we can help more!

Comment by Carl Quinn [ 06/Jul/11 ]

I have gotten past my resolver issues, and mostly using GradleIBiblioResolver. I am not sure what I had wrong before, but creating various resolvers from scratch and then adding them to the repositories containers works fine for me now.

The next question is regarding some of the other settings that are generally used in ivysettings.xml:

  • Ivy repositoryCacheDir (per workspace) and resolutionCacheDir (per project)
  • The circularDependencyStrategy: 'warn' is a fine default, but it would be nice to set 'error' for strictness.
  • It might be nice to set defaultLatestStrategy and defaultConflictManager, but I'm happy to work with Gradle implementations if that makes more sense.
  • How do the status enums get defined? We use a Ivy/Mavin hybrid of snapshot, candidate and release.
Comment by Leonard Brünings [ 08/Jul/11 ]

I extended the snippet to include transitive, artifacts, and exclude settings.
There are still some things not mapped from the ivy.xml but this is another step
in the right direction.

import org.apache.ivy.core.module.descriptor.DependencyArtifactDescriptor
import org.gradle.api.internal.artifacts.dependencies.DefaultExternalModuleDependency
import org.gradle.api.internal.artifacts.dependencies.DefaultDependencyArtifact;
import org.gradle.api.internal.artifacts.DefaultExcludeRule;

//...//

moduleDescriptor.getDependencies().each { DependencyDescriptor descriptor ->
    def mappableConfiguration = descriptor.moduleConfigurations.find { moduleConf ->
        rootProjectConfigurations.containsKey(moduleConf)
    }
    
    if (mappableConfiguration) {
        ModuleRevisionId id = descriptor.dependencyRevisionId
        DefaultExternalModuleDependency dep = new DefaultExternalModuleDependency(id.organisation, id.name, id.revision, descriptor.getDependencyConfigurations(mappableConfiguration)[0]);
		dep.setChanging (descriptor.changing)
		dep.setTransitive(descriptor.transitive)
		
		descriptor.getAllDependencyArtifacts().each { DependencyArtifactDescriptor depArt ->
			dep.addArtifact(new DefaultDependencyArtifact (depArt.name, depArt.type, depArt.ext, null, depArt.url));
		}
		
		def excRuleContainer = dep.excludeRules;
		descriptor.excludeRules?.values().each { def ruleList ->
			ruleList.each { def rule ->
				excRuleContainer.add(new DefaultExcludeRule(rule.attributes))
			}			
		}
		rootProjectConfigurations[mappableConfiguration].addDependency(dep)
	}
}
Comment by Rodion Moiseev [ 17/Aug/11 ]

Could anyone please describe the exact steps needed to integrate the above script into a simple project (with a single build.gradle, for instance). Preferably for the 1.0-milestone-3 or above.

I have tried using apply from: 'import-ivy.gradle' and copying the contents to ../build.gradle but neither approach works. Well, it half-works: I get appropriate output from dependencies task, but running compileJava or eclipse seems to behave as if no dependencies were resolved (e.g. I get an empty .classpath file).

Is there a certain order in which this script, 'java' plugin and 'eclipse' plugin have to be evaluated?

Update:

I have experimented and found the following:

1. Doing project.configurations.myConf.addDependency(new DefaultExternalModuleDependency(...)) does not work
2. Doing project.dependencies.add("myConf", new DefaultExternalModuleDependency(...)) does not work
3. Doing project.dependencies.add("myConf", "${id.organisation}:${id.name}:${id.revision}") works!

I am guessing DefaultExternalModuleDependency is not being initialised correctly, but not sure what exactly is the problem.

No one else having issues with this??

Comment by Blaine Simpson [ 18/Nov/11 ]

I just published Ivyxml Plugin, that loads Ivy dependency files. The code snippets here got me started, but it's been much broadened and hardened since.

It does not load ivysettings files, only ivy.xml files.

Comment by Szczepan Faber [ 20/Nov/11 ]

@Blaine, thanks for the plugin and the reference!

Comment by Chad Walz [ 08/Dec/11 ]

Thanks for addressing this and placing a plugin out there. After going through the readme here: https://github.com/unsaved/gradle-ivyxml-plugin/blob/master/README.txt, I'm failing to see what a proper ['confName'] will be, so my result is a BUILD FAILED "Cause: Configuration with name 'confName' not found."

Comment by Blaine Simpson [ 08/Dec/11 ]

Chad: If you have any other issue with the plugin, please register an... Issue for it.

You can use any Gradle Configuration name there, but it would demonstrate something useful regarding Ivy configuration if you entered an Ivy conf name (which will automatically mirror a Gradle Configuration name).

Comment by Blaine Simpson [ 19/Oct/14 ]

I updated my Ivyxml plugin to support Gradle 2.x and have registered it on the Plugin Portal

Comment by Benjamin Muschko [ 15/Nov/16 ]

As announced on the Gradle blog we are planning to completely migrate issues from JIRA to GitHub.

We intend to prioritize issues that are actionable and impactful while working more closely with the community. Many of our JIRA issues are inactionable or irrelevant. We would like to request your help to ensure we can appropriately prioritize JIRA issues you’ve contributed to.

Please confirm that you still advocate for your JIRA issue before December 10th, 2016 by:

  • Checking that your issues contain requisite context, impact, behaviors, and examples as described in our published guidelines.
  • Leave a comment on the JIRA issue or open a new GitHub issue confirming that the above is complete.

We look forward to collaborating with you more closely on GitHub. Thank you for your contribution to Gradle!

Comment by Benjamin Muschko [ 10/Feb/17 ]

Thanks again for reporting this issue. We haven't heard back from you after our inquiry from November 15th. We are closing this issue now. Please create an issue on GitHub if you still feel passionate about getting it resolved.

Generated at Wed Jun 30 11:24:07 CDT 2021 using Jira 8.4.2#804003-sha1:d21414fc212e3af190e92c2d2ac41299b89402cf.