[GRADLE-2123] WTP Plugin should add org.eclipse.jst.component.dependency to all WEB-INF/lib deps and not add them to the WTP component file Created: 23/Feb/12  Updated: 25/Jan/17  Resolved: 03/Jun/16

Status: Resolved
Project: Gradle
Affects Version/s: 1.0-milestone-7, 1.0-milestone-8
Fix Version/s: 3.0-milestone-1, 3.0-rc-2

Type: Bug
Reporter: Andrew Oberstar Assignee: Donát Csikós
Resolution: Fixed Votes: 5

Attachments: Zip Archive Test multiproject for GRADLE-2123.zip    

 Description   

Currently Gradle is generating classpath entries like this for JAR dependencies in the WAR:

<classpathentry kind="lib" path="C:\Documents and Settings\it403\.gradle\caches\artifacts\ch.qos.logback\logback-core\ef28c670c723aafc18a1f247a00c89bb\jars\logback-core-0.9.29.jar" exported="true" />

It should be generating:

<classpathentry kind="lib" path="C:\Documents and Settings\it403\.gradle\caches\artifacts\ch.qos.logback\logback-core\ef28c670c723aafc18a1f247a00c89bb\jars\logback-core-0.9.29.jar" exported="true">
  <attributes>
    <attribute name="org.eclipse.jst.component.dependency" value="WEB-INF/lib"/>
  </attributes>
</classpath>

And once it starts using "org.eclipse.jst.component.dependency", the entries from the WTP component file should be removed. Without removing them the JARs may be included in the classloader twice when using a server within Eclipse.

<dependent-module deploy-path="/WEB-INF/lib" handle="module:/classpath/lib/C:\Documents and Settings\it403\.gradle\caches\artifacts\ch.qos.logback\logback-core\ef28c670c723aafc18a1f247a00c89bb\jars\logback-core-0.9.29.jar">
  <dependency-type>uses</dependency-type>
</dependent-module>

This seems related to a bunch of issues that have been reported (and some of them fixed before):
http://issues.gradle.org/browse/GRADLE-1134
http://issues.gradle.org/browse/GRADLE-1275
http://issues.gradle.org/browse/GRADLE-1412
http://issues.gradle.org/browse/GRADLE-1707

This Maven issue was linked from one of them, which recommends the approach I documented above:
http://jira.codehaus.org/browse/MECLIPSE-264

Bear in mind that one difference in my use case is that we are using SelfResolvingDependencies. I don't know if these are handled differently in the WTP plugin than the module dependencies you would get from using a Maven/Ivy repo.

Even if that is the issue, they should be handled consistently.



 Comments   
Comment by Szczepan Faber [ 10/Mar/12 ]

Hello,

Thanks a lot for this breakdown!

1. Is it safe to remove dependent-module from the wb component? The other linked issues don't mention that it should be removed.
2. Is it always "WEB-INF/lib" at <attribute name="org.eclipse.jst.component.dependency" value="WEB-INF/lib"/>? What about ear archives? Some other linked issues use a different value, e.g. value="../".

I'm not an expert on the wtp stuff so I'm leaning on you how this should work

Comment by Andrew Oberstar [ 12/Mar/12 ]

One additional requirement... Any entries in the WAR's classpath that aren't going to WEB-INF/lib (i.e. providedRuntime) should have the following attribute instead:
<attribute name="org.eclipse.jst.component.nodependency" value=""/>

Now for your questions:

1. If you look at this comment in the Maven issue, he says it can be removed from WtpComponent. In my tests, this has also been the case.

http://jira.codehaus.org/browse/MECLIPSE-264?focusedCommentId=103194&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-103194

The initial issue I ran into was warnings on all dependencies in the WAR. The warning said that the dependencies weren't going to be exported or published. The issue was that while the WtpComponent was directing those dependencies to be assembled into the WEB-INF/lib folder, the separate entry in the Classpath was not being directed anywhere. That's where the jst.component.dependency comes in. But once you have that defined in both places, Eclipse is essentially being told to copy the same JAR twice. In most cases that isn't a practical issue, because they would overwrite each other. However, there are cases when deploying to a server managed by Eclipse that the same JAR ends up on your classpath twice. That's why it needs to be removed from WtpComponent.

2. Now you're getting into the stuff I'm fuzzy on; I'm no Eclipse expert either.

In the EAR case I don't know that we need to do anything different. I'm not sure what the correct approach is, but continuing to use dependent-modules does not seem to cause any issues. I assume that's because the dependencies aren't actually listed in the .classpath right now. You'd have to talk to someone who understands WTP better to be sure.

As for the value of "WEB-INF/lib", the issues that reference the "../" are referring to classpath entries in other projects that the WAR depends on. That is to say that if my WAR depends on another JAR project its dependencies should be included the same directory as the JAR. This issue is purely regarding the dependencies declared on the WAR.

I believe all "WEB-INF/lib" dependencies in the WAR do require the value to be "WEB-INF/lib". This is the same value mentioned in the Maven issue.

Comment by Szczepan Faber [ 14/Mar/12 ]

I did some research in the area and I have a different solution to the problem, via the 'web app libraries' classpath container.

After adding the 'web app libraries' container, the dependencies are managed automatically via the container. This means that there is no reason to keep dependencies explicitly in .classpath. They will be automatically managed - provided the jars/modules are still configured in the WTP component file.

Configuring the 'web libraries container' adds value on it's own - see GRADLE-1974.

So my current plan is: when 'eclipse-wtp' and 'war' plugins combo is applied I'm adding the 'web app libraries' container by default. I'm also removing explicit classpath entries from .classpath. The only thing I can imagine we lose with that approach is that it no longer is possible to selectively decide whether given dependency is exported or not. You can only export/unexport the whole container. I don't think this is a big issue, though. Also, I'm planning to make this behavior configurable enough so that one can turn it off should he need to.

How does it sound?

Comment by Andrew Oberstar [ 15/Mar/12 ]

Ah, I didn't realize that container pulled in things from the WTP component. I thought it was purely things currently in the WEB-INF/lib folder.

While this does handle the management of getting the files on the classpath, it doesn't seem to allow for source or javadoc attachment, which is a big weakness.

Even with this approach there would need to be something to handle the providedRuntime dependencies which aren't going to get pulled in by the container. These dependencies still throw the Classpath Dependency Validator message. (BTW I was wrong. It would be <attribute name="org.eclipse.jst.component.nondependency" value=""/> notice the non instead of no)

Unless there are ways around those two things, it still seems like the jst component configuration in the classpath file is the more robust option.

Comment by Szczepan Faber [ 15/Mar/12 ]

Hey Andrew,

Very good points regarding the provided and sources. The provided dependencies can be fixed quickly as we can just keep them in the .classpath. However, with the sources things are more complicated. The sources to the libs under a container can be attached, but this information is stored in eclipse workspace .metadata. And gradle does not know where client's workspace lives. In theory at the point of generating eclipse metadata the workspace might not even exist yet.

This means that we'll have to stick with dependencies on the classpath for now.

Comment by Szczepan Faber [ 15/Mar/12 ]

I think the easiest implementation (without lots of refactorings on my side) is:

  1. Every library dependency in the .classpath is marked as 'nondependency'.
  2. The component file configures the exports and the deploy path.

If it's reasonably easy I could do:

  1. Some library dependencies in the .classpath are marked with 'nondependency' or 'WEB-INF/lib', depending on the configuration.
  2. The dependencies are dropped from the component file.

I'm not sure which solution is better. The latter feels slightly better but it requires more work and it is also a bigger change. Both solve the problem of warnings and work well with sources/javadocs and provided dependencies.

Comment by Andrew Oberstar [ 15/Mar/12 ]

Both will solve the warnings, but the first solution leaves the potential for duplication in the classloader when deployed to a Eclipse managed server environment.

If possible, I would prefer #2, which I believe is the entirety of what I was originally proposing.

Comment by Adam Murdoch [ 22/Jul/12 ]

I think the correct solution is this, for a web project in Eclipse:

  • No external dependencies are included in the component file.
  • Dependencies to be deployed (i.e. runtime - providedRuntime classpath) are included in the .classpath file with the org.eclipse.jst.component.dependency attribute.
  • All other dependencies (e.g. provided, test) are included in the .classpath file with the org.eclipse.jst.component.nondependency attribute.

@Andrew, this is pretty much what you were originally suggesting?

Comment by Adam Murdoch [ 22/Jul/12 ]

To round this out, for a wtp java project in Eclipse (i.e. java + eclipse-wtp plugins):

  • No external dependencies are included in the component file.
  • Runtime dependencies are included in the .classpath file with the org.eclipse.jst.component.dependency attribute.
  • All other dependencies (e.g. test dependencies) are included in the .classpath file with the org.eclipse.jst.component.nondependency attribute.

For a wtp ear project in Eclipse (i.e. ear + eclipse-wtp plugins):

  • Runtime dependencies are included in the component file.
Comment by Adam Murdoch [ 22/Jul/12 ]

No idea what to do for ear + war + eclipse-wtp plugins or ear + java + eclipse-wtp plugins. Needs some research.

Comment by Andrew Oberstar [ 23/Jul/12 ]

Yes, Adam, that is what I was suggesting. All 3 flavors you describe (war, java, and ear) look right to me. I do want to point out another bug I submitted, GRADLE-2396, that describes some issues with the way the classpath component.dependency attribute is set. That would be worth consideration if any work is planned on this issue.

Comment by Mauro Molinari [ 25/Oct/12 ]

There's another problem with dependencies in war+eclipse-wtp multiprojects. Consider the attached zip. It contains a root project and a subproject. The root project depends at compile time on the subproject. Moreover, the root project depends on stax-1.0.1, while the subproject depends on stax-1.0. Right now, when you set up projects in Eclipse using Gradle 1.2, you can see that:

  • the deployment assembly of prj1 contains stax-1.0.jar
  • the deployment assembly of root contains both prj1 (with its stax-1.0.jar) and stax-1.0.1.jar

If you then deploy "root" on a server with WTP, when you publish the server you'll find both stax-1.0.jar and stax-1.0.1.jar in WEB-INF/lib, so you lose the ability of Gradle to resolve classpath conflicts.

I think the easiest way of fixing this problem is doing the following:

  • do not add jars as wtp dependencies in utility projects (i.e.: projects other depend on)
  • make things so that the war projects (i.e.: projects dependent on other projects) have a deployment assembled in this way:
    • subprojects they depend on
    • its own compile+runtime dependencies
    • the compile+runtime dependencies of the projects they depend on (with conflicts resolved)

In other words, when assembling a webapp project (i.e.: WTP Dynamic Web Projects) classpath, dependencies that must deployed should all be set by Gradle on the webapp project itself and NOT on the dependency (Utility) projects, because Eclipse is not able to do the needed conflict resolutions.
However, in order to make Utility projects compile, it's also needed that their JARs are added to their own buildpath too, but NOT declared for deploying.
I'm not a WTP internals expert, so I don't know how this should be actually handled. Maybe the solution is in GRADLE-2396, which talks about a strictly related problem: with the current behaviour, not only dependency conflict resolution is lost, but also the mechanism to exclude dependency deployment through the "provided" configurations. With my suggested solution, Gradle will handle the deployment assembly of the webapp project directly, so it should be able to honour "provided" configurations too, thus fixing also GRADLE-2396.

Comment by Adam Murdoch [ 30/Oct/12 ]

@Mauro, this is exactly how we want to solve the problem.

Comment by Mauro Molinari [ 31/Oct/12 ]

Glad to know Adam, thank you for your feedback!

Comment by Mauro Molinari [ 16/Jul/13 ]

Hi,
is there any progress or plan for this? Unfortunately I can't find any item in the "roadmap" page regarding a better IDE integration.
This would be very important for Gradle adoption, especially now that Eclipse is improving on Maven project support.

Comment by Andreas Schmid [ 09/Jan/15 ]

Another problem that this bug causes is shadowing source files, see http://forums.gradle.org/gradle/topics/war_eclipsewtpplugin_shadows_source_attachments_with_web_app_libraries_container.

I am working on it

Comment by Andreas Schmid [ 11/Jan/15 ]

I fixed this issue, see pull request https://github.com/gradle/gradle/pull/383.

However this not the cleanest fix, IMHO the complete Eclipse plugin has to be rewritten such that the org.gradle.plugins.ide.eclipse.model.internal.ClasspathFactory knows about the wtp component dependencies and org.gradle.plugins.ide.eclipse.model.internal.WtpComponentFactory knows about the classpath dependencies. This would give the ability to do not even create the incorrect entries which later on have to be clean up within EclipseWtpPlugin using their whenMerged features.

Furthermore, I am just not sure about the following from this comment of Adam Murdoch on this issue and the statement

For a wtp ear project in Eclipse (i.e. ear + eclipse-wtp plugins):

  • Runtime dependencies are included in the component file.

What do anybody mentioned and Andrew Oberstar think?

Comment by Andrew Oberstar [ 30/Jan/15 ]

I think this covers the necessary behavior. I added some comments to the pull request and associated commits.

Comment by Donát Csikós [ 01/Jun/16 ]

As part of an ongoing effort to improve Gradle tooling support we implemented a fix for this issue.
From now on the eclipse-wtp plugin places all library dependencies to the .classpath file and declares the deployment information as classpath attributes.
To try it out just use the latest Gradle 3.0 snapshot. Any feedback is greatly appreciated.

Generated at Wed Jun 30 12:13:25 CDT 2021 using Jira 8.4.2#804003-sha1:d21414fc212e3af190e92c2d2ac41299b89402cf.