[GRADLE-2396] EclipseWtpPlugin incorrectly sets all utility JAR dependencies to be included in the parent WAR's WEB-INF/lib Created: 19/Jul/12  Updated: 25/Jan/17  Resolved: 03/Jun/16

Status: Resolved
Project: Gradle
Affects Version/s: 1.0
Fix Version/s: 3.0-milestone-1, 3.0-rc-2

Type: Bug
Reporter: Andrew Oberstar Assignee: Donát Csikós
Resolution: Fixed Votes: 4


 Description   

The EclipseWtpPlugin will currently mark all dependencies of web utility JARs (JARs projects depended upon by a WAR), so that they are included in the WAR's WEB-INF/lib. This causes a problem for JAR's that are also a part of the WAR's providedCompile/Runtime configurations (or are part of the EAR's earlib configuration).

The current behavior results in these JARs being placed in the WAR's WEB-INF/lib regardless of whether they are provided elsewhere on the classpath.

To fix this the plugin should set any provided dependencies (either from EAR or WAR) as:

<attribute name="org.eclipse.jst.component.nondependency" value=""/>

Other dependencies should continue to be set as:

<attribute name="org.eclipse.jst.component.dependency" value="../"/>

This should result in the correct behavior.



 Comments   
Comment by Andrew Oberstar [ 23/Jul/12 ]

This could actually be helped with "provided" configurations for the JavaPlugin (see GRADLE-784).

Comment by Mauro Molinari [ 23/Jul/12 ]

IMHO, if I understood the bug right, the problem is rather that the WAR project is not filtering the utility JAR dependencies that are also in the providedCompile/providedRuntime configuration of the WAR... isn't it?
Because the fact that the dependencies of the Utility projects are also deployed is actually the desired behaviour in my case...

Comment by Andrew Oberstar [ 23/Jul/12 ]

Yes, that would be another way to put it. I agree that the current behavior is good in most cases. Doesn't work with some EAR configurations however.

Comment by Mauro Molinari [ 25/Oct/12 ]

Regarding my previous comment, I said: "the fact that the dependencies of the Utility projects are also deployed is actually the desired behaviour in my case". However, this is partially true. I mean, I need for their dependencies to be deployed (unless they are provided dependencies), but the current way Gradle sets dependencies for WTP projects (see my comment of 25/ott/12 5:11 PM in GRADLE-2123) is not appropriate when you need dependency version conflict resolution and provided dependencies.

Comment by Andreas Schmid [ 14/Dec/14 ]

Is this anyhow related to http://forums.gradle.org/gradle/topics/all-provided-dependencies-appear-unexpectedly-in-default-configuration?

Comment by Mauro Molinari [ 14/Dec/14 ]

Hi Andreas,
it has passed a lot of time since my experiments with these issues and since things haven't changed, I learnet to avoid to use Gradle own's way of managing dependencies for Eclipse projects and rather use Gradle IDE from STS to handle dependency management and apply some custom adjustments to the Deployment Descriptor to make things works more-or-less.
Unfortunately, I think the current way dependencies for Eclipse projects are set up is wrong and causes problems. I think this is one of them and I don't know whether it's related to the one you're linking. In GRADLE-2123 I expressed my opinion on how Gradle should set up and handle dependencies in Eclipse projects to make the Eclipse projects depend on the correct artifacts and match the dependency resolution that Gradle performs when building the war on its own..

Comment by Andreas Schmid [ 14/Dec/14 ]

I'm already working on GRADLE-2123 but this not that easy. Nevertheless, the results building and deploying the project from within Eclipse does work with the current as well as the correct version described in GRADLE-2123.

Comment by Andrew Oberstar [ 15/Dec/14 ]

Andreas, I don't believe this is related to the forum post you linked to. It is unique to the way that Eclipse files are generated.

As an FYI, we've been using some not so pretty implementations in our Gradle plugins to resolve both this issue and GRADLE-2123 with the approaches I've described. These have been working for our use cases over the 2+ years that these issues have been open.

Comment by Andreas Schmid [ 15/Dec/14 ]

Andrew, would you be allowed to give me the scripts to get some hints and ideas from it? Would help me a lot!

Comment by Andrew Oberstar [ 16/Dec/14 ]

Yeah, I can try to piece our stuff together. It won't be too pretty.

Comment by Andreas Schmid [ 16/Dec/14 ]

doesn't matter, just to get an idea
Can you see my email in my profile if you don't want to post it here?

Comment by Andrew Oberstar [ 18/Dec/14 ]

I guess 2396 was the only one of the two we put a fix in for:

	private void fixGradle2396(Project project) {
		project.plugins.withType(EarPlugin) {
			project.gradle.projectsEvaluated {
				project.eclipse.wtp.component.file.whenMerged { WtpComponent component ->
					def libs = component.wbModuleEntries.findAll { it instanceof WbDependentModule }.collect { it.handle - 'module:/classpath/lib/' }.findAll { !it.startsWith('module:') }
					Configuration deploy = project.configurations[EarPlugin.DEPLOY_CONFIGURATION_NAME]
					withEachEclipseWtpProjectDependency(deploy) { Project depProject ->
						if (!depProject.plugins.hasPlugin(SecurianWarPlugin)) {
							depProject.eclipse.classpath.file.whenMerged { Classpath classpath ->
								classpath.entries.each {
									if (it instanceof AbstractLibrary && libs.contains(it.library.path)) {
										it.entryAttributes.remove(AbstractClasspathEntry.COMPONENT_DEPENDENCY_ATTRIBUTE)
										it.entryAttributes[AbstractClasspathEntry.COMPONENT_NON_DEPENDENCY_ATTRIBUTE] = ''
									}
								}
							}
						}
					}
				}
			}
		}
	}
Comment by Andreas Schmid [ 20/Dec/14 ]

Thanks for the snippet, I will try to provide a fix which will be accepted and still released with 2.3 during the next weeks

Comment by Andreas Schmid [ 11/Jan/15 ]

Also the issue which GRADLE-2123 (and even provided a reproducer) should be considered if fixing this!

Comment by Andreas Schmid [ 11/Jan/15 ]

Andrew, I have almost fixed GRADLE-2123 and thereby I had a look at this as well. As far as I understand, you are removing all the dependencies from wtp components of all depended 'jar' projects (= utility jars). Is that correct (unfortunately the code of withEachEclipseWtpProjectDependency(deploy, closure) is missing)?

However, IMHO if you have a compile/runtime dependency within an utility jar and if you add the utility jar as compile/runtime dependency to the war, you want to have the compile/runtime dependencies of the utility jar added as well to the war, don't you? At least as the default case ...

Therefore, I think the only clean solution to this issue, is to add a provided configuration to your utility jars. And for sure this configuration is excluded from wtp component dependencies. (I just also don't really understand why the provided configuration(s) will not be added by default to a java plugin as I almost always use it as soon as I have a war or ear project.) As you most certainly already know, you can achieve this could be done by

   configurations {
        provided // should not be included in war file where common is a dependency of
    }
    sourceSets {
        main {
            compileClasspath += configurations.provided
            runtimeClasspath += configurations.provided
        }
        test {
            compileClasspath += testenv.output + configurations.provided
            runtimeClasspath += testenv.output + configurations.provided
        }
    }
    dependencies {
        // provided => should not be used in *.war due to transitive dependency resolution
    }
    eclipse.classpath {
        plusConfigurations += [ configurations.provided ]
        noExportConfigurations += [ configurations.provided ]
    }

If you are using this custom provided configuration also leads IMHO to lesser and clearer dependencies as you have to declare them just once, at the point were they first appear.

What do you think? Maybe I just haven't got your point, so please tell me my fallacy.

Comment by Mauro Molinari [ 11/Jan/15 ]

Hi Andreas,
I take the liberty to repeat once again my point of view. As stated previously, the Utility module dependencies SHOULD in some way go on the Dynamic Web Projects classpath (by default compile and runtime dependencies are transitive), but the way in which Gradle should handle this is by recomputing the dependencies of each project independently and not "deploy" the Utility project dependencies as they are in the dependent projects. The same problem is present with flat Java projects, which are setup in Eclipse with all the dependencies "exported".

Otherwise, if my Utility project depends on lib "foo" in version "1.2" and my Dynamic Web Project depends on lib "foo" in version "1.3", Eclipse will deploy both foo-1.2.jar and foo-1.3.jar when publishing the Dynamic Web Project, because Eclipse does not apply any version conflict resolution.
So, I think Gradle should compute the dependencies of the Dynamic Web Project by its own (including its transitive dependencies, of course) and deploy just those JARs.
In a comment in Gradle-2123 (link: https://issues.gradle.org/browse/GRADLE-2123?focusedCommentId=16307&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-16307) I specified how I think the problem of Eclipse/WTP dependencies should be handled by Gradle and Adam Murdoch said he agreed with my point of view on this issue.

Hope this helps the discussion.

Comment by Mauro Molinari [ 11/Jan/15 ]

Sorry Andreas, I started to write my message before you posted your latest changes :-P

Comment by Andreas Schmid [ 11/Jan/15 ]

Mauro, partly it is working as you expect, namely if you add a dependency in the same version to utility jar compile and war providedCompile, see test case org.gradle.plugins.ide.eclipse.EclipseWtpWebAndJavaProjectIntegrationTest#generates configuration files for web project and java project it depends on.

Comment by Andrew Oberstar [ 12/Jan/15 ]

Andreas, didn't realize I forgot that method in my post. Here's the rest:

	private void withEachEclipseWtpProjectDependency(Configuration configuration, Closure closure) {
		configuration.allDependencies.withType(ProjectDependency) { ProjectDependency dep ->
			Project project = dep.dependencyProject
			project.plugins.withType(EclipseWtpPlugin) { closure(project) }
			Configuration runtime = project.configurations[JavaPlugin.RUNTIME_CONFIGURATION_NAME]
			if (runtime) {
				withEachEclipseWtpProjectDependency(runtime, closure)
			}
		}
	}
Comment by Andrew Oberstar [ 12/Jan/15 ]

I've forgotten some of the context in the two years since reporting this and I'm having trouble parsing this on a Monday morning. Here's what I think the code is doing:

1. On the EAR project grab a list of all of the earlib stuff in the component file.
2. For each project dependency in the EAR's deploy configuration (and for each of their dependencies in the runtime configuration):
2.1. Check if it's not a WAR. If it is, skip it.
2.2 Look through its dependencies and set any that are in the earlib (as determined by #1) and set them to "non-dependency" so they don't end up in the WEB-INF/lib.

It's possible a provided configuration would resolve this, but I can't say for certain with how fuzzy my memory is. I wish I had described the issue better originally.

Comment by Donát Csikós [ 01/Jun/16 ]

The latest Gradle 3.0 snapshot sets the <attribute name="org.eclipse.jst.component.nondependency" value=""/> classpath attribute for all dependencies on a utility project.

Comment by Andrew Oberstar [ 09/Jun/16 ]

Hi Donát, I haven't had a chance yet, but I plan to review this compared to our customizations to see if we get the same (or more likely better) results from your changes (and dropping our own). Right now, I probably wouldn't do this until later this month. Would that be too late to provide feedback in time to make any tweaks, if I discover anything from that testing? If you have a preferred timeline, please let me know.

Comment by Donát Csikós [ 09/Jun/16 ]

Giving feedback is never too late, thanks for looking into it.
To be serious, you have time since this feature will be released as part of Gradle 3.0 and even 2.14 is not out. I'd say just keep an eye on the release candidates on the Gradle forum.

Comment by Andrew Oberstar [ 17/Jun/16 ]

Hey Donat, got a chance to try this out today. It was just a really simple test WAR with a JAR subproject, but it worked great! I think we're going to be able to drop essentially all of our custom eclipse code from our plugins due to the fixes you made here and the coming 2.0 Buildship features.

Certainly could run into some things when people use it for real, but I doubt anyone will move to 3.0 here until the final release. But with the simple test and manually look through the config files, it all looks to be configured much more appropriately than it was by Gradle 2 or our custom code.

I do have a couple other issues that aren't related to these fixes, but I'll post those on the forums. Thanks for fixing this!

Comment by Donát Csikós [ 19/Jun/16 ]

I'm glad the changes work well for you. Thank you for the feedback!

Generated at Wed Jun 30 12:20:47 CDT 2021 using Jira 8.4.2#804003-sha1:d21414fc212e3af190e92c2d2ac41299b89402cf.