[GRADLE-1275] EclipsePlugin does not properly support WTP project dependencies Created: 27/Dec/10  Updated: 25/Sep/13  Resolved: 24/Feb/11

Status: Resolved
Project: Gradle
Affects Version/s: 0.9
Fix Version/s: 1.0-milestone-1

Type: Bug
Reporter: Matt Accola Assignee: Peter Niederwieser
Resolution: Fixed Votes: 1

Attachments: Zip Archive Eclipse-Screenshots.zip     Text File UtilityModulesWorkaround-v2.txt     Text File UtilityModulesWorkaround.txt     Text File WebModulesWorkaround.txt     Zip Archive test-v2.zip     Zip Archive test.zip     File wtp-fix.gradle    

 Description   

Gradle's support for generating Eclipse project files does not cover all cases for WTP (Web Tools Platform).

  • All Java projects which a Dynamic Web Project depends must be configured as with the WTP Utility facet. This includes
    • Adding the jst.utility facet to the org.eclipse.wst.common.project.facet.core.xml file.
    • Adding the following natures to the .project file: org.eclipse.wst.common.project.facet.core.nature, org.eclipse.jem.workbench.JavaEMFNature, org.eclipse.wst.common.modulecore.ModuleCoreNature.
    • Adding the following builders to the .project file: org.eclipse.wst.common.project.facet.core.builder, org.eclipse.wst.validation.validationbuilder.
    • An org.eclipse.wst.common.component file must be added
  • Each 1st level JAR dependency of each Utility project must be flagged as a runtime dependency. This is accomplished by adding the org.eclipse.jst.component.dependency attribute to the appropriate classpathentry in the .classpath file.
  • Each project dependency (both 1st level and transitive) of a Dynamic Web Project must be adding as a runtime dependency. This is accomplished by adding an entry to the org.eclipse.wst.common.component file in the web project.

I have attached a very simple multi-module project which demonstrates the proper Eclipse files. The sample project contains the TestWeb Dynamic Web project which depends directly on the TestModuleA Utility project. The TestModuleA Utility project depends on the TestModuleB Utility project. Both TestModuleA and TestModuleB each have one JAR dependency. The result of deploying or exporting the WAR file from Eclipse should result in the following contents:

META-INF/MANIFEST.MF
WEB-INF/lib/TestModuleA.jar
WEB-INF/lib/TestModuleB.jar
WEB-INF/lib/commons-collections-3.2.jar
WEB-INF/lib/commons-lang-2.5.jar



 Comments   
Comment by Matt Accola [ 30/Dec/10 ]

I am attaching a workaround which I scripted using the task hooks built into the Ecilpse Plugin. This script works for my specific set of projects but might not cover all cases. UtilityModulesWorkaround.txt contains script to cover bullet points #1 and #2 in the bug report. WebModulesWorkaround.txt contains two snippets of script...together they cover bullet point #3 in the bug report.

Comment by Peter Niederwieser [ 21/Jan/11 ]

Are you sure about #1 ("All Java projects which a Dynamic Web Project depends ...")? I fail to see how UtilityModulesWorkaround.txt addresses this. I had hoped that test.zip contains the .settings directories but it doesn't. The dynamic web project I've created with Eclipse doesn't look like you describe in #1. So I'm not sure at this point what the correct behavior is.

Comment by Matt Accola [ 21/Jan/11 ]

To answer your first question, I am pretty sure of my findings. I have verified that (a) Eclipse WTP does not work in all cases with the base gradle-0.9 and (b) that applying the changes I described does cause Eclipse WTP to work ... at least in the set of use cases I have tried.

Just to make sure I was clear on what test.zip contained. It does not contain Eclipse files which were generated by Gradle. It contains Eclipse files which were generated by exercising the Eclipse UI to setup the projects. You mentioned that you would like to see the files in the .settings folder. I am uploading a new attachment, test-v2.zip, which contains the .settings directories for all projects.

Regarding your comment that UtilityModulesWorkaround.txt is not addressing the problem...you have a point there. It does NOT modify the .settings files or the .project file. I actually failed to include some additional script I have which uses the eclipseProject hook to add the necessary natures to make the project a Utility project. I am attaching the full script which includes this, UtilityModulesWorkaround-v2.txt. I do not have a scripted solution to actually add the Utility Module facet to the project because I am not aware of any Gradle hooks that allow me to write out the necessary files to .settings. What the script does provide is the capability to flag all runtime dependencies with the org.eclipse.jst.component.dependency attribute. This is critical to the function of Eclipse WTP. Without this attribute Eclipse WTP does not include the dependency JARs in the WAR (under WEB-INF/lib).

Finally, to address your comment that your project that you created from the Eclipse UI does not look like mine. I have attached a couple of screen shots which show the various project configuration screens which need to be modified to achieve a proper project configuration. If I can be of any other assistance please let me know.

Comment by Peter Niederwieser [ 25/Jan/11 ]

Thanks for the explanations. ProjectB's .project file doesn't contain a org.eclipse.wst.validation.validationbuilder. Should it always be added?

Comment by Matt Accola [ 25/Jan/11 ]

That's a good question that I don't have a definitive answer for. I thought the Eclipse GUI always added it so I'm surprised it is not there. However, in my experience it is not integral to the function of WTP. Personally I usually disable all unnecessary validators for large code bases because they just drain resources.

Comment by Rob Winch [ 29/Jan/11 ]

For what it's worth this I have attached a file, wtp-fix.gradle, containing the changes that I apply to my wars get my WTP projects to work. I am certain that the code is not as efficient as it could be nor does it produce the optimal eclipse configurations, but it seems to get the job done.

Comment by Peter Niederwieser [ 14/Feb/11 ]

Most changes that you (Matt) described are now implemented. Maybe you can answer a few more questions:

1. You said "All Java projects which a Dynamic Web Project depend [...]". What about Groovy, Scala, etc. projects? Any reason for not including all depended-upon projects? And are we only talking about first-level project dependencies here?

2. The org.eclipse.wst.common.component file seems to define the deployment structure. Are you sure that it has to be created for depended-upon projects as well (and not just for war projects)? If so, what exactly should go in there (only project or also lib dependencies, transitive or not, what deploy-path)? I wonder why even the component file in your web project doesn't contain lib dependencies.

Thanks for your help!

Comment by Matt Accola [ 14/Feb/11 ]

Answer to question #1...I believe all projects including Scala and Groovy projects should have these changes applied. Basically any project which produces a JAR artifact that needs to be included in the WAR file (both Scala and Groovy projects would meet that criteria). We are not talking about only 1st level dependencies. My experience has been that you must declare all project dependencies at all levels in the Dynamic Web Project...Eclipse WTP is not smart enough to follow transitive dependencies...you will see that my Gradle script covered that case.

Answer to question #2...your comment "The org.eclipse.wst.common.component file seems to define the deployment structure" is exactly correct. In fact on the Eclipse Helios GUI the screen is called "Deployment Assembly". I am not sure that it needs to be created for depended-upon projects. I know that when you build up the projects/dependencies from the Eclipse GUI that is does write out this file even for non-WAR projects but everything seems to function just fine without it. This kind of goes back to question #1. If Eclipse WTP DID support resolving transitive dependencies then the org.eclipse.wst.common.component file would presumably be the mechanism for projects to expose their own dependencies...however, as I noted Eclipse WTP doesn't seem to recognize transitive project dependencies and you are forced to declare them at the Dynamic Web Project level. Hopefully this doesn't confuse things. Bottom line is...no, I don't think the org.eclipse.wst.common.component file is required for non-Dynamic Web Projects at the current time.

Comment by Peter Niederwieser [ 14/Feb/11 ]

>> My experience has been that you must declare all project dependencies at all levels in the Dynamic Web Project

Right, but 1. ("All Java projects ...") is about how to configure depended-upon projects. Are you saying this also applies to transitively depended-upon projects?

Comment by Matt Accola [ 15/Feb/11 ]

Yeah, I was wrong in the initial write-up to say "Java projects"...I wasn't considering that other types of projects also need the same functionality. I think it makes sense to add this behavior to Scala and Groovy projects as well.

Take a look again at the sample Eclipse projects I attached (test-v2.zip). There are three projects with the following dependencies. TestWeb depends on -> TestModuleA depends on -> TestModuleB. So TestModuleB is a 2nd level transitive dependency of TestWeb.

If you import these projects into Eclipse and do a File...Export...Web...WAR File you will see that both TestModuleA.jar and TestModuleB.jar are present in the WEB-INF/lib directory of the exported WAR. The reason why this occurs is because there are entries for both TestModuleA and TestModuleB in the TestWeb/.settings/org.eclipse.wst.common.component file. One would think that an entry for TestModuleB would not be necessary here but if you remove it and then re-export the WAR file you will find that the WEB-INF/lib folder of the exported WAR file only contains TestModuleA.jar, NOT TestModuleB.jar. Long story short...you must have an entry for all project dependencies at all levels, NOT just the 1st level project dependencies.

What I am describing above applies to project dependencies (kind="src" in .classpath file). The rules are different for non-project dependencies (kind="var" or kind="lib" in .classpath file). For non-project dependencies Eclipse DOES support resolving n-level transitive dependencies via the org.eclipse.jst.component.dependency attribute which has been discussed in this JIRA issue.

I want revise my comment from yesterday regarding the necessity of creating a "org.eclipse.wst.common.component" file for Utility projects. In my comment yesterday I stated that I didn't think it was necessary. I believe this is true but I would recommend at least creating the file with the outer tags just in case Eclipse checks for the file to be present. Also, I think it would be wise to provide a Gradle task hook along the lines of the eclipseWtp task hook to allow Gradle users to write scripts to modify the contents. This could be useful if Eclipse changes anything with future releases.

Comment by Peter Niederwieser [ 22/Feb/11 ]

I think we've been talking cross purposes. I was only asking if the statement "All Java projects which a Dynamic Web Project depends" also applies to transitive project dependencies. In 3. you explicitly say "Each project dependency (both 1st level and transitive)", but in 1. you just say "All Java projects which a Dynamic Web Project depends". I assume the answer is nevertheless 'yes'?

Comment by Matt Accola [ 22/Feb/11 ]

Yes

Comment by Peter Niederwieser [ 23/Feb/11 ]

I've implemented all your suggestions to the best of my knowledge. Now trying to get hold of a real-world web project to do some manual testing. If you could build Gradle on your own ("gradle install") and give it a spin, it would be greatly appreciated. Only very little time left to fix stuff before 1.0-milestone-1 gets released.

Comment by Rob Winch [ 23/Feb/11 ]

The changes made do not appear to fix the Spring Security's sample web applications running in STS 2.6.0.M1 against Tomcat 7.0.6. The error that I see in Eclipse's problems view is something on the lines of:

Project 'spring-security-config' is missing required library: '/home/rwinch/workspaces/springsecurity/spring-security/core/build/classes/test' spring-security-config

If I try and run the spring-security-samples-jaas project I get an error stating it cannot resolve classpath dependencies in spring-security-config. This makes sense since spring-security-config could not be built. Unfortunately, the earliest I expect to have time to look into the exact cause won't be until the weekend. If you want you can try for yourself using git://git.springsource.org/spring-security/spring-security.git Please note that I would not expect that any of the aspectj projects work in eclipse because it is not configuring eclipse properly. Let me know if you prefer me to look into the issue and/or help out or if you think you have the situation under control yourself.

Comment by Matt Accola [ 24/Feb/11 ]

I tested this out this morning and so far looks good...with one exception.

The problem I found is that 2nd-level dependencies are not getting the Utility Module facet applied. This prevents them from being included in the WAR file. Take the multi-module project in the test-v2.zip attachment. With the new Gradle version I see that TestModuleA correctly has the Utility Module facet applied. I see that both TestModuleA and TestModuleB are listed in TestWeb's org.eclipse.wst.common.component file. The only problem is that TestModuleB does not have the Utility Module facet applied. The end result is that the exported WAR file only includes the JAR file for TestModuleA and not TestModuleB. If we can just get the facet applied to TestModuleB we should be good.

Also, I found that the new version is a breaking change. My script that previously used the "eclipseWtp" task hook no longer runs. I had to change to use the new "eclipseWtpComponent" task hook. This is perfectly fine and the documentation in the User Guide is good. However, I might suggest that you provide some kind of notes for developers who are migrating from 0.9 to 1.0 so they don't have to figure it out on their own. Let me know when any new code is committed and I can run some more tests. Thanks!

Comment by Adam Murdoch [ 24/Feb/11 ]

@matt, there is a 'breaking changes' page on the wiki which mentions this (though it could do with a little more detail): http://docs.codehaus.org/display/GRADLE/Gradle+1.0-milestone-1+Breaking+Changes

Usually, we link to this page from the release notes.

Comment by Peter Niederwieser [ 24/Feb/11 ]

> The problem I found is that 2nd-level dependencies are not getting the Utility Module facet applied.

Matt, which commit did you test with? Both automated test and manual test with test-v2.zip project indicates that transitive project dependencies also have the jst.utility facet applied.

Comment by Peter Niederwieser [ 24/Feb/11 ]

I'm resolving this issue because all mentioned points have been fixed to the best of our understanding. Please open new issues as needed.

Comment by Matt Accola [ 24/Feb/11 ]

I will open another issue tomorrow when I get to work. Sorry I didn't respond tonight...I don't have my workspace with me at home.

I believe the issue is that the 2nd level project dependency does not have the appropriate project natures applied...most importantly ModuleCoreNature. The reason I referred to it as "the Utility Module facet not being applied" is because that is how it is manifested in the Eclipse GUI.

I will provide details in the new JIRA issue.

Comment by Peter Niederwieser [ 25/Feb/11 ]

Fixed in master, but might not make it into 1.0-milestone-1.

Comment by Matt Accola [ 25/Feb/11 ]

Retested with this commit, https://github.com/gradle/gradle/commit/fd16c01478fc50599bedbda75a7895bd8696b4a0. Everything looks great now as far as I can tell! Thanks for your persistence on this.

Comment by Matt Accola [ 25/Feb/11 ]

Upon further testing I have found what appears to be a serious issue. I will write up a new JIRA issue with details.

Comment by David Koch [ 19/Dec/11 ]

With Gradle 1.0-milestone-6 on Windows I can't create a proper eclipse wtp project by calling "gradle eclipse" neither with my own project nor with the to this issue attached test.zip project.
In both cases the required eclipse files are missing in the ".settings" folder of the projects the war module depends on.
Those files are generated by eclipse if I configure the dependencies manually:
TestModuleA/.settings/org.eclipse.wst.common.component
TestModuleA/.settings/org.eclipse.wst.common.project.facet.core.xml
TestModuleB/.settings/org.eclipse.wst.common.component
TestModuleB/.settings/org.eclipse.wst.common.project.facet.core.xml
Without those files the "Web Deployment Assembly" config window indicates the following errors for test.zip projects:
Cannot find entry: "src/main/webapp".
Cannot find entry: "TestModuleA".
Cannot find entry: "TestModuleB".
The TestWeb module itself is generated as wtp project because it has a "Deployment Assembly" config and the right icon.

Comment by Matt Accola [ 19/Dec/11 ]

You need to apply the 'eclipse-wtp' plugin for the web project instead of 'eclipse'. See http://wiki.gradle.org/display/GRADLE/Gradle+1.0-milestone-4+Breaking+Changes.

In the case of the attached test-v2.zip this would mean changing line 2 of TestWeb/build.gradle to:

apply plugin: 'eclipse-wtp'

Comment by David Koch [ 20/Dec/11 ]

I did it already. Otherwise the TestWeb module of the test.zip wouldn't become a wtp project.
I did this change to the test.zip and I removed all eclipse files manually. Than I started

gradle eclipse

with the result I've posted yesterday.

Comment by Truong Xuan Tinh [ 24/Dec/11 ]

I confirm the behavior that David Koch described.
I am also using Gradle 1.0 milestone 6 on Linux with Eclipse Indigo Service Release 1

Comment by Truong Xuan Tinh [ 06/Jan/12 ]

I solved my problem with the information from this https://issuetracker.springsource.com/browse/STS-2184
Basically, I just apply the 'eclipse' and the 'eclipse-wtp' to every of the projects that the Dynamic Web Project depending on.

Comment by Szczepan Faber [ 10/Mar/12 ]

I've scanned quickly the discussion and I probably missed some stuff. Can I assume that fixing: GRADLE-1880 would resolve the problem? If I don't hear from you I'll assume yes

Btw. eclipse-wtp does implicitly apply the eclipse plugin so you don't have to apply both.

Comment by Szczepan Faber [ 10/Mar/12 ]

Ok... Ignore my earlier question I haven't noticed this issue has been closed.

Comment by Sebastien Tardif [ 24/Sep/13 ]

The part about generating custom MANIFEST.MF when 100% running in Eclipse is NOT working.

The issue is easy to demonstrate with the sample provider by Gradle at gradle-1.7\samples\webApplication\quickstart

You just need to add the following to build.gradle:
apply plugin: 'eclipse-wtp'

Then when building the war on the command line you can see that gradle-1.7\samples\webApplication\quickstart\build\libs\META-INF\MANIFEST.MF is populated but if you use Eclipse WTP integration, and deploy the project to the server all from Eclipse, MANIFEST.MF is NOT populated. I tested that with GlassFish Server 3.1. The problem occurs with or without "Use Jar Archives for Deployment" enabled, a GlassFish integration option.

Doesn't the same thing in Eclipse using Tomcat 7.0.42 doesn't work better.

Comment by Peter Niederwieser [ 25/Sep/13 ]

That's not something that the Gradle Eclipse tasks support, at least not out-of-the-box. Such features will be tackled longer-term in the Eclipse Gradle tooling by having Eclipse delegate all build activities to Gradle. Meanwhile, you may be able to work around by configuring the Gradle Eclipse tasks to include an output directory containing the Gradle-generated files, but I can't tell for sure. Alternatively, you could deploy the Gradle-generated War.

Generated at Wed Jun 30 11:51:24 CDT 2021 using Jira 8.4.2#804003-sha1:d21414fc212e3af190e92c2d2ac41299b89402cf.