Forked groovyc compilation can create a classpath line too long for Windows

Details

  • Type: Bug Bug
  • Status: Resolved Resolved
  • Resolution: Fixed
  • Affects Version/s: 1.0-milestone-1
  • Fix Version/s: 1.0-rc-1

Description

We hit the same issue documented here:

http://gradle.1045684.n5.nabble.com/Groovyc-fork-failing-on-Windows-td3357060.html

When gradle tries to compile (in our case) a mixed groovy/java environment on a Windows machine it returns an exception:

java.io.IOException: Cannot run program "c:\PROGRA~1\Java\jdk1.6.0_10
\jre\bin\java": CreateProcess error=87, The parameter is incorrect"

This is due to the command exceeding the size limit on command-line arguments in Windows. The build works fine in Linux.

Peter suggests turning off forking on the above thread as a workaround:

[compileGroovy, compileTestGroovy]*.groovyOptions.fork = false

The process indeed does not fork with that line, but it just hangs (even with GRADLE_OPTS set to 1G).

Issue Links

Trackbacks

[JIRA: Architecture Agile] [ARCH-138] Build fails on fresh checkout on windows machines Reproduction Steps ================== - checkout new branch - run gradlew Symptoms ======== FAILURE: Build failed with an exception. * What went wrong: Execution failed for task ':scriptexporter:compileTestGroovy'. Cause: Error runni...

[JIRA: Bespin] [ARCH-313] Gradle :webservice:compileJava task fails on Windows because classpath is too long Related Gradle issue: http://issues.gradle.org/browse/GRADLE-1420

Activity

Ascending order - Click to sort in descending order
Hide
Permalink
Peter Niederwieser added a comment -

Gradle currently puts too much on the compile class path. Once that is fixed, the "too long class path" problem will probably go away automatically.

Show
Peter Niederwieser added a comment - Gradle currently puts too much on the compile class path. Once that is fixed, the "too long class path" problem will probably go away automatically.
Hide
Permalink
Ray Nicholus added a comment - - edited

As Ben mentioned, preventing the fork (which should cut the classpath size in half), causes the process to hang, so that is not an option.

An easy workaround, theoretically, would be to create an empty jar with the project's classpath specified in the jar's Class-Path manifest attribute. I attempted to do this with Gradle, but it seems Gradle specifies absolute paths for each classpath entry. Apparently, absolute paths are not valid in the manifest's Class-Path attribute. This appears to be true based on Sun's manifest attribute spec at http://download.oracle.com/javase/1.4.2/docs/guide/jar/jar.html, as well as from my personal experience.

Putting the pathing jar concept aside for a moment, it would be nice if Gradle specified relative paths for classpath entries, instead of absolute paths, wherever possible. This, for one, would decrease the size of the classpath. Back to pathing jars: If this behavior is introduced, a pathing jar can be used to address long classpath issues such as this. Note, that for cases where the external dependency cache is not on the same drive as the build/source initially, the GRADLE_USER_HOME env variable can be set to the build/source drive, making it possible to reference these classpath entries relative to the pathing jar.

Show
Ray Nicholus added a comment - - edited As Ben mentioned, preventing the fork (which should cut the classpath size in half), causes the process to hang, so that is not an option. An easy workaround, theoretically, would be to create an empty jar with the project's classpath specified in the jar's Class-Path manifest attribute. I attempted to do this with Gradle, but it seems Gradle specifies absolute paths for each classpath entry. Apparently, absolute paths are not valid in the manifest's Class-Path attribute. This appears to be true based on Sun's manifest attribute spec at http://download.oracle.com/javase/1.4.2/docs/guide/jar/jar.html, as well as from my personal experience. Putting the pathing jar concept aside for a moment, it would be nice if Gradle specified relative paths for classpath entries, instead of absolute paths, wherever possible. This, for one, would decrease the size of the classpath. Back to pathing jars: If this behavior is introduced, a pathing jar can be used to address long classpath issues such as this. Note, that for cases where the external dependency cache is not on the same drive as the build/source initially, the GRADLE_USER_HOME env variable can be set to the build/source drive, making it possible to reference these classpath entries relative to the pathing jar.
Hide
Permalink
Peter Niederwieser added a comment -

Is it you who asked the Gradle pathing Jar question on stackoverflow.com? I recently fixed my answer and it now works fine for me. Absolute paths don't seem to be a problem here.

Relative paths typically won't be shorter because dependencies come from Gradle's Ivy cache, which doesn't share a common parent with the build's working directory. What we could do is to try to substitute a part of the path with $GRADLE_USER_HOME.

How big are the projects that suffer from this problem?

Show
Peter Niederwieser added a comment - Is it you who asked the Gradle pathing Jar question on stackoverflow.com? I recently fixed my answer and it now works fine for me. Absolute paths don't seem to be a problem here. Relative paths typically won't be shorter because dependencies come from Gradle's Ivy cache, which doesn't share a common parent with the build's working directory. What we could do is to try to substitute a part of the path with $GRADLE_USER_HOME. How big are the projects that suffer from this problem?
Hide
Permalink
Ray Nicholus added a comment -

Hi Peter. Before you posted the adjustment on stack, I realized that spaces were required and tried this instead - no luck. I attempted this again in case I missed something, and, as before, the compile fails due to an inability to find dependencies.

Show
Ray Nicholus added a comment - Hi Peter. Before you posted the adjustment on stack, I realized that spaces were required and tried this instead - no luck. I attempted this again in case I missed something, and, as before, the compile fails due to an inability to find dependencies.
Hide
Permalink
Jesper Skov added a comment -

This should be fixed in Groovy. I've created http://jira.codehaus.org/browse/GROOVY-5024 which includes a partial fix (and hints for a complete fix).

Show
Jesper Skov added a comment - This should be fixed in Groovy. I've created http://jira.codehaus.org/browse/GROOVY-5024 which includes a partial fix (and hints for a complete fix).
Hide
Permalink
Ray Nicholus added a comment -

Thanks for the update. Which version of groovy includes this fix?

Show
Ray Nicholus added a comment - Thanks for the update. Which version of groovy includes this fix?
Hide
Permalink
Guillaume Laforge added a comment -

Ray, it's not in a released version of Groovy yet.
I tried the patch indicated in the Groovy JIRA issue, but alas, it's failing the Groovy build.
So it seems the patch needs a bit more work.

Show
Guillaume Laforge added a comment - Ray, it's not in a released version of Groovy yet. I tried the patch indicated in the Groovy JIRA issue, but alas, it's failing the Groovy build. So it seems the patch needs a bit more work.
Hide
Permalink
Ray Nicholus added a comment -

Tasks for gradle build file to address groovyc "classpath too long" issue.

Show
Ray Nicholus added a comment - Tasks for gradle build file to address groovyc "classpath too long" issue.
Hide
Permalink
Ray Nicholus added a comment - - edited

I finally got the "pathing jar" idea to work. I consider this to be a permanent workaround. This could be considered a solution if it is made part of gradle itself.

The original pathing jar code was provided by Peter, but it didn't work. The problem: classpath elements referenced in the pathing jar must be relative to the location of the pathing jar. So, this appears to work for me (see attached classpath-too-long.txt).

Show
Ray Nicholus added a comment - - edited I finally got the "pathing jar" idea to work. I consider this to be a permanent workaround. This could be considered a solution if it is made part of gradle itself. The original pathing jar code was provided by Peter, but it didn't work. The problem: classpath elements referenced in the pathing jar must be relative to the location of the pathing jar. So, this appears to work for me (see attached classpath-too-long.txt).
Hide
Permalink
Kirk Rasmussen added a comment -

For us unfortunate developers stuck on the Windows platform I have opened the following bug report (might take a few days to show up):

http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=7162307

It turns out that javac has supported @argfile mechanism for a long time (since at least 1.4):

http://docs.oracle.com/javase/1.4.2/docs/tooldocs/windows/java.html#commandlineargfile

I'm not sure why the "java" launcher was over looked. I have problems running certain TestNG unit tests because the -classpath command line becomes too long (in Windows anyways)

This could be easily solved by Oracle if they added support for @argfile! Or if Windows ever enters the 21st century. I won't hold my breath...

Show
Kirk Rasmussen added a comment - For us unfortunate developers stuck on the Windows platform I have opened the following bug report (might take a few days to show up): http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=7162307 It turns out that javac has supported @argfile mechanism for a long time (since at least 1.4): http://docs.oracle.com/javase/1.4.2/docs/tooldocs/windows/java.html#commandlineargfile I'm not sure why the "java" launcher was over looked. I have problems running certain TestNG unit tests because the -classpath command line becomes too long (in Windows anyways) This could be easily solved by Oracle if they added support for @argfile! Or if Windows ever enters the 21st century. I won't hold my breath...
Hide
Permalink
Kirk Rasmussen added a comment -

FYI...

Surefire uses the empty JAR technique too but they also use the CLASSPATH environment variable as an option which doesn't have some the shortcomings of the MANIFEST techique:

http://maven.apache.org/plugins/maven-surefire-plugin/examples/class-loading.html

Show
Kirk Rasmussen added a comment - FYI... Surefire uses the empty JAR technique too but they also use the CLASSPATH environment variable as an option which doesn't have some the shortcomings of the MANIFEST techique: http://maven.apache.org/plugins/maven-surefire-plugin/examples/class-loading.html

People

  • Assignee:
    Unassigned
    Reporter:
    Ben Dotte
Vote (5)
Watch (7)

Dates

  • Created:
    Updated:
    Resolved: