Gradle
  1. Gradle
  2. GRADLE-1420

Forked groovyc compilation can create a classpath line too long for Windows

    Details

    • Type: Bug Bug
    • Status: Resolved Resolved
    • Resolution: Fixed
    • Affects Version/s: 1.0-milestone-1
    • Fix Version/s: 1.0-rc-1

      Description

      We hit the same issue documented here:

      http://gradle.1045684.n5.nabble.com/Groovyc-fork-failing-on-Windows-td3357060.html

      When gradle tries to compile (in our case) a mixed groovy/java environment on a Windows machine it returns an exception:

      java.io.IOException: Cannot run program "c:\PROGRA~1\Java\jdk1.6.0_10
      \jre\bin\java": CreateProcess error=87, The parameter is incorrect"

      This is due to the command exceeding the size limit on command-line arguments in Windows. The build works fine in Linux.

      Peter suggests turning off forking on the above thread as a workaround:

      [compileGroovy, compileTestGroovy]*.groovyOptions.fork = false

      The process indeed does not fork with that line, but it just hangs (even with GRADLE_OPTS set to 1G).

        Issue Links

          Activity

          Hide
          Peter Niederwieser
          added a comment -

          Gradle currently puts too much on the compile class path. Once that is fixed, the "too long class path" problem will probably go away automatically.

          Show
          Peter Niederwieser
          added a comment - Gradle currently puts too much on the compile class path. Once that is fixed, the "too long class path" problem will probably go away automatically.
          Contegix Support
          made changes -
          Field Original Value New Value
          Project Import Sat Mar 19 09:23:24 CDT 2011 [ 1300544604020 ]
          Hide
          Ray Nicholus
          added a comment - - edited

          As Ben mentioned, preventing the fork (which should cut the classpath size in half), causes the process to hang, so that is not an option.

          An easy workaround, theoretically, would be to create an empty jar with the project's classpath specified in the jar's Class-Path manifest attribute. I attempted to do this with Gradle, but it seems Gradle specifies absolute paths for each classpath entry. Apparently, absolute paths are not valid in the manifest's Class-Path attribute. This appears to be true based on Sun's manifest attribute spec at http://download.oracle.com/javase/1.4.2/docs/guide/jar/jar.html, as well as from my personal experience.

          Putting the pathing jar concept aside for a moment, it would be nice if Gradle specified relative paths for classpath entries, instead of absolute paths, wherever possible. This, for one, would decrease the size of the classpath. Back to pathing jars: If this behavior is introduced, a pathing jar can be used to address long classpath issues such as this. Note, that for cases where the external dependency cache is not on the same drive as the build/source initially, the GRADLE_USER_HOME env variable can be set to the build/source drive, making it possible to reference these classpath entries relative to the pathing jar.

          Show
          Ray Nicholus
          added a comment - - edited As Ben mentioned, preventing the fork (which should cut the classpath size in half), causes the process to hang, so that is not an option. An easy workaround, theoretically, would be to create an empty jar with the project's classpath specified in the jar's Class-Path manifest attribute. I attempted to do this with Gradle, but it seems Gradle specifies absolute paths for each classpath entry. Apparently, absolute paths are not valid in the manifest's Class-Path attribute. This appears to be true based on Sun's manifest attribute spec at http://download.oracle.com/javase/1.4.2/docs/guide/jar/jar.html , as well as from my personal experience. Putting the pathing jar concept aside for a moment, it would be nice if Gradle specified relative paths for classpath entries, instead of absolute paths, wherever possible. This, for one, would decrease the size of the classpath. Back to pathing jars: If this behavior is introduced, a pathing jar can be used to address long classpath issues such as this. Note, that for cases where the external dependency cache is not on the same drive as the build/source initially, the GRADLE_USER_HOME env variable can be set to the build/source drive, making it possible to reference these classpath entries relative to the pathing jar.
          Hide
          Peter Niederwieser
          added a comment -

          Is it you who asked the Gradle pathing Jar question on stackoverflow.com? I recently fixed my answer and it now works fine for me. Absolute paths don't seem to be a problem here.

          Relative paths typically won't be shorter because dependencies come from Gradle's Ivy cache, which doesn't share a common parent with the build's working directory. What we could do is to try to substitute a part of the path with $GRADLE_USER_HOME.

          How big are the projects that suffer from this problem?

          Show
          Peter Niederwieser
          added a comment - Is it you who asked the Gradle pathing Jar question on stackoverflow.com? I recently fixed my answer and it now works fine for me. Absolute paths don't seem to be a problem here. Relative paths typically won't be shorter because dependencies come from Gradle's Ivy cache, which doesn't share a common parent with the build's working directory. What we could do is to try to substitute a part of the path with $GRADLE_USER_HOME. How big are the projects that suffer from this problem?
          Hide
          Ray Nicholus
          added a comment -

          Hi Peter. Before you posted the adjustment on stack, I realized that spaces were required and tried this instead - no luck. I attempted this again in case I missed something, and, as before, the compile fails due to an inability to find dependencies.

          Show
          Ray Nicholus
          added a comment - Hi Peter. Before you posted the adjustment on stack, I realized that spaces were required and tried this instead - no luck. I attempted this again in case I missed something, and, as before, the compile fails due to an inability to find dependencies.
          Daz DeBoer
          made changes -
          Workflow jira [ 12642 ] jira with pivotal tracker [ 13502 ]
          Hide
          Jesper Skov
          added a comment -

          This should be fixed in Groovy. I've created http://jira.codehaus.org/browse/GROOVY-5024 which includes a partial fix (and hints for a complete fix).

          Show
          Jesper Skov
          added a comment - This should be fixed in Groovy. I've created http://jira.codehaus.org/browse/GROOVY-5024 which includes a partial fix (and hints for a complete fix).
          Hide
          Ray Nicholus
          added a comment -

          Thanks for the update. Which version of groovy includes this fix?

          Show
          Ray Nicholus
          added a comment - Thanks for the update. Which version of groovy includes this fix?
          Hide
          Guillaume Laforge
          added a comment -

          Ray, it's not in a released version of Groovy yet.
          I tried the patch indicated in the Groovy JIRA issue, but alas, it's failing the Groovy build.
          So it seems the patch needs a bit more work.

          Show
          Guillaume Laforge
          added a comment - Ray, it's not in a released version of Groovy yet. I tried the patch indicated in the Groovy JIRA issue, but alas, it's failing the Groovy build. So it seems the patch needs a bit more work.
          Hide
          Ray Nicholus
          added a comment -

          Tasks for gradle build file to address groovyc "classpath too long" issue.

          Show
          Ray Nicholus
          added a comment - Tasks for gradle build file to address groovyc "classpath too long" issue.
          Ray Nicholus
          made changes -
          Attachment classpath-too-long.txt [ 10349 ]
          Hide
          Ray Nicholus
          added a comment - - edited

          I finally got the "pathing jar" idea to work. I consider this to be a permanent workaround. This could be considered a solution if it is made part of gradle itself.

          The original pathing jar code was provided by Peter, but it didn't work. The problem: classpath elements referenced in the pathing jar must be relative to the location of the pathing jar. So, this appears to work for me (see attached classpath-too-long.txt).

          Show
          Ray Nicholus
          added a comment - - edited I finally got the "pathing jar" idea to work. I consider this to be a permanent workaround. This could be considered a solution if it is made part of gradle itself. The original pathing jar code was provided by Peter, but it didn't work. The problem: classpath elements referenced in the pathing jar must be relative to the location of the pathing jar. So, this appears to work for me (see attached classpath-too-long.txt).
          Pivotal Tracker Integration
          made changes -
          Status Open [ 1 ] In Progress [ 3 ]
          Adam Murdoch
          made changes -
          Fix Version/s 1.0-rc-1 [ 10062 ]
          Adam Murdoch
          made changes -
          Status In Progress [ 3 ] Resolved [ 5 ]
          Resolution Fixed [ 1 ]
          Hide
          Kirk Rasmussen
          added a comment -

          For us unfortunate developers stuck on the Windows platform I have opened the following bug report (might take a few days to show up):

          http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=7162307

          It turns out that javac has supported @argfile mechanism for a long time (since at least 1.4):

          http://docs.oracle.com/javase/1.4.2/docs/tooldocs/windows/java.html#commandlineargfile

          I'm not sure why the "java" launcher was over looked. I have problems running certain TestNG unit tests because the -classpath command line becomes too long (in Windows anyways)

          This could be easily solved by Oracle if they added support for @argfile! Or if Windows ever enters the 21st century. I won't hold my breath...

          Show
          Kirk Rasmussen
          added a comment - For us unfortunate developers stuck on the Windows platform I have opened the following bug report (might take a few days to show up): http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=7162307 It turns out that javac has supported @argfile mechanism for a long time (since at least 1.4): http://docs.oracle.com/javase/1.4.2/docs/tooldocs/windows/java.html#commandlineargfile I'm not sure why the "java" launcher was over looked. I have problems running certain TestNG unit tests because the -classpath command line becomes too long (in Windows anyways) This could be easily solved by Oracle if they added support for @argfile! Or if Windows ever enters the 21st century. I won't hold my breath...
          Hide
          Kirk Rasmussen
          added a comment -

          FYI...

          Surefire uses the empty JAR technique too but they also use the CLASSPATH environment variable as an option which doesn't have some the shortcomings of the MANIFEST techique:

          http://maven.apache.org/plugins/maven-surefire-plugin/examples/class-loading.html

          Show
          Kirk Rasmussen
          added a comment - FYI... Surefire uses the empty JAR technique too but they also use the CLASSPATH environment variable as an option which doesn't have some the shortcomings of the MANIFEST techique: http://maven.apache.org/plugins/maven-surefire-plugin/examples/class-loading.html
          Pivotal Tracker Integration
          made changes -
          Status Resolved [ 5 ] Closed [ 6 ]
          Luke Daley
          made changes -
          Workflow jira with pivotal tracker [ 13502 ] jira with pivotal tracker (no resolved, only closed) [ 18213 ]
          Luke Daley
          made changes -
          Workflow jira with pivotal tracker (no resolved, only closed) [ 18213 ] Copy of jira with pivotal tracker (no closed, only resolved) [ 20824 ]
          Status Closed [ 6 ] Resolved [ 5 ]

            People

            • Assignee:
              Unassigned
              Reporter:
              Ben Dotte
            • Votes:
              5 Vote for this issue
              Watchers:
              7 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: