[GRADLE-2964] When Gradle makes very large JAR files, they're corrupt Created: 26/Nov/13 Updated: 03/Mar/14 Resolved: 03/Mar/14 |
|
Status: | Resolved |
Project: | Gradle |
Affects Version/s: | None |
Fix Version/s: | 1.12-rc-1 |
Type: | Bug | ||
Reporter: | Gradle Forums | Assignee: | Luke Daley |
Resolution: | Fixed | Votes: | 0 |
Description |
This build script generates a load of dummy resources, packs them into a jar, and verifies the jar using the command-line zip tool: apply plugin: 'java' def generatedResourcesDir = new File(project.buildDir, "generated-resources") task generateResources << { sourceSets { task verifyJar(dependsOn: jar) << { def generatedResourcesDir = new File(project.buildDir, "generated-resources") task generateResources << { sourceSets { task verifyJar(dependsOn: jar) << { def generatedResourcesDir = new File(project.buildDir, "generated-resources") task generateResources << { sourceSets { task verifyJar(dependsOn: jar) << { When i run this with gradle 1.9, it fails with: FAILURE: Build failed with an exception.
zip error: Zip file structure invalid (/home/tanderson/workspace/GradleZipBug/build/libs/GradleZipBug.jar)". Expression: (status == 0). Values: status = 3 I get the same error with 1.7, and a slightly different one with 1.8. This is a pretty stupid build, but it's a distillation of a genuine case i have where i am building an uberjar containing an application and all its dependencies that ends up being absolutely gigantic - about 90 MB. That is also a stupid build, but it's the build i am currently chained to, and it's not likely to get substantially less stupid any time soon. However, with that build, it works fine under 1.7, and fails under 1.8 ( i have yet to try it with 1.9). When i examine the jar files produced by that build using 1.7 and 1.8, i see that the corrupt file produced by 1.8 has a truncated end of central directory record at the end; rather than being the 22-byte structure described in the specification, it is only 8 bytes, suggesting a buffering issue somewhere. If you play with the parameters of the dummy resource generation to make more, smaller, files, you can get different results: 1.7 produces jars with valid central directories, whereas 1.8 does not. Sadly, this test still fails, because the command-line zip program cannot handle archives with more than 65535 entries, even though the various bits of jar support in Java can. My Java is: java version "1.6.0_26" My lsb_release -a is: lsb_release -a Is this a known bug? Is there a workaround? Is this something to do with gradle, or a problem with the JDK? |
Comments |
Comment by Szczepan Faber [ 23/Jan/14 ] |
This may be related to the limitations of the ZipOutputStream that we use internally for handling zips. Large zips (>4g, >65k entries) need extra handling (Zip64). Typically this is handled automatically by ZipOutputStream, but not always (see org.apache.tools.zip.ZipOutputStream#setUseZip64). The example project produces about 62k files that weight 8g, hence it might be related. If my theory stands, a change in Gradle is needed to support those cases. |