Uploaded image for project: 'Gradle'
  1. Gradle
  2. GRADLE-2959

zipTree unable to overwrite temporary read-only files

    Details

    • Type: Bug
    • Status: Resolved
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 2.9-rc-1
    • Gradle Forums topic Reference:

      Description

      Hi,
      environment debian linux 6.0, gradle 1.1 and 1.5 rc3
      I'm building a release task similar to a fatJar. This is my task (the relevant part):

      task fatJar(type: Jar) {
      from configurations.runtime.filter{
      it.path.matches(".com\.amadego.") // include only few internal jars
      }.collect {
      print("including $

      {it}

      (TODO:exclude META-INF/**)\n")
      zipTree(it)
      }

      from files(sourceSets.main.output.classesDir)
      classifier = 'fatJar'
      }

      I'm facing a strange behaviour:
      If I run this task the first time it works, but the second time it doesn't. Going through debugging I've spotted that there is a read-only property file that cannot be overwritten in the second run.

      What is the proper way to manage this issue?
      Should I delete the temporary dir before every build? Shouldn't it be done automatically at every build?

      Thanks for your help!

        Issue Links

          Activity

          Hide
          forums Gradle Forums added a comment -

          It's hard to say what's going on, but the task declaration has some problems. In particular, it resolves the `runtime` configuration in the configuration phase instead of the execution phase. Try this instead:

          task fatJar(type: Jar) {
          from { configurations.runtime.filter {
          it.path.matches("...") // adapt as necessary (couldn't get this to display right in forums)
          }.collect

          { zipTree(it) }}

          from sourceSets.main.output
          classifier = 'fatJar'
          }

          Show
          forums Gradle Forums added a comment - It's hard to say what's going on, but the task declaration has some problems. In particular, it resolves the `runtime` configuration in the configuration phase instead of the execution phase. Try this instead: task fatJar(type: Jar) { from { configurations.runtime.filter { it.path.matches("...") // adapt as necessary (couldn't get this to display right in forums) }.collect { zipTree(it) }} from sourceSets.main.output classifier = 'fatJar' }
          Hide
          forums Gradle Forums added a comment -

          Thanks Peter,
          I will explain it better with a simple example.

          Imagine that this is the layout of the project:

          src/main/java/HelloWorld.java
          lib/zipped_readonly_property_file.zip
          build.gradle

          where the file "lib/zipped_readonly_property_file.zip" is a zip/jar/whatever compressed archive including a readonly file (in the example, it contains simply a HelloWorld.properties file) and that at least one of the file inside the zip is a read only file.

          This is the build.gradle:

          apply plugin: "java"

          dependencies {
          runtime files("lib/zipped_readonly_property_file.zip")
          }

          task fatJar(type: Jar) {
          from {
          configurations.runtime.filter {
          it.path.matches(".*")
          }.collect {
          zipTree(it)
          }
          }
          from sourceSets.main.output
          classifier = 'fatJar'
          }

          Now, if you run "gradle fatJar", gradle will expand the zip file in a temporary folder, and then compress it into the fat jar. Fine, it works as expected. This is the command output:

          :compileJava
          :processResources UP-TO-DATE
          :classes
          :fatJar

          BUILD SUCCESSFUL

          Now, for a second time, try to run "gradle fatJar". This time the zip file will be expanded again OVER the previously extracted folder, but since the property file was read-only, gradle will fail complaining that:

          :compileJava UP-TO-DATE
          :processResources UP-TO-DATE
          :classes UP-TO-DATE
          :fatJar

          FAILURE: Build failed with an exception.

          • What went wrong:
            Could not expand ZIP '/home/larzeni/.eclipseWorkspace/gradleReadOnlyTest/lib/zipped_readonly_property_file.zip'.
          • Try:
            Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output.

          BUILD FAILED

          And if you go through using --debug option, the problem cause can be traced to:

          org.gradle.api.GradleException: Could not copy zip entry /home/larzeni/.eclipseWorkspace/gradleReadOnlyTest/lib/zipped_readonly_property_file.zip!HelloWorld.properties to '/home/larzeni/.eclipseWorkspace/gradleReadOnlyTest/build/tmp/expandedArchives/zipped_readonly_property_file.zip_67dpdot7v9plqhjmhvtlkg3gjd/HelloWorld.properties'

          That is, gradle cannot overwrite a zip that includes a read-only file. Obviously if you run gradle clean , the tmp/zip-expanded directory will be removed and the build will work fine again.

          Possible solutions coud be:
          A) do not re-expand a zipped file which was already expanded if it's not changed or,
          B) delete the temporary dir that contains the expanded zip after the build.

          I hope now the problem is more understandable.

          If you want, I've put the files for the example at the following link:
          [1]https://www.box.com/s/sb5obht20jhmgjg...

          Thanks for your help!
          ----------------------------------------------------------------------------------------
          [1] https://www.box.com/s/sb5obht20jhmgjg7c1t5

          Show
          forums Gradle Forums added a comment - Thanks Peter, I will explain it better with a simple example. Imagine that this is the layout of the project: src/main/java/HelloWorld.java lib/zipped_readonly_property_file.zip build.gradle where the file "lib/zipped_readonly_property_file.zip" is a zip/jar/whatever compressed archive including a readonly file (in the example, it contains simply a HelloWorld.properties file) and that at least one of the file inside the zip is a read only file. This is the build.gradle: apply plugin: "java" dependencies { runtime files("lib/zipped_readonly_property_file.zip") } task fatJar(type: Jar) { from { configurations.runtime.filter { it.path.matches(".*") }.collect { zipTree(it) } } from sourceSets.main.output classifier = 'fatJar' } Now, if you run "gradle fatJar", gradle will expand the zip file in a temporary folder, and then compress it into the fat jar. Fine, it works as expected. This is the command output: :compileJava :processResources UP-TO-DATE :classes :fatJar BUILD SUCCESSFUL Now, for a second time, try to run "gradle fatJar". This time the zip file will be expanded again OVER the previously extracted folder, but since the property file was read-only, gradle will fail complaining that: :compileJava UP-TO-DATE :processResources UP-TO-DATE :classes UP-TO-DATE :fatJar FAILURE: Build failed with an exception. What went wrong: Could not expand ZIP '/home/larzeni/.eclipseWorkspace/gradleReadOnlyTest/lib/zipped_readonly_property_file.zip'. Try: Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. BUILD FAILED And if you go through using --debug option, the problem cause can be traced to: org.gradle.api.GradleException: Could not copy zip entry /home/larzeni/.eclipseWorkspace/gradleReadOnlyTest/lib/zipped_readonly_property_file.zip!HelloWorld.properties to '/home/larzeni/.eclipseWorkspace/gradleReadOnlyTest/build/tmp/expandedArchives/zipped_readonly_property_file.zip_67dpdot7v9plqhjmhvtlkg3gjd/HelloWorld.properties' That is, gradle cannot overwrite a zip that includes a read-only file. Obviously if you run gradle clean , the tmp/zip-expanded directory will be removed and the build will work fine again. Possible solutions coud be: A) do not re-expand a zipped file which was already expanded if it's not changed or, B) delete the temporary dir that contains the expanded zip after the build. I hope now the problem is more understandable. If you want, I've put the files for the example at the following link: [1] https://www.box.com/s/sb5obht20jhmgjg... Thanks for your help! ---------------------------------------------------------------------------------------- [1] https://www.box.com/s/sb5obht20jhmgjg7c1t5
          Hide
          forums Gradle Forums added a comment -

          Could someone answer for this because I'm facing the EXACT same problem? Simply put I want gradle to either skip complaining about not being able to overwrite the file or to clean up the temp area before trying to write. Either solution would be fine. I'm surprised this doesn't come up more often.

          Show
          forums Gradle Forums added a comment - Could someone answer for this because I'm facing the EXACT same problem? Simply put I want gradle to either skip complaining about not being able to overwrite the file or to clean up the temp area before trying to write. Either solution would be fine. I'm surprised this doesn't come up more often.
          Hide
          forums Gradle Forums added a comment -

          Same issue here. Just upgraded to 1.9, thinking surely this would be fixed by now. Still same issue.

          Show
          forums Gradle Forums added a comment - Same issue here. Just upgraded to 1.9, thinking surely this would be fixed by now. Still same issue.
          Hide
          lhotari Lari Hotari added a comment -

          zipTree/tarTree as input no more creates temp directory by default. A temp directory still gets created if the task calls the getFiles method on the zipTree/tarTree instance. However this doesn't happen in most cases.

          Show
          lhotari Lari Hotari added a comment - zipTree/tarTree as input no more creates temp directory by default. A temp directory still gets created if the task calls the getFiles method on the zipTree/tarTree instance. However this doesn't happen in most cases.

            People

            • Assignee:
              lhotari Lari Hotari
              Reporter:
              forums Gradle Forums
            • Votes:
              4 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development