Uploaded image for project: 'Gradle'
  1. Gradle
  2. GRADLE-3122

Copy spec expand() fails on files bigger than 64K

    Details

    • Type: Bug
    • Status: Resolved
    • Resolution: Won't Fix
    • Affects Version/s: 2.0
    • Fix Version/s: None
    • Gradle Forums topic Reference:

      Description

      While attempting to migrate our first project from Maven to Gradle we are running in the following issue.

      Here is a simple gradle script:

      task filterBigFile(type: Copy)

      { from '.' into 'build/big_file_filtered.txt' include 'big_file.txt' expand(expand_me: 'test') }

      If 'big_file' is bigger than 64K, the copy task will fail with the following stacktrace: [1]https://gist.github.com/anonymous/20b...

      Reproduced with Gradle 2.0 and 1.12.

      Filtering with maven is working as expected.

        Activity

        forums Gradle Forums created issue -
        ldaley Luke Daley made changes -
        Field Original Value New Value
        Summary Cannot filter files bigger than 64K Copy spec expand() fails on files bigger than 64K
        Affects Version/s 2.0 [ 10961 ]
        Description While attempting to migrate our first project from Maven to Gradle we are running in the following issue.

        Here is a simple gradle script:

        task filterBigFile(type: Copy) {
        from '.'
        into 'build/big_file_filtered.txt'
        include 'big_file.txt'
        expand(expand_me: 'test')
        }


        If 'big_file' is bigger than 64K, the copy task will fail with the following stacktrace: [1]https://gist.github.com/anonymous/20b...

        Reproduced with Gradle 2.0 and 1.12.

        Filtering with maven is working as expected.

        Any feedback is welcome.

        Thanks,
        Marc
        ----------------------------------------------------------------------------------------
        [1] https://gist.github.com/anonymous/20b64ee5992c312bc01d
        While attempting to migrate our first project from Maven to Gradle we are running in the following issue.

        Here is a simple gradle script:

        task filterBigFile(type: Copy) {
        from '.'
        into 'build/big_file_filtered.txt'
        include 'big_file.txt'
        expand(expand_me: 'test')
        }


        If 'big_file' is bigger than 64K, the copy task will fail with the following stacktrace: [1]https://gist.github.com/anonymous/20b...

        Reproduced with Gradle 2.0 and 1.12.

        Filtering with maven is working as expected.
        ldaley Luke Daley made changes -
        Comment [ btw, this is the Groovy issue: [1]http://jira.codehaus.org/browse/GROOV...
        ----------------------------------------------------------------------------------------
        [1] http://jira.codehaus.org/browse/GROOVY-3457 ]
        ldaley Luke Daley made changes -
        Comment [ Hi Marc,

        Can you try with Gradle 2.0 please to see if this is fixed by the newer Groovy version.

        If it doesn't work what we might look at doing is use [1]https://github.com/mbjarland/groovy-s... as the basis for this templating to get around this limit.
        ----------------------------------------------------------------------------------------
        [1] https://github.com/mbjarland/groovy-streaming-template-engine ]
        ldaley Luke Daley made changes -
        Comment [ Hello Luke, thanks for the hint!

        However, it seems that the filter method as other shortcomings:
        - Using the ReplaceTokens ant filter (as proposed in the documentation) requires a Hashtable, which does not accept null as key or value. So, I cannot directly use the HashMap project.properties.
        - ReplaceTokens only allows a single character delimiter, in turn, we cannot use the maven default syntax (${...}) for filtering (we have some shared resource files that are used both by maven and gradle).

        The documentation only sparingly describes the differences between expand and filter. The 64k limit should really be explicitly stated IMHO.

        I could probably work around these limitations one way or another, but I will submit a ticket on the subject I think. (filtering is such an important feature, we should at least have parity with maven) ]
        ldaley Luke Daley made changes -
        Comment [ expand() just won't work on a file this big. The filter() methods will work though, and will be more efficient. ]
        Hide
        ldaley Luke Daley added a comment -
        Show
        ldaley Luke Daley added a comment - This is fundamentally http://jira.codehaus.org/browse/GROOVY-3457
        Hide
        etienne Etienne Studer added a comment - - edited

        I've looked into https://github.com/mbjarland/groovy-streaming-template-engine which might be a suitable option, but it is licensed under GNU. Also, it currently does not handle $var, but only ${var}. If those two things can be 'fixed', we should look into it again. I've contacted Matias and I've also asked him to consider creating a pull request with his contribution for the Groovy project.

        Show
        etienne Etienne Studer added a comment - - edited I've looked into https://github.com/mbjarland/groovy-streaming-template-engine which might be a suitable option, but it is licensed under GNU. Also, it currently does not handle $var, but only ${var}. If those two things can be 'fixed', we should look into it again. I've contacted Matias and I've also asked him to consider creating a pull request with his contribution for the Groovy project.
        Hide
        ldaley Luke Daley added a comment -

        I think we should push getting this “fixed” in Groovy first.

        This may mean that we need to work with the Groovy folks and contribute the “fix”.

        Show
        ldaley Luke Daley added a comment - I think we should push getting this “fixed” in Groovy first. This may mean that we need to work with the Groovy folks and contribute the “fix”.
        Hide
        etienne Etienne Studer added a comment -

        Pull request that is ready to be merged into groovy-core has been submitted by Matias to the Groovy project.

        Show
        etienne Etienne Studer added a comment - Pull request that is ready to be merged into groovy-core has been submitted by Matias to the Groovy project.
        Hide
        bmuschko Benjamin Muschko added a comment -

        As announced on the Gradle blog we are planning to completely migrate issues from JIRA to GitHub.

        We intend to prioritize issues that are actionable and impactful while working more closely with the community. Many of our JIRA issues are inactionable or irrelevant. We would like to request your help to ensure we can appropriately prioritize JIRA issues you’ve contributed to.

        Please confirm that you still advocate for your JIRA issue before December 10th, 2016 by:

        • Checking that your issues contain requisite context, impact, behaviors, and examples as described in our published guidelines.
        • Leave a comment on the JIRA issue or open a new GitHub issue confirming that the above is complete.

        We look forward to collaborating with you more closely on GitHub. Thank you for your contribution to Gradle!

        Show
        bmuschko Benjamin Muschko added a comment - As announced on the Gradle blog we are planning to completely migrate issues from JIRA to GitHub. We intend to prioritize issues that are actionable and impactful while working more closely with the community. Many of our JIRA issues are inactionable or irrelevant. We would like to request your help to ensure we can appropriately prioritize JIRA issues you’ve contributed to. Please confirm that you still advocate for your JIRA issue before December 10th, 2016 by: Checking that your issues contain requisite context, impact, behaviors, and examples as described in our published guidelines . Leave a comment on the JIRA issue or open a new GitHub issue confirming that the above is complete. We look forward to collaborating with you more closely on GitHub. Thank you for your contribution to Gradle!
        Hide
        bmuschko Benjamin Muschko added a comment -

        The Groovy issue https://issues.apache.org/jira/browse/GROOVY-3457 has been fixed in Groovy 2.4.0-beta-4. The latest version of Gradle uses Groovy 2.4.7. I am assuming this is fixed. Please open an issue on GitHub if it still occurs.

        Show
        bmuschko Benjamin Muschko added a comment - The Groovy issue https://issues.apache.org/jira/browse/GROOVY-3457 has been fixed in Groovy 2.4.0-beta-4. The latest version of Gradle uses Groovy 2.4.7. I am assuming this is fixed. Please open an issue on GitHub if it still occurs.
        bmuschko Benjamin Muschko made changes -
        Status Open [ 1 ] Resolved [ 5 ]
        Resolution Won't Fix [ 2 ]

          People

          • Assignee:
            Unassigned
            Reporter:
            forums Gradle Forums
          • Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development