Sorry, I've made a mistake in one critical sentence:

"I first had it with the snapshot 5, and after installing the snapshot 6 it did not repeat."

should be:

"I first had it with the snapshot 5, and after installing the snapshot 6 IT REPEATED."

Can you run with -XX:+HeapDumpOnOutOfMemoryError in your GRADLE_OPTS, and send us the resulting heap dump?

I tried to build the spock framework with gradle and got OOM. Should be easy to repro using current master. Took a heap dump in case it is the similar problem since it fails while running tests.

Heap dumps are too large to attach here ( you have 10MB attach limit).

So get from my dropbox:
http://dl.dropbox.com/u/14591789/java_pid9295.hprof.zip

BTW, I'm constantly running with HeapDumpOnOutOfMemoryError, but meanwhile the problem did not reproduce on my machine. I'll probably try limiting JVM heap size, just for the sake of creating that heap dump.

I've got OOM, but a heap dump file was not created. I only got the exception (attached). I've searched everywhere, but could not find a dump file. Is it possible that the exception was wrapped in another exception and no dump file was created?

OOM error stack trace

I am also running into this. With the max heap set to 640M .. gradle slows to a crawl spending at least half the time in GC. I need to run on 32-bit Java and sometimes when I up the heap to 1024M there can be problems allocating such a large block of memory.
The thing is, my build is just invoking gradle (from a gradle script) multiple times on a bunch of other projects... each project on it's own doesn't need much memory so after each one is done the heap usage should fall back and stabilize.. but the memory usage grows and seems excessive.

This is with 1.0-rc-3 (has been a problem since I can remember with milestone-3)
Is this really a 'minor' issue? I causes builds to fail and seems to be a real leak of some sort.

BTW, in my case the problem was with the unit tests runner - my unit tests were not optimized and accumulated too much memory during their execution.

I worked around it by forking a separate processes per test class. It can be configured in the master/build.gradle like this (see the first line with forkEvery):

        test {
          forkEvery = 1 // prevent Java OOM error on over 1 GB of mem usage

          excludes = ["**/Abstract*.class"]

          // reports about too long unit tests
          afterTest { descr, result ->
                        def time = result.endTime - result.startTime
                        if( time > 3000 )
                                logger.warn( "Test ${descr.className} ${descr.name} takes too much time: $time msec" )
          }
        }

Forking separate test processes did not increase the total execution time by more than 5-10% (AFAIR), and it significantly reduced the memory usage.

As announced on the Gradle blog we are planning to completely migrate issues from JIRA to GitHub.

We intend to prioritize issues that are actionable and impactful while working more closely with the community. Many of our JIRA issues are inactionable or irrelevant. We would like to request your help to ensure we can appropriately prioritize JIRA issues you’ve contributed to.

Please confirm that you still advocate for your JIRA issue before December 10th, 2016 by:

• Checking that your issues contain requisite context, impact, behaviors, and examples as described in our published guidelines.
• Leave a comment on the JIRA issue or open a new GitHub issue confirming that the above is complete.

We look forward to collaborating with you more closely on GitHub. Thank you for your contribution to Gradle!

Thanks again for reporting this issue. We haven't heard back from you after our inquiry from November 15th. We are closing this issue now. Please create an issue on GitHub if you still feel passionate about getting it resolved.