[GRADLE-1957] tests in milestone 5 or 6 hang forever Created: 24/Nov/11  Updated: 19/Jan/13  Resolved: 17/Jan/13

Status: Resolved
Project: Gradle
Affects Version/s: None
Fix Version/s: 1.4-rc-1

Type: Bug
Reporter: Gradle Forums Assignee: Unassigned
Resolution: Duplicate Votes: 5

Issue Links:
Duplicate
Duplicates GRADLE-2609 Build hangs forever when test JVM get... Resolved

 Description   

when I switch to milestone 5 or 6 from milestone 3 the tests hang for ever. They work fine in milestone 3.



 Comments   
Comment by Gradle Forums [ 24/Nov/11 ]

David,

We can't do anything without some more information. We really need a sample app that has this problem that we can run.

Comment by Gradle Forums [ 24/Nov/11 ]

Thanks Luke - I am trying to track it down.

We have stripped it down to use only the java plugin.

It appears to work with --debug enabled.

I'll try to get a sample app with the problem.

Thanks again
David

Comment by Gradle Forums [ 24/Nov/11 ]

Great, thanks.

Comment by Gradle Forums [ 24/Nov/11 ]

FYI:

When we fork a JVM per test the tests execute, we have also tried it forking every 100 tests and it works, every 2048 does not..

There are around 6k tests, they are all unit tests.

Has anything changed in the way JUnit is invoked to explain this behaviour?

Still trying to produce a sample.

Comment by Gradle Forums [ 24/Nov/11 ]

I have a sample that reproduces the problem. Basically if you run a lot of tests (somewhere between 1000 and 10000), running gradle test will hang forever.

I wrote a perl script to generate the tests:

use File::Path qw(mkpath);

my $package = "mars";
my $path="src/test/java/".$package;
mkpath $path;

for($i=0;$i<10000;$i++) {
open ( F, ">".$path."/Spaceman".$i."Test.java") or die $!;
print F "package ".$package.";\n public class Spaceman".$i."Test \n{\n @org.junit.Test public void gradleRunsMe()

{System.out.println(\"HI\");}

\n}";
close(F);
}

>
> and then I used the following build.gradle:
>

apply plugin: 'java'

dependencies {
testCompile (group: 'junit',name: 'junit-dep',version: '4.10') {
exclude module: 'hamcrest-core'
}
}
repositories {
mavenCentral()
}

>
> and then I run
> % gradle test
>

Comment by Gradle Forums [ 24/Nov/11 ]

Hi David,

Thanks for going to the trouble to create that. I can indeed reproduce it here with that script. We'll take a look and let you know.

Comment by Mike M. [ 13/Dec/11 ]

We also have this issue. However, we only have about 130 Tests that use Spring and Hibernate.
In our case, it's one specific JUnit Test Class that seems to cause the issue. This test creates new threads and interrupts them after some time. It was doing that with plain old java.lang.Thread's until now. I migrated the code to the java.util.concurrent.Executor framework with Future<?>'s because I suspected the main thread interruption to cause the Gradle executor to behave strangely. However, switching to Executors didn't solve the problem. Also, the execution doesn't always hang, it only appears sporadically. But at least in our case, it definitely seems to be thread / concurrency related.

When I keep the build running, after some time, I get an OOM error, maybe the stacktrace helps:

Running test: test testSomeConcurrencyStuff(com...SomeConcurrencyTestCase)
Could not dispatch message [Request consumer: dd826ee8-8632-4c38-b999-edfbd4126069, payload: org.gradle.messaging.remote.internal.protocol.RemoteMethodInvocation@1]. Discarding message. 
java.lang.OutOfMemoryError: GC overhead limit exceeded
        at java.io.ObjectOutputStream$BlockDataOutputStream.<init>(ObjectOutputStream.java:1724)
        at java.io.ObjectOutputStream.<init>(ObjectOutputStream.java:225)
        at org.gradle.messaging.remote.internal.Message$ExceptionReplacingObjectOutputStream.<init>(Message.java:136)
        at org.gradle.messaging.remote.internal.Message.send(Message.java:27)
        at org.gradle.messaging.remote.internal.DefaultMessageSerializer.write(DefaultMessageSerializer.java:35)
        at org.gradle.messaging.remote.internal.inet.SocketConnection.dispatch(SocketConnection.java:96)
        at org.gradle.messaging.remote.internal.DelegatingConnection.dispatch(DelegatingConnection.java:35)
        at org.gradle.messaging.dispatch.FailureHandlingDispatch.dispatch(FailureHandlingDispatch.java:29)
        at org.gradle.messaging.dispatch.AsyncDispatch.dispatchMessages(AsyncDispatch.java:132)
        at org.gradle.messaging.dispatch.AsyncDispatch.access$000(AsyncDispatch.java:33)
        at org.gradle.messaging.dispatch.AsyncDispatch$1.run(AsyncDispatch.java:72)
        at org.gradle.messaging.concurrent.DefaultExecutorFactory$StoppableExecutorImpl$1.run(DefaultExecutorFactory.java:64)
        at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
        at java.lang.Thread.run(Thread.java:662)
> Building > :my-project:test > 67 tests completed
Comment by Adam Murdoch [ 13/Dec/11 ]

Could you try out a milestone-7 snapshot, to confirm that this issue is fixed: http://repo.gradle.org/gradle/distributions-snapshots/gradle-1.0-milestone-7-20111209194225+0100-bin.zip

Comment by Adam Murdoch [ 13/Dec/11 ]

Actually, I suspect the original issue isn't fixed.

Comment by Mike M. [ 14/Dec/11 ]

Adam,
I don't know what you mean by the "original issue", but I tried the 1.0-milestone-7 build you suggested and it definitely fixes the issue for our build (had our whole test suite running about 10 times now without it blocking a single time).
So the only thing for me to do before re-enabling the threaded test in CI and for our team is to wait for the 1.0-milestone-7 release
Thank you!

Comment by Mauro Molinari [ 10/Jan/12 ]

I also experienced this problem with milestone-5 (and 6). I solved it by enabling forking every x tests (where x is not too large). Looking at the thread dumps of the running Gradle process, I noticed that the Gradle threads were waiting, while my test code was not executing at all.

I didn't try milestone-7.

Comment by Connor Garvey [ 20/Jan/12 ]

I'm migrating from Maven to Gradle and have this problem. I don't know whether a certain test is causing it, but the hang point is deterministic. The tests always hang at the same point.

> Building > :test > 1786 tests completed, 157 failures

Following Mauro's advice and setting forkEvery to 100 allowed all 3080 tests to finish.

Also, the build went from 0 failures to 201.

Comment by Connor Garvey [ 23/Jan/12 ]

For me, the actual cause was permgen space running out, as in GRADLE-1149. This worked for me, although I hope to fix the project and not need extra permgen space.

test {
  jvmArgs '-XX:MaxPermSize=256m'
}
Comment by Adam Murdoch [ 27/Dec/12 ]

This should be fixed now. Can you try a recently nightly build from http://gradle.org/nightly and let us know if the problem is fixed for you or not?

Generated at Wed Jun 30 12:08:55 CDT 2021 using Jira 8.4.2#804003-sha1:d21414fc212e3af190e92c2d2ac41299b89402cf.