[GRADLE-3146] Checkstyle task hangs during the parallel build Created: 01/Aug/14  Updated: 24/Jan/17  Resolved: 24/Jan/17

Status: Resolved
Project: Gradle
Affects Version/s: None
Fix Version/s: None

Type: Bug
Reporter: Szczepan Faber Assignee: Lari Hotari
Resolution: Won't Fix Votes: 1


 Description   

Checkstyle task is not thread safe and may cause the --parallel build to hang. Relevant stack trace of a couple of worker threads:

"Daemon" prio=10 tid=0x00007fcf4803d000 nid=0x479c runnable [0x00007fcf646c0000]
   java.lang.Thread.State: RUNNABLE
	at java.util.HashMap.get(HashMap.java:303)
	at com.puppycrawl.tools.checkstyle.api.Utils.getPattern(Utils.java:161)
	at com.puppycrawl.tools.checkstyle.api.Utils.getPattern(Utils.java:144)
	at com.puppycrawl.tools.checkstyle.filters.SuppressWithNearbyCommentFilter.setCommentFormat(SuppressWithNearbyCommentFilter.java:334)
	at com.puppycrawl.tools.checkstyle.filters.SuppressWithNearbyCommentFilter.<init>(SuppressWithNearbyCommentFilter.java:312)
	at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
	at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:39)
	at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:27)
	at java.lang.reflect.Constructor.newInstance(Constructor.java:513)
	at java.lang.Class.newInstance0(Class.java:355)
	at java.lang.Class.newInstance(Class.java:308)
	at com.puppycrawl.tools.checkstyle.PackageObjectFactory.createObject(PackageObjectFactory.java:113)
	at com.puppycrawl.tools.checkstyle.PackageObjectFactory.doMakeObject(PackageObjectFactory.java:91)
	at com.puppycrawl.tools.checkstyle.PackageObjectFactory.createModule(PackageObjectFactory.java:147)
	at com.puppycrawl.tools.checkstyle.Checker.setupChild(Checker.java:153)
	at com.puppycrawl.tools.checkstyle.api.AutomaticBean.configure(AutomaticBean.java:184)
	at com.puppycrawl.tools.checkstyle.CheckStyleTask.createChecker(CheckStyleTask.java:374)
	at com.puppycrawl.tools.checkstyle.CheckStyleTask.realExecute(CheckStyleTask.java:300)
	at com.puppycrawl.tools.checkstyle.CheckStyleTask.execute(CheckStyleTask.java:262)
	at org.apache.tools.ant.UnknownElement.execute(UnknownElement.java:292)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
	at java.lang.reflect.Method.invoke(Method.java:597)
	at org.apache.tools.ant.dispatch.DispatchUtils.execute(DispatchUtils.java:106)
	at groovy.util.AntBuilder.performTask(AntBuilder.java:260)
	at groovy.util.AntBuilder.nodeCompleted(AntBuilder.java:220)
	at org.gradle.api.internal.project.ant.BasicAntBuilder.nodeCompleted(BasicAntBuilder.java:71)
	at sun.reflect.GeneratedMethodAccessor1131.invoke(Unknown Source)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
	at java.lang.reflect.Method.invoke(Method.java:597)
	at org.codehaus.groovy.runtime.callsite.PojoMetaMethodSite$PojoCachedMethodSiteNoUnwrapNoCoerce.invoke(PojoMetaMethodSite.java:230)
	at org.codehaus.groovy.runtime.callsite.PojoMetaMethodSite.call(PojoMetaMethodSite.java:53)
	at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:42)
	at org.codehaus.groovy.runtime.callsite.PojoMetaMethodSite.call(PojoMetaMethodSite.java:55)
	at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:120)
	at org.gradle.api.internal.project.AntBuilderDelegate.nodeCompleted(DefaultIsolatedAntBuilder.groovy:176)
	at groovy.util.BuilderSupport.doInvokeMethod(BuilderSupport.java:147)
	at groovy.util.BuilderSupport.invokeMethod(BuilderSupport.java:64)
	at org.codehaus.groovy.runtime.callsite.PogoMetaClassSite.call(PogoMetaClassSite.java:45)
	at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:120)
	at org.gradle.api.plugins.quality.Checkstyle$_run_closure1.doCall(Checkstyle.groovy:162)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
	at java.lang.reflect.Method.invoke(Method.java:597)
	at org.codehaus.groovy.reflection.CachedMethod.invoke(CachedMethod.java:90)
	at groovy.lang.MetaMethod.doMethodInvoke(MetaMethod.java:233)
	at org.codehaus.groovy.runtime.metaclass.ClosureMetaClass.invokeMethod(ClosureMetaClass.java:272)
	at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:877)
	at groovy.lang.Closure.call(Closure.java:412)
	at groovy.lang.Closure.call(Closure.java:425)
	at org.gradle.api.internal.ClosureBackedAction.execute(ClosureBackedAction.java:58)

Checkstyle is using a static HashMap instance internally, without any synchronization and this does not work very well in concurrent scenario. The build hangs, CPU is spinning, just like the symptoms described at http://stackoverflow.com/questions/17070184/hashmap-stuck-on-get

We should probably use Checkstyle command line api instead of the ant builder. This way we could fork the operation (just like we do for findbugs).

 Comments   
Comment by Lari Hotari [ 23/Sep/15 ]

It looks like this problem was fixed in checkstyle some time ago (checkstyle 5.8 contains the fix):
https://github.com/checkstyle/checkstyle/commit/902fa730
The cache has been removed in checkstyle 6.5:
https://github.com/checkstyle/checkstyle/commit/ebd4afde

Comment by Lari Hotari [ 23/Sep/15 ]

The default version of Checkstyle was upgraded to 5.9 in Gradle 2.4 (https://github.com/gradle/gradle/commit/cd861155). I assume we can close this issue.

Comment by Lari Hotari [ 23/Sep/15 ]

There are more concurrency problems in Checkstyle.

some examples:
https://github.com/checkstyle/checkstyle/blob/78cec4f822cd966a1293ce0c743d4514517385dc/src/main/java/com/puppycrawl/tools/checkstyle/checks/javadoc/AbstractJavadocCheck.java#L81

Comment by Lari Hotari [ 23/Sep/15 ]

It looks like they won't fix the concurrency bugs in Checkstyle:
https://github.com/checkstyle/checkstyle/pull/1122#issuecomment-105398453
"Checkstyle is single threaded by design."
"We are single threaded for now."

Comment by Benjamin Muschko [ 15/Nov/16 ]

As announced on the Gradle blog we are planning to completely migrate issues from JIRA to GitHub.

We intend to prioritize issues that are actionable and impactful while working more closely with the community. Many of our JIRA issues are inactionable or irrelevant. We would like to request your help to ensure we can appropriately prioritize JIRA issues you’ve contributed to.

Please confirm that you still advocate for your JIRA issue before December 10th, 2016 by:

  • Checking that your issues contain requisite context, impact, behaviors, and examples as described in our published guidelines.
  • Leave a comment on the JIRA issue or open a new GitHub issue confirming that the above is complete.

We look forward to collaborating with you more closely on GitHub. Thank you for your contribution to Gradle!

Comment by Benjamin Muschko [ 24/Jan/17 ]

Given the statement from the Checkstyle project I am going to mark the issue as "Won't fix".

Generated at Wed Jun 30 12:41:10 CDT 2021 using Jira 8.4.2#804003-sha1:d21414fc212e3af190e92c2d2ac41299b89402cf.