[GRADLE-3209] SonarRunner should respect sources in root project in Multi-Project Hierarchies Created: 08/Dec/14  Updated: 16/Jan/17  Resolved: 16/Jan/17

Status: Resolved
Project: Gradle
Affects Version/s: None
Fix Version/s: None

Type: Bug
Reporter: Gradle Forums Assignee: Unassigned
Resolution: Won't Fix Votes: 1


 Description   

I've attempted to use the new Sonar Runner support, but have hit a problem analyzing an existing, multi-project hierarchy: the root Gradle project is a Java project with code, but the underlying Sonar-Runner refuses to analyze it.

From Sonar's documentation on the runner[1]: "A project that defines modules (or a module that defines sub-modules) cannot define a source code folder to be analyzed by Sonar."

The existing "Sonar" plugin for Gradle, from what I recall, handled this case acceptably and included the root project in the analysis.

I came across a somewhat related issue (SONARPLUGINS-2295 [2]) that refers to some discussion on the topic, and a note that they don't, "see any real life use case so we will not support this situation." However, the issue also notes that, "On Sonar side it seems to be perfectly supported to have an aggregator project (ie with sub modules) also having sources," indicating that the constraint is only enforced by the runner.

If I've understood, then this means that using the new Sonar Runner imposes constraints on how the project hierarchy is organized. Can anyone recommend a potential solution to allow the new Sonar Runner to be used in this scenario without having to reorganize the hierarchy (which could be difficult for existing projects)?

[1] [1]http://docs.codehaus.org/display/SONA...
[2] [2]https://jira.codehaus.org/browse/SONA...
----------------------------------------------------------------------------------------
[1] http://docs.codehaus.org/display/SONAR/Analyzing+with+Sonar+Runner
[2] https://jira.codehaus.org/browse/SONARPLUGINS-2295



 Comments   
Comment by Gradle Forums [ 08/Dec/14 ]

The new sonar-runner plugin shouldn't impose any constraints other than those imposed by the Sonar Runner API (otherwise it's a Gradle bug). If Sonar Runner can't analyze multi-project builds where the root project contains code, there's nothing we can do on our side. Can you post the exact error message and stack trace (`-s`)? Please use HTML code tags.

One thing you could do is to overwrite the default sonar-runner configuration for the root project and don't tell it about the source code. Of course this means that this code won't get analyzed.

Comment by Gradle Forums [ 08/Dec/14 ]

Currently, there is no error generated. The analysis quietly ignores the root project. The issue I referenced, SONARPLUGINS-2295, was a request to have the Sonar's Runner at least generate a warning message in this scenario. I think that change exists in a newer version of the Sonar Runner than is currently used by the Gradle 1.6 plugin.

One possibility I started looking at was to pass additional properties to the Sonar configuration to add an additional, custom Sonar module to include the code from root, including the 'sonar.modules' property. However, it seems that the Gradle plugin is explicitly overriding the 'sonar.modules' property based on the project hierarchy, so I'm unable to inject a new module that way.

Comment by Gradle Forums [ 08/Dec/14 ]

The issue was reported in 2012 and discussed in January 2013, which means that the affected Sonar Runner version isn't newer than the one used by Gradle.

It's true that `sonar.modules` is set later than all other defaults and cannot be overwritten in the `sonarProperties` block. That's something we might be able to change. In any case, it should be possible to overwrite `sonar.modules` on the task level. Something like:

tasks.sonarRunner {
doFirst

{ sonarProperties += ["sonar.modules": "foo,bar,baz"] }

}

Comment by Gradle Forums [ 08/Dec/14 ]

Noticed this discussion only after I posted [1]http://forums.gradle.org/gradle/topic...

While this is definitely a problem with Sonar Runner, it's also a conflict between what Gradle supports, and what Sonar-Runner does not. Gradle tries to enforce the project module hierarchy using `sonar.modules`, even though this hierarchy will result in modules (that are supported in gradle) being left out.

If gradle were to use `sonar.modules` to setup a flat hierarchy instead, it would (hopefully) solve the problem. You'd lose the ability to drilldown, but it would be better than missing whole projects.
----------------------------------------------------------------------------------------
[1] http://forums.gradle.org/gradle/topics/sonar_runner_gradle_plugin_tries_to_run_reports_on_multi_level_multi_projects_but_sonar_runner_only_analyzes_leaf_projects

Comment by Gradle Forums [ 08/Dec/14 ]

I just tested this using sonar-runner (the standalone program, not from gradle) and it works, even if the Sonar dashboard loses information about the nested nature of the modules.

sonar.modules=Parent1,Parent1/Child1,Parent2,Parent2/Child2

Parent1.sonar.sources=src
Parent1/Child1.sonar.sources=src
Parent2.sonar.sources=src
Parent2/Child2.sonar.sources=src

Parent1.sonar.binaries=build/classes/main
Parent1/Child1.sonar.binaries=build/classes/main
Parent2.sonar.binaries=build/classes/main
Parent2/Child2.sonar.binaries=build/classes/main

If the Gradle sonar-runner plugin can be configured to call sonar in this way, the problem will be solved.

How do I override the sonar.modules properties of the root as well as child projects, and also set each module/project's sonar.sources and sonar.binaries properties?

Comment by Gradle Forums [ 08/Dec/14 ]

Not sure if anyone has overcome this hurdle yet, but we use Oracle Commerce (ATG) and by default all its software is packaged in a multi-module hierarchy with source in the root and sub-parents. Traditionally customisation follows the same pattern. As of SR 2.4 and Gradle 2.2.1 the problem still exists with Gradle correctly generating the sonar properties according to its model but invalid for Sonar which still only supports leaf nodes with source.
So the first steps I've taken are:
1. Hand crank the sonar-project.properties in the project root and run the runner manually. This uses the pattern of making all modules appear to be leaf-nodes, even the root. This works without errors.
2. Generate the default gradle build and copy the generated properties from build/tmp/sonarRunner/sonar-project.properties into the root.
a. Remove all sub-module modules settings and modify the root's to include all the individual modules plus a new one for the root.
b. Copy the root (default) block of settings and give it a prefix as per the project name (and the new module added to modules)
c. Strip all source related properties from the default root settings
d. Set the name of each module's projectName to the pattern .Parent.Child, e.g. Common.Datamanagement.Versioned
e. Run the runner manually against this file and it all works and the Sonar dashboard breaks the code down into the modules as I want to see them.
We could actually live like this but running the sonar runner manually in CI instead of the gradle task and modifying it each time the project dependencies/structure changes is too cumbersone and error prone - but at least a fallback option if necessary.
Now the challenge is to get Gradle to generate the same ... If I crack it I will add it back here

Comment by Gradle Forums [ 08/Dec/14 ]

I've been able to do everything except set the sonar.modules in the root and parent sub-projects as Gradle writes the modules list after all other settings including system overrides.
The sonarProperties Map is read only so as Peter suggested above the setting in doFirst is not possible.
Finally we can't even manipulate the generated file as its written to the file system the line before the execution of the runner - so no hook to get at it.
Sonar Runner still executes so I guess the only possibility (which is a real hack - given I can't change the code structure) is to execute the sonarRunner (using a dryRun option if it works), get at the generated file and change the module settings then execute the runner manually via another task.

Comment by Gradle Forums [ 08/Dec/14 ]

So end result is now working for multi-module projects with source at all levels:
1. Some manipulation of various sonar properties using gradle configuration
2. Run the task in preview so no results written to database:
gradle sonarRunner -Dsonar.analysis.mode=preview
3. Create a new task that reads the gradle generated sonar properties at build/tmp/sonarRunner/sonar-project.properties and does all the final manipulation for module, module renaming, fixing the root project,removing the preview option, etc. and rewrites the output back to the same file.
4. In CI a separate step is run to copy that project properties file into the root and invoke the sonar-runner command line and then delete the properties file.
5. The dashboard nicely shows all the modules and their source.
Its a bit of a hack but works nicely

Comment by Gradle Forums [ 08/Dec/14 ]

I had worked on this a while back. Your technique (the one in the previous post, not the one you JUST posted) is exactly what I settled for. (I'm using ATG too).

The permanent solution is to fix the Sonar Runner Gradle plugin to do what you are doing manually. You can fork the Gradle source code and change this file:

[1]https://github.com/gradle/gradle/blob...

String projectPrefix = project.getPath().substring(targetProject.getPath().length()).replace(":", ".");

should become

String projectPrefix = project.getPath().substring(targetProject.getPath().length()).replace(":", ".").replace("/", ".");

I never got around to submitting a patch (the above suggestion is untested) as I couldn't use Gradle as my build tool because of this problem: [2]http://forums.gradle.org/gradle/topic...

I'm curious how you are using Gradle with ATG, but I guess that's a question for a different forum.
----------------------------------------------------------------------------------------
[1] https://github.com/gradle/gradle/blob/master/subprojects/sonar/src/main/groovy/org/gradle/sonar/runner/plugins/SonarRunnerPlugin.java#L157
[2] http://forums.gradle.org/gradle/topics/is-the-repository-extensibility-design-change-complete

Comment by Gradle Forums [ 08/Dec/14 ]

I got a few future deprecation warnings accessing the sonarRunner task in the configuration phase so had to move more of the solution to post run task.
FYI - Here is a generic solution that is currently working on 5 different projects:
[1]https://github.com/hallatech/gradle-s...
----------------------------------------------------------------------------------------
[1] https://github.com/hallatech/gradle-sonar-multi-atg-project

Comment by René Gröschke (Inactive) [ 08/Dec/14 ]

we should implement the same behaviour as the original sonar plugin. Using the sonar plugin, sources of the root project are collected in a module named "[default]" It shouldn't be hard to model that with the sonar-runner

Comment by Benjamin Muschko [ 15/Nov/16 ]

As announced on the Gradle blog we are planning to completely migrate issues from JIRA to GitHub.

We intend to prioritize issues that are actionable and impactful while working more closely with the community. Many of our JIRA issues are inactionable or irrelevant. We would like to request your help to ensure we can appropriately prioritize JIRA issues you’ve contributed to.

Please confirm that you still advocate for your JIRA issue before December 10th, 2016 by:

  • Checking that your issues contain requisite context, impact, behaviors, and examples as described in our published guidelines.
  • Leave a comment on the JIRA issue or open a new GitHub issue confirming that the above is complete.

We look forward to collaborating with you more closely on GitHub. Thank you for your contribution to Gradle!

Comment by Julien HENRY [ 15/Nov/16 ]

The SonarQube plugin was moved to a dedicated plugin. Related issue is tracked here: https://jira.sonarsource.com/browse/SONARGRADL-5

Comment by Benjamin Muschko [ 16/Jan/17 ]

The Sonar Runner plugin was removed from Gradle core and is developed as external plugin.

Generated at Wed Jun 30 12:42:51 CDT 2021 using Jira 8.4.2#804003-sha1:d21414fc212e3af190e92c2d2ac41299b89402cf.