[GRADLE-421] buildSrc can't be used inside settings.gradle Created: 11/Mar/09 Updated: 04/Jan/13 Resolved: 11/Aug/09
Currently buildSrc is built after settings.gradle is evaluated, but before all the build.gradles are evaluated. It is sometimes desirable to simplify a settings.gradle file by using classes defined in buildSrc, but this is not currently possible. Although most customization of gradle can be done in the root build.gradle (or in a plugin), some things (like programatically calling include) must be done from the settings.gradle file.
The attached patch makes this change in the simplest way I could see to do this. Currently (prior to this patch) BaseSettings.createClassLoader doesn't just create a new classLoader, but modifies the current context classloader using reflection. I continued in this manner for my patch, but I think it would be cleaner for BuildSourceBuilder and DefaultSettings to return new classloaders without using reflection to modify the current one. I will be glad to create a patch to work this way if desired, but it involves changes to more files.
This patch changes org.gradle.Gradle to explicitly build buildSrc (instead of a side effect of BaseSettings.getClassLoader), then add the buildSrc results to the classpath. It then processes the settings file and adds the resulting dependencies to the classpath. Lastly it runs the project build.gradles as before. I added a use of a buildSrc file to the settings.gradle of the javaproject sample.
|Comment by Steve Appling [ 16/Mar/09 ]
I would like to get some feedback on this suggestion. Is this likely to make it into version 0.6? If not, I would like any suggestions on alternate approaches that might be accepted. I tried to discuss this on the developer list but didn't get much of a response, so I'm not quite sure how to proceed. I need some way of using our own custom code to build up the list of modules to be included.
|Comment by Hans Dockter [ 05/May/09 ]
We postpone this issue to 0.7. In 0.7 we will work on a more general approach for adding project to the build script classpath which is planned to replace buildsrc. But your use case points to an important feature we might have overseen otherwise. Basically it means that there is an additional build lifecycle phase that needs logic to be build. For 0.7 we want to merge the settings.gradle functionality into the build.gradle file. We will use Groovy AST transformations for this. Basically this means that we preprocess the build.gradle to extract the information that was in the settings.gradle and then do the normal build script evaluation in the next phase. We will provide a way to use 'custom code' for a dynamic usage of the preprocessing step.
Our current classloading mechanism are pretty basic and not good enough. For 0.8 we want to change this. Our current plan is to base Gradle on OSGi. Tom has already worked on this. Therefore we might keep the classloading as is until then.
|Comment by John Murph [ 03/Aug/09 ]
I have an implementation of this. It is quite a bit of refactoring, however. I felt that finding and running the settings file needed to be explicitly separate (finding is done as part of processing the file right now, by ScriptLocatingSettingsProcessor). Also, buildSrc was being built as a side effect of getting the classpath from the settings object. I needed that to happen at a different time (this is the meat of this change), and therefore wanted this to also be more explicit.
To do this I have created a new abstraction called SettingsHandler. This is given to Gradle during construction instead of the ISettingsFinder and SettingsProcessor objects (they are now given to this SettingsHandler). This gives Gradle one abstraction to deal with. Internally, this new class uses the settings finder to find the settings.gradle file, then builds the buildSrc module (if any), and lastly processes (executes) settings.gradle. This change, of course, rippled though several classes and their tests.
One concern I have is that it is not currently possible for us to know if we have found the correct settings.gradle file until after it is executed. Now that buildSrc is built before that time, we might build buildSrc just to learn that we have the wrong settings.gradle file. Then we have to try an empty settings file, and possibly rebuild buildSrc (because we've now found a different one). I'm not sure how to fix this with the current design other than compile settings.gradle into two parts (normal logic, and buildSrc dependent stuff), similar to how build.gradle is currently done. This is messy, however. Of course, if defining included projects gets moved into build.gradle files (how?!) or accomplished some other way, that might stop being a problem.
These changes are available for review at git://github.com/sappling/gradle.git in the build_buildSrc_before_settings branch. (Once again, I'll add javadocs and tests later.) Please take a look and give me feedback.
|Comment by Adam Murdoch [ 09/Aug/09 ]
This looks good.
If you get a chance, could you move the rest of BaseSettings.createClassLoader() to live in BuildSrcBuilder.buildAndCreateClassLoader()?
Also, could you fix the copyright notice in the headers of the new files?
|Comment by John Murph [ 10/Aug/09 ]
Good, I'm glad you like it.
As for the copyright notices... oops! I thought I had fixed those. I need to play around with IDEA 8's new copyright plugin. That might fix this for me so it won't keep happening. Those will be fixed shortly.
As for BaseSettings.createClassLoader() issue, I would rather remove the need for this method. I do not like that it injects the class into an existing classloader (using ClasspathUtil.addUrl() which uses reflection to call into non-public APIs). Why not just add tools.jar to the nonLoggingJars in BootStrapMain? I think this would be much cleaner and would simply the only other use of ClasspathUtil (which is GradleUtil.executeIsolatedAntScript() which currently has to know about the special handling of tools.jar). If you are fine with this change, I can include it in this patch or make it a new patch.
|Comment by Adam Murdoch [ 10/Aug/09 ]
I'd rather solve the tools.jar problem as a separate issue. We can get buildSrc working in settings.gradle without changing how tools.jar is handled.
We should tidy up tools.jar handling as part of a larger Gradle embedding classloader cleanup task. It's a good idea to move tools.jar into the BootStrapMain.nonLoggingJar. But, this also forces every embedder to add tools.jar to the appropriate classloader. We should also move the classloader setup from BootstrapMain to somewhere behind the Gradle class, so embedders don't need to care.
|Comment by John Murph [ 10/Aug/09 ]
I've fixed the copyright headers, and it sounds like you want to hold off on the BaseSettings.createClassLoader() issue. So, I think this patch is ready for inclusion. Do you mind taking care of it, Adam?
The general cleanup of tools.jar sounds nice. I know that embedding Gradle has been a pain because of the classloader issues. This probably needs to get it's own JIRA issue (if it doesn't already have one).
|Comment by Adam Murdoch [ 11/Aug/09 ]
Your changes have been applied. Thank you for the patch.