[GRADLE-114] Improve the speed of the build script evaluation Created: 11/Jun/08 Updated: 04/Jan/13 Resolved: 19/Aug/08 |
|
Status: | Resolved |
Project: | Gradle |
Affects Version/s: | 0.1.4 |
Fix Version/s: | 0.4 |
Type: | Improvement | ||
Reporter: | Hans Dockter | Assignee: | Hans Dockter |
Resolution: | Fixed | Votes: | 0 |
Issue Links: |
|
Comments |
Comment by Ittay Dror [ 12/Jun/08 ] |
Maybe not improve the speed of evaluation, but save the result so it can be quickly reused. Build logic doesn't change that much. How about this: if gradle sees a file (or set of files) withe extension .gradlec, it loads them instead of the .gradle files. Otherwise, it loads the regular files, does the evaluation and works on that. Say there's a 'gradlec' target to create this file (or files). Then, if a user wants to, he can run this target, put the file in his version control and all users use it. When the build logic changes, run 'gradlec' again etc. Of course, there can be a timestamp (or md5) check and warnings if the .gradle files have changed after .gradlec and a way to tell gradle to ignore gradlec. In fact, maybe the .gradlec file(s) can be compiled already to .class file(s)? I have reasons for wishing drastic measures to improve speed: |
Comment by Hans Dockter [ 12/Jun/08 ] |
I'm completely dedicated to improving the performance as much as possible. The important thing, at least as I understand it right now, is that the groovy compilation of the gradlefile is only one part of the story. But running this compiled gradlefile also takes surprisingly long when all what it does is executing a createTask method (and the closure is not even evaluated then). For example: createTask('helloWorld') { println 'hello' } This takes Gradle on my machine something like 0.4 seconds (just the running of the compiled gradlefile, not the execution of the build). I think this is not a Groovy problem but a Gradle problem. I gonna have a look with a profiler about what is going on here. But I'm also completely positive about caching the compiled gradlefiles. Although in Groovy 1.6 the compile speed has improved a lot. I try to get more numbers and then we can discuss what measures are worth doing. |
Comment by Ittay Dror [ 12/Jun/08 ] |
I'm not talking about just the compilation from groovy, but the whole phase of reading all gradle files, creating project objects for them, creating the dependencies between targets. for a large project, this may take some time. |
Comment by Ittay Dror [ 12/Jun/08 ] |
In case I wasn't clear, my point was that after creating the DAG, gradle should save the result (DAG and closures of code) to a file (.gradlec), probably next to the root project file. That way, the process is not repeated every build. Compiling the file to .class is another step in optimization. |
Comment by Hans Dockter [ 12/Jun/08 ] |
I see. But one has to keep in mind that a Gradle build is more dynamic than XML. For example: if (today = 'monday') { createTask('x') { // do something } } else { createTask('x') { // do something else } createTask('y') { // do something } } Depending on the day the dag is different. And I'm sure as people will get acquainted with the dynamicity of Gradle they will use such features more and more. I'm not opposed to your idea, but it is not a silver bullet. Here a summary of strategies (or outlooks) I see for improving the performance:
|
Comment by Ittay Dror [ 12/Jun/08 ] |
about the second strategy, i think that for those dynamic buildscripts, just offer a flag that can be turned on in the script to prevent saving the dag. i think that a user that uses dynamic creation of targets will be able to figure out that he needs to turn on this flag (and pay the performance penalty). also, what happens for large projects, with many modules, where i want to build only a specific module? assuming the build uses gradle's configuration injection and inter project dependencies, so that the module cannot be built without realization of the whole multi module graph. |
Comment by Hans Dockter [ 13/Jun/08 ] |
Or we do it the other way round. You have to set the flag if you want DAG caching. That way we prevent faulty builds. Right. We always need to build the whole DAG. First I would implement caching the compiled script and see whether we can improve the performance of the DAG generation (including giving a try to Groovy 1.6-beta2). The next step would be DAG caching. |
Comment by Ittay Dror [ 18/Jun/08 ] |
Another suggestion is to have gradle always running, with the user triggering targets from the command line. The benefit is that the initial bootstrap is done only once in a while (whenever the user decides to relaunch gradle). This can also be good for CI - the targets can trigger themselves periodically, or when changes have been made to the code. It will also be good for continuing a build after errors have been fixed by the user. |