Gradle

Oops, naturally for gradle A1, the end of the output contains A1 and not A2.

We have to discuss this on the dev list. I will trigger this discussion when we start the work on 0.7.

I have now run into this issue as well. I have posted something to the dev list.

Hi Marc, would you tell us the real world problem where you needed those task relationships?

I don't remember exactly but it was nearly as simple as in the provided dummy example. I think that I had 2 tasks: compile and copy compiled stuff and I wanted to be able to execute both per default (in the right order) as well as to have the possibility to call each one individually.

We have discussed this on the dev list. The full thread can be found here: http://markmail.org/thread/wn6ifkng6k7os4qn

Quote from Adam:

I feel pretty strongly that ordering of task dependencies is the wrong
way to go.

I have 2 main objections. Firstly, it really muddies the semantics of
the dependency graph. By ordering the edges, it stops being a DAG, and
becomes something else. It becomes unclear what it is and what it means.

Consider:

task a
task b
task c(dependsOn: b,a)

Why should task c have anything to say about the relationship between
tasks a and b, and how they should be ordered? If b really needs to
happen before a, then why is this not declared on b or a?

It becomes worse if you add a few indirections:

task a
task b
task c(dependsOn: b,a)
task d(dependsOn: a,c)
task e(dependsOn: c,d)

which of the above incompatible orderings do we choose? do we fail the
build? what if a,b and c are in a plugin outside our control? How does a
human understand what will happen here?

My second objection is that task ordering is, in practice, almost always
lossy. That is, there's something missing from the model to express the
actual reason why a particular task needs to be completed before another
one can start. Ordered dependencies are a poor approximation of this.

We should, instead, identify what it is that's missing from the model,
and add it in.

Hi guys.

I read the conversation on the mailing list (understanding most of it =). Allow me to put my comment here.

>Why should task c have anything to say about the relationship between
>tasks a and b, and how they should be ordered?

I think I get it though sometime it's just natural... example:

clean
build
rebuild(dependsOn: clean, build)

My real life example is:

task 'build-all' (dependsOn: [jar, 'sign-jar', 'build-tiddly', 'zip']) << {}

The order matters in my case. The reason I don't have explicit dependencies in some tasks is because it is way easier to work & debug tasks that don't have dependency. Every time I make changes to 'build-tiddly' task I can easily just run this task without triggering its dependency that is just time-consuming.

>My second objection is that task ordering is, in practice, almost always
>lossy.

Could be. However, not respecting the order feels counter intuitive. I spent some time and I couldn't understand why my build didn't work. I couldn't find it in the docs quickly so I ended up writing tiny gradle script to find out in what order dependencies are processed. dependsOn already offers predictable order (alphabetical, right?) but it's just not intuitive. I think processing dependencies alphabetically is not the right way of coaching build masters how to do proper DAG.

I don't want to argue what's the best solution - you guys decide on it. Just wanted you to know my feedback.

Keep on doing great stuff with Gradle!

I also found this behavior a bit unexpected.
I was trying to reproduce the maven lifecycle when I discovered this issue.
http://permalink.gmane.org/gmane.comp.programming.tools.gradle.user/4309

IMHO, the dependency graph should be processed using an in-order traversal with the additional restriction that each nodes are executed only once.
As Adam had noted on the discussion this could result in different execution order, however I do not find this confusing or problematic for my purposes.
What I do find confusing is why the dependencies are reordered, what is the advantage in doing this? or is it purely idealogical?

@Kurt We see that there is a problem to solve that is not solved well yet by Gradle. We don't think that dependency order is a good solution, as pointed out. We want to address this issue in 1.0. But I see your point re the reordering and why to do it at all. Could you raise this issue on the dev list?

I reopen this issue as it represents a problem still to be solved, although we want to solve it in a different ways then proposed.

Just my two cents: I was enthusiastic about gradle until I hit this issue. This basically prevents transitional migration of our project, as I cannot import most of our ant targets due to their reliance on ordered "depends" list. So I would have been stuck with rewriting all ant targets.
IMHO atomicity and small targets/tasks you talk about in your discussion actually encourages using "depends" list, since you want to build bigger targets from small, reusable ones, executed in specific order

Another real life example:
releaseVersion = "test.release.version"
mailMessage = ""
task getVersion {
// sets releaseVersion property to user input
}
task releaseLogic {
//does release logic with releaseVersion
//customize mailMessage
}
task sendMail {
//sends customized mail to interested users
}

task release(dependsOn: ["getVersion", "releaseLogic", "sendMail"] {
//do regular release
}
I don't want any of subtasks to be dependend on other (I want to be able to do test release without forcing user input, and to send mail for tasks others then release) but I want release task to happen in exact order.

Hi Jakub,

thanks a lot for your feedback. You are right, the ant migration aspect is another important thing to consider. We will solve this is in one way or the other.

Hi guys, I was just wondering if there was any more progress on this.

I found this issue after encountering a similar problem to Jakub while trying to set our Gradle project. Our use case is that we want to let our developers just run the war task for their grails app, but when we want to actually upload the war to our nexus server, we'd like to run clean before the war task. The most natural way to do this seemed to be having no dependsOn specified for the war task, but then have ["clean", "war"] for the dependsOn for the upload task, but from this discussion I see that that's not correct or reliable.

We're currently on 1.0 M3 and I was wondering if this is still going to be addressed for 1.0, as per the comment from Hans on June 7th.

Thanks,

Mark

@mark: We want to solve the 'clean must run first' use case. We're yet to decide exactly what is in or out for 1.0 release, but I'd like to include it if we can. However, we're very unlikely to solve the use case by changing Task.dependsOn to imply any ordering between the dependencies. Perhaps you could raise a separate jira issue for your particular use case, so we can tackle it separately.

>However, we're very unlikely to solve the use case by changing Task.dependsOn to imply

Too bad Albeit controversial, I find it the most natural thing to do It even feels consistent with our current approach to task names given via the start parameters - task names hint at order but ultimately it's the dependsOn on that is the master information for the DAG.

Anyways, I'm eager to see how we approach that issue

IMO the objections about implications to graph maintenance aren't justified. A preferred sequence would be just an additional constraint on the DAG. It's like saying that color-coding DAG nodes to indicate some aspect would harm its authenticity. As a matter of fact, the graph nodes already have a sequence, and it is the alphabetical ordering of the Task names, so we just want to apply a different ordering. In the real world we have practical needs, and the graph structures need to accommodate them. Even if DAG could not manage dependency execution sequence in a natural way, that would indicate that DAG is inadequate for a build system, not a need to preserve the structure by hacking weak and limited work-arounds (like command line switches to force a specific task to run first).

Just like I experienced with a different project a year ago, after championing conversion to Gradle and finally getting approval for the HyperSQL project I have to withdraw my recommendation due the embarrassing inability to control dependency sequencing in a manageable way like we have done easily and intuitively with Ant for 10 years.

I think I worked around the issue with something like this:
taskb.doFirst { taska.execute() }

I added bit of sugar adding to list then adding logic to each task automagically.
I was working elsewhere then and don't have the code anymore.
I have been meaning reinvent it and report back but not had time.

I just completed my work-around to preserve Ant build file dependency precedence specifications. Here you are:
http://hsqldb.svn.sourceforge.net/viewvc/hsqldb/base/trunk/build/build.gradle
This is the new Gradle wrapper for HyperSQL's main Ant build file, and most of it is my dependency-sequence work-around.

There is some unrelated stuff in there. The methods and functions of interest here are

• recursivePush(...)
• resequenceAntDependencies()
• The top-level code under the comment "// RESEQUENCE"
• tasks

My 'tasks' override makes up in large part for the lack of a private task mechanism to hide tasks similarly to Ant non-described and -* tasks.

My system supports nested Ant imports, but you must specify all file names (as you can see I have done). Would be easy to pull in the imported file names automatically, but I though it best to leave that decision up to the invoker.
Supports custom Ant build file adjustments by an optional closure parameter to resequenceAntDependencies().
Your Ant targets may not be named with 3 leading digits, because I use that name space for my renaming targets to accomplish the desired ordering.
As you can see by running "gradlew tasks" (or similar), I add wrappers for the renamed tasks, copying over the descriptions, then hiding the renamed ones, so that "gradle tasks" looks just like "ant -p" before you introduced Gradle.

Checks are made for dependency cycles, where in one dependency spec we have A... B and in another B... A. Doesn't handle this, but my highly-nested 74-target Ant build file doesn't have any, so you probably won't either.

Unrelated to the dependency sequencing, but of interest to others wrapping Ant, I use the custom closure and the resequence.destDir option to work around Gradle's multiple terrible bugs that prevent the basedir specified in the Ant build file from working.

@Kurt, execute() method only executes 'actions' attached to the task via doFirst() / doLast(). So your workaround may not work for all kinds tasks. For example, most built-in tasks/custom tasks don't have any doFirst/doLast actions by default.

It is very important to set the order in which tasks should be executed. I came across this bug while I was creating a task for testing our web-app. Before webtests can be executed, the schema has to be created, the jetty must be configured and so on:

task createSchema << {}
task dropSchema << {}
task startJetty << {}
task runWebtests (dependsOn: [dropSchema, createSchema, startJetty] << {}

In this case, the order of execution is very important, i.e. the task dropSchema is executed before createSchema, which is currently impossible. And I prefer not to change the task createSchema by adding a dependOn dropShema. This would be just a workaround. Also in some scenarios I don't want to drop the schema before creating one, because in this way some other tasks will cause problems.

There are many scenarios in which the order is important and has to be set by dependsOn. Currently we have to create a workaround to solve this issue, but it would be very nice, if this "bug" is fixed in one of the next versions.

Wanted to add my sentiment, like others, this may prevent us from moving to Gradle. We have a large, complex, existing ant build that is highly dependent on the tasks being executed in order of dependency declaration. Requirement #1 for any build changes is the ability to do it incrementally. I don't even know how anyone can say in good-faith that Gradle supports existing Ant builds when it doesn't respect the order of dependency declaration. I've seen dozens of Ant scripts and the reliance of this ordering is everywhere.

Certainly the dependent tasks need to be executed in some order. Is the arbitrary choice of alphabetical order provide any benefit whatsoever?

Was there any progress on this issue?

I read Adam comment on the mailing list and I can't imagine how he came up with it. I guess view really depends on what you want to use gradle for. For his question:

It becomes worse if you add a few indirections:

task a
task b
task c(dependsOn: b,a)
task d(dependsOn: a,c)
task e(dependsOn: c,d)

which of the above incompatible orderings do we choose? do we fail the
build? what if a,b and c are in a plugin outside our control? How does a
human understand what will happen here?

I'm not sure what Adam means by this list, but I assume it's a list of tasks ran by the user. If so [for me] it's quite simple, you run task a, then you run task b, then you run task c [since b and a were already ran], then you run task d [since a and c were already ran] and finally you run task e [since c and d were already done].

If it's just a definition of tasks and user decides to run d and e, then you run a, c, d, e.

I think you want to add some intelligence to gradle, which is not required. Let users decide what to run and in what order. If something was ran then don't do it again [or allow user to specify dependencies which have to be re-run even if they were already].
If there's an error, let the USER decide what to do and ask them to rewrite the dependencies. If they have a tasks:

task a
task b
task c
task d(dependsOn: a, b)
task e(dependsOn: d, c)

and they want to run task e without running a, what would you tell them? Just rewrite the tasks.

Currently, I've got tasks like:

checkout from VS
clean
build
package(dependsOn: clean, build)
stop server
do some cleanup on server
copy package to server
start server

So I have all this little tasks, with their dependencies and in the end I wanted to have a deploy task:

deploy(dependsOn: git, war, stop, cleanup, copy, start)

I was pretty shocked, when I saw gradle reordering all the tasks and trying to execute:

cleanup
copy
git
start
stop
war

I understand that what is going on behind the scenes is complex and there are things about gradle I have no clue about, but in the end it should be me who decides the order of things.

If I could give this issue more votes I would.
I'd love for Gradle to honor my settings and execute tasks in the order I specified them. I don't want to rename task names nor reorder task dependencies just to please the tool (that's the maven way, remember?)

I did some debugging and found a wrong usage of TreeSet<Task>. org.gradle.api.DefaultTask.Task is a Comparator that sorts alphabetically by its path.

Just changing it to a List and reverse its original order helps a lot.

The attached patch is based on branch REL_1.0-milestone-7.

IMO, this is NOT a bug, though it does point up a deficiency in Gradle.

Dependency order should not be respected; doing so breaks Gradle the same way ant is broken, and keeps it from being able to (in the future) do parallel builds.

Instead, Gradle needs a facility to easily invoke one task from another, procedurally. Saying that 'a depends on b and c' says nothing about the order of b and c. Saying 'a needs to do first b, then c' does, and Gradle needs to be able to express both.
The first should be:

task a(dependsOn: [b, c])

but the second could be either:

task a {
    b.execute()
    c.execute()
}

or:

task a(dependsOn: b) {
    c.execute()
}

...though I'm enough of a noob to not know if there's actually an 'execute()' method on each task (or what it's actually called if it does exist).

I think at heart this is about the ability to specify that:
A should always happen before B, even though b doesn't depend on a.

You could have something like Task.orderAfter to specify this. This method wouldn't imply a dependency, just that if both tasks are scheduled, one should happen after the other. I guess you end up with two DAGs - one for dependencies, and one for explicit ordering (which will usually be small). The commandline list of tasks could also add edges to the ordering DAG (e.g. a commandline of "clean build" will implicitly call "build.orderAfter clean"). Merging the two might be as easy as this: for each edge in the ordering DAG, if the two ends are in the dependency DAG, add the edge in from the ordering one into the dependency one (it's probably harder though!).

In my case, I just want to get a continuous integration build to do things in the right order. My workaround is:

def whenScheduled(task, closure) {
    if (gradle.startParameter.taskNames.contains(task)) {
        closure(task)
    }
}

task cibuild {
    project.buildType = 'build'
    dependsOn 'clean', 'test', 'artifactoryPublish'
    whenScheduled(name) {
        gradle.projectsEvaluated {
            project.artifactoryPublish.dependsOn 'test'
            project.compileJava.dependsOn 'clean'
        }
    }
}

which is pretty much applying a limited version of the DAG merging manually!

@Tom Widmer

I think the way you are hacking a solution leads to a possible fix that doesn't require a second DAG: Add a command that says, effectively, dependsOnIfActive(task)

This would allow the current DAG to order the events, but if the task is not requested explicitly or implicitly, it's not included in the tree. I don't know if this would work for every case, but it might be a good enough compromise. It also doesn't require any explicit task ordering.