[GRADLE-2582] Parallel resolution of artifacts Created: 02/Dec/12  Updated: 02/Feb/17  Resolved: 02/Feb/17

Status: Resolved
Project: Gradle
Affects Version/s: 1.3
Fix Version/s: None

Type: Improvement
Reporter: Ben Manes Assignee: Unassigned
Resolution: Duplicate Votes: 26

Issue Links:
Related
Related to GRADLE-2788 parallel build may download the same ... Resolved
Related to GRADLE-3420 uploadArchives task is not parallel safe Resolved
Related to GRADLE-3309 Excessive downloads during resolution Resolved

 Description   

Gradle resolves dependencies serially, which can be time consuming if the repository is slow. This could be improved by resolving dependencies in parallel, similar to Maven's support [1].

Currently we use use ArtifactoryOnline as a hosted repository. It provides internal artifacts and proxies to external repositories. Unfortunately it is slow, so we use an Artifactory instance as an on-site cache. This reduces a resolution from 15m to 30s! AO is now used for EC2 machines and developers working remotely.

In a project reduced to a subset of public dependencies the time took:

  • Maven Central: 1m
  • JFrog's mirror: 3m
  • Artifactory Online: 6m

I'm unsure how to optimize Artifactory Online, but I think that parallel downloads would greatly help speed up the process.

[1] http://jira.codehaus.org/browse/MNG-3379



 Comments   
Comment by Adam Murdoch [ 06/Dec/12 ]

This is certainly something we want to do. We might also look at running tasks in parallel to resolving, so that, for example, we can be compiling while downloading the stuff to run the tests.

Also, if you run in parallel execution mode with --parallel, you can get parallel downloads to some degree (but sometimes it will download an artifact twice - parallel mode is still a work in progress).

Comment by Ben Manes [ 13/Sep/13 ]

Parallel resolution would vastly increase performance of the Gradle Versions Plugin, which resolves the latest version of every dependency to check for potential updates. While not run frequently, its embarrassingly slow due to sequential resolution.

https://github.com/ben-manes/gradle-versions-plugin

Comment by Mike Richardson [ 08/Jul/14 ]

The slow speed definitely doesn't help make the case for switching from Maven when I present the 2...

Comment by John Lonergan [ 07/Jul/16 ]

Yes please - really a grind

Comment by Benjamin Muschko [ 15/Nov/16 ]

As announced on the Gradle blog we are planning to completely migrate issues from JIRA to GitHub.

We intend to prioritize issues that are actionable and impactful while working more closely with the community. Many of our JIRA issues are inactionable or irrelevant. We would like to request your help to ensure we can appropriately prioritize JIRA issues you’ve contributed to.

Please confirm that you still advocate for your JIRA issue before December 10th, 2016 by:

  • Checking that your issues contain requisite context, impact, behaviors, and examples as described in our published guidelines.
  • Leave a comment on the JIRA issue or open a new GitHub issue confirming that the above is complete.

We look forward to collaborating with you more closely on GitHub. Thank you for your contribution to Gradle!

Comment by Ray [ 15/Nov/16 ]

I definitely feel that this is still a very important feature for Gradle. A number of recent improvements have been made in order to improve build times and this would go a long way to help in that direction as well.

Comment by Benjamin Muschko [ 02/Feb/17 ]

The feature is now tracked here: https://github.com/gradle/gradle/issues/1309

Generated at Wed Jun 30 12:25:51 CDT 2021 using Jira 8.4.2#804003-sha1:d21414fc212e3af190e92c2d2ac41299b89402cf.