[GRADLE-2723] Gradle wrapper duplicates identical Gradle distributions in user home Created: 25/Mar/13  Updated: 02/Feb/17  Resolved: 02/Feb/17

Status: Resolved
Project: Gradle
Affects Version/s: None
Fix Version/s: None

Type: Bug
Reporter: Gradle Forums Assignee: Unassigned
Resolution: Won't Fix Votes: 1


 Description   

The Gradle Wrapper uses a hash of the distributionUrl to isolate gradle distributions downloaded from different locations. This has the unfortunate effect of creating many duplicate distributions in GRADLE_USER_HOME when the Gradle wrapper in several projects points at a Gradle distribution included in that project via SVN Externals.

Gradle wrapper should really be using the hash of the downloaded distribution for isolation, not the url used to download gradle.

Original report:
--------------------

In our team we have many projects which are using Gradle. Every project have in its root directory 'externals'.

-hpa
-- externals
-shardconnector
-- externals

Externals folder is shared between all projects (It's implemented using Subversion externals property, so it is same for every project). In 'externals' folder we Gradle distribution: gradle-1.4-bin.zip. Gradle wrapper is using this exact distribution. Now, when I am building projects I can see that every project is creating it's own distribution in my home:
c:\Users\me\.gradle\wrapper\dists\gradle-1.4-bin\

It causes big problems on CI server, as we have 7 projects using gradle. Every project is taking in CI server home about 80 MB for same files: zipped gradle distribution and its unpacked version. Currently its about 500 MB on server and will be even more if we will add another projects. Also other (human) users can use Gradle and the same problem will appear by them.

My question is if it is necessary to take into consideration Gradle distribution path? If there is no essential reasons behind current behavior, I would say that this is a bug.

I would be thankful for your comments and eventually solution for this problem.



 Comments   
Comment by Gradle Forums [ 25/Mar/13 ]

The Gradle wrapper is designed to download Gradle from a known location and make it available for local use. It does this by storing the distribution in the `~/.gradle` directory. As far as I know, there's no way to prevent the Gradle wrapper from copying the distribution zip file and unzipping it.

I assume that you have your Gradle Wrapper configured to 'download' the distribution from the SVN externals directory instead?
Are all of your CI jobs running with different user home directories?

Comment by Gradle Forums [ 25/Mar/13 ]

Storing Gradle distribution in home directory is ok. I assume it is necessary for Gradle to work. Problem is that same distribution is copied multiple times.

CI jobs are run by same user. It's not root cause of problem.

I think that current Gradle behavior is like below (I didn't look at source code, though):
1. Gradle wrapper is calculating hash code from whole path to Gradle distribution.
2. If there is directory named with same hash in c:\Users\me\.gradle\wrapper\dists\gradle-1.4-bin\ it is assumed that distribution is already installed. If there is different hash code of distribution path, then distribution is installed again.

Assumption behind above algorithm works in many cases. But in general you can not assume that different distribution location always mean different distribution. E.g. in my case different distribution paths point to same distribution.

I think it would be better to make distribution resolution in 2 steps:
1. Gradlew should first check distribution path (as it is fast)
2. Then if there is already information in cache it should use installed distribution
3. If there is not information in gradlew cache about distribution, then there should be calculated hash from whole distribution zip (or at least from distribution name).
a. if according to calculated hash there is already installed gradle distribution in dists directory, cache should be updated and existing distribution should be used.
b. if there is no distribution in dists, then distribution should be downloaded, unpacked and installed.

Above algorithm should be fast and should not cause duplicated distributions, even if distribution paths are different.

Comment by Gradle Forums [ 25/Mar/13 ]

It seems odd that you're experiencing this behaviour. What does your `grade-wrapper.properties` file look like? Are you configuring the `distributionPath` or the `distributionUrl`?

Comment by Gradle Forums [ 25/Mar/13 ]

Well, it works this way. It's simple to reproduce, I have even created test case:

1. Create 3 projects in some directory: project1, project2, project3
2. Put into projects following 'gradle.build' file:

apply plugin: 'java'
task wrapper(type: Wrapper) {
gradleVersion = '1.5-rc-3'
distributionUrl = 'gradle-1.5-rc-3-bin.zip'
}

3. In every project's root directory execute: gradle wrapper
4. Put bin distribution of gradle 1.5 rc3 into created /gradle/wrapper directory in every project
5. Execute ./gradlew test in root of every project
6. You will see that gradle is unzipped and installed 3 times - once for every project

I have a zipped files, but can't attach it to this thread.

Comment by Benjamin Muschko [ 15/Nov/16 ]

As announced on the Gradle blog we are planning to completely migrate issues from JIRA to GitHub.

We intend to prioritize issues that are actionable and impactful while working more closely with the community. Many of our JIRA issues are inactionable or irrelevant. We would like to request your help to ensure we can appropriately prioritize JIRA issues you’ve contributed to.

Please confirm that you still advocate for your JIRA issue before December 10th, 2016 by:

  • Checking that your issues contain requisite context, impact, behaviors, and examples as described in our published guidelines.
  • Leave a comment on the JIRA issue or open a new GitHub issue confirming that the above is complete.

We look forward to collaborating with you more closely on GitHub. Thank you for your contribution to Gradle!

Comment by Benjamin Muschko [ 02/Feb/17 ]

I don't understand the use case of wanting to host the Wrapper ZIP file in each of the projects. That defeats the purpose of the Wrapper. I am going to mark the issues as "Won't fix".

Generated at Wed Jun 30 12:29:44 CDT 2021 using Jira 8.4.2#804003-sha1:d21414fc212e3af190e92c2d2ac41299b89402cf.