Uploaded image for project: 'Gradle'
  1. Gradle
  2. GRADLE-1964

Provide a mechanism for caching local filesystem dependencies when access to the filesystem is slow

    Details

    • Type: Improvement
    • Status: Resolved
    • Resolution: Won't Fix
    • Affects Version/s: None
    • Fix Version/s: None
    • Gradle Forums topic Reference:

      Description

      As a part of upgrading to milestone-5, we changed the way repositories are defined.
      The problem we are facing with the new ivy {} repository notation is that dependencies are not being cached in the gradle cache. Repositories are currently defined as follows:

      repositories {
      ivy

      { artifactPattern "/[organisation]/[module]/[type]s/[artifact]-[revision].[ext]" ivyPattern "/[organisation]/[module]/ivy-[revision].xml" }

      }

      On using FileSystemResolver dependency caching works fine.

        Issue Links

          Activity

          Hide
          forums Gradle Forums added a comment -

          When a dependency artifact is available on the local filesystem, it usually doesn't make sense to copy this file into the cache. The purpose of the cache is to keep a temporary copy of any artifacts that would be expensive to re-fetch: local files usually don't meet this criteria and keeping an additional copy would be wasteful.

          However, there are times when the filesystem is slow to access (eg a network share), and caching would be useful. Currently there is no mechanism for telling Gradle to cache the artifacts in these cases.

          A way to instruct Gradle to cache files from a local repository is something we'd like to add. (And perhaps a way to instruct Gradle NOT to cache files from a remote repository)

          Show
          forums Gradle Forums added a comment - When a dependency artifact is available on the local filesystem, it usually doesn't make sense to copy this file into the cache. The purpose of the cache is to keep a temporary copy of any artifacts that would be expensive to re-fetch: local files usually don't meet this criteria and keeping an additional copy would be wasteful. However, there are times when the filesystem is slow to access (eg a network share), and caching would be useful. Currently there is no mechanism for telling Gradle to cache the artifacts in these cases. A way to instruct Gradle to cache files from a local repository is something we'd like to add. (And perhaps a way to instruct Gradle NOT to cache files from a remote repository)
          Hide
          vampire Björn Kautler added a comment -

          This is a major flaw in my opinion.
          We have a network share as corporate repository.
          There are multiple reasons why it is a bad idea not to cache those files.

          • The network may be slow sometimes or for some people that are connected via VPN always
          • The network may be down, e. g. if you travel with your laptop
          • The server is under more load than necessary
          • The network is under more load than necessary
          • ...

          Please add this possibility.
          My current workaround is to have a local copy of the whole repository on my disk and have the repository location configurable.

          Show
          vampire Björn Kautler added a comment - This is a major flaw in my opinion. We have a network share as corporate repository. There are multiple reasons why it is a bad idea not to cache those files. The network may be slow sometimes or for some people that are connected via VPN always The network may be down, e. g. if you travel with your laptop The server is under more load than necessary The network is under more load than necessary ... Please add this possibility. My current workaround is to have a local copy of the whole repository on my disk and have the repository location configurable.
          Hide
          vampire Björn Kautler added a comment -

          With 1.0-rc1 using FileSystemResolver also does not cache the libraries anymore. How do I get the libraries to be cached again?

          Show
          vampire Björn Kautler added a comment - With 1.0-rc1 using FileSystemResolver also does not cache the libraries anymore. How do I get the libraries to be cached again?
          Hide
          vampire Björn Kautler added a comment -

          Just for others if they have the same issue, I found a new workaround:

          repositories {
             add(new org.apache.ivy.plugins.resolver.ChainResolver()) {
                add new org.apache.ivy.plugins.resolver.FileSystemResolver()
                resolvers[0].identity {
                   name = 'ivy'
                   addIvyPattern "$publicRepoDir/ivy/$GRADLE_IVY_PATTERN"
                   addArtifactPattern "$publicRepoDir/ivy/$GRADLE_ARTIFACT_PATTERN"
                   checkmodified = true
                }
             }
          }
          
          Show
          vampire Björn Kautler added a comment - Just for others if they have the same issue, I found a new workaround: repositories { add( new org.apache.ivy.plugins.resolver.ChainResolver()) { add new org.apache.ivy.plugins.resolver.FileSystemResolver() resolvers[0].identity { name = 'ivy' addIvyPattern "$publicRepoDir/ivy/$GRADLE_IVY_PATTERN" addArtifactPattern "$publicRepoDir/ivy/$GRADLE_ARTIFACT_PATTERN" checkmodified = true } } }
          Hide
          bjakke Bjarne Boström added a comment -

          It would seem the ivy resolvers are deprecated in 2.x : https://discuss.gradle.org/t/custom-ivy-resolver-deprecation-in-gradle-2-x/7492 and the workaround posted here does not work (or maybe I failed something, I get unable to resolve class org.apache.ivy.plugins.resolver.ChainResolver). I think this feature would be important because a network shared ivyrepo is very convinient in companies.

          Show
          bjakke Bjarne Boström added a comment - It would seem the ivy resolvers are deprecated in 2.x : https://discuss.gradle.org/t/custom-ivy-resolver-deprecation-in-gradle-2-x/7492 and the workaround posted here does not work (or maybe I failed something, I get unable to resolve class org.apache.ivy.plugins.resolver.ChainResolver). I think this feature would be important because a network shared ivyrepo is very convinient in companies.
          Hide
          vampire Björn Kautler added a comment -

          Bjarne Boström it is deprecated since some versions and is supposed to be removed in 2.x
          That is mainly what prevents me from upgrading our build past 1.12.

          Show
          vampire Björn Kautler added a comment - Bjarne Boström it is deprecated since some versions and is supposed to be removed in 2.x That is mainly what prevents me from upgrading our build past 1.12.
          Hide
          bmuschko Benjamin Muschko added a comment -

          As announced on the Gradle blog we are planning to completely migrate issues from JIRA to GitHub.

          We intend to prioritize issues that are actionable and impactful while working more closely with the community. Many of our JIRA issues are inactionable or irrelevant. We would like to request your help to ensure we can appropriately prioritize JIRA issues you’ve contributed to.

          Please confirm that you still advocate for your JIRA issue before December 10th, 2016 by:

          • Checking that your issues contain requisite context, impact, behaviors, and examples as described in our published guidelines.
          • Leave a comment on the JIRA issue or open a new GitHub issue confirming that the above is complete.

          We look forward to collaborating with you more closely on GitHub. Thank you for your contribution to Gradle!

          Show
          bmuschko Benjamin Muschko added a comment - As announced on the Gradle blog we are planning to completely migrate issues from JIRA to GitHub. We intend to prioritize issues that are actionable and impactful while working more closely with the community. Many of our JIRA issues are inactionable or irrelevant. We would like to request your help to ensure we can appropriately prioritize JIRA issues you’ve contributed to. Please confirm that you still advocate for your JIRA issue before December 10th, 2016 by: Checking that your issues contain requisite context, impact, behaviors, and examples as described in our published guidelines . Leave a comment on the JIRA issue or open a new GitHub issue confirming that the above is complete. We look forward to collaborating with you more closely on GitHub. Thank you for your contribution to Gradle!
          Hide
          bmuschko Benjamin Muschko added a comment -

          Thanks again for reporting this issue. We haven't heard back from you after our inquiry from November 15th. We are closing this issue now. Please create an issue on GitHub if you still feel passionate about getting it resolved.

          Show
          bmuschko Benjamin Muschko added a comment - Thanks again for reporting this issue. We haven't heard back from you after our inquiry from November 15th. We are closing this issue now. Please create an issue on GitHub if you still feel passionate about getting it resolved.

            People

            • Assignee:
              Unassigned
              Reporter:
              forums Gradle Forums
            • Votes:
              4 Vote for this issue
              Watchers:
              7 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development