[GRADLE-3091] HTTP dependency resolvers should support preemptive authentication Created: 18/May/14  Updated: 03/Aug/15  Resolved: 03/Aug/15

Status: Resolved
Project: Gradle
Affects Version/s: None
Fix Version/s: 2.7-rc-1

Type: Improvement
Reporter: Gradle Forums Assignee: Mark Vieira (Inactive)
Resolution: Fixed Votes: 3


 Description   

Certain repos (GitHub being one of them) return 404 for unauthorized resources, instead of 401.

If the user supplies credentials, we should just preemptively include them in the request.



 Comments   
Comment by Gradle Forums [ 18/May/14 ]

I put the credentials into "gradle.properties" and in "build.gradle" I have this:

maven {
url 'http://mycompany.ycy.de/artifactory/repo'
credentials {
username artifactoryUser
password artifactoryPassword
}
}

This works (at least in 1.11).

Comment by Gradle Forums [ 18/May/14 ]

I wonder if it depends on the way that the server does auth. Perhaps github does it differently than your server.

On thing that's sort of unusual about github: if you don't auth with raw.githubusercontent.com, it returns a 404 Not Found rather than a 401 Unauthorized. In the debug logs I can see that gradle is trying to do an HTTP HEAD on the artifact url, where it gets a 404 (which implies that it didn't send auth info).

Perhaps gradle doesn't auth until it does an HTTP GET, or it only auths if the server returns 404 Not Found?

Comment by Gradle Forums [ 18/May/14 ]

Looking more closely at the debug log, the line that says Target auth state: UNCHALLENGED is really suspicious. Why isn't it sending along auth info?

14:23:23.927 [DEBUG] [org.gradle.api.internal.externalresource.transport.http.HttpClientHelper] Performing HTTP HEAD: https://raw.githubusercontent.com/foobar/repo/master/releases/com/foobar/myartifact/0.0.1/mylib-0.0.1.pom
14:23:24.116 [DEBUG] [org.apache.http.impl.conn.PoolingClientConnectionManager] Connection request: [route:

{s}->https://raw.githubusercontent.com][total kept alive: 0; route allocated: 0 of 5; total allocated: 0 of 10]
14:23:24.128 [DEBUG] [org.apache.http.impl.conn.PoolingClientConnectionManager] Connection leased: [id: 0][route: {s}

->https://raw.githubusercontent.com][total kept alive: 0; route allocated: 1 of 5; total allocated: 1 of 10]
14:23:24.990 [DEBUG] [org.apache.http.impl.conn.DefaultClientConnectionOperator] Connecting to raw.githubusercontent.com:443
14:23:25.319 [DEBUG] [org.apache.http.client.protocol.RequestAddCookies] CookieSpec selected: best-match
14:23:25.331 [DEBUG] [org.apache.http.client.protocol.RequestAuthCache] Auth cache not set in the context
14:23:25.331 [DEBUG] [org.apache.http.client.protocol.RequestTargetAuthentication] Target auth state: UNCHALLENGED
14:23:25.332 [DEBUG] [org.apache.http.client.protocol.RequestProxyAuthentication] Proxy auth state: UNCHALLENGED
14:23:25.333 [DEBUG] [org.apache.http.impl.client.SystemDefaultHttpClient] Attempt 1 to execute request
14:23:25.334 [DEBUG] [org.apache.http.impl.conn.DefaultClientConnection] Sending request: HEAD /foobar/repo/master/releases/com/foobar/myartifact/0.0.1/mylib-0.0.1.pom HTTP/1.1
14:23:25.336 [DEBUG] [org.apache.http.headers] >> HEAD /foobar/repo/master/releases/com/foobar/myartifact/0.0.1/mylib-0.0.1.pom HTTP/1.1
14:23:25.336 [DEBUG] [org.apache.http.headers] >> Accept-Encoding: gzip,deflate
14:23:25.337 [DEBUG] [org.apache.http.headers] >> Host: raw.githubusercontent.com
14:23:25.338 [DEBUG] [org.apache.http.headers] >> Connection: Keep-Alive
14:23:25.338 [DEBUG] [org.apache.http.headers] >> User-Agent: Gradle/1.12 (Mac OS X;10.9.2;x86_64) (Oracle Corporation;1.7.0_45;24.45-b08)
14:23:25.464 [DEBUG] [org.apache.http.impl.conn.DefaultClientConnection] Receiving response: HTTP/1.1 404 Not Found
14:23:25.464 [DEBUG] [org.apache.http.headers] << HTTP/1.1 404 Not Found
14:23:25.465 [DEBUG] [org.apache.http.headers] << Date: Thu, 15 May 2014 20:23:25 GMT
14:23:25.465 [DEBUG] [org.apache.http.headers] << Server: Apache
14:23:25.466 [DEBUG] [org.apache.http.headers] << Content-Security-Policy: default-src 'none'
14:23:25.466 [DEBUG] [org.apache.http.headers] << Access-Control-Allow-Origin: https://render.githubusercontent.com
14:23:25.467 [DEBUG] [org.apache.http.headers] << X-XSS-Protection: 1; mode=block
14:23:25.467 [DEBUG] [org.apache.http.headers] << X-Frame-Options: deny
14:23:25.468 [DEBUG] [org.apache.http.headers] << X-Content-Type-Options: nosniff
14:23:25.468 [DEBUG] [org.apache.http.headers] << Strict-Transport-Security: max-age=31536000
14:23:25.469 [DEBUG] [org.apache.http.headers] << Content-Length: 9
14:23:25.469 [DEBUG] [org.apache.http.headers] << Accept-Ranges: bytes
14:23:25.470 [DEBUG] [org.apache.http.headers] << Via: 1.1 varnish
14:23:25.470 [DEBUG] [org.apache.http.headers] << X-Served-By: cache-sv95-SJC3
14:23:25.471 [DEBUG] [org.apache.http.headers] << X-Cache: MISS
14:23:25.471 [DEBUG] [org.apache.http.headers] << X-Cache-Hits: 0
14:23:25.472 [DEBUG] [org.apache.http.headers] << Vary: Authorization,Accept-Encoding
14:23:25.472 [DEBUG] [org.apache.http.headers] << Expires: Thu, 15 May 2014 20:28:25 GMT
14:23:25.472 [DEBUG] [org.apache.http.headers] << Source-Age: 0
14:23:25.473 [DEBUG] [org.apache.http.headers] << Keep-Alive: timeout=10, max=50
14:23:25.473 [DEBUG] [org.apache.http.headers] << Connection: Keep-Alive

Just to be clear, even tho its showing 404 Not Found, the file really is there. I can curl it with my credentials and it pulls down fine:

curl -u myusername https://raw.githubusercontent.com/foobar/repo/master/releases/com/foobar/myartifact/0.0.1/mylib-0.0.1.pom

I think the problem is that gradle isn't sending auth info for HTTP HEAD, and github returns 404 Not Found when it doesn't get auth for private repos.

Comment by Gradle Forums [ 18/May/14 ]

More digging: [Why am I getting a 404 error on a repository that exists?]([1]https://developer.github.com/v3/troub...)
----------------------------------------------------------------------------------------
[1] https://developer.github.com/v3/troubleshooting/#why-am-i-getting-a-404-error-on-a-repository-that-exists

Comment by Aaron Simmons [ 25/May/14 ]

This may be related to GRADLE-2968

Comment by Aaron Simmons [ 25/May/14 ]

Hm, it seems like preemptive auth was removed on purpose :
http://www.gradle.org/docs/1.1/release-notes#dependency-resolution-supports-http-digest-authentication
It's not a bug, its a feature!

Comment by Aaron Simmons [ 25/May/14 ]

From what I can gather, the best practice is to not send auth info if you don't have to.

I can see a couple ways of handling this problem:

  1. Turn pre-emptive auth back on (probably not happening, since it was turned off on purpose)
  2. Have gradle consider a 404 equivalent to a 401. ie, if it gets a 404, it will retry the GET/HEAD with an auth
  3. Add a flag to the gradle dsl to turn pre-emptive auth on, something like this:
    maven { 
        credentials {
            username "${GITHUB_USERNAME}"
            password "${GITHUB_PASSWORD}"
            preemptive true
        }
        url "https://raw.githubusercontent.com/foobar/repo/master/releases"
    }
      
Comment by Rodrigo B. de Oliveira [ 03/Nov/14 ]

An explicit preemptive flag as suggested by Aaron Simmons above seems to fit the bill nicely.

Comment by Aaron Simmons [ 03/Nov/14 ]

Can we please get some movement on this? I can't simply go in and unilaterally change this behavior-- gradle removed it for some reason and the devs need to discuss the options.

Comment by Christian Bahl [ 04/Nov/14 ]

i also prefer option 3, because i´m using jenkins with maven repository plugin. if jenkins is secured and auth is not send, it returns 403 (forbidden). so not to forget or add each status code the flag "preemptive" is the best solution and please add it to one of the next versions!

Comment by Ryan Harter [ 28/Jan/15 ]

I've submitted pull request 386 which implements option 3 above.

Comment by Daz DeBoer [ 02/Mar/15 ]

Seems like the workaround of embedding username/password in the repository URL is no longer an option, with Gradle 2.x:

http://forums.gradle.org/gradle/topics/gradle-2-x-no-longer-supports-embedding-username-and-password-in-maven-url

Comment by Ryan Harter [ 02/Mar/15 ]

I've been using a custom build with my pending PR for several months now successfully.

Comment by Aaron Simmons [ 02/Mar/15 ]

Sadly I've resorted to using workaround plugins like https://github.com/layerhq/gradle-git-repo-plugin.

I would build my own gradle with Ryan Harter's PR, but our team is too spread out (contractors, etc) to require everyone to build a custom gradle.

Generated at Wed Jun 30 12:39:42 CDT 2021 using Jira 8.4.2#804003-sha1:d21414fc212e3af190e92c2d2ac41299b89402cf.