How do I make an HTTPS request to a GitLab page protected by OAuth2?
Background: A GL repo assembles draft documentation from multiple development repos and publishes on GitLab Pages. All of this is protected by OAuth2. As we near a release date I need to make the draft visible to a wider internal audience, for whom we do not buy GL seats. The public documentation server has a preview URL, lightly protected by Basic Auth. I should be able to reverse proxy to the GitLab Pages draft – if I could only get the HTTPS request to GL to work.
Testing with curl and wget and a Personal Access Token (PAT) XYZXYZXYZXYZXYZXYZXY (no, not really) with every role granted.
❯ wget 'https://ourdev.gitlab.io/this/is/my/documentation?access_token=XYZXYZXYZXYZXYZXYZXY'
--2021-09-24 13:33:32-- https://ourdev.gitlab.io/this/is/my/documentation?access_token=XYZXYZXYZXYZXYZXYZXY
Resolving ourdev.gitlab.io (ourdev.gitlab.io)... 35.185.44.232
Connecting to ourdev.gitlab.io (ourdev.gitlab.io)|35.185.44.232|:443... connected.
HTTP request sent, awaiting response... 302 Found
Location: https://projects.gitlab.io/auth?domain=https://ourdev.gitlab.io&state=bCN5ylZVNsjKt33cFslzNw== [following]
--2021-09-24 13:33:33-- https://projects.gitlab.io/auth?domain=https://ourdev.gitlab.io&state=bCN5ylZVNsjKt33cFslzNw==
Resolving projects.gitlab.io (projects.gitlab.io)... 35.185.44.232
Connecting to projects.gitlab.io (projects.gitlab.io)|35.185.44.232|:443... connected.
HTTP request sent, awaiting response... 302 Found
Location: https://gitlab.com/oauth/authorize?client_id=5059a88907e1b093c23df47d996183b101a862a5e53e099b563120d1308db2c1&redirect_uri=https://projects.gitlab.io/auth&response_type=code&state=bCN5ylZVNsjKt33cFslzNw==&scope=api [following]
--2021-09-24 13:33:33-- https://gitlab.com/oauth/authorize?client_id=5059a88907e1b093c23df47d996183b101a862a5e53e099b563120d1308db2c1&redirect_uri=https://projects.gitlab.io/auth&response_type=code&state=bCN5ylZVNsjKt33cFslzNw==&scope=api
Resolving gitlab.com (gitlab.com)... 172.65.251.78, 2606:4700:90:0:f22e:fbec:5bed:a9b9
Connecting to gitlab.com (gitlab.com)|172.65.251.78|:443... connected.
HTTP request sent, awaiting response... 302 Found
Location: https://gitlab.com/users/sign_in [following]
--2021-09-24 13:33:34-- https://gitlab.com/users/sign_in
Reusing existing connection to gitlab.com:443.
HTTP request sent, awaiting response... 503 Service Temporarily Unavailable
2021-09-24 13:33:34 ERROR 503: Service Temporarily Unavailable.
Similar results from:
wget 'https://oauth2:XYZXYZXYZXYZXYZXYZXY#https://ourdev.gitlab.io/this/is/my/documentation'
curl -ILs --header "Authorization: Bearer XYZXYZXYZXYZXYZXYZXY" "https://ourdev.gitlab.io/this/is/my/documentation"
Everything seems to lead to https://gitlab.com/users/sign_in followed by the 503.
Many articles show how to use PATs to reach the GitLab API. Perhaps they are not the way to simply request a page?
GitLab devs say (a year ago) it’s “not possible currently”:
Programmatic access to private pages
Related
I want to block some sites with my proxy by responding with 403. I succeeded with responding in such a way in an HTTP proxy, but when I get the CONNECT method, I respond with:
HTTP/1.1 403 Forbidden
Status: 403 Forbidden
Proxy-agent: smth
Connection: close
But users still get the ERR_TUNNEL_CONNECTION_FAILED. What can I do to provide users with a nice 403 error?
RFC 2817 does not disallow 4xx codes, and the Draft has the section "4. Extensibility" that states that
"The tunneling handshake is freely extensible using the HTTP/1.x headers;"
But it looks like all browsers decided to just ignore almost every non-2xx codes for security reasons.
Here are some bugs reports:
https://learn.microsoft.com/en-us/previous-versions/windows/internet-explorer/ie-developer/compatibility/dd565641(v=vs.85)
Internet Explorer 8 has a feature that ensures that the secure connection is made all the way to the target server. If it isn't, then no page is displayed.
https://bugzilla.mozilla.org/show_bug.cgi?id=479880
I realize that I'm way late to the party here, but [...] this fix is terrible for proxy admins (myself included). Our proxy returns a 403 forbidden for filtered SSL content and an error page about why the content is blocked, and Firefox just drops all of it on the floor
https://bugs.chromium.org/p/chromium/issues/detail?id=7338
So that's the way it is.
I am sending a HTTPS request which needs a Bearer token. I have added a Header Manager to my HTTP request in JMeter and defined NAMEand VALUE in the Header Manager as Authorization and "Bearer xxxxyxyxyxz" respectively.
The request is running for 21 seconds each time and then its failed with below text
org.apache.http.conn.HttpHostConnectException: Connect to abcd.uat.xyz.com:443 [abcd.uat.xyz.com/11.222.250.10] failed: Connection timed out: connect.
Am I sending the bearer token in right way with Header Manager or is there some other way to send this kind of requests?
Note: Its(The same request and Bearer token) working perfectly in Postman and I am getting the correct response as well.
The issue is resolved after adding Proxy Server details to the request. I have added those in Advanced section of HTTP request.
In Postman, the tool using default system proxy settings, hence working without any issues but when it comes to Jmeter we need to add Proxy Server details explicitly.
Based on the exception it seems, you are trying to access HTTP server with HTTPS request.
Make sure to confirm whether "abcd.uat.xyz.com:443" support https or not..If not simply make your request HTTP by updating Protocol field.
I am a rookie when it comes to Jmeter and I am struggling to get a HTTP Request to run successfully (even for 1 iteration).
The Site being tested is an Intranet site used by staff. In terms of accessing via any browser, the homepage loads (having authenticated me as a user through my Domain Login/Password - through logging in to my laptop). From what I can gather from tools such as FIDDLER:
GET request to the server sent, Server response is 401 unauthorized (WWW-Authenticate: Negotiate)
GET REQUEST header sent back with Authorization: Negotiate HASH, Server response is 401 Unauthorized with a different WWW-Authenticate: Negotiate HASH
GET REQUEST sent back with same Authorization: Negotiate HASH as received in previous SERVER response. Server responds with HTTP 200.
I am struggling to get the above working in JMETER. I have tried by capturing the output from Blazemeter Chrome plugin and saving as .jmx file but replaying this results in
div id="header">h1>Server Error
h2>401 - Unauthorized: Access is denied due to invalid credentials.
h3>You do not have permission to view this directory or page using the credentials that you supplied.
Any ideas as to how I should go about getting this to work would be greatly appreciated.........
JMeter comes with HTTP Authorization Manager which you can use to bypass NTLM authentication challenge. Just add it to your Test Plan and provide the following values:
Username: your Windows domain user name
Password: your Windows domain password
Domain: your Windows domain
Mechanism: BASIC_DIGEST
The HTTP Authorization Manager should automatically handle auth flow. See Windows Authentication with Apache JMeter article for more details if required.
My questions is simple, but I cannot find an answer and I haven't got any resources to test it myself.
Can I make HTTPS CORS request from one domain to another HTTPS domain?
Can I make HTTP CORS request from one domain to another HTTPS domain?
I know that I can do HTTP CORS request from one domain to another HTTP domain, but I don't know if there is any difference when I use HTTPS.
Yes you can do a CORS request from a HTTPS domain to another HTTPS domain.
The only difference is because HTTPS is a secure origin, you can only make call to secure origin, so not to HTTP, the browser will block it with a message like:
Mixed Content: The page at 'https://example.com/index.html' was loaded over HTTPS, but requested an insecure XMLHttpRequest endpoint 'http://example2.com/endpoint'. This request has been blocked; the content must be served over HTTPS.
Warning: If you allow http requests to call your https webpage, it will be insecure because it means an attacker can force requests to your https webpage with the cookies of a victim and read the answer
Beware if you still need to support IE8/IE9 and are using XDomainRequest as it does not support cross-protocol requests. As per MDN:
The origin's security protocol must match that of the requested URL. (http to http, https to https). If these do not match, the request will error "Access is Denied".
I've written a proxy that provides tunnelling for secured connections. However, the proxy tends to redirect (non-secured, HTTP GET) connections to a redirected page if the site being accessed is invalid. It does it by returning a "HTTP/1.0 302 Moved Temporarily" message. Will returning a HTTP/1.0 302 Moved Temporarily to a HTTP CONNECT also result in the same browser behaviour?
I've tried and it doesn't work as expected.
Therefore I was wondering if anyone could suggest the correct way of asking a HTTP CONNECT request to redirect to a different page?
If the site being accessed is invalid, you probably want to do something other than a 302 redirect. Where would you redirect to? A 302 redirect should probably only be passed back to the client from the destination site itself, but in that case it would still be a valid site.
If the site the client is requesting is not accepting connections, you should probably return a 502 Bad Gateway error to the client. See:
https://www.rfc-editor.org/rfc/rfc2616#section-10.5.3
If you can actually connect to the destination site, you should return a 200 OK to the client and then whatever the destination site returns over the proxied connection.