We are trying to make a call to web service (nonsecure i.e. HTTP) in Xamarin.iOS. As it is an http URL we have added NSAppTransportSecurity to bypass the secure access.
Currently, we are using HttpClient with NSUrlSessionHandler so that ATS config specified in info.plist gets applied.
client = new HttpClient(new NSUrlSessionHandler());
Problem #1
Now, the issue is whenever we are firing the web service request and after that, if we switch to a different app and come back to our app, the request gets stuck and does not return anything. if we stay on our app, it returns the response successfully.
Problem #2
In order to resolve Problem#1 issue of switching app, we tried client = new HttpClient(new CFNetworkHandler()); i.e. using CFNetworkHandler and that resolves our issue but this eventually creates another issue of accessing http URL. As we have used CFNetworkHandler , configurations related to NSAppTransportSecurity in info.plist gets ignored. As Apple Developer Document says
ATS doesn’t apply to calls your app makes to lower-level networking
interfaces like the Network framework or CFNetwork.
Error that we get while calling webservice with CFNetwork is as follows and it looks like info.plist config is getting ignored
StatusCode: 505, ReasonPhrase: 'HTTP/1.1 505 HTTP Version Not Supported', Version: 1.1, Content: System.Net.Http.CFContentStream, Headers:
{
Date: Fri, 03 Jul 2020 06:15:19 GMT
Connection: close
Server: Microsoft-HTTPAPI/2.0
Content-Type: text/html; charset=us-ascii
Content-Length: 350
}
However, on the other end,Microsoft Xamarin Says
App Transport Security does not apply to Xamarin apps using Managed HTTPClient implementations. It applies to connections using CFNetwork HTTPClient implementations or NSURLSession HTTPClient implementations only.
So, if we use NSUrlSessionHandler the request got stuck while switching app and if we use CFNetworkHandler then ATS won't allow to access the services.
ATS Configurations are as follows
set NSAppTransportSecurity with NSAllowsArbitraryLoads set to YES
set NSExceptionDomains with following parameters
a. naaxdev.cloudapp.net
b. NSIncludesSubdomains set to YES
c. NSExceptionAllowsInsecureHTTPLoads set to YES
d. NSExceptionRequiresForwardSecrecy set to NO
Related
I don't know much about Webflux / Reactor / Netty. I'm using Spring's WebClient to do all the heavy lifting. But it appears not to work correctly when a server responds back early with an error.
My understanding is when you are POSTing data to a server, the server can respond at any time with an HTTP 4XX error. The client is supposed to stop sending the HTTP body and read that error.
I have a very simply WebClient that POSTs data to a server. It looks like this:
FileResponse resp = client.post().uri(uri)
.contentType(MediaType.APPLICATION_OCTET_STREAM)
.header("Authorization", "Bearer " + authorizationToken)
.accept(MediaType.APPLICATION_JSON)
.bodyValue(data)
.retrieve()
.bodyToMono(FileResponse.class)
.block();
The body can contain a large amount of data (100+KB). Apparently the server looks at the header, validates the authorization token, and only if it's valid, reads the body. If the authorization token is not valid (expired, etc) it immediately responds with an "HTTP 401 Unauthorized" with the response body "{"message": "Invalid user/password"}" while the client is still sending the body. The server then closes the socket which results in the WebClient throwing this:
2022-08-10 15:56:03,474 WARN [reactor.netty.http.client.HttpClientConnect] (reactor-http-nio-1) [id: 0xa7b48bb8, L:/5.6.7.8:51122 - R:dubcleoa030/1.2.3.4:5443] The connection observed an error: java.io.IOException: An existing connection was forcibly closed by the remote host
at java.base/sun.nio.ch.SocketDispatcher.read0(Native Method)
at java.base/sun.nio.ch.SocketDispatcher.read(SocketDispatcher.java:43)
at java.base/sun.nio.ch.IOUtil.readIntoNativeBuffer(IOUtil.java:276)
at java.base/sun.nio.ch.IOUtil.read(IOUtil.java:233)
at java.base/sun.nio.ch.IOUtil.read(IOUtil.java:223)
at java.base/sun.nio.ch.SocketChannelImpl.read(SocketChannelImpl.java:358)
at deployment.bp-global.war//io.netty.buffer.PooledByteBuf.setBytes(PooledByteBuf.java:253)
at deployment.bp-global.war//io.netty.buffer.AbstractByteBuf.writeBytes(AbstractByteBuf.java:1133)
at deployment.bp-global.war//io.netty.channel.socket.nio.NioSocketChannel.doReadBytes(NioSocketChannel.java:350)
at deployment.bp-global.war//io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:148)
at deployment.bp-global.war//io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:714)
at deployment.bp-global.war//io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:650)
at deployment.bp-global.war//io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:576)
at deployment.bp-global.war//io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:493)
at deployment.bp-global.war//io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989)
at deployment.bp-global.war//io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
at deployment.bp-global.war//io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
at java.base/java.lang.Thread.run(Thread.java:834)
I've made the same request with curl, and it's handled properly. Curl sees the server's early response, stops sending the body and processes the response from the server. I've chopped out a lot of fluff from the curl output but here is the important stuff...
Trying 1.2.3.4...
* TCP_NODELAY set
* Connected to 1.2.3.4 port 5443 (#0)
> POST /api/folders/file/?path=/out HTTP/1.1
> Host: 1.2.3.4:5443
> User-Agent: curl/7.61.1
> accept-encoding: gzip
> Content-Type: application/octet-stream
> Authorization: Bearer youshallnotpass
> accept: application/json
> Content-Length: 298190
>
< HTTP/1.1 401 Unauthorized
< Server: Cleo Harmony/5.7.0.3 (Linux)
< Date: Wed, 10 Aug 2022 21:59:23 GMT
< Content-Length: 36
< Content-Language: en
< Content-Type: application/json
< Connection: keep-alive
* HTTP error before end of send, stop sending
* Closing connection 0
{"message": "Invalid user/password"}
I'm not sure if the issue is with Spring's WebClient or the underlying reactor-netty stuff. But am I crazy or does it just look broken if the server responds early? If I am correct that it's broken, any thoughts on a work-around?
Thank you!
Todd
I setup a small stand-alone command line Spring Boot program so I could test various aspects of this issue. What I found is that if I send the body from memory (byte[]) the issue occurs. If I send the body from a file resource as shown below, everything works correctly.
Our current very large product is using Spring Boot 2.3.3. My stand-alone test program gave me the ability to quickly upgrade Spring Boot to 2.7.2. Everything works correctly in Spring Boot 2.7.2. So it was definitely a bug that was fixed at some point.
Unfortunately our large project cannot be upgraded to Spring Boot 2.7.2 overnight. It will be a large effort which will require extensive testing from our QA department. So for now, as a work-around, I'll write the body payload to a temporary file so I can get WebClient to read it as a file resource which works in 2.3.3 as shown below. Just in case any other poor developer runs into this and needs a work-around, try this...
InputStreamResource resource = new InputStreamResource(new FileInputStream(tempFilename));
String resp = client.post().uri(uri)
.contentType(MediaType.APPLICATION_OCTET_STREAM)
.header("Authorization", "Bearer youshallnotpass")
.accept(MediaType.APPLICATION_JSON)
.body(BodyInserters.fromResource(resource))
.retrieve()
.bodyToMono(String.class)
.block();
EDIT: I started going through the releases to determine when this was fixed. This issues appears to be fixed in Spring Boot 2.3.4 (Netty 4.1.52). I cannot find any reference to this bug in the release notes of Spring Boot or Netty. So I'm not sure where the issue was. Perhaps it was in a deeper dependency. But it was definitely fixed with Spring Boot 2.3.4.
It may happened because there is a conflict in dependencies (may create conflict with azure dependency for same package):
update your spring boot version I ahve updted to latest and works for me.
I am working on a RESTful web application using Go + gorilla/mux v1.4 framework. Some basic security testing after a release revealed an Open URL Redirection vulnerability in the app that allows user to submit a specially crafted request with an external URL that causes server to response with a 301 redirect.
I tested this using Burp Suite and found that any request that redirects to an external URL in the app seems to be responding with a 301 Moved Permanently. I've been looking at all possible ways to intercept these requests before the 301 is sent but this behavior seems to be baked into the net/http server implementation.
Here is the raw request sent to the server (myapp.mycompany.com:8000):
GET http://evilwebsite.com HTTP/1.1
Accept: */*
Cache-Control: no-cache
Host: myapp.mycompany.com:8000
Content-Length: 0
And the response any time is:
HTTP/1.1 301 Moved Permanently
Location: http://evilwebsite.com/
Date: Fri, 13 Mar 2020 08:55:24 GMT
Content-Length: 0
Despite putting in checks for the request.URL to prevent this type of redirect in the http.handler, I haven't had any luck getting the request to reach the handler. It appears that the base http webserver is performing the redirect without allowing it to reach my custom handler code as defined in the PathPrefix("/").Handler code.
My goal is to ensure the application returns a 404-Not Found or 400-Bad Request for such requests. Has anybody else faced this scenario with gorilla/mux. I tried the same with a Jetty web app and found it returned a perfectly valid 404. I've been at this for a couple of days now and could really use some ideas.
This is not the claimed Open URL redirect security issue. This request is invalid in that the path contains an absolute URL with a different domain than the Host header. No sane client (i.e. browser) can be lured into issuing such an invalid request in the first place and thus there is no actual attack vector.
Sure, a custom client could be created to submit such a request. But a custom client could also be made to interpret the servers response in a non-standard way or visit a malicious URL directly without even contacting your server. This means in this case the client itself would be the problem and not the servers response.
I am in the process of upgrading our jetty from 9.2.24 to 9.4.10, for an app that works extensively with websockets.
I have an existing test (junit) that sets embedded jetty, registers to it rest resource and websocket servlet and then tests to see if they can be accessed.
The test works perfectly well when jetty is at version 9.2.24. An attempt to move to version 9.4.10 with the very same code fails with
java.io.IOException: Connect failure
at org.eclipse.jetty.websocket.jsr356.ClientContainer.connect(ClientContainer.java:232)
at org.eclipse.jetty.websocket.jsr356.ClientContainer.connectToServer(ClientContainer.java:255)
...
Caused by: org.eclipse.jetty.websocket.api.UpgradeException: 400 Bad Request
at org.eclipse.jetty.websocket.client.WebSocketUpgradeRequest.onComplete(WebSocketUpgradeRequest.java:522)
at org.eclipse.jetty.client.ResponseNotifier.notifyComplete(ResponseNotifier.java:193)
The websocket definition on server side is based o JSR356 spec (i.e. extends EndPoint). The websocket client used to access the websocket is also based on the javax.websocket (i.e. ContainerProvider.getWebSocketContainer().connectToServer(Endpoint instance...)) - where the websocket container is effectively a jetty one...
The server sets up perfectly. The problem is only when trying to access the websocket. I have debugged and could not find any difference in the way the client initiates and sends the websocket request. In particular the request has a the 'upgrade' header set to 'websocket' as expected.
So I could only assume that the problem is in the way the websocket resource is registered in the embedded jetty. I have debugged the working flow (with 9.2.24) and found the most early place where the connection is accepted in jetty (one of the selector threads at AbstractConnection). but from some reason I am not getting to that point for the websocket when working with 9.4.10
I have read several resources and SO question (e.g. this question) and could not found anything that will help me with this problem.
I am in a dead end.
Here is the key elements in the relevant code of the server registration (I also have another rest resource along with the websocket one):
// web socket
ServletContextHandler wsContext = new ServletContextHandler(ServletContextHandler.SESSIONS);
wsContext.setContextPath("/ws_api");
ServerContainer container = WebSocketServerContainerInitializer.configureContext(servletContextHandler);
container.addEndpoint(new BasicServerEndpointConfig(container.getClient(), endpointClassObject, path)
// rest handler
ServletContextHandler restContext = new ServletContextHandler(ServletContextHandler.SESSIONS);
restContext.setContextPath("/rest_api");
...
ServletHolder sh = new ServletHolder(...);
restContext.addServlet(sh, "/*");
final HandlerList handlers = new HandlerList();
handlers.setHandlers(new Handler[]{wsContext, restContext, new DefaultHandler()});
server.setHandler(handlers);
Help...
Update (additional information per Joakim Erdfelt request):
I am at class HTTPConnection class, in BP at onComplete() method, fetching the request headers from the _channel._fields object I get:
The response object's status is 200 (and not 101 as expected):
My endpoint object is part of a large inheritance chain. It is full of boilerplate business logic code that I need to remove before I can upload it, but in the root stands the javax.websocket.Endpont class, where we implemented only the onOpen(Session session, EndpointConfig config) method. I am not getting to that method when debugging, seems to fail long before...
Your request headers looks like this ...
Accept: application/json, application/*+json
Accept-Encoding: gzip
Cache-Control: no-cache
Connection: keep-alive
Content-Type: application/json
Host: 127.0.0.1:8080
Pragma: no-cache
Sec-WebSocket-Key: sMQPm6Cf00itLII3QBb4w==
Sec-WebSocket-Version: 13
Upgrade: websocket
User-Agent: Java/1.8.0_144
That is an invalid WebSocket Upgrade Request.
The most glaring omission is
Connection: upgrade
But there's also other fields that a compliant WebSocket Client would never set.
Content-Type: application/json
Accept: application/json, application/*+json
Accept-Encoding: gzip
I am developing an OpenCMIS client using BindingType.Browser. Creating a session passing the required parameters for USER, PASSWORD, BROWSER_URL, BINDING_TYPE and REPOSITORY_ID works as expected. The session is created and further steps can be executed.
Now I want to make my client run on machines using HTTP proxies to access the internet. To specify the proxy to access I set the system properties http.proxyUrl and http.proxyPort. This works too as long as the proxy does not require authentication.
Well, that's the point where I am struggeling right now. I activated the authentication in my test proxy and in the client code I added the parameters PROXY_USER and PROXY_PASSWORD to create the session. But this doesn't seem to work. I already debugged the used StandardAuthenticationProvider to verify what happens. The HTTP-Header "Proxy-Authenticate" is created by the authentication provider but it seems that the chemistry framework is sending the request without that header. In the log of my proxy the received requests do not contain any security headers.
CONNECT my.server.org:443 HTTP/1.1
User-Agent: Java/1.8.0_111
Host: my.server.org
Accept: text/html, image/gif, image/jpeg, *; q=.2, */*; q=.2
Proxy-Connection: keep-alive
Any suggestions?
I found the solution in the following thread: Java Web Start: Unable to tunnel through proxy since Java 8 Update 111.
Since Java 8 Update 111 the Basic authentication scheme has been deactivated, by default, in the Oracle Java Runtime, by adding "Basic" to the jdk.http.auth.tunneling.disabledSchemes networking property when proxying HTTPS. For more information refer to 8u111 Update Release Notes.
To re-activate the proxying you need to set jdk.http.auth.tunneling.disabledSchemes="". This cann be done via VM Argument when starting the application
-Djdk.http.auth.tunneling.disabledSchemes=""
or at runtime by setting the property in the main method of the application
System.setProperty("jdk.http.auth.tunneling.disabledSchemes", "");
What we have?
Client : win8, ie11, logged in into system using domain credentials.
Server : 3 tomcat7 nodes run beyond apache 2.2.22. application uses waffle library to authenticate windows users who are logged in into domain in sso manner.
Application uses spring security and the main thing regarding this topic is that filter that handles login via form comes before filter that handles authentication headers.
NegotiateSecurityFilterProvider supports only Negotiate protocol, not NTLM
What we do?
Go into application via direct link : https://app.domain.com/app_name/subordinates.do.
It is ok, we are bearing valid kerberos header (it is good and big one kerberos token which fiddler describes as 'Authorization Header (Negotiate) appears to contain a kerberos ticket' ^^) and waffle on application side passes us inside with kerberos reply.
Logout.
Login via form on login page: we make post request with user_name and password, again we bearing same kerberos token. Application uses user_name and password to login us with help of waffle WindowsAuthenticationProvider. Here we get authenticate before we rich NegotiateSecurityFilter, so there is no any kerberos header within reply from server. Anyway everything is ok.
Now we are log into MS account via OS. And magic happens.
When trying to login via direct link we get 'The handle specified is invalid' error on login page as SPRING_SECURITY_LAST_EXCEPTION constant. my guess here is that we send some kind of invalid authorization header
And when trying to login via form we get 'The parameter is incorrect'. here i think we send ntlm type 1 POST request with empty body but we still have invalid header so application does not recognize it and does not sent 401 reply and thereafter waffle sends null name to AD and here error comes (just guess)
BUT when I turn fiddler on to see what is really happened then everything begins to work fine as before login into MS account. Ok, to figure out what header are sent to the server I used some code inside cmd file:
UDPATED add code and output
var cookieContainer = new CookieContainer();
var authRequest = (HttpWebRequest) WebRequest.Create("https://app.domain.com/app_name/home.do");
var credentials = CredentialCache.DefaultNetworkCredentials;
authRequest.Credentials = credentials;
authRequest.CookieContainer = cookieContainer;
authRequest.AllowAutoRedirect = false;
var authResponse = (HttpWebResponse)authRequest.GetResponse();
Console.WriteLine("Request headers:");
foreach (string header in authRequest.Headers.AllKeys) {
Console.WriteLine("\t{0}: {1}", header, authRequest.Headers.Get(header));
}
Console.WriteLine("\nResponse: {0} {1}", (int)authResponse.StatusCode, authResponse.StatusDescription);
Console.WriteLine("Response headers:");
foreach (string header in authResponse.Headers)
Console.WriteLine("\t{0}: {1}", header, authResponse.GetResponseHeader(header));
foreach (var cookie in cookieContainer.GetCookies(new Uri("https://app.domain.com/app_name/")))
Console.WriteLine("Received cookie: {0}", cookie);
Console.WriteLine("\nPress ENTER to exit");
Console.ReadLine();
Here what I get:
Request headers:
Authorization: Negotiate oTMwMaADCgEBoioEKE5UTE1TU1AAAQAAAJeCCOIAAAAAAAAAAAAAAAAAAAAABgOAJQAAAA8=
Host: {host}
Cookie: JSESSIONID={sessionId}
Response: 302 Found
Response headers:
Vary: Accept-Encoding
Content-Length: 0
Content-Type: text/ plain; charset=UTF-8
Date: Tue, 04 Feb 2014 11:44:15 GMT
Location: https://app.domain.com/app_name/login.do?error_code=1
Server: Apache/2.2.22 (Win32) mod_ssl/2.2.22 OpenSSL/0.9.8t mod_jk/1.2.37
Received cookie: JSESSIONID={sessionId}
It is definitely much smaller header than kerberos one what fiddler sees when authentication works.
So questions are:
1. Why does loging into MS account affect what headers are sent to server?
2. Why it begins to work when fiddler on?
3. What type of this header : Negotiate oTMwMaADCgEBoioEKE5UTE1TU1AAAQAAAJeCCOIAAAAAAAAAAAAAAAAAAAAABgOAJQAAAA8= and how should it be handled by server?
UPDATE 17 March 2014:
wireshark capture shows KRB5KDC_ERR_S_PRINCIPAL_UNKNOWN error after tgs request, server mentioned - machine name with apache.
After investigation with support team we found out that special user which is used to run tomcat servers on different nodes didn't have spn for domain name of machine with apache (it had spn for resource domain name but not for current machine). After spn was added problem disappeared.
After decoding oTMwMaADCgEBoioEKE5UTE1TU1AAAQAAAJeCCOIAAAAAAAAAAAAAAAAAAAAABgOAJQAAAA8= we can see that it contains NTLMSSP (a new version).
Check browsers configuration:
In Internet Explorer: webpage should be in "Local intranet" zone (in a zone in which user is logged automatically) and that IWA, Integrated Windows Authentication is enabled.
If that's not the case, please take a look in Wireshark for dns and kerberos packets.
Check DNS:
IE uses dns to resolve webserver address into principal name. CNAME address is resolved into A address. If not found, IE will not ask for Kerberos service ticket at all (and will fallback to NTLM).
Check SPNs:
When Active Directory can't find requested principal (or there or two, or more). Then IE falls back into NTLM.