I'm using the Spring WebClient from spring-boot-starter-webflux 2.1.3.RELEASE to check the anonymity level of proxy servers. After I made some requests with the WebClient to a custom node.js http server through some proxy servers, there are no proxy related HTTP headers in my requests. I'm missing e.g. x-forwarded-for, via, x-proxy-id.. just the remote-address is exposed.
As far as I understood it, the netty client connects to the proxy via tcp CONNECT for every proxy type (HTTP,SOCKS4/5) and this is the reason for the missing headers.
Question:
Is there a way to get classic proxy HTTP headers with this approach or is there another way to configure a proxy server with the WebClient?
My example configuration:
HttpClient httpClient = HttpClient.create()
.tcpConfiguration(tcpClient ->
tcpClient
.proxy(proxy -> {
proxy.type(ProxyProvider.Proxy.HTTP).address(new InetSocketAddress(ip, port));})
.option(ChannelOption.CONNECT_TIMEOUT_MILLIS, 10000)
.doOnConnected(connection ->
connection
.addHandlerLast(new ReadTimeoutHandler(10000))
.addHandlerLast(new WriteTimeoutHandler(10000))));
ReactorClientHttpConnector connector = new ReactorClientHttpConnector(httpClient);
WebClient build = WebClient.builder()
.baseUrl(baseUrl)
.clientConnector(connector)
.build();
Expected headers
{"user-agent":"ReactorNetty/0.8.4.RELEASE","host":"21X.8X.XX.145:8XX","accept":"*/*","x-proxy-id":"719306848","x-forwarded-for":"21X.8X.XX.145","via":"1.1 101.XX.8X.11X (Mikrotik HttpProxy)"}
Actual headers
{"user-agent":"ReactorNetty/0.8.4.RELEASE","host":"21X.8X.XX.145:8XX","accept":"*/*"}
The reason for the missing headers is the CONNECT request, which establishes a TCP tunnel instead of sending HTTP requests to the proxy.
But unfortunately as stated in
https://github.com/netty/netty/issues/8269
and
https://github.com/spring-projects/spring-framework/issues/21767
the underlying netty client which is used by the webclient issuing a CONNECT request for every type of proxy and this will not be changed.
Related
I have a restemplate which can make call to multiple external systems over https.
I configured the resttemplate like so :
SSLContext sslContext = SSLContextBuilder
.create()
.loadTrustMaterial(key, keyPassword)
.build();
HttpClient client = HttpClients
.custom()
.setSSLContext(sslContext)
.setSSLHostnameVerifier(NoopHostnameVerifier.INSTANCE)
.build();
HttpComponentsClientHttpRequestFactory httpComponentsClientHttpRequestFactory = new HttpComponentsClientHttpRequestFactory(
client);
httpComponentsClientHttpRequestFactory.setConnectTimeout(5000);
httpComponentsClientHttpRequestFactory.setReadTimeout(30000);
return new RestTemplate(httpComponentsClientHttpRequestFactory);
I have setup 2 mock services on 2 separate VMs with ssl enabled and I am testing this restemplate by calling those services over https. And it works.
What I want to confirm is that configuring the restemplate as shown in the above code and importing the certificates of the different services to be called in by the client truststore should work without any additional configurations right?
I am confused because in the code in this example here : https://github.com/jonashackt/spring-boot-rest-clientcertificates-docker-compose the author has used 2 different factories i.e. serverTomClientHttpRequestFactory & serverAliceClientHttpRequestFactory, however mine works with a single restemplate. can someone please shed some light on this topic?
I am building a spring boot application that runs on #websphere-liberty profile. I am using a Spring webclient for outbound connections. All good except that I am not able to specify liberty SSL context to the netty httpclient used by Spring Webclient.
Any help on how how to specify pre-configured libery ssl context to Netty httpclient is greatly appreciated.
IBM documentation says that I should use SSLSocketFactory.getDefault() to get liberty SSL context. But I don't see any way to pass this to Netty HttpClient.
Ref: [https://www.ibm.com/support/knowledgecenter/en/SSEQTP_liberty/com.ibm.websphere.wlp.doc/ae/twlp_sec_ssl.html]
This is what I tried, but it is not pointing to the SSL context configured by #Websphere-liberty. Instead, it seems to create a new one. Because of that, the trust/keystores are not available to the netty HttpClient and I get an error saying that the peer is not trusted.
#Bean
public WebClient createWebClient() throws SSLException {
JdkSslContext jdkSslContext = new
JdkSslContext(SSLContext.getInstance("SSL"),true,
null, IdentityCipherSuiteFilter.INSTANCE,null,ClientAuth.NONE,
null,false);
ClientHttpConnector httpConnector = HttpClient.create().secure { t -> t.sslContext(jdkSslContext) }
return WebClient.builder().clientConnector(httpConnector).build();
}
I am using springboot webclient to call rest api from remote server. First request works fine. If I made subsequent request after sometime, the server throws 500 server error. The error I got is " onError(java.io.IOException: An existing connection was forcibly closed by the remote host)".
I want to test the behavior with disabling connection pool since I believe it uses the previous connection. Can you help me how to disable the connection pool when creating webclient?
TcpClient tcpClient = TcpClient.create()
.option(ChannelOption.CONNECT_TIMEOUT_MILLIS, 30000)
.option(ChannelOption.SO_KEEPALIVE, false)
.doOnConnected(connection ->
connection.addHandlerLast(new ReadTimeoutHandler(30))
.addHandlerLast(new WriteTimeoutHandler(30))
);
ReactorClientHttpConnector httpConnector = new ReactorClientHttpConnector(HttpClient.from(tcpClient));
final WebClient webClient = WebClient
.builder()
.clientConnector(httpConnector)
.baseUrl("http://customer.service.api.internal.cloud.qa.intranet.pagseguro.uol")
.exchangeStrategies(strategies)
.build()
You can disable the connection pooling using the code below
TcpClient tcpClient = TcpClient.newConnection()
Use wiretap(true) to inspect the traffic between the server and the client.
You can also enable the logging for Reactor Netty and trace whether the connection is reused (in the connection pool scenario)
logging.level.reactor.netty=debug
If the connection is reused you will see
2019-04-11 18:52:10.049 DEBUG 98105 --- [ctor-http-nio-5] r.n.resources.PooledConnectionProvider : [id: 0x897584fa, L:/<IP>:<PORT> - R:/<IP>:<PORT>] Channel acquired, now 1 active connections and 0 inactive connections
Also using the channel id id: 0x897584fa you can track what's happening with the channel.
I want to use Spring 5 Web client for REST connections.
WebClient.Builder builder = WebClient.builder().baseUrl(gatewayUrl);
ClientHttpConnector httpConnector = new ReactorClientHttpConnector(opt -> opt.sslContext(sslContext));
builder.clientConnector(httpConnector);
How I can implement handler for connection timeout?
I checked the code here but I can't find a solution.
I need to access a JAX-WS webservice protected by SSL using Jersey Client. I have successfully got this working with limited configuration and just letting the Client use the default HTTPURLConnection API.
This approach wont work however for my overall solution because it does not allow me the flexibility to change the credentials used when making a request to the WS. I'm trying to use DefaultHTTPClient instead and then passing it to the Client object on intialization.
NTCredentials credentials = new NTCredentials("username", "password",
computerName, domainName);
DefaultHttpClient httpClienttemp = new DefaultHttpClient();
DefaultHttpClient httpClient = wrapClient(httpClienttemp);
httpClient.getCredentialsProvider().setCredentials(AuthScope.ANY, credentials );
ClientConfig config = new DefaultClientConfig();
The wrapClient method creates an X509TrustManager and overrides the necessary methods so that all certificates are accepted. It also creates a SchemeRegistry entry for https access on port 443. This configuration results in a Connection refused exception.
The strange thing is, if i add an additional entry in the SchemeRegistry for http and give it a port of 443 then the request does get sent however a Connection Reset exception then gets thrown.
The Url i use to create the WebResource object is https however the SOAPAction i declare in the header uses http. Any ideas where im going wrong?
This is a limitation of the default HTTP Client (com.sun.jersey.api.client.Client) documented in the Jersey docs. You will have to use Apache HTTP Client to achieve this functionality.
Looks like someone already recommended doing this in the answer to your previous question: Jersey Client API - authentication.
EDIT: Corrected reference to the default Jersey HTTP Client to avoid confusion.