Proxy setting not working with Spring WebClient - spring-boot

Question

My following WebClient is working fine with internet connection but not through our proxy connection.
WebClient webClient = WebClient.builder()
.baseUrl("https://targetsite.com")
.build();
webClient.post()
.uri("/service/serviceName")
.body(BodyInserters.fromObject(reqData))
.retrieve()
.bodyToMono(WebServiceResponse.class)
Event though, the same client is working through proxy, if I set it as mentioned below,
HttpClient httpClient = HttpClient.create()
.tcpConfiguration(tcpClient -> tcpClient
.proxy(proxy -> proxy
.type(ProxyProvider.Proxy.HTTP)
.host("ourproxy.com")
.port(8080)));
ReactorClientHttpConnector connector = new ReactorClientHttpConnector(httpClient);
WebClient webClient = WebClient.builder()
.clientConnector(connector)
.baseUrl("https://targetsite.com")
.build();
webClient.post()
.uri("/service/serviceName")
.body(BodyInserters.fromObject(reqData))
.retrieve()
.bodyToMono(WebServiceResponse.class)
But if I set the same Proxy details either using System.setProperty("http.proxyHost","ourproxy.com");
System.setProperty("http.proxyPort","8080");
or
JVM run-time arguments -Dhttp.proxyHost=ourproxy.com -Dhttp.proxyPort=8080
WebClient webClient = WebClient.builder()
.baseUrl("https://targetsite.com")
.build();
System.setProperty("http.proxyHost", "ourproxy.com");
System.setProperty("http.proxyPort", "8080");
webClient.post()
.uri("/service/serviceName")
.body(BodyInserters.fromObject(reqData))
.retrieve()
.bodyToMono(WebServiceResponse.class)
The calls are getting failed with UnknownHostException like,
[04/11/2019 12:32:43.031 IST] DEBUG [reactor-http-epoll-3] [PooledConnectionProvider:254] - Creating new client pool [http] for targetsite.com:443
[04/11/2019 12:32:43.033 IST] DEBUG [reactor-http-epoll-3] [PooledConnectionProvider:254] - [id: 0xe4a0dc15] Created new pooled channel, now 0 active connections and 1 inactive connections
[04/11/2019 12:32:43.045 IST] DEBUG [reactor-http-epoll-3] [SslProvider:254] - [id: 0xe4a0dc15] SSL enabled using engine SSLEngineImpl and SNI targetsite.com:443
[04/11/2019 12:32:43.046 IST] DEBUG [reactor-http-epoll-3] [BootstrapHandlers:254] - [id: 0xe4a0dc15] Initialized pipeline DefaultChannelPipeline{(reactor.left.sslHandler = io.netty.handler.ssl.SslHandler), (reactor.left.sslReader = reactor.netty.tcp.SslProvider$SslReadHandler), (BootstrapHandlers$BootstrapInitializerHandler#0 = reactor.netty.channel.BootstrapHandlers$BootstrapInitializerHandler), (SimpleChannelPool$1#0 = io.netty.channel.pool.SimpleChannelPool$1), (reactor.left.httpCodec = io.netty.handler.codec.http.HttpClientCodec), (reactor.left.decompressor = io.netty.handler.codec.http.HttpContentDecompressor), (reactor.right.reactiveBridge = reactor.netty.channel.ChannelOperationsHandler)}
[04/11/2019 12:32:43.165 IST] ERROR [reactor-http-epoll-2] [AbstractErrorWebExceptionHandler:117] - [13ebf1eb] 500 Server Error for HTTP POST "/service/serviceName"
java.net.UnknownHostException: targetsite.com: Name or service not known
Please help, why my code is not functioning if I set the proxy details via JVM run-time arguments or system properties.
Actually I want to avoid code level proxy setting. So please guide me to correct my code or approach, so that I can use JVM run-time argument option.

Answer 1

Later when I raised this concern with reactor-netty team(https://github.com/reactor/reactor-netty/issues/887#issuecomment-549439355), it was confirmed by them that, system properties are not supported by reactor.netty.http.client.HttpClient. The only option is setting it through tcpConfiguration, which I mentioned in my question already; so I am closing this question with this remarks.
Thank you all

Answer 2

You need to set Proxy Host and Port while creating HttpClient instance per bellow.
HttpClient httpClient =
HttpClient.create()
.proxy(proxy -> proxy.type(ProxyProvider.Proxy.HTTP)
.host(sasConfig.getProxyHost())
.port(Integer.parseInt(sasConfig.getProxyPort())));
ReactorClientHttpConnector conn = new ReactorClientHttpConnector(httpClient);
WebClient webClient = WebClient.builder().clientConnector(conn).build();

Answer 3

I don't have enough reputation to comment on other answers so i have to start my own answer here.
Since reactor-netty v1.0.8, proxy can be configured from system properties.
Example:
WebClient
.builder()
.clientConnector(
new ReactorClientHttpConnector(
HttpClient.create().proxyWithSystemProperties()))
.build()

Answer 4

I had similar situation and struggled to find what in the end was a stupid mistake :
I enabled wiretap :
val httpClient = HttpClient.create()
.wiretap(
MyClient::class.java.toString(), LogLevel.DEBUG, AdvancedByteBufFormat.TEXTUAL)
webClient=WebClient.builder()
.clientConnector(ReactorClientHttpConnector(httpClient))
.codecs(consumerCodecForIncreasedPayloadSize)
.build()
I was then able to spot something strange in the logs :
[21119d8d] CONNECT: http://myProxy.my.company.com/<unresolved>:8080
what was that <unresolved> thing when issuing a CONNECT command to the proxy ?
then I finally realized : the https.proxyHost value I was setting at startup should not be http://myProxy.my.company.com , but simply myProxy.my.company.com, without the http://
then it worked.

Answer 5

Here is what worked for me
Step 1 : Define proxy environment variables
-Dhttp.proxyHost=<proxyHost>
-Dhttp.proxyPort=8080
-Dhttps.proxyHost=<proxyHost>
-Dhttps.proxyPort=8080
-Dhttps.nonProxyHosts=localhost
Configuration of proxy on webClient
#Configuration
public class WebClientConfiguration {
#Bean
public WebClient webClient() {
return WebClient.builder() //
.defaultHeader(ACCEPT, APPLICATION_JSON_VALUE) //
.clientConnector(new ReactorClientHttpConnector(httpClient())) //
.build();
}
private HttpClient httpClient() {
return HttpClient //
.create() //
.proxyWithSystemProperties();
}
}
Set the spring cloud proxy properties (In the application start)
static {
String nonProxyHosts = System.getProperty("http.nonProxyHosts");
if (nonProxyHosts != null) {
String regexProxyList = nonProxyHosts.replaceAll("\\.", "\\\\.").replaceAll("\\/", "\\\\/").replaceAll("\\*", ".\\*");
System.setProperty("spring.cloud.gateway.httpclient.proxy.non-proxy-hosts-pattern", regexProxyList);
}
String proxyHost = System.getProperty("https.proxyHost");
String proxyPort = System.getProperty("https.proxyPort");
if (proxyHost != null && proxyPort != null) {
System.setProperty("spring.cloud.gateway.httpclient.proxy.host", proxyHost);
System.setProperty("spring.cloud.gateway.httpclient.proxy.port", proxyPort);
}
}