Zuul routing the requests through an external proxy server - spring-boot

Our current project requirement is to to route some requests to third-party external api servers. For this we are using spring zuul based router service.
zuul:
routes
test:
path: /test/**
serviceId: test
url: http://my.api.server.com
test2:
path: /test2/**
serviceId: test2
url: http://my.api.server.com // but through an external proxy
Now the requirement is that for some endpoints, the requests to the external api server has be routed through some external proxy server, not owned by us,
How to do this via a curl is:
curl <external_api_server>/api/v1/user -k \
-x tntqyhnhjym.sandbox.verygoodproxy.com:8080 \
-H "Content-type: application/json" \
-d '{"card_number": "tok_sandbox_t8VSoovCuHA779eJGZhKvg", ... }'
-x <proxy> redirects the request through the given proxy.
How to do this via spring-zuul server?
There is one lead, I have gotten? https://github.com/spring-cloud/spring-cloud-netflix/issues/2320. Not clean, in the sense that I would need to extendSimpleHostRoutingFilter of zuul

Option 1 - Reverse Proxy Server ( that uses the proxy)
You could setup a reverse proxy - that is configured to go through the proxy. Your reverse proxy would be started with the parameters (either in e.g. java or nodejs) to use the external proxy. This reverse proxy would be a different process that would pass all requests through the proxy you want.
You could do it either through setting up a second zuul proxy application or through a nodejs reverse proxy (express or node-http-proxy).
Second zuul application (only for externals)
So if you used zuul, you would make a second application with the following:
test2:
path: /proxied-test2/**
serviceId: test2
url: http://my.api.server.com
You would then start this server on the same server with the parameters of your proxy and on a specific port (something like e.g. 9200) so e.g.
-Dhttp.proxyHost=localhost -Dhttp.proxyPort=8888
Original Application
In your original application, you would replace your route to now be the following.
zuul:
routes
test:
path: /test/**
serviceId: test
url: http://my.api.server.com
test2:
path: /test2/**
serviceId: test2
url: http://localhost:9200/proxied-test2/
Option 2: Use a scriptable http proxy server
You could setup a proxy server and then setup some exceptions and rules about which requests should be routed through the proxy and which requests should work without the proxy.
The second step is to configure your application to use the local proxy server specified in step 1. For this you can use the the following command-line parameters:
-Dhttp.proxyHost=localhost -Dhttp.proxyPort=8888
I have configured exclude lists for proxy servers in the past, but I never configured/scripted include lists. In your case, it would make more sense to have include lists so I would test scriptable/programmable proxy servers, e.g.:
https://mitmproxy.org/ - scriptable via mitmdump

I also spent hours looking for a solution for this issue. I found a way which is more flexible and easier than those described previously.
You can implement a custom ProxySelector and define different proxies for every route you want.
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import java.io.IOException;
import java.net.*;
import java.net.Proxy.Type;
import java.util.ArrayList;
import java.util.List;
public class CustomProxySelector extends ProxySelector {
ProxySelector defaultProxySelector = ProxySelector.getDefault();
ArrayList<Proxy> noProxy = new ArrayList<>();
ArrayList<Proxy> proxies = new ArrayList<>();
public CustomProxySelector(String proxyHost, int proxyPort) {
noProxy.add(Proxy.NO_PROXY);
proxies.add(new Proxy(Type.HTTP, new InetSocketAddress(proxyHost, proxyPort)));
}
//define your custom proxy selection here
#Override
public List<Proxy> select(URI uri) {
if (uri.getHost().matches("external.address.com")) {
return proxies;
}
if (defaultProxySelector != null) {
return defaultProxySelector.select(uri);
}
return noProxy;
}
#Override
public void connectFailed(URI arg0, SocketAddress arg1, IOException arg2) {
//error handling
}
}
Then add a bean of type CloseableHttpClient which overwrites the standard HttpClient of Zuul and configure it with your custom implementation of ProxySelector.
#Bean
#Primary
public CloseableHttpClient customHttpClientForZuulWithHttpProxyConfig(#Value("${your.proxy.host}") String proxyHost, #Value("${your.proxy.port}") int proxyPort) {
SystemDefaultRoutePlanner routePlanner = new SystemDefaultRoutePlanner(new CustomProxySelector(proxyHost, proxyPort));
return HttpClientBuilder.create().setRoutePlanner(routePlanner).build();
}
Please also take note that zuul is not longer supported by spring cloud. so the recommended way is to switch to spring gateway.

Related

Wiremock does not receive any requests

I am writing an integrationtest for a resource (Jax-rs with resteasy) which invokes a thirdparty request to the Trustpilot Api to get a review link for a product.
In my tests I wanted to mock this call by using Wiremock and invoke the request by using resteasy Dispatcher.
The code below shows my Setup (names etc are changed).
#RunWith(SpringJUnit4ClassRunner.class)
#ContextConfiguration(classes = { MyResourceTestContext.class })
public class MyResourceIntegrationTest {
#Rule
public WireMockRule wireMockRule = new WireMockRule(wireMockConfig().dynamicPort().dynamicHttpsPort());
#Autowired
private MyResource myResource;
private Dispatcher dispatcher;
private MockHttpResponse response;
#Before
public void setUp() throws Exception {
dispatcher = MockDispatcherFactory.createDispatcher();
dispatcher.getRegistry().addSingletonResource(myResource);
response = new MockHttpResponse();
}
#Test
public void testResourceShouldReturnRedirectToTrustpilotreview() throws Exception {
URI apiRequestURI = UriBuilder
.fromUri("https://api.trustpilot.com/v1/private/product-reviews/business-units"
+ "/{businessUnitId}}/invitation-links")
.build(BUSINESS_UNIT_ID);
URI expectedRedirectURI = UriBuilder.fromUri("https://products.trustpilot.com/#evaluate/{reviewLinkId}")
.build(REVIEW_LINK_ID);
stubFor(post(apiRequestURI.toString())
.willReturn(okJson("{\"reviewLinkId\":\"" + REVIEW_LINK_ID + "\","
+ "\"reviewUrl\":\"" + expectedRedirectURI.toString() + "\"}")));
MockHttpRequest request = MockHttpRequest.get("/myResource");
dispatcher.invoke(request, response);
WireMock.verify(1, postRequestedFor(urlEqualTo(apiRequestURI.toString())));
Assert.assertEquals(HttpStatus.TEMPORARY_REDIRECT.value(), response.getStatus());
Assert.assertEquals(expectedRedirectURI.toString(), response.getOutputHeaders().getFirst("Location"));
}
But what I'm getting is (with my actual bid in the url):
com.github.tomakehurst.wiremock.client.VerificationException: Expected at least one request matching: {
"url" : "https://api.trustpilot.com/v1/private/product-reviews/business-units/<bid>/invitation-links",
"method" : "POST"
}
Requests received: [ ]
Stepping into the request with the Debugger I can say this request and other requests are actually executed successfully. But they are not recognized/mocked by Wiremock.
Is there something wrong with my Wiremock setup? Any Ideas? Thank you!
There are a couple of reasons why this isn't sending anything to WireMock at the moment:
WireMock only accepts relative URLs when stubbing so passing the full URL including protocol and domain won't work. Try something equivalent to stubFor(post(urlPathEqualTo("/v1/private/product-reviews/business-units/<bid>/invitation-links") instead.
There doesn't seem to be any way for your class under test to know which host/port WireMock is running on and no forward proxying set up so the code is presumably still trying to call the real Trustpilot API.
To solve 2. you have a couple of options:
Set up your class/code under test to use localhost as the destination host and the result of wireMockRule.port() as the port number, or wireMockRule.httpsPort() if you're on HTTPS.
Use the new auto JVM proxy setup feature to proxy your app's calls through WireMock. This assumes you're on WireMock 2.31.0+ and that your app's HTTP client respects the JVM proxy system properties.

Wrong "Generated server url" in springdoc-openapi-ui (Swagger UI) deployed behind proxy

Spring Boot 2.2 application with springdoc-openapi-ui (Swagger UI) runs HTTP port.
The application is deployed to Kubernetes with Ingress routing HTTPS requests from outside the cluster to the service.
In this case Swagger UI available at https://example.com/api/swagger-ui.html has wrong "Generated server url" - http://example.com/api. While it should be https://example.com/api.
While Swagger UI is accessed by HTTPS, the generated server URL still uses HTTP.
I had same problem. Below worked for me.
#OpenAPIDefinition(
servers = {
#Server(url = "/", description = "Default Server URL")
}
)
#SpringBootApplication
public class App {
// ...
}
If the accepted solution doesn't work for you then you can always set the url manually by defining a bean.
#Bean
public OpenAPI customOpenAPI() {
Server server = new Server();
server.setUrl("https://example.com/api");
return new OpenAPI().servers(List.of(server));
}
And the url can be defined via a property and injected here.
springdoc-openapi FAQ has a section How can I deploy the Doploy springdoc-openapi-ui, behind a reverse proxy?.
The FAQ section can be extended.
Make sure X-Forwarded headers are sent by your proxy (X-Forwarded-For, X-Forwarded-Proto and others).
If you are using Undertow (spring-boot-starter-undertow), set property server.forward-headers-strategy=NATIVE to make a Web server natively handle X-Forwarded headers. Also, consider switching to Undertow if you are not using it.
If you are using Tomcat (spring-boot-starter-tomcat), set property server.forward-headers-strategy=NATIVE and make sure to list IP addresses of all internal proxies to trust in the property server.tomcat.internal-proxies=192\\.168\\.\\d{1,3}\\.\\d{1,3}. By default, IP addresses in 10/8, 192.168/16, 169.254/16 and 127/8 are trusted.
Alternatively, for Tomcat set property server.forward-headers-strategy=FRAMEWORK.
Useful links:
Running Behind a Front-end Proxy Server
Customize Tomcat’s Proxy Configuration
ServerProperties.ForwardHeadersStrategy
In case you have non-default context path
#Configuration
public class SwaggerConfig {
#Bean
public OpenAPI openAPI(ServletContext servletContext) {
Server server = new Server().url(servletContext.getContextPath());
return new OpenAPI()
.servers(List.of(server))
// ...
}
}
Below worked for me.
#OpenAPIDefinition(servers = {#server(url = "/", description = "Default Server URL")})
#SpringBootApplication
class App{
// ...
}
or
#OpenAPIDefinition(servers = {#server(url = "/", description = "Default Server URL")})
#Configuration
public class OpenAPIConfig {
#Bean
public OpenAPI customOpenAPI() {
return new OpenAPI()
.info(new Info().title("App name")
.termsOfService("http://swagger.io/terms/")
.license(new License().name("Apache 2.0").url("http://springdoc.org")));
}
}
Generated server url is HHTP - issue

Create a Custom Spring Cloud Netflix Ribbon Client

I am using Spring Cloud Netflix Ribbon in combination with Eureka in a Cloud Foundry environment.
The use case I am trying to implement is the following:
I have a running CF application named address-service with several instances spawned.
The instances are registering to Eureka by the service name address-service
I have added custom metadata to service instances using
eureka.instance.metadata-map.applicationId: ${vcap.application.application_id}
I want to use the information in Eureka's InstanceInfo (in particular the metadata and how many service instances are available) for setting a CF HTTP header "X-CF-APP-INSTANCE" as described here.
The idea is to send a Header like "X-CF-APP-INSTANCE":"appIdFromMetadata:instanceIndexCalculatedFromNoOfServiceInstances" and thus "overrule" CF's Go-Router when it comes to load balancing as described at the bottom of this issue.
I believe to set headers, I need to create a custom RibbonClient implementation - i.e. in plain Netflix terms a subclass of AbstractLoadBalancerAwareClient as described here - and override the execute() methods.
However, this does not work, as Spring Cloud Netflix Ribbon won't read the class name of my CustomRibbonClient from application.yml. It also seems Spring Cloud Netflix wraps quite a bit of classes around the plain Netflix stuff.
I tried implementing a subclass of RetryableRibbonLoadBalancingHttpClient and RibbonLoadBalancingHttpClient which are Spring classes. I tried giving their class names in application.yml using ribbon.ClientClassName but that does not work. I tried to override beans defined in Spring Cloud's HttpClientRibbonConfiguration but I cannot get it to work.
So I have two questions:
is my assumption correct that I need to create a custom Ribbon Client and that the beans defined here and here won't do the trick?
How to do it properly?
Any ideas are greatly appreciated, so thanks in advance!
Update-1
I have dug into this some more and found RibbonAutoConfiguration.
This creates a SpringClientFactory which provides a getClient() method that is only used in RibbonClientHttpRequestFactory (also declared in RibbonAutoConfiguration).
Unfortunately, RibbonClientHttpRequestFactory hard-codes the client to Netflix RestClient. And it does not seem possible to override either SpringClientFactory nor RibbonClientHttpRequestFactory beans.
I wonder if this is possible at all.
Ok, I'll answer this question myself, in case someone else may need that in the future.
Actually, I finally managed to implement it.
TLDR - the solution is here: https://github.com/TheFonz2017/Spring-Cloud-Netflix-Ribbon-CF-Routing
The solution:
Allows to use Ribbon on Cloud Foundry, overriding Go-Router's load balancing.
Adds a custom routing header to Ribbon load balancing requests (including retries) to instruct CF's Go-Router to route requests to the service instance selected by Ribbon (rather than by its own load balancer).
Shows how to intercept load balancing requests
The key to understanding this, is that Spring Cloud has its own LoadBalancer framework, for which Ribbon is just one possible implementation. It is also important to understand, that Ribbon is only used as a load balancer not as an HTTP client. In other words, Ribbon's ILoadBalancer instance is only used to select the service instance from the server list. Requests to the selected server instances are done by an implementation of Spring Cloud's AbstractLoadBalancingClient. When using Ribbon, these are sub-classes of RibbonLoadBalancingHttpClient and RetryableRibbonLoadBalancingHttpClient.
So, my initial approach to add an HTTP header to the requests sent by Ribbon's HTTP client did not succeed, since Ribbon's HTTP / Rest client is actually not used by Spring Cloud at all.
The solution is to implement a Spring Cloud LoadBalancerRequestTransformer which (contrary to its name) is a request interceptor.
My solution uses the following implementation:
public class CFLoadBalancerRequestTransformer implements LoadBalancerRequestTransformer {
public static final String CF_APP_GUID = "cfAppGuid";
public static final String CF_INSTANCE_INDEX = "cfInstanceIndex";
public static final String ROUTING_HEADER = "X-CF-APP-INSTANCE";
#Override
public HttpRequest transformRequest(HttpRequest request, ServiceInstance instance) {
System.out.println("Transforming Request from LoadBalancer Ribbon).");
// First: Get the service instance information from the lower Ribbon layer.
// This will include the actual service instance information as returned by Eureka.
RibbonLoadBalancerClient.RibbonServer serviceInstanceFromRibbonLoadBalancer = (RibbonLoadBalancerClient.RibbonServer) instance;
// Second: Get the the service instance from Eureka, which is encapsulated inside the Ribbon service instance wrapper.
DiscoveryEnabledServer serviceInstanceFromEurekaClient = (DiscoveryEnabledServer) serviceInstanceFromRibbonLoadBalancer.getServer();
// Finally: Get access to all the cool information that Eureka provides about the service instance (including metadata and much more).
// All of this is available for transforming the request now, if necessary.
InstanceInfo instanceInfo = serviceInstanceFromEurekaClient.getInstanceInfo();
// If it's only the instance metadata you are interested in, you can also get it without explicitly down-casting as shown above.
Map<String, String> metadata = instance.getMetadata();
System.out.println("Instance: " + instance);
dumpServiceInstanceInformation(metadata, instanceInfo);
if (metadata.containsKey(CF_APP_GUID) && metadata.containsKey(CF_INSTANCE_INDEX)) {
final String headerValue = String.format("%s:%s", metadata.get(CF_APP_GUID), metadata.get(CF_INSTANCE_INDEX));
System.out.println("Returning Request with Special Routing Header");
System.out.println("Header Value: " + headerValue);
// request.getHeaders might be immutable, so we return a wrapper that pretends to be the original request.
// and that injects an extra header.
return new CFLoadBalancerHttpRequestWrapper(request, headerValue);
}
return request;
}
/**
* Dumps metadata and InstanceInfo as JSON objects on the console.
* #param metadata the metadata (directly) retrieved from 'ServiceInstance'
* #param instanceInfo the instance info received from the (downcast) 'DiscoveryEnabledServer'
*/
private void dumpServiceInstanceInformation(Map<String, String> metadata, InstanceInfo instanceInfo) {
ObjectMapper mapper = new ObjectMapper();
String json;
try {
json = mapper.writerWithDefaultPrettyPrinter().writeValueAsString(metadata);
System.err.println("-- Metadata: " );
System.err.println(json);
json = mapper.writerWithDefaultPrettyPrinter().writeValueAsString(instanceInfo);
System.err.println("-- InstanceInfo: " );
System.err.println(json);
} catch (JsonProcessingException e) {
System.err.println(e);
}
}
/**
* Wrapper class for an HttpRequest which may only return an
* immutable list of headers. The wrapper immitates the original
* request and will return the original headers including a custom one
* added when getHeaders() is called.
*/
private class CFLoadBalancerHttpRequestWrapper implements HttpRequest {
private HttpRequest request;
private String headerValue;
CFLoadBalancerHttpRequestWrapper(HttpRequest request, String headerValue) {
this.request = request;
this.headerValue = headerValue;
}
#Override
public HttpHeaders getHeaders() {
HttpHeaders headers = new HttpHeaders();
headers.putAll(request.getHeaders());
headers.add(ROUTING_HEADER, headerValue);
return headers;
}
#Override
public String getMethodValue() {
return request.getMethodValue();
}
#Override
public URI getURI() {
return request.getURI();
}
}
}
The class is looking for the information required for setting the CF App Instance Routing header in the service instance metadata returned by Eureka.
That information is
The GUID of the CF application that implements the service and of which several instances exist for load balancing.
The index of the service / application instance that the request should be routed to.
You need to provide that in the application.yml of your service like so:
eureka:
instance:
hostname: ${vcap.application.uris[0]:localhost}
metadata-map:
# Adding information about the application GUID and app instance index to
# each instance metadata. This will be used for setting the X-CF-APP-INSTANCE header
# to instruct Go-Router where to route.
cfAppGuid: ${vcap.application.application_id}
cfInstanceIndex: ${INSTANCE_INDEX}
client:
serviceUrl:
defaultZone: https://eureka-server.<your cf domain>/eureka
Finally, you need to register the LoadBalancerRequestTransformer implementation in the Spring configuration of your service consumers (which use Ribbon under the hood):
#Bean
public LoadBalancerRequestTransformer customRequestTransformer() {
return new CFLoadBalancerRequestTransformer();
}
As a result, if you use a #LoadBalanced RestTemplate in your service consumer, the template will call Ribbon to make a choice on the service instance to send the request to, will send the request and the interceptor will inject the routing header. Go-Router will route the request to the exact instance that was specified in the routing header and not perform any additional load balancing that would interfere with Ribbon's choice.
In case a retry were necessary (against the same or one or more next instances), the interceptor would again inject the according routing header - this time for a potentially different service instance selected by Ribbon.
This allows you to use Ribbon effectively as the load balancer and de-facto disable load balancing of Go-Router, demoting it to a mere proxy. The benefit being that Ribbon is something you can influence (programmatically) whereas you have little to no influence over Go-Router.
Note: this was tested for #LoadBalanced RestTemplate's and works.
However, for #FeignClients it does not work this way.
The closest I have come to solving this for Feign is described in this post, however, the solution described there uses an interceptor that does not get access to the (Ribbon-)selected service instance, thus not allowing access to the required metadata.
Haven't found a solution so far for FeignClient.

Get the hostname/IP:port of the server to which the request is to be forwarded from Zuul+ribbon

I'm using Eureka for service discovery and Zuul+ribbon as reverse proxy and load balancer.
I have 2 instances registered under Eureka as follows:
MYSERVICE n/a (2) (2) UP (2) - MYHOST:MyService:8888 , MYHOST:MyService:9999
Below is my zuul config:
#EnableZuulProxy
#EnableDiscoveryClient
zuul:
debug.request: true
sensitiveHeaders:
routes:
ecm:
path: /myservice/**
serviceId: MYSERVICE
stripPrefix: false
host:
maxTotalConnections: 200
maxPerRouteConnections: 30
RibbonRoutingFilter:
route.disable: false
I want a filter or interceptor which would help me log my request URL, my request params and the server chosen by Zuul.
I tried extending the following:
#Component
public class RibbonInterceptor extends ZoneAvoidanceRule {
#Override
public Server choose(Object key) {
Server choose = super.choose(key);
System.out.println(choose);
return choose;
}
But, this just gave me the server info from Ribbon, and here ribbon was just choosing a server. I wanted this info from Zuul along with the request details.
Please help!!
For the request URL and the server chosen by Zuul, you can set the loglevel of the LoadBalancerContext to DEBUG in application.properties.
#logging load balancing information
logging.level.com.netflix.loadbalancer.LoadBalancerContext=DEBUG
This will create a log statement like:
2017-09-11T12:59:09.746-07:00: [DEBUG] hystrix-myserviceV3-2 com.netflix.loadbalancer.LoadBalancerContext - myserviceV3 using LB returned Server: myservice-2.abc.com:8080 for request http:///myservice/auth/users
Not sure though, how you can handle the request params.
Assuming you use Apache HttpClient, there are many ways to do this but I think the most simple is to add an HttpRequestInterceptor to the CloseableHttpClient used by Ribbon. You can customize the client by providing a bean of type CloseableHttpClient as mentioned in the documentation [1]. You then have the request actually used by HttpClient so you log the details.
#Bean
public HttpClient delegate(IClientConfig clientConfig)
{
HttpClientBuilder builder = HttpClientBuilder.create();
//set connection pool configuration
HttpRequestInterceptor interceptor = (request, context) -> logger.info("Server : {}, headers : {}", request.getRequestLine().getUri(), request.getAllHeaders());
builder.addInterceptorFirst(interceptor);
return builder.build();
}
You could also extend HttpClientRibbonCommand and override the run() method to print what you want. You would use your new class by providing a bean of type RibbonCommandFactory<YourExtendedRibbonCommand> and it should wire automatically to the RibbonRoutingFilter.
Finally, if you're using the semaphore isolation strategy in hystrix, you could use your RibbonInterceptor like you indeed in addition to com.netflix.zuul.context.RequestContext. In the RequestContext, you'll find the original HttpServletRequest along with parsed parameters and headers that were processed in the pre filters.
[1] https://cloud.spring.io/spring-cloud-netflix/single/spring-cloud-netflix.html#_zuul_http_client

Spring Boot - Integration Testing with HTTPS

I have a Spring Boot app that normally runs on tomcat (as a normal WAR), and for some URL patterns I have Spring-Security setup to force it to be HTTPS. All the HTTPS/certificates stuff is done outside of my application, so all this step (at least appears) to be doing is redirecting to the HTTPS address (and port).
My security configuration:
http.requiresChannel().requestMatchers( "/**" ) .requiresSecure()
It all works as expected on tomcat, and I get redirected correctly.
Now I want to write some Spring #IntegrationTests, but am not able to get it to make requests to the HTTPS URL - I suspect I am missing something fairly obvious, but not having any joy with my attempts.
My basic test case:
#RunWith( SpringJUnit4ClassRunner )
#SpringApplicationConfiguration( classes = Application )
#WebAppConfiguration
#IntegrationTest( [ "server.port:0" ] )
class MultipleUserAuthenticationTest {
#Value('${local.server.port}') private int port
private URL base
private RestTemplate template
#Before public void setUp() throws Exception {
this.base = new URL("https://localhost:" + port + "/api")
template = new RestTemplate()
}
#Test public void apiRequest() throws Exception {
HttpHeaders headers = //set custom headers
HttpEntity<String> entity = new HttpEntity<String>( headers )
ResponseEntity<String> exchange = template.exchange( base.toString(), HttpMethod.GET, entity, String )
}
I have tried the approach outlined here https://stackoverflow.com/a/34411279/258813 and https://stackoverflow.com/a/24491820/258813
But no joy - It still seems to be failing because the port it is attempting to access is the HTTP port, so I basically get:
org.springframework.web.client.ResourceAccessException: I/O error on GET request for "https://localhost:52002/api":Unrecognized SSL message, plaintext connection?; nested exception is javax.net.ssl.SSLException: Unrecognized SSL message, plaintext connection?
Am I missing something obvious? Is there another way to get the HTTPS port (or tell spring to use one)?

Resources