AntPathRequestMatcher is not matching string with wildcards - spring

I have a SpringBoot application and want to detect the liveness endpoint being invoked so I can allow the request to be unauthenticated.
When I make this request:
http://localhost:8080/001/MyApp/actuator/health/liveness
The code below prints false.
Shouldn't new AntPathRequestMatcher("/**liveness") cause isWhitelisted to return true?
public RequestMatcher getRequestMatcher() {
return new OrRequestMatcher(
new AntPathRequestMatcher("/error"),
new AntPathRequestMatcher("/login**"),
new AntPathRequestMatcher("/**logout"),
new AntPathRequestMatcher("/**liveness"));
}
public boolean isWhitelisted(HttpServletRequest httpServletRequest) {
return getRequestMatcher().matches(httpServletRequest);
}
System.out.println(requestPropertyResolver.isWhitelisted(httpServletRequest));

Needed this instead:
return new OrRequestMatcher(
new AntPathRequestMatcher("/error"),
new AntPathRequestMatcher("/login**"),
new AntPathRequestMatcher("/**logout"),
new AntPathRequestMatcher("/**/liveness"));
/**/ - matches zero or more 'directories' in a path, so needed to end the directory tree

Related

Camel Routes - How to return the body response as xml

First of all, I am new with Spring Boot.
I am not sure if it is possible, but I would like to return the xml response from the external url.
I have this code:
#GetMapping("/myPage")
public void myPage() {
restConfiguration().host("localhost").port(8080);
from("timer://runOnce?repeatCount=1&delay=0")
.to("rest:get:/external-page")
.to("stream:out");
}
myPage() is returning a XML (that's OK). So, now I would like to return the same XML when I do:
curl http://localhost/myPage
I am not sure if I have to use .to("stream:out"), but the curl is returning an empty result.
Can someone help me?
Thanks in advance.
I found the solution, this is how to get the response.
CamelContext context = new DefaultCamelContext();
context.addRoutes(new RouteBuilder() {
public void configure() {
restConfiguration().host(sHost).port(iPort);
from("direct:start")
.setHeader(Exchange.HTTP_METHOD,simple("GET"))
.to("rest:get:/external-page");
}
});
context.start();
ProducerTemplate template = context.createProducerTemplate();
String headerValue = "application/xml";
Map<String, Object> headers = new HashMap<String,Object>();
headers.put("Content-Type", headerValue);
Object result = template.requestBodyAndHeaders("direct:start", null, headers, String.class);
Exchange exchange = new DefaultExchange(context);
String response = ExchangeHelper.convertToType(exchange, String.class, result);
context.stop();
return response;

How to read/modify form data that goes through Spring Cloud Gateway?

I am trying to validate and log form data that goes through Spring Cloud Gateway. I have tried a few methods and encounter a few problems and I could not read it properly. I have tried:
#Component
public class GatewayRequestFilter {
#Bean
public GlobalFilter apply() {
return (exchange, chain) -> {
MediaType contentType = exchange.getRequest().getHeaders().getContentType();
ModifyRequestBodyGatewayFilterFactory.Config modifyRequestConfig = new ModifyRequestBodyGatewayFilterFactory.Config();
/// Method 1
if (contentType.includes(MediaType.MULTIPART_FORM_DATA)) {
modifyRequestConfig.setRewriteFunction(String.class, String.class, (exchange1, originalRequestBody) -> {
validateAndAuditLog(exchange1, originalRequestBody);
return Mono.just(originalRequestBody);
});
}
/// Method 2
if (contentType.includes(MediaType.MULTIPART_FORM_DATA)) {
return exchange.getMultipartData().flatMap(originalRequestBody -> {
validateAndAuditLog(exchange1, originalRequestBody);
return chain.filter(exchange);
});
}
/// Method 3:
/// https://github.com/spring-cloud/spring-cloud-gateway/issues/1307#issuecomment-553910834
return new ModifyRequestBodyGatewayFilterFactory().apply(modifyRequestConfig).filter(exchange, chain);
};
}
}
For the 1st and 3rd method, if I set inClass as String.class then I can see data in some kind of http format. The problem is that I don't know how to parse it into hashMap or LinkedMultiValueMap to access each of value using key. Here is the output I get:
----------------------------162653831591335516327921
Content-Disposition: form-data; name="simple-text"
text
----------------------------162653831591335516327921
Content-Disposition: form-data; name="simple-file"; filename="simple-file"
Content-Type: application/octet-stream
Simple file
----------------------------162653831591335516327921--
If I change inClass as Object.class then there is another error:
{
"timestamp": "2020-04-03T02:37:57.096+0000",
"path": "/tc/test/test",
"status": 500,
"error": "Internal Server Error",
"message": "Content type 'multipart/form-data;boundary=--------------------------537619313111072161580699' not supported for bodyType=java.lang.Object",
"requestId": "0592497a-1"
}
For the 2nd method I can get data in LinkedMultiValueMap which is good because I can read each data using key value and I can also get uploaded files name, but the problem is that, it hang for 10s before pass the request to down stream.
Anyone has any idea what should I do to read or modify form data that goes through Spring Cloud Gateway?
Rewriting the answer with example.
Basic approach is defined here, though it needs lot of refinement to work for multi-part.
https://developpaper.com/question/how-to-modify-the-request-parameters-of-multipart-form-data-format-in-spring-cloud-gateway/
For any approach to work once you read the data, you need to set a modified request object to exchange downstream to be processed again. Setting the new multi-part object downstream is bit tricky because there is not a straightforward way to convert string->multi-part->string.
Here is a sample code based on the approach. Note that this for now works only if multi-part contains form fields and not file type fields, because in later case we are dealing with a stream, which can be embedded anywhere within the entire multi-part request, and it is not possible to modify such request without blocking calls, which the netty does not allow.
private final List<HttpMessageReader<?>> messageReaders = HandlerStrategies.withDefaults().messageReaders();
public GatewayFilter apply(Config config) {
return new OrderedGatewayFilter((exchange, chain) -> {
ServerRequest serverRequest = ServerRequest.create(exchange, messageReaders);
// get modified body from original body o
Mono<MultiValueMap<String, String>> modifiedBody = serverRequest.bodyToMono(String.class).flatMap(o -> {
// create mock request to read body
SynchronossPartHttpMessageReader synchronossReader = new SynchronossPartHttpMessageReader();
MultipartHttpMessageReader reader = new MultipartHttpMessageReader(synchronossReader);
MockServerHttpRequest request = MockServerHttpRequest.post("").contentType(exchange.getRequest().getHeaders().getContentType()).body(o);
Mono<MultiValueMap<String, Part>> monoRequestParts = reader.readMono(MULTIPART_DATA_TYPE, request, Collections.emptyMap());
// modify parts
return monoRequestParts.flatMap(requestParts -> {
Map<String, List<String>> modifedBodyArray = requestParts.entrySet().stream().map(entry -> {
String key = entry.getKey();
LOGGER.info(key);
List<String> entries = entry.getValue().stream().map(part -> {
LOGGER.info("{}", part);
// read the input part
String input = ((FormFieldPart) part).value();
// return the modified input part
return new String(modifyRequest(config, exchange, key, input));
}).collect(Collectors.toList());
return new Map.Entry<String, List<String>>() {
#Override
public String getKey() {
return key;
}
#Override
public List<String> getValue() {
return entries;
}
#Override
public List<String> setValue(List<String> param1) {
return param1;
}
};
}).collect(Collectors.toMap(k -> k.getKey(), k -> k.getValue()));
return Mono.just(new LinkedMultiValueMap<String, String>(modifedBodyArray));
});
});
// insert the new modified body
BodyInserter bodyInserter = BodyInserters.fromPublisher(modifiedBody, new ParameterizedTypeReference<MultiValueMap<String, String>>() {});
HttpHeaders headers = new HttpHeaders();
headers.putAll(exchange.getRequest().getHeaders());
// the new content type will be computed by bodyInserter
// and then set in the request decorator
headers.remove(HttpHeaders.CONTENT_LENGTH);
CachedBodyOutputMessage outputMessage = new CachedBodyOutputMessage(exchange, headers);
return bodyInserter.insert(outputMessage, new BodyInserterContext())
.then(Mono.defer(() -> {
ServerHttpRequest decorator = decorate(exchange, headers, outputMessage);
return chain.filter(exchange.mutate().request(decorator).build());
}));
}, RouteToRequestUrlFilter.ROUTE_TO_URL_FILTER_ORDER + 1);
}
// some of the helper methods
private String modifyRequest(Config config, ServerWebExchange exchange, String key, String input) {
// do your thing in here !!!
return input;
}
private ServerHttpRequestDecorator decorate(ServerWebExchange exchange, HttpHeaders headers, CachedBodyOutputMessage outputMessage) {
return new ServerHttpRequestDecorator(exchange.getRequest()) {
#Override
public HttpHeaders getHeaders() {
long contentLength = headers.getContentLength();
HttpHeaders httpHeaders = new HttpHeaders();
httpHeaders.putAll(headers);
if (contentLength > 0) {
httpHeaders.setContentLength(contentLength);
} else {
// TODO: this causes a 'HTTP/1.1 411 Length Required' // on httpbin.org
httpHeaders.set(HttpHeaders.TRANSFER_ENCODING, "chunked");
}
return httpHeaders;
}
#Override
public Flux<DataBuffer> getBody() {
return outputMessage.getBody();
}
};
}

Streaming upload via #Bean-provided RestTemplateBuilder buffers full file

I'm building a reverse-proxy for uploading large files (multiple gigabytes), and therefore want to use a streaming model that does not buffer entire files. Large buffers would introduce latency and, more importantly, they could result in out-of-memory errors.
My client class contains
#Autowired private RestTemplate restTemplate;
#Bean
public RestTemplate restTemplate(RestTemplateBuilder restTemplateBuilder) {
int REST_TEMPLATE_MODE = 1; // 1=streams, 2=streams, 3=buffers
return
REST_TEMPLATE_MODE == 1 ? new RestTemplate() :
REST_TEMPLATE_MODE == 2 ? (new RestTemplateBuilder()).build() :
REST_TEMPLATE_MODE == 3 ? restTemplateBuilder.build() : null;
}
and
public void upload_via_streaming(InputStream inputStream, String originalname) {
SimpleClientHttpRequestFactory requestFactory = new SimpleClientHttpRequestFactory();
requestFactory.setBufferRequestBody(false);
restTemplate.setRequestFactory(requestFactory);
InputStreamResource inputStreamResource = new InputStreamResource(inputStream) {
#Override public String getFilename() { return originalname; }
#Override public long contentLength() { return -1; }
};
MultiValueMap<String, Object> body = new LinkedMultiValueMap<String, Object>();
body.add("myfile", inputStreamResource);
HttpHeaders headers = new HttpHeaders();
headers.setContentType(MediaType.MULTIPART_FORM_DATA);
HttpEntity<MultiValueMap<String, Object>> requestEntity = new HttpEntity<>(body,headers);
String response = restTemplate.postForObject(UPLOAD_URL, requestEntity, String.class);
System.out.println("response: "+response);
}
This is working, but notice my REST_TEMPLATE_MODE value controls whether or not it meets my streaming requirement.
Question: Why does REST_TEMPLATE_MODE == 3 result in full-file buffering?
References:
How to forward large files with RestTemplate?
How to send Multipart form data with restTemplate Spring-mvc
Spring - How to stream large multipart file uploads to database without storing on local file system -- establishing the InputStream
How to autowire RestTemplate using annotations
Design notes and usage caveats, also: restTemplate does not support streaming downloads
In short, the instance of RestTemplateBuilder provided as an #Bean by Spring Boot includes an interceptor (filter) associated with actuator/metrics -- and the interceptor interface requires buffering of the request body into a simple byte[].
If you instantiate your own RestTemplateBuilder or RestTemplate from scratch, it won't include this by default.
I seem to be the only person visiting this post, but just in case it helps someone before I get around to posting a complete solution, I've found a big clue:
restTemplate.getInterceptors().forEach(item->System.out.println(item));
displays...
org.SF.boot.actuate.metrics.web.client.MetricsClientHttpRequestInterceptor
If I clear the interceptor list via setInterceptors, it solves the problem. Furthermore, I found that any interceptor, even if it only performs a NOP, will introduce full-file buffering.
public class SimpleClientHttpRequestFactory { ...
I have explicitly set bufferRequestBody = false, but apparently this code is bypassed if interceptors are used. This would have been nice to know earlier...
#Override
public ClientHttpRequest createRequest(URI uri, HttpMethod httpMethod) throws IOException {
HttpURLConnection connection = openConnection(uri.toURL(), this.proxy);
prepareConnection(connection, httpMethod.name());
if (this.bufferRequestBody) {
return new SimpleBufferingClientHttpRequest(connection, this.outputStreaming);
}
else {
return new SimpleStreamingClientHttpRequest(connection, this.chunkSize, this.outputStreaming);
}
}
public abstract class InterceptingHttpAccessor extends HttpAccessor { ...
This shows that the InterceptingClientHttpRequestFactory is used if the list of interceptors is not empty.
/**
* Overridden to expose an {#link InterceptingClientHttpRequestFactory}
* if necessary.
* #see #getInterceptors()
*/
#Override
public ClientHttpRequestFactory getRequestFactory() {
List<ClientHttpRequestInterceptor> interceptors = getInterceptors();
if (!CollectionUtils.isEmpty(interceptors)) {
ClientHttpRequestFactory factory = this.interceptingRequestFactory;
if (factory == null) {
factory = new InterceptingClientHttpRequestFactory(super.getRequestFactory(), interceptors);
this.interceptingRequestFactory = factory;
}
return factory;
}
else {
return super.getRequestFactory();
}
}
class InterceptingClientHttpRequest extends AbstractBufferingClientHttpRequest { ...
The interfaces make it clear that using InterceptingClientHttpRequest requires buffering body to a byte[]. There is not an option to use a streaming interface.
#Override
public ClientHttpResponse execute(HttpRequest request, byte[] body) throws IOException {

How can we remove x-forwarded-* header in Netflix Zuul filter?

I m building a simple PoC to strip-off the X-FORWARDED-* header while invoking microservice through Zuul gateway. I tried with route filter, but I couldnt find those headers there.
Where are the default headers are added and how can we remove it?
Found out. Add the below in application.properties
zuul.add-proxy-headers=false
Works for me
RequestContext.getCurrentContext().getZuulRequestHeaders().remove("name of the header to be remove");
or
return new HttpServletRequestWrapper(request) {
private Set<String> headerNameSet;
#Override
public Enumeration<String> getHeaderNames() {
if (headerNameSet == null) {
// first time this method is called, cache the wrapped request's header names:
headerNameSet = new HashSet<>();
Enumeration<String> wrappedHeaderNames = super.getHeaderNames();
while (wrappedHeaderNames.hasMoreElements()) {
String headerName = wrappedHeaderNames.nextElement();
if (!headerName.contains("x-forwarded")) {
headerNameSet.add(headerName);
}
}
}
return Collections.enumeration(headerNameSet);
}
#Override
public Enumeration<String> getHeaders(String name) {
if (name.contains("x-forwarded")) {
return Collections.<String>emptyEnumeration();
}
return super.getHeaders(name);
}
#Override
public String getHeader(String name) {
if (name.contains("x-forwarded")) {
return null;
}
return super.getHeader(name);
}
};
Simple one through the configuration file
zuul:
#mentioned header are ignored while build the request in zuul
ignored-header: header_name1, header_name2
#Will ignore all the x-forward-* headers
add-proxy-headers: false

Does Spring XwsSecurityInterceptor remove the Security element from Header?

I have implemented Spring XwsSecurityInterceptor and receiving soap message with <wsse:UsernameToken/> inside the <wsse:Security/> tag (OASIS WS-Security). It works fine.
Now I am trying to implement a logging interceptor to log the request/response soap messages in DB.
I can get the Security element in getSource() method of my custom logging interceptor (which extends org.springframework.ws.server.endpoint.interceptor.PayloadLoggingInterceptor):
#Override
protected Source getSource(WebServiceMessage message) {
SaajSoapMessage soap = (SaajSoapMessage) message;
logger.info(Utils.getSoapEnvelopeAsString(soap));
// this envelop contains the <wsse:Security/> element as expected
// ...
// ...
}
But My problem is, when I extract the envelop inside my endpoint method, I don't get the <wsse:Security/> element in the header anymore.
public JAXBElement<MyResponseType> getRecepientInfo(#RequestPayload JAXBElement<MyRequestType> request, MessageContext messageContext) {
SaajSoapMessage soapReq = (SaajSoapMessage) messageContext.getRequest();
logger.info(Utils.getSoapEnvelope(soapReq));
// this envelop doesn't contain the <wsse:Security/> element
}
Here is the code for Utils.getSoapEnvelope(soap):
public static String getSoapEnvelope(SaajSoapMessage soapMessage) {
SoapEnvelope envelope = soapMessage.getEnvelope();
String envelopeMessge = "";
try {
envelopeMessge = Utils.getSourceAsString(envelope.getSource());
} catch (Exception e) {
// TODO handle Exception here.
}
return envelopeMessge;
}
public static String getSourceAsString(Source source) throws Exception{
TransformerFactory tfactory = TransformerFactory.newInstance();
Transformer xform = tfactory.newTransformer();
StringWriter writer = new StringWriter();
Result result = new StreamResult(writer);
xform.transform(source, result);
return writer.toString();
}
Does spring remove the <wsse:Security/> element from the header after authentication has been completed? Or, I am doing something wrong here?
How should I get the <wsse:Security/> element from header inside endpoint method?
I know this is a late answer but for whom it may interest I found out how to solve this.
You need to modify your securityPolicy.xml file so that the security header is kept. Simply set the attribute retainSecurityHeader to true. Here is an example of such a file:
<xwss:SecurityConfiguration retainSecurityHeader="true" dumpMessages="false" xmlns:xwss="http://java.sun.com/xml/ns/xwss/config">
<xwss:UsernameToken digestPassword="false" useNonce="false" id="someId"/>
</xwss:SecurityConfiguration>

Resources