Is there any way to handle application/stream+json content with the old fashioned RestTemplate the way webClient does?
As far as my attempts go, wrapping the results of something like restTemplate.getForEntity in Flux.just(<convert response entity to mono here>) would just return the first element and stop at that, while webClient handles it properly, populating that resulting json with new entries as they appear. Haven't tried working with inputStream yet, but at first glance it doesn't seem to be what i need, despite having a "stream" in its name.
Unfortunately, using webClient would be a rather costly option in this case (still waiting for https://github.com/spring-projects/spring-security/issues/4921). I'd rather implement things that would "soon" appear officially only if there's absolutely no other way.
RestTemplate is exposing an API which is not meant to be used to stream the HTTP response. The underlying HTTP response is read and closed after each call, whereas the "application/stream+json" media type is meant for streaming responses.
I don't see any way to properly handle this use case (reading streaming responses) with RestTemplate, by design (check out SPR-14882 for another example of that). So you'll probably have to wait for that issue to be resolved or use another HTTP client with such features.
Related
I want to append some data to the incoming request. Like random generated token or uuid to the incoming request. And then I want to process it through the controller. I came to know about ClientHttpRequestInterceptor. But looking at this doc, it seems like it only intercept the response, it doesn't intercept the request. Which is what I am not looking. Is there any other way to do this ?
And how can I register this intercept in my RestController ? So that before controller process the request, the request should already has the data.
EDIT:
I just found out, I can directly set the data in controller using set method in request body. And this is working. But I am not sure if this is recommended way. Because as far as I know the request has to be modified in dispatcher servlet.
Please advice.
If you don't want to do it this way (How to modify request body before reaching controller in spring boot),
you might do one of the following:
OncePerRequestFilter (as mentioned in the #doctore answer) and add a parameter to the request. This would allow you to add data to the request, but not change anything sent by the client.
Add a method in the controller and call it at the start of processing. I don't like this as much because unlike the filter approach, this requires you to call the method.
[Note: I've never tried this, but it should work] Add a method [somewhere] and use Spring AOP to call it before entering the handler method in the controller. This is fine, but is essentially just you creating your own way of processing a OncePerRequestFilter.
There are surely other ways of doing this with Spring,
I just don't know them.
You need to add your own OncePerRequestFilter implementation. In the next link you will be able to see an example of that:
Filter example
In this case, it uses TheadContext (MDC) to include the information you want to use in your controller layer (do not include "something similar" to MDC.remove(mdcTokenKey); in your code, you want to keep the information on MDC to access it in your controller).
PD: The internal server of Spring MVC: Tomcat, Jetty, etc reuses the threads so, if you don't want to have some problems it is important you include always a value in your "TheadContext cache". In that way, you will avoid to find "old values", I mean, values included in the current thread but in a "previous Http request".
UPDATE (modify the request body):
Take a look to the following link if you want to modify the request itself:
Modify request content before manage it in controller
I want to implement a Liferay Portlet that downloads a ~1GB file from a separate server, and serves it to the website visitor who clicked the link.
The file must be streamed in a memory-efficient way (so no loading everything into memory), and the user should see the download progress shortly after clicking (so no storing everything onto the local disk).
I must use WebClient because it seems to be the standard for making web requests within Liferay 7 (RestTemplate will be deprecated).
I started writing something like this, inspired by an example from the javadoc:
Mono<DataBuffer> bodyMono = client.get()
.uri("https://theotherserver.com/file94875.pdf")
.retrieve()
.bodyToMono(DataBuffer.class);
... which I would feed into the portlet's MVCResourceCommand.serveResource() via PortletResponseUtil.sendFile, which expects a java.io.InputStream.
Unfortunately WebClient gives me a Mono<DataBuffer> (or Flux<DataBuffer>), and another answer claims that reconstructing the InputStream defeats the purpose of using WebClient in the first place.
What would be the most efficient and WebClient-savvy way to implement this?
In case of Liferay, the documentation states, that you can use ....getPortletOutputStream() to retrieve an OutputStream.
After setting contentlengh (so browser knows how much to expect), you can use this: Convert writes to OutputStream into a Flux<DataBuffer> usable by ServerResponse
To write your data to the OutputStream
I'm using the new 2.X NEST client. That part is important, because there were a great many breaking changes which will effect potential answers here.
Previously, I used the Glimpse Elasticsearch plugin to see the underlying queries being generated by NEST. However, it would appear that that plugin is no longer compatible with 2.X NEST. As a result, I'm trying to find a workaround to see the JSON query. The problem here is that the old way of accessing response.RequestInformation to get at the request body is gone. It seems to have been replaced with a combination of ApiCall, CallDetails, and DebugInformation. The problem here is that in all of these the request byte array is null unless you add .DisableDirectStreaming() to the ConnectionSettings instance you pass into ElasticClient. The problem there is that I'm handling all that using dependency injection with Ninject, so in something like a controller action, I have no access to the ConnectionSettings instance to make such a change. I suppose I could just add .DisableDirectStreaming() globally, but I have no idea what the potential consequences of that is and the documentation on this is frustratingly sparse.
So, there's a few avenues for an answer here, any of which I'll accept. First, if anyone has manage to get the Glimpse plugin functioning with 2.X, I'd love to know what you did. However, based on the fact that the underlying API has changed dramatically, my assumption is that the plugin is fundamentally broken until someone branches it for 2.X or Elastic comes out with their own version (which is supposedly coming at some undetermined point in the future).
Second, if there's some way to get at the request body without disabling direct streaming, and I simply missed it. I'd appreciate guidance there.
Third, if anyone has any ideas for how I can disable direct streaming at the controller action level, without affecting my Ninject setup or applying it globally, feel free to chime in.
Finally, it would be great if someone from the Elastic team can enlighten me to the effects of disabling direct streaming and what potential problems can arise from that, so I can make a determination about whether it would be wise to apply it globally or not.
With .DisableDirectStreaming() set to true, the request bytes and response bytes are buffered in memory streams to enable them to be available on the response via response.RequestBodyInBytes and response.ResponseBodyInBytes, respectively.
By default, it is set to false so the request type e.g. SearchDescriptor<T>, SearchRequest<T>, etc. is serialized directly to the request stream of the http request and similarly, the response type is deserialized from the response stream. The overhead of setting it to true is therefore keeping the request and response bytes around in memory for the lifetime of the response (and GC kicking in).
With Connection Settings, it's best to have one instance for the lifetime of the application; Serialization settings are cached per connection settings as well as caches for field and property expressions. There's no way currently to keep the request and response bytes around on a per request basis i.e. ad-hoc introspection, but I think this would be a useful addition; I'll add an issue for it :)
I'm not personally overly familiar with the Glimpse integration but I would expect it would require updating to work with NEST 2.x because of some of the changes. Having just given it a brief look, the changes look pretty straightforward. Looks like this can be done without having to set .DisableDirectStreaming() to true, but only grabbing the request bytes before they're written to the request stream.
I'm attempting to port over an Internet Explorer plugin to Firefox, but I'm not sure where to look for what I need.
Basically I need to be able to filter all content that is received by the browser with a certain Content-Type header. I tried implementing a stream converter, and this works, but only for the top-level document in the page, frame, or iframe. I had the same problem with IE, and getting around it was really hacky, and since I would ideally like this to be cross platform I would really like to be able to do this in Firefox without resorting to vtable hacks.
The content is served compressed with a proprietary compression format. So I need to receive the data, decompress it, and change the Content-Type back to what the original uncompressed file should have.
If there is a way to just filter all data received, that would probably be acceptable, I could handle parsing the header myself.
Thanks
I think I may have found what I needed. I came across this link which is used for tracing HTTP calls: http://blues.ath.cx/firekeeper/resources/http_tracer.html
There seems to be some problems with the JavaScript implementation for some reason, and I'm not a JavaScript guru to figure it out, but I've implemented it in C++ and initial results suggest that I should be able to modify it for my needs.
Basically we're replacing the nsIHttpProtocolHandler service with our own implementation, which keeps a reference to the initial implementation. When a call is made to the service, we just proxy it over to the saved original implementation. Then we provide our own implementation of nsIHttpChannel and nsIStreamListener which we use as proxies too.
Again we proxy most of the calls back off to the original handlers. But in OnDataAvailable, instead of passing the data on to the underlying nsIStreamListener, we save it using nsIStorageStream. Then in OnStopRequest, after we've gotten all of the data, we can decompress it and then call OnDataAvailable on the original handler, followed by OnStopRequest.
It has worked on some small simple tests so far, but I'll have to put it through some more rigorous tests... I'll also have to figure out if I can do the same thing with HTTPS.
The biggest problem I see at the moment is that it relies on some unfrozen interfaces such as nsIHttpChannelInternal. Can't be helped though as far as I can tell, and my version compatibility requirements are pretty small, so I can live with it if I have to.
In the meantime, if anybody has any other suggestions, I'm all ears :D
I'm familiar enough with Ajax and JSON that I can send a request and get a parse a JSON request. Ideally I'd like to receive multiple response to periodically update a progress bar. This way clients can have a positive feedback.
I've heard of JSON streams but have not found a good resource on how to implement this. Does anyone know of a good resource or how to do this?
JSON is just yet another format of data going over the HTTP protocol (like text, html, pdf, etc). You are probably referring to cometd.
This allows you to open a persistent connection and push data from the server to the client (ie stream it). Any format is valid to push, the client just needs to understand it.
Found a technique called page streaming.
Basically you write <script>some js</script> entries into the persistent connection and flush them into the network interface. As browser receives that, it will parse and execute the script.
Try looking into the library "comet." It's implements what's known as "reverse AJAX." It'll allow you to send events from the server to the client easily.
The polling suggestion made just before mine, is also perfectly valid.
<script language="JavaScript">
function doSomething() {
// do something here...
}
setInterval('doSomething()',10000);
<script>
That will call a function every 10 seconds. So you can poll the server every 10 seconds (or 1 second) to get a response on the status of whatever event you're trying to track. Simply put your AJAX call inside that function and it'll send.