Spring MVC OutputStream buffer size / flush not blocking? - spring

I have been trying to investigate the buffering behaviour of the HttpServletResponse OutputStream (Using tomcat in spring-boot). My expectation that flushing on the server side blocks until all data has been transmitted to the client.
I am trying to understand this, because I am noticing that clients seem to be processing much longer than the processing on the server.
My setup looks something like this:
Server:
#RequestMapping(path = "/some/url", method = RequestMethod.POST)
public void call(HttpServletRequest request, HttpServletResponse response)
{
OutputStream outputStream = response.getOutputStream();
... writing data here ...
outputStream.flush();
response.flushBuffer();
}
Some dummy client:
HttpURLConnection con = (HttpURLConnection) urlobj.openConnection();
con.setRequestMethod("POST");
con.setDoInput(true);
con.setDoOutput(true);
int nresp = 0;
final InputStream in = con.getInputStream();
while (true)
{
if (in.read() < 0)
break;
nresp++;
if ((nresp & 31) == 0)
Thread.sleep(1l); // slow down read
}
In this example, depending on the response size, the server finishes in 40ms, but the client is reading data for 35 seconds. Also in Wireshark I can see data being transmitted the entire time. Which means that the server is buffering the data.
I also tried with StreamingResponseBody as return value or with "response.setBufferSize(1024)", but the result is the same.
Does anybody understand where this buffering is occurring and if its behaving correctly?
Is there any way on the server to completely flush all buffers and blocking until data is really flushed to the client?
cheers, Christian

Related

Spring WebClient hangs when posting file / inputstream / resource to Minio

I am developing a proxy service to a Minio server using WebClient that handles all Minio/S3 API endpoints. Most of them work fine, but I have encountered one case in which the PUT operation seems to get hung up when trying to set the body of the request to either an InputStream, a File, or a Resource pointing to it. (See epilogue at the bottom, as I'm left wondering where the problem really is.)
The only way I've found to make it work is to read the file contents to an in-memory byte array. The following baseline works, for example:
WebClient.UriSpec<WebClient.RequestBodySpec> uriSpec = client.method(request.getMethod());
WebClient.RequestBodySpec bodySpec = uriSpec.uri(uri);
WebClient.RequestHeadersSpec<?> headersSpec = bodySpec;
try {
// read file to byte array; works fine
byte[] bytes = Files.readAllBytes(Path.of(file.get().getFile().toURI()));
// set it to the request body
headersSpec = bodySpec.bodyValue(bytes);
} catch (IOException e) {
throw new UncheckedIOException(e);
}
// manipulate some headers
headersSpec = headersSpec.headers(httpHeaders -> ...);
// execute the request; works fine in this scenario
return headersSpec.exchangeToMono(resp -> ...)
.doOnError(throwable -> log.error("Trouble proxying request: " + throwable.getMessage(), throwable));
However, every alternative that I try to stream this content instead, results in a request that seems to hang in the headersSpec.exchangeToMono invocation. I don't see any errors on the proxy service, and the client socket eventually gives up:
java.net.SocketTimeoutException: timeout
client-tester_1 | at okio.SocketAsyncTimeout.newTimeoutException(JvmOkio.kt:143) ~[okio-jvm-2.8.0.jar:na]
client-tester_1 | Suppressed: reactor.core.publisher.FluxOnAssembly$OnAssemblyException:
Some examples of failure (or, paraphrasing Edison, I've successfully found at least a dozen ways that do not work):
// Use same byte array as above; Hangs
Resource resource = new ByteArrayResource(bytes);
headersSpec = bodySpec.bodyValue(resource);
// Read an input stream from the file (this one relies on a HttpMessageWriter<InputStream> that I configured on the client); Hangs
InputStream bodyStream = new BufferedInputStream(Files.newInputStream(Path.of(file.get().getFile().toURI())));
headersSpec = bodySpec.bodyValue(bodyStream);
// Resource for the file; Hangs
Resource resource = new FileSystemResource(Path.of(file.get().getFile().toURI()));
Flux<DataBuffer> flux = DataBufferUtils.read(resource, DefaultDataBufferFactory.sharedInstance, 4096);
headersSpec = bodySpec.body(flux, DataBuffer.class);
// Different resource; Hangs
Resource resource = new UrlResource(file.get().getFile().toURI());
headersSpec = bodySpec.bodyValue(resource);
// Try BodyInserters; Hangs
Flux<DataBuffer> flux = DataBufferUtils.read(Path.of(file.get().getFile().toURI()), DefaultDataBufferFactory.sharedInstance, 4096);
headersSpec = bodySpec.body(BodyInserters.fromDataBuffers(flux));
// Yet another attempt; Take a guess...
InputStream bodyStream = new BufferedInputStream(Files.newInputStream(Path.of(file.get().getFile().toURI())));
headersSpec = bodySpec.body(BodyInserters.fromResource(resource));
I'm using recent versions of the relevant libraries:
org.springframework.boot:spring-boot-starter-webflux -> 2.7.5
org.springframework.boot:spring-boot-starter-reactor-netty:2.7.5
org.springframework:spring-core:5.3.23
Epiloge I'm wondering if the problem is not necessarily with Spring/WebClient/Netty -- as many of these code samples were inspired by other examples I've found -- but rather by some nuance on the Minio server?

How is HttpServletResponse entangled with the fetch API when making a GET request for a BLOB?

Using Spring Boot, I am trying to implement a REST controller, which can handle a GET request asking to return a BLOB object from my database.
Googling around a little bit, and putting pieces together, I have created the following code snippet:
#GetMapping("student/pic/studentId")
public void getProfilePicture(#PathVariable Long studentId, HttpServletResponse response) throws IOException {
Optional<ProfilePicture> profilePicture;
profilePicture = profilePictureService.getProfilePictureByStudentId(studentId);
if (profilePicture.isPresent()) {
ServletOutputStream outputStream = response.getOutputStream();
outputStream.write(profilePicture.get().getPicture());
outputStream.close();
}
}
I am sending the GET request using VanillaJS and the fetch-API:
async function downloadPicture(profilePic, studentId) {
const url = "http://localhost:8080/student/pic/" + studentId;
const response = await fetch(url);
const responseBlob = await response.blob();
if (responseBlob.size > 0) {
profilePic.src = URL.createObjectURL(responseBlob);
}
}
Somehow, this works. That's great, but now I would like to understand the usage of HttpServletResponse in this context, which I am not familiar with. It seems to me that the fetch-API makes use of HttpServletResponse (maybe even creates it), since I am not creating this object or do anything with it.
What is very strange to me is that the return-type of my controller method getProfilePicture() is void, and still I am sending a response, which is most definitely not void.
Also, if the profilePicture was not found in my database, for example due to a non-existing studentId being passed, my controller-method does not do anything. But still, I am getting a response code of 200. That's why I have added the responseBlob.size > 0 part in my Javascript to check for a positive response.
Can someone explain this magic to me, please?
response.getOutputStream(); javadoc says "Returns a ServletOutputStream suitable for writing binary data in the response." It's literally the response stream and you write the picture bytes into it. It's not related to the client reading the response. Alternatively you could just return a byte array which will be automatically written into the response stream and the result will be the same.
To return a different http status code you should change the method return type to ResponseEntity<byte[]>:
#GetMapping("student/pic/studentId")
public ResponseEntity<byte[]> getProfilePicture(#PathVariable Long studentId, HttpServletResponse response) throws IOException {
Optional<ProfilePicture> profilePicture = profilePictureService.getProfilePictureByStudentId(studentId);
if (profilePicture.isPresent()) {
return ResponseEntity.ok(profilePicture.get().getPicture()); //status code 200
} else {
return ResponseEntity.notFound().build(); //status code 404
}
}
ResponseEntity is basically springs way to return different status codes/messages.
Is there a reason why you are manually downloading the image via javascript? You could just create a img element with the http link to the image and the browser will automatically display the image content: <img src="http://localhost:8080/student/pic/studentId">

stream was reset: PROTOCOL_ERROR

I've written a web API for sending and receiving data between a server and an app I'm working on.
It works fine for everything, but I've now found it won't let me send large strings. The strings are base64 strings which represent images, typically around 100kb in size.
The method I'm attempting is a multipart post, breaking the base64 string into chunks that can be sent successfully.
When I use this method, I get an error:
stream was reset: PROTOCOL_ERROR
Upon checking the database it seems the first string chunk sends successfully, but nothing more after that.
Can anyone shed some light on what's causing this to happen?
Relevant code is here:
First is the process for breaking the image into manageable chunks and posting it:
HTTPRequest req = new HTTPRequest();
IEnumerable<string> imgStrSplit = Split(img1byteArrayStr, 1000);
foreach (string s in imgStrSplit)
{
response = await req.SubmitImage("Test", s, "1");
}
And below is the SubmitImage() method of the HTTPRequest class:
public async Task<int> SubmitImage(string name, string imageString, string imgNum)
{
using (System.Net.Http.HttpClient client = new System.Net.Http.HttpClient(new NativeMessageHandler()))
{
client.DefaultRequestHeaders.Add("Accept", "application/json");
int successResponse;
string address = $"https://myURL/SubmitImage?name=" + name + "&imageStr=" + imageString + "&imgNum=" + imgNum + "&curl=AYZYBAYZE143";
HttpResponseMessage response = await client.GetAsync(address);
successResponse = JsonConvert.DeserializeObject<int>(response.Content.ReadAsStringAsync().Result);
return successResponse;
}
Thanks.
Found a solution via this discussion.
The problem was in the fact that I was using the ModernHTTPClint NuGet package, as seen in this line of the SubmitImage method:
using (System.Net.Http.HttpClient client = new System.Net.Http.HttpClient(new NativeMessageHandler()))
The NativeMessageHandler() is said to make connections work much faster, but apparently in some cases results in the PROTOCOL_ERROR I had encountered. By removing this part, I fixed the problem.

GZip .NET Compact Framework 3.5

I am trying to send and receive process gzip-ed data from server on my client device application (not web).
I am sending gzip-ed content and on client side, I have following method that returns WebResponse:
protected override WebResponse GetWebResponse(WebRequest request)
{
WebResponse res = base.GetWebResponse(request);
if (((System.Net.HttpWebResponse)(res)).ContentEncoding.Contains("gzip"))
{
Stream responseStream = res.GetResponseStream();
responseStream = new GZipStream(responseStream, CompressionMode.Decompress);
}
//This returns g-ziped content as WebResponse, but I need to return
//above decompressed responseStream as WebResponse, how do I do that?
return res;
}
I am new to this but I am thinking that intercepting every response comming to my app in GetWebResponse is excellent centralized spot to decompress all responses. But the problem is how to pass the decompressed stream as response back?
Much appreciated

Async sends in .NET ActiveMQ

I'm looking to increase the performance of a high-throughput producer that I'm writing against ActiveMQ, and according to this useAsyncSend will:
Forces the use of Async Sends which adds a massive performance boost;
but means that the send() method will return immediately whether the
message has been sent or not which could lead to message loss.
However I can't see it making any difference to my simple test case.
Using this very basic application:
const string QueueName = "....";
const string Uri = "....";
static readonly Stopwatch TotalRuntime = new Stopwatch();
static void Main(string[] args)
{
TotalRuntime.Start();
SendMessage();
Console.ReadLine();
}
static void SendMessage()
{
var session = CreateSession();
var destination = session.GetQueue(QueueName);
var producer = session.CreateProducer(destination);
Console.WriteLine("Ready to send 700 messages");
Console.ReadLine();
var body = new byte[600*1024];
Parallel.For(0, 700, i => SendMessage(producer, i, body, session));
}
static void SendMessage(IMessageProducer producer, int i, byte[] body, ISession session)
{
var message = session.CreateBytesMessage(body);
var sw = new Stopwatch();
sw.Start();
producer.Send(message);
sw.Stop();
Console.WriteLine("Running for {0}ms: Sent message {1} blocked for {2}ms",
TotalRuntime.ElapsedMilliseconds,
i,
sw.ElapsedMilliseconds);
}
static ISession CreateSession()
{
var connectionFactory = new ConnectionFactory(Uri)
{
AsyncSend = true,
CopyMessageOnSend = false
};
var connection = connectionFactory.CreateConnection();
connection.Start();
var session = connection.CreateSession(AcknowledgementMode.AutoAcknowledge);
return session;
}
I get the following output:
Ready to send 700 messages
Running for 2430ms: Sent message 696 blocked for 12ms
Running for 4275ms: Sent message 348 blocked for 1858ms
Running for 5106ms: Sent message 609 blocked for 2689ms
Running for 5924ms: Sent message 1 blocked for 2535ms
Running for 6749ms: Sent message 88 blocked for 1860ms
Running for 7537ms: Sent message 610 blocked for 2429ms
Running for 8340ms: Sent message 175 blocked for 2451ms
Running for 9163ms: Sent message 89 blocked for 2413ms
.....
Which shows that each message takes about 800ms to send and the call to session.Send() blocks for about two and a half seconds. Even though the documentation says that
"send() method will return immediately"
Also these number are basically the same if I either change the parallel for to a normal for loop or change the AsyncSend = true to AlwaysSyncSend = true so I don't believe that the async switch is working at all...
Can anyone see what I'm missing here to make the send asynchronous?
After further testing:
According to ANTS performance profiler that vast majority of the runtime is being spent waiting for synchronization. It appears that the issue is that the various transport classes block internally through monitors. In particular I seem to get hung up on the MutexTransport's OneWay method which only allows one thread to access it at a time.
It looks as though the call to Send will block until the previous message has completed, this explains why my output shows that the first message blocked for 12ms, while the next took 1858ms. I can have multiple transports by implementing a connection-per-message pattern which improves matters and makes the message sends work in parallel, but greatly increases the time to send a single message, and uses up so many resources that it doesn't seem like the right solution.
I've retested all of this with 1.5.6 and haven't seen any difference.
As always the best thing to do is update to the latest version (1.5.6 at the time of this writing). A send can block if the broker has producer flow control enabled and you've reached a queue size limit although with async send this shouldn't happen unless you are sending with a producerWindowSize set. One good way to get help is to create a test case and submit it via a Jira issue to the NMS.ActiveMQ site so that we can look into it using your test code. There have been many fixes since 1.5.1 so I'd recommend giving that new version a try as it could already be a non-issue.

Resources