ServerDoneResponse: 06:32:46.776
ClientBeginResponse: 06:32:47.934
As you can see here, it takes 1.2 second for Fiddler to start sending response to the client. What could possibly cause this to happen? My guess is that it has something to do with CustomRules.js but I am not 100% sure here.
Related
Trying to see if I can get a response from ctrader server.
Getting no response and seems to hang at "s.recv(1024)". So not sure what could be going wrong here. I have limited experience with sockets and network coding.
I have checked my login credentials and all seems ok.
Note: I am aware of many FIX engines that are available for this purpose but wanted to
try this on my own.
ctrader FIX guides
require 'socket'
hostname = "h51.p.ctrader.com"
port = 5201
#constructing a fix message to see what ctrader server returns
#8=FIX.4.4|9=123|35=A|49=demo.ctrader.*******|56=cServer|57=QUOTE|50=QUOTE|34=1|52=20220127-16:49:31|98=0|108=30|553=********|554=*******|10=155|
fix_message = "8=FIX.4.4|9=#{bodylengthsum}|" + bodylength + "10=#{checksumcalc}|"
s = TCPSocket.new(hostname, port)
s.send(fix_message.force_encoding("ASCII"),0)
print fix_message
puts s.recv(1024)
s.close
Sockets are by default blocking on read. When you call recv that call will block if no data is available.
The fact that your recv call is not returning anything, would be an indication that the server did not send you any reply at all; the call is blocking waiting for incoming data.
If you would use read instead, then the call will block until all the requested data has been received.
So calling recv(1024) will block until 1 or more bytes are available.
Calling read(1024) will block until all 1024 bytes have been received.
Note that you cannot rely on a single recv call to return a full message, even if the sender sent you everything you need. Multiple recv calls may be required to construct the full message.
Also note that the FIX protocol gives the msg length at the start of each message. So after you get enough data to see the msg length, you could call read to ensure you get the rest.
If you do not want your recv or read calls to block when no data (or incomplete data) is available, then you need to use non-blocking IO instead for your reads. This is complex topic, which you need to research, but often used when you don't want to block and need to read arbitary length messages. You can look here for some tips.
Another option would be to use something like EventMachine instead, which makes it easier to deal with sockets in situations like this, without having to worry about blocking in your code.
This question already has answers here:
Why HTTP/2 does multiplexing altough tcp does same thing?
(3 answers)
Closed last year.
This passage claims that the binary frame layer becomes the base for multiplexing in http for TCP connection, which is confusing to me.
https://developers.google.com/web/fundamentals/performance/http2#design_and_technical_goals
The confusing part is the HTTP client can just send more requests in one TCP connection without waiting for the response and receive the response for the corresponding request. That is the "frame" is the request and response. So why should it add the binary frame?
Let's have a look at what you're suggesting:
the HTTP client can just send more requests in one TCP connection without waiting for the response
So far, so good: I can send "GET /foo" and then immediately "GET /bar" on the same connection.
and receive the response for the corresponding request
So, the server replies "200 OK" with some HTML content, and ... wait, is that for "/foo" or "/bar"? The key word in your own description is "corresponding" - we need some way of saying "this response corresponds to request #1".
And then, halfway through sending the first response, the server finishes handling the other request, and is ready to send part of a different response; but if it jumps in with "200 OK", that's going to appear to be part of the response it's already sending. So we also need to be able to say "this is the start of a new response", and "this content is the continuation of response #2".
To do that, we need a new abstraction: a frame, with a header which can encode details like "the next 100 bytes are the start of response #2, which corresponds to request #1". (I'm not sure if that's exactly how an HTTP/2 frame works, but I think it's roughly the principle.)
We could do that and still keep the protocol human readable (which is what we really mean by "text-based" vs "binary") but there's going to be a lot of these frame headers, so the shorter we can make them, the better. So if we're interested in performance, we can give up on "human readable" as a requirement, and we end up with a binary framing protocol like HTTP/2.
I have code, that make http-requests to sites (using aiohttp) with async_timeout. If I run all requests together, then some requests are raising TimeoutError (even if timeout=20s.). But if I run one request -- it works.
def coro(url):
with async_timeout.timeout(TIMEOUT, loop=loop):
async with session.get(url) as response:
text, status = (await response.text()), response.status
...
Is this async_timeout problem/bug or my?
I tried to use TCPConnector (aiohttp.TCPConnector(limit=None, verify_ssl=False, loop=loop)), but it doesn't work
There is nothing strange if a request takes more than 20 sec in case of very large requests amount (and this request is much faster when executed alone).
To make sure just insert timestamp printouts before and after .get()/.text() execution.
Timeout's code is deadly simple and highly tested, don't suspect an error in it.
I want to send notifications to apple devices in batches (1.000 device tokens in batch for example). Ant it seems that I can't know for sure that message was delivered to APNs.
Here is the code sample:
ssl_connection(bundle_id) do |ssl, socket|
device_tokens.each do |device_token|
ssl.write(apn_message_for device_token)
# I can check if there is an error response from APNs
response_has_an_error = IO.select([socket],nil,nil,0) != nil
# ...
end
end
The main problem is if network is down after the ssl_connection is established
ssl.write(...)
will never raise an error. Is there any way to ckeck that connection still works?
The second problem is in delay between ssl.write and ready error answer from APNs. I can pass timeout parameter to IO.select after last messege was sent. Maybe It's OK to wait for a few seconds for 1.000 batch, but wat if I have to send 1.000 messages for differend bundle_ids?
At https://zeropush.com, we use a gem named grocer to handle our communication with Apple and we had a similar problem. The solution we found was to use the socket's read_non_block method before each write to check for incoming data on the socket which would indicate an error.
It makes the logic a bit funny because read_non_block throws IO::WaitReadable if there is no data to read. So we call read_non_block and catch IO::WaitReadable before continuing as normal. In our case, catching the exception is the happy case. You may be able to use a similar approach rather than using IO.select(...).
One issue to be aware of is that Apple may not respond immediately and any notifications sent between a failing notification and reading from the socket will be lost.
You can see the code we are using in production at https://github.com/SymmetricInfinity/grocer/blob/master/lib/grocer/connection.rb#L30.
If a website user submits an HTML form with: (1) a post method; (2) a multipart/form-data enctype; and, (3) a large attached file, can the server upload a posted file, and send a server generated HTTP response before the file upload is completed, without using AJAX?
That's pretty dense. So, I wrote an example to illustrate what I mean. Let's say there is an image upload form with a caption field.
<form action="upload-with-caption/" method="post" enctype="multipart/form-data">
<input type="hidden" id="hiddenInfo" name="hiddenInfo" />
File: <input type="file" name="imgFile" id="imgFile" /><br />
Caption: <input type="text" name="caption" id="caption" />
<input type="submit" />
</form>
I want to store the caption in a database table with the the definition:
[files_table]
file_id [uniqueidentifier]
file_caption [varchar(500)]
file_status [int]
Then I want to upload the file to /root/{unique-id}/filename.ext.
file_status is mapped to a C# enum with the following definition:
enum FileUploadStatus{
Error = 0,
Uploading = 1,
Uploaded = 2
}
When the form submits, if the file is too large to process in 1 second, I want to send the webpage back a response that says it is currently uploading.
Can I do this with a single synchronous HTTP post?
Note: I will obviously want to check for the status updates later using AJAX, but that is not what this question is asking. I am specifically asking if the file can continue to upload after the response is sent.
HTTP is a synchronous protocol.
You cannot send a response until you receive the entire request.
Looking at the HTTP specifications alone (RFC's 753x), then the answer is Yes (and, the currently accepted answer is wrong). HTML specifically I don't think have anything to add.
The HTTP/1.1 protocol "relies on the order of response arrival to correspond exactly to the order in which requests are made on the same connection" (RFC 7230 §5.6). Timing has nothing to do with it.
Not only does the protocol allow for early responses, but some message semantics from categories 4xx (Client Error) and 5xx (Server Error) actually expects the response to be sent before the request has completed.
Let's take an example. If you intend to send five trillion billion million gigabytes to a web server (let's assume this number fit whatever data types are in use for the Content-Length header), when would you expect to receive a "413 Payload Too Large" response back? As soon as possible or only after a couple of decades when the request transfer completes? Obviously the sooner the better!
2xx (Successful) responses are a bit different. These responses "indicates that the client's request was successfully received, understood, and accepted" (RFC 7231 §6.3). Sending back this type of response early is likely to confuse the client.
Instead, what you probably want to send back as an early response belongs to the 1xx (Informational) category. These are referred to as "interim responses" meant to supersede but not obsolete the final response.
RFC 7231 §6.2:
The 1xx (Informational) class of status code indicates an interim
response for communicating connection status or request progress
prior to completing the requested action and sending a final
response.
RFC 7230 §5.6:
More than one response message per request only occurs
when one or more informational responses precede a
final response to the same request.
RFC 7231 §5.1.1 has a great example where a client is about to send a "presumably large" message but instead of immediately sending the body after the head, the client includes an Expect: 100-continue header and then goes into a short paus whilst expecting the server to either reject the message or welcoming the client to carry on by means of responding a "100 Continue" interim response. This then potentially avoids the client having to transmit bytes for nothing. Smart!
Finally, I thought long and hard about when would we ever want to send a 2xx (Successful) response back to the client before the request has completed? I can only come up with one single scenario - and this is certainly not a common case, but I am going to have it stated: If the server has consumed enough of the request in order to take action and the server wish to discard the remaining body because the residue is sufficiently large and at the same time of no more use to the server, then respond 202 Accepted and include a "Connection: close" header.
This is obviously not good for connection re-use and could also easily lead to confused clients and so the payoff why we're responding early should be 1) advantageous enough to mitigate the overhead of establishing a new connection, 2) advantageous enough to offset the danger of crashing clients that was not prepared for an early response, and 3) be well documented.
The "Connection: close" header will explicitly instruct the client to stop sending the request (RFC 7230 §6.3). And due to message framing, the connection is dead anyways as there is no way for the communication to resume with a new message exchange pair over the same connection. Technically speaking, the client can cleanly abort a chunked transfer (RFC 7230 §4.1) and thus save the connection, but this is details and not applicable in the general case.