This question already has answers here:
Why HTTP/2 does multiplexing altough tcp does same thing?
(3 answers)
Closed last year.
This passage claims that the binary frame layer becomes the base for multiplexing in http for TCP connection, which is confusing to me.
https://developers.google.com/web/fundamentals/performance/http2#design_and_technical_goals
The confusing part is the HTTP client can just send more requests in one TCP connection without waiting for the response and receive the response for the corresponding request. That is the "frame" is the request and response. So why should it add the binary frame?
Let's have a look at what you're suggesting:
the HTTP client can just send more requests in one TCP connection without waiting for the response
So far, so good: I can send "GET /foo" and then immediately "GET /bar" on the same connection.
and receive the response for the corresponding request
So, the server replies "200 OK" with some HTML content, and ... wait, is that for "/foo" or "/bar"? The key word in your own description is "corresponding" - we need some way of saying "this response corresponds to request #1".
And then, halfway through sending the first response, the server finishes handling the other request, and is ready to send part of a different response; but if it jumps in with "200 OK", that's going to appear to be part of the response it's already sending. So we also need to be able to say "this is the start of a new response", and "this content is the continuation of response #2".
To do that, we need a new abstraction: a frame, with a header which can encode details like "the next 100 bytes are the start of response #2, which corresponds to request #1". (I'm not sure if that's exactly how an HTTP/2 frame works, but I think it's roughly the principle.)
We could do that and still keep the protocol human readable (which is what we really mean by "text-based" vs "binary") but there's going to be a lot of these frame headers, so the shorter we can make them, the better. So if we're interested in performance, we can give up on "human readable" as a requirement, and we end up with a binary framing protocol like HTTP/2.
Related
Trying to see if I can get a response from ctrader server.
Getting no response and seems to hang at "s.recv(1024)". So not sure what could be going wrong here. I have limited experience with sockets and network coding.
I have checked my login credentials and all seems ok.
Note: I am aware of many FIX engines that are available for this purpose but wanted to
try this on my own.
ctrader FIX guides
require 'socket'
hostname = "h51.p.ctrader.com"
port = 5201
#constructing a fix message to see what ctrader server returns
#8=FIX.4.4|9=123|35=A|49=demo.ctrader.*******|56=cServer|57=QUOTE|50=QUOTE|34=1|52=20220127-16:49:31|98=0|108=30|553=********|554=*******|10=155|
fix_message = "8=FIX.4.4|9=#{bodylengthsum}|" + bodylength + "10=#{checksumcalc}|"
s = TCPSocket.new(hostname, port)
s.send(fix_message.force_encoding("ASCII"),0)
print fix_message
puts s.recv(1024)
s.close
Sockets are by default blocking on read. When you call recv that call will block if no data is available.
The fact that your recv call is not returning anything, would be an indication that the server did not send you any reply at all; the call is blocking waiting for incoming data.
If you would use read instead, then the call will block until all the requested data has been received.
So calling recv(1024) will block until 1 or more bytes are available.
Calling read(1024) will block until all 1024 bytes have been received.
Note that you cannot rely on a single recv call to return a full message, even if the sender sent you everything you need. Multiple recv calls may be required to construct the full message.
Also note that the FIX protocol gives the msg length at the start of each message. So after you get enough data to see the msg length, you could call read to ensure you get the rest.
If you do not want your recv or read calls to block when no data (or incomplete data) is available, then you need to use non-blocking IO instead for your reads. This is complex topic, which you need to research, but often used when you don't want to block and need to read arbitary length messages. You can look here for some tips.
Another option would be to use something like EventMachine instead, which makes it easier to deal with sockets in situations like this, without having to worry about blocking in your code.
I am working on an assignment where I have to develop a web server in Ruby using the socket library. I was able to get a simple web server up and running as seen in this thread here .
I am currently working on getting and storing the body of an HTTP request into a variable in my web server. The problem I am running into is trying to define a while loop that gets the entire body of a HTTP request.
I am attempting to get the body of a HTTP request by using the gets method. I could not find any documentation on this method (I saw it being used here)
and was wondering if there were more documentation online.
In my first post here, someone suggested that I use the Content-Length header to determine the size of the body and how much data to read from the socket. I don't really understand how I would go about implementing this because I am unsure how the gets method functions.
Since this is for an assignment, I don't think posting code would be a good idea. I am looking for more information on the gets method and any tips to point me towards the right direction.
You shouldn't be using gets. gets tries to read complete lines (ie it reads up to a line separator), but there is no guarantee that an http request body ends with a line separator.
Instead you should be using read - this allows you to read an arbitrary amount of data (as you mentioned you can use the content length header to know how much to read)
Your ultimate problem isn't related to gets, or even really anything in your code. But before we get to that, let's answer this question & explore sockets a little bit.
If you follow the chain up, you find that Ruby's TCPSocket class inherits from its IO class. It's IO that provides gets. gets will read, line-by-line, until there's nothing more to read. Let's create a simple client that connects to a port, spits out 4 lines of poetry, and then quits:
# poetry_sender.rb
require 'socket'
poem = ["'God save thee, ancient Mariner!",
"From the fiends, that plague thee thus!—",
"Why look'st thou so?'—With my cross-bow",
"I shot the ALBATROSS."]
puts "Client establishing connection..."
s = TCPSocket.new 'localhost', 2000
puts "Client sending poetry..."
poem.each { |line| s.puts line } # Print each line out on the socket
s.close # Close our socket
puts "All done."
And a simple server, that displays what the client sends us:
# poetry_receiver.rb
require 'socket'
server = TCPServer.new 2000 # Server bind to port 2000
loop do
puts "Server now awaiting some poetry..."
socket = server.accept # Wait for a client to connect
while line = socket.gets
puts "A client sent us this beautiful line: #{line}"
end
puts "They had nothing more to say; let's disconnect them."
socket.close
end
If you run the server (poetry_receiver.rb) first, and then the client, you'll see some output like this:
Server now awaiting a connection...
A client sent us this beautiful line: 'God save thee, ancient Mariner!
A client sent us this beautiful line: From the fiends, that plague thee thus!—
A client sent us this beautiful line: Why look'st thou so?'—With my cross-bow
A client sent us this beautiful line: I shot the ALBATROSS.
They had nothing more to say; let's disconnect them.
Server now awaiting a connection...
The last two lines are the important ones; they indicate that socket.gets returned nil and we exited the while loop.
So, how can we modify our poetry_sender.rb so the server doesn't detect the end of the poem? You might think it's got something to do with blank lines, but if you set poem = [] or poem = ["", "", ""] then you'll find that it still gets disconnected OK. But what if we added a delay before closing the socket in poetry_sender.rb?
sleep 60
s.close # Close our socket
puts "All done."
Now you'll see a big delay in the server output. The TCP server doesn't break out of its while loop until the TCP client closes its socket.
Now we can turn to your broader problem: you're trying to implement a simple HTTP server, but your server is getting hung up in a while loop when you try to connect via your web browser. It's because your web browser is keeping that socket open; but it has to, otherwise it has no way to send you back a response. So, how do we know when a client has finished sending us a response? The HTTP 1.1 spec says:
A client sends an HTTP request to a server in the form of a request message... followed by header fields... an empty line to indicate the end of the header section, and finally a message body containing the payload body (if any).
Let's not worry about the message body; how could we write a while loop that terminates if it has no more impact, or if it receives a blank line? Here's one way, in a simple HTTP server that just sends back "Hello world" no matter what request it receives:
require 'socket'
server = TCPServer.new('localhost', 2345)
http_request = [] # We'll store the lines of our incoming request here.
loop do
socket = server.accept
while (line = socket.gets) && line.chomp != '' # While the client is connected, and hasn't sent us a blank line yet...
http_request << line
end
# Send response headers
socket.print "HTTP/1.1 200 OK\r\n" +
"Content-Type: text/plain\r\n" +
"Connection: close\r\n" +
"\r\n"
# Send response body
socket.print "Hello world!"
socket.close
end
Quite late to the party, but I'm currently implementing my own rack app server (for fun).
Here you can see how I do it: https://github.com/tak1n/reifier/blob/master/lib/reifier/request.rb
The first line of a HTTP request is always the request line, which is basically something like GET /test HTTP/1.1
After the request line until \r\n you get the headers.
After that you are able to read the body (if PUT / POST request) with just using the CONTENT_LENGTH you parsed from the headers.
I have a function that just makes a get request to check the status code. It does not read anything from the body. Should I still end the function with resp.Body.Close() ?
Callers should close resp.Body when done reading from it. If resp.Body is not closed, the Client's underlying RoundTripper (typically Transport) may not be able to re-use a persistent TCP connection to the server for a subsequent "keep-alive" request.
Yes. When you call http.Get, the function returns a response as soon as all the HTTP headers have been read. The body of the response has not been read yet. The Response.Body is a wrapper around the network connection to the server. When you read from it, it downloads the body of the response.
.Close() tells the system that you're done with the network connection. If you have not read the response body, the default http transport closes the connection. (The transport can only re-use the connection if the body has been read, because if it reused a connection with an unread body the next request made using that connection would receive the previous request's response!)
So reading the Body is often more efficient than simply Close()ing if you're making more than one request - especially with TLS connections which are relatively expensive to create.
If you don't need the body of the response, you should use Head instead of Get. Head doesn't require reading or closing the response body.
If a website user submits an HTML form with: (1) a post method; (2) a multipart/form-data enctype; and, (3) a large attached file, can the server upload a posted file, and send a server generated HTTP response before the file upload is completed, without using AJAX?
That's pretty dense. So, I wrote an example to illustrate what I mean. Let's say there is an image upload form with a caption field.
<form action="upload-with-caption/" method="post" enctype="multipart/form-data">
<input type="hidden" id="hiddenInfo" name="hiddenInfo" />
File: <input type="file" name="imgFile" id="imgFile" /><br />
Caption: <input type="text" name="caption" id="caption" />
<input type="submit" />
</form>
I want to store the caption in a database table with the the definition:
[files_table]
file_id [uniqueidentifier]
file_caption [varchar(500)]
file_status [int]
Then I want to upload the file to /root/{unique-id}/filename.ext.
file_status is mapped to a C# enum with the following definition:
enum FileUploadStatus{
Error = 0,
Uploading = 1,
Uploaded = 2
}
When the form submits, if the file is too large to process in 1 second, I want to send the webpage back a response that says it is currently uploading.
Can I do this with a single synchronous HTTP post?
Note: I will obviously want to check for the status updates later using AJAX, but that is not what this question is asking. I am specifically asking if the file can continue to upload after the response is sent.
HTTP is a synchronous protocol.
You cannot send a response until you receive the entire request.
Looking at the HTTP specifications alone (RFC's 753x), then the answer is Yes (and, the currently accepted answer is wrong). HTML specifically I don't think have anything to add.
The HTTP/1.1 protocol "relies on the order of response arrival to correspond exactly to the order in which requests are made on the same connection" (RFC 7230 §5.6). Timing has nothing to do with it.
Not only does the protocol allow for early responses, but some message semantics from categories 4xx (Client Error) and 5xx (Server Error) actually expects the response to be sent before the request has completed.
Let's take an example. If you intend to send five trillion billion million gigabytes to a web server (let's assume this number fit whatever data types are in use for the Content-Length header), when would you expect to receive a "413 Payload Too Large" response back? As soon as possible or only after a couple of decades when the request transfer completes? Obviously the sooner the better!
2xx (Successful) responses are a bit different. These responses "indicates that the client's request was successfully received, understood, and accepted" (RFC 7231 §6.3). Sending back this type of response early is likely to confuse the client.
Instead, what you probably want to send back as an early response belongs to the 1xx (Informational) category. These are referred to as "interim responses" meant to supersede but not obsolete the final response.
RFC 7231 §6.2:
The 1xx (Informational) class of status code indicates an interim
response for communicating connection status or request progress
prior to completing the requested action and sending a final
response.
RFC 7230 §5.6:
More than one response message per request only occurs
when one or more informational responses precede a
final response to the same request.
RFC 7231 §5.1.1 has a great example where a client is about to send a "presumably large" message but instead of immediately sending the body after the head, the client includes an Expect: 100-continue header and then goes into a short paus whilst expecting the server to either reject the message or welcoming the client to carry on by means of responding a "100 Continue" interim response. This then potentially avoids the client having to transmit bytes for nothing. Smart!
Finally, I thought long and hard about when would we ever want to send a 2xx (Successful) response back to the client before the request has completed? I can only come up with one single scenario - and this is certainly not a common case, but I am going to have it stated: If the server has consumed enough of the request in order to take action and the server wish to discard the remaining body because the residue is sufficiently large and at the same time of no more use to the server, then respond 202 Accepted and include a "Connection: close" header.
This is obviously not good for connection re-use and could also easily lead to confused clients and so the payoff why we're responding early should be 1) advantageous enough to mitigate the overhead of establishing a new connection, 2) advantageous enough to offset the danger of crashing clients that was not prepared for an early response, and 3) be well documented.
The "Connection: close" header will explicitly instruct the client to stop sending the request (RFC 7230 §6.3). And due to message framing, the connection is dead anyways as there is no way for the communication to resume with a new message exchange pair over the same connection. Technically speaking, the client can cleanly abort a chunked transfer (RFC 7230 §4.1) and thus save the connection, but this is details and not applicable in the general case.
It appears that when you make an XMLHttpRequest from a script in a browser, if the browser is set to work offline or if the network cable is pulled out, the request completes with an error and with status = 0. 0 is not listed among permissible HTTP status codes.
What does a status code of 0 mean? Does it mean the same thing across all browsers, and for all HTTP client utilities? Is it part of the HTTP spec or is it part of some other protocol spec? It seems to mean that the HTTP request could not be made at all, perhaps because the server address could not be resolved.
What error message is appropriate to show the user? "Either you are not connected to the internet, or the website is encountering problems, or there might be a typing error in the address"?
I should add to this that I see the behavior in FireFox when set to "Work Offline", but not in Microsoft Internet Explorer when set to "Work Offline". In IE, the user gets a dialog giving the option to go online. FireFox does not notify the user before returning the error.
I am asking this in response to a request to "show a better error message". What Internet Explorer does is good. It tells the user what is causing the problem and gives them the option to fix it. In order to give an equivalent UX with FireFox I need to infer the cause of the problem and inform the user. So what in total can I infer from Status 0? Does it have a universal meaning or does it tell me nothing?
Short Answer
It's not a HTTP response code, but it is documented by WhatWG as a valid value for the status attribute of an XMLHttpRequest or a Fetch response.
Broadly speaking, it is a default value used when there is no real HTTP status code to report and/or an error occurred sending the request or receiving the response. Possible scenarios where this is the case include, but are not limited to:
The request hasn't yet been sent, or was aborted.
The browser is still waiting to receive the response status and headers.
The connection dropped during the request.
The request timed out.
The request encountered an infinite redirect loop.
The browser knows the response status, but you're not allowed to access it due to security restrictions related to the Same-origin Policy.
Long Answer
First, to reiterate: 0 is not a HTTP status code. There's a complete list of them in RFC 7231 Section 6.1, that doesn't include 0, and the intro to section 6 states clearly that
The status-code element is a three-digit integer code
which 0 is not.
However, 0 as a value of the .status attribute of an XMLHttpRequest object is documented, although it's a little tricky to track down all the relevant details. We begin at https://xhr.spec.whatwg.org/#the-status-attribute, documenting the .status attribute, which simply states:
The status attribute must return the response’s status.
That may sound vacuous and tautological, but in reality there is information here! Remember that this documentation is talking here about the .response attribute of an XMLHttpRequest, not a response, so this tells us that the definition of the status on an XHR object is deferred to the definition of a response's status in the Fetch spec.
But what response object? What if we haven't actually received a response yet? The inline link on the word "response" takes us to https://xhr.spec.whatwg.org/#response, which explains:
An XMLHttpRequest has an associated response. Unless stated otherwise it is a network error.
So the response whose status we're getting is by default a network error. And by searching for everywhere the phrase "set response to" is used in the XHR spec, we can see that it's set in five places:
To a network error, when:
the open() method is called, or
the response's body's stream is errored (see the algorithm described in the docs for the send() method)
the timed out flag is set, causing the request error steps to run
the abort() method is called, causing the request error steps to run
To the response produced by sending the request using Fetch, by way of either the Fetch process response task (if the XHR request is asychronous) or the Fetch process response end-of-body task (if the XHR request is synchronous).
Looking in the Fetch standard, we can see that:
A network error is a response whose status is always 0
so we can immediately tell that we'll see a status of 0 on an XHR object in any of the cases where the XHR spec says the response should be set to a network error. (Interestingly, this includes the case where the body's stream gets "errored", which the Fetch spec tells us can happen during parsing the body after having received the status - so in theory I suppose it is possible for an XHR object to have its status set to 200, then encounter an out-of-memory error or something while receiving the body and so change its status back to 0.)
We also note in the Fetch standard that a couple of other response types exist whose status is defined to be 0, whose existence relates to cross-origin requests and the same-origin policy:
An opaque filtered response is a filtered response whose ... status is 0...
An opaque-redirect filtered response is a filtered response whose ... status is 0...
(various other details about these two response types omitted).
But beyond these, there are also many cases where the Fetch algorithm (rather than the XHR spec, which we've already looked at) calls for the browser to return a network error! Indeed, the phrase "return a network error" appears 40 times in the Fetch standard. I will not try to list all 40 here, but I note that they include:
The case where the request's scheme is unrecognised (e.g. trying to send a request to madeupscheme://foobar.com)
The wonderfully vague instruction "When in doubt, return a network error." in the algorithms for handling ftp:// and file:// URLs
Infinite redirects: "If request’s redirect count is twenty, return a network error."
A bunch of CORS-related issues, such as "If httpRequest’s response tainting is not "cors" and the cross-origin resource policy check with request and response returns blocked, then return a network error."
Connection failures: "If connection is failure, return a network error."
In other words: whenever something goes wrong other than getting a real HTTP error status code like a 500 or 400 from the server, you end up with a status attribute of 0 on your XHR object or Fetch response object in the browser. The number of possible specific causes enumerated in spec is vast.
Finally: if you're interested in the history of the spec for some reason, note that this answer was completely rewritten in 2020, and that you may be interested in the previous revision of this answer, which parsed essentially the same conclusions out of the older (and much simpler) W3 spec for XHR, before these were replaced by the more modern and more complicated WhatWG specs this answers refers to.
status 0 appear when an ajax call was cancelled before getting the response by refreshing the page or requesting a URL that is unreachable.
this status is not documented but exist over ajax and makeRequest call's from gadget.io.
Know it's an old post. But these issues still exist.
Here are some of my findings on the subject, grossly explained.
"Status" 0 means one of 3 things, as per the XMLHttpRequest spec:
dns name resolution failed (that's for instance when network plug is pulled out)
server did not answer (a.k.a. unreachable or unresponding)
request was aborted because of a CORS issue (abortion is performed by the user-agent and follows a failing OPTIONS pre-flight).
If you want to go further, dive deep into the inners of XMLHttpRequest. I suggest reading the ready-state update sequence ([0,1,2,3,4] is the normal sequence, [0,1,4] corresponds to status 0, [0,1,2,4] means no content sent which may be an error or not). You may also want to attach listeners to the xhr (onreadystatechange, onabort, onerror, ontimeout) to figure out details.
From the spec (XHR Living spec):
const unsigned short UNSENT = 0;
const unsigned short OPENED = 1;
const unsigned short HEADERS_RECEIVED = 2;
const unsigned short LOADING = 3;
const unsigned short DONE = 4;
from documentation http://www.w3.org/TR/XMLHttpRequest/#the-status-attribute
means a request was cancelled before going anywhere
Since iOS 9, you need to add "App Transport Security Settings" to your info.plist file and allow "Allow Arbitrary Loads" before making request to non-secure HTTP web service. I had this issue in one of my app.
Yes, some how the ajax call aborted. The cause may be following.
Before completion of ajax request, user navigated to other page.
Ajax request have timeout.
Server is not able to return any response.