I have a TIBCO BW process which starts with the 'HTTP Receiver' Activity.
I'd like to obtain (via a custom Java Code activity) the size of the original HTTP request.
(The purpose is to collect statistics, measure response times, request/response sizes etc.)
I'd like to measure the data in bytes and not in characters, is it possible to get the request
as a byte array, ServletInputStream or something similar?
I don't think you can get the original request from Java code - simply because you cannot access the HTTPServletRequest object of the HTTP Receiver from a Java activity. If you'd like to access the raw request data, you may write a proxy servlet and access BusinessWorks via this proxy servlet.
If your task is only to get stats on the request size, there is a simpler solution. You can use the "Content-length" header parameter for this reason (http://www.w3.org/Protocols/rfc2616/rfc2616-sec14.html).
The Content-Length entity-header field indicates the size of the entity-body,
in decimal number of OCTETs, sent to the recipient
Content-length is an output parameter of the HTTP receiver. You may need to add the length of "RequestURI", "PostData" and "Header" parameters as they are not part of the entity body.
Update: PostData is part of entity-body, therefore its size is included in Content-length
Related
I have a use case where we need to persist Http Response (https://square.github.io/okhttp/4.x/okhttp/okhttp3/-response/) into a Chronicle Queue. Since this response object is coming from OkHttp module, I can't make it Marshallable so that I can write it to a Chronicle Queue.
I only care about the HTTP Status code (Integer), HTTP Header values (Array of String) and ByteStream or byte[]. I like to minimize the number of objects getting created if possible (to reduce GC). If I need to extract data from "Response" object to a custom POJO that is Marshallable, won't that add more objects to be GCed?
This will be a great example as I can think of many applications having similar use cases (of handling the rate of message production/consumption in HTTP applications).
I tried to create a POJO from the OkHttp response object. I am not able to persist the data into the Chronicle Queue as it complaints about java.net classes are not Marshallable.
I am creating an application that allows users to submit JSON or Base64 image data via socket.io
The goal I am trying to achieve is:
if JSON is submitted, the message can have a maximum size of 1MB
if a Base64 image is submitted, the message can have a maximum size of 5MB
From the socket.io docs I can see that:
you can specify a maxHttpBufferSize option value that allows you to limit the maximum message size
namespaces allow you to split logic over a single connection
However, I can't figure out the correct way to get the functionality to work the way I have described above.
Would I need to:
set up 2 separate io instances on the server, one for JSON data and the other for Base64 images (therefore allowing me to set separate maxHttpBufferSize values for each), and then the client can use the correct instance, depending on what they want to submit (if so, what is the correct way of doing this?)
set up 1 instance with a maxHttpBufferSize of 5MB, and then add in my own custom logic to determine message sizes and prevent further actions if the data is JSON and over 1MB in size
set this up in some totally different way that I haven't thought of
Many thanks
From what I can see in the API, maxHttpBufferSize is a parameter for the underlying Engine.IO server (of which there is one instance per Socket.IO Server Instance). Obviously you're free to set up two servers but I doubt it makes sense to separate the system into two entirely different applications.
Talk of using Namespaces to separate logic is more about handling different messages at different endpoints (for example you would register a removeUserFromChat message handler to a user connecting via an /admin namespace, but you wouldn't want to register this to a user connecting via the /user namespace).
In the most recent socket server I set up, I defined my own protocol where part of the response would contain a HTTP status code, as well as a description that could be displayed to the user. For example I would return 200 on success. If I was uploading a file via a REST HTTP Interface, I would expect a 400 (BAD REQUEST) response if my request couldn't be processed - and I believe that this makes sense for your use case. Alternatively you could define your own custom 4XX error code if the file is too large, and handle this in your UI purely based on the code returned. Obviously you don't need to follow the HTTP protocol, and the design decisions are ultimately up to you, but in my opinion it makes sense to return some kind of error response in your message handler.
I suspect that the maxHttpBufferSize has different use at lower levels than your use case. When sending content over network, content is split into 'n bytes' of packets and when a application writes 'n' bytes, the network sends a packet over network (the less the n, more overhead due to network headers. The more the n, high latency because of waiting involved in accumulating n bytes before sending). Documentation is not clear about maxHttpBufferSize but it could be the packet size (n) configuration, not limit on the max data on connection.
It seems, http request header Content-Length might serve your purpose. It gives the actual object size based on that you can make a decision.
I'm building a web-based reporting tool that queries but does not change large amounts of data.
In order to verify the reporting query, I am using a form for input validation.
I know the following about HTTP GET:
It should be used for idempotent requests
Repeated requests may be cached by the browser
What about the following situations?
The data being reported changes every minute and must not be cached?
The query string is very large and greater than the 2000 character URL limit?
I know I can easily just use POST and "break the rules", but are there definitive situations in which POST is recommended for idempotent requests?
Also, I'm submitting the form via AJAX and the framework is Python/Django, but I don't think that should change anything.
I think that using POST for this sort situation is acceptable. Citing the HTTP 1.1 RFC
The action performed by the POST method might not result in a
resource that can be identified by a URI. In this case, either 200
(OK) or 204 (No Content) is the appropriate response status,
depending on whether or not the response includes an entity that
describes the result.
In your case a "search result" resource is created on the server which adheres to the HTTP POST request specification. You can either opt to return the result resource as the response or as a separate URI to the just created resource and may be deleted as the result resource is no longer necessary after one minute's time(i.e as you said data changes every one minute).
The data being reported changes every minute
Every time you make a request, it is going to create a new resource based on your above statement.
Additionally you can return 201 status and a URL to retrieve the search result resource but I m not sure if you want this sort of behavior but I just provided as a side note.
Second part of your first question says results must not be cached. Well this is something you configure on the server to return necessary HTTP headers to force intermediary proxies and clients to not cache the result, for example, with If-Modified-Since, Cache-control etc.
Your second question is already answered as you have to use POST request instead of GET request due to the URL character limit.
All,
I want to send a large-ish JSON string to my server using an Ajax request with POST. I haven't done Ajax+POST before (only GET) so I looked for some info on the web. What confuses me is that some resources say to include those three lines:
request.setRequestHeader("Content-type", "application/x-www-form-urlencoded");
request.setRequestHeader("Content-length", params.length);
request.setRequestHeader("Connection", "close");
Others only say to use the first (Content-type). So, are those 3 necessary? What do they do?
Content-type
This indicates to the server, the type of data you are sending in the request body (ie POST data).
The value in your question application/x-www-form-urlencoded is used for submitting HTML forms, where the input data is sent in key-value form for example:
xmlHttp.send("firstName=bob&lastName=Smith&age=43")
The values should be URL Encoded (aka Percent Encoded) to prevent any special characters breaking the format. If you send this type of data in a POST request without setting the content type, most servers will detect the data type and handle it anyway. It's good practice to set it though (leaves no ambiguity), even though it's not always required.
JSON data can be sent as application/x-www-form-urlencoded, you simply use a key and set the value to the JSON.
myData={"firstName":"bob","age":"43"}
The above JSON should be URL Encoded, in Javascript you can use encodeURIComponent(), I have not encoded it in the example, for the purpose of making it human readable here. This is the most common format for sending POST data and most server side languages will automatically parse the key-value pairs into a native array or construct of some sort.
Alternatively, you can send your JSON literally as the request body, with no key or other data labels. Example:
xmlHttp.send('{"firstName":"bob","age":"43"}');
Using this way, you should set the Content-type to application/json but again, most servers will handle it without that because as far as the HTTP request is concerned, it is just text data. On the server side, you would need to get access to the raw POST data to obtain the JSON. Most server side languages will give you access to the raw post data.
Content-length
As with content-type, it's good practice to set the content-length. It tells the server how much data you are sending and so makes it easier for the server to know when all data is received. That said, it won't usually cause a problem if it's not set, but the W3 docs state that it SHOULD be used.
Connection: close
Sending close for the connection header indicates to the server that you don't support, or don't want to use a persistent connection for the request. The server may respond with a Connection: Keep-alive header, which is not telling the client to keep the connection open, it is just indicating to the client that it supports Persistent Connections. Persistent connections allow the reuse of the same TCP connection for multiple requests, instead of opening and closing for each request. There is no need to set this header in ajax requests. If you don't set it, the browser will decide and will set it itself.
Summary
To summarize, Connection is not needed, Content-length should be used. Content-type should be used but depending on the server and the server side code, it's probably not needed.
Content-type RFC (W3.org)
Content-length RFC (W3.org)
Connection RFC (W3.org)
URL Encoding (Percent Encoding) Wikipedia
HTTP Persistent Connection (Wikipedia)
AJAX on the Mozilla Development Network
You do it when you "open" the request object:
request.open("POST", <url>);
Hi,
I am getting a "400 Bad HTTP request " error when form has large data.
I am using Ajax to make a request. I read that the max limit of the HTTP POST data limit
varies from 2MB to 8MB that depends on the client and Server.But my data is about to 15Kb only.
What I was not able to understand that is there any limit for individual parameters also?
Let's say:
I have two parameters which has to be sent to my Ajax request.
refHttp.open('POST','main.do?param1='+para1+'¶m2='+para2,true);
Data of the each parameter para1 and para2 may be lager(around to 6KB) sometimes.
Is there any restriction on max data size of the individual parameters?
Is there any other alternative to send large data to POST method?
Thanks.
If you send your parameters as part of the url you will suffer that limitation.
In your case, use the send method of the XMLHTTPRequest object, so as it is sent in the body part of the http request:
refHttp.open('POST','main.do',true);
refHttp.send('param1='+para1+'¶m2='+para2);