I need to include the content-length of a image /png file posted in an upload to a webservice.
But how do I calculate the content-length to include in the header?
Thanks.
I am submitting it using rest-client.
The webservice for the upload is Postful: and the documentation has been unclear: http://www.postful.com/developer/guide#uploading_attachments
Because I am writing the payload and headers, seems like I need to input that value.
I am also looking at postalmethods which says that the content-length is the user input:
http://postalmethods.com/method/2009-02-26/UploadFile
The files themselves are .PNG. I am going to attach them to a model using Paperclip, so will have a filepath from that.
The file that I need the content-length to post is stored as an attachment using paperclip, so the specific code generating problems is:
File.size(#postalcard.postalimage.url)
Well, you know how you're reading and posting the data, presumably - so you know how much data you're sending. That's the content length. If you're just sending it directly in binary as the body of the post, it's just the length of the file. If you're base-64 encoding it, then the content length will be the ((file length + 2) / 3) * 4. If it's going in a SOAP envelope etc, you'll need to take account of that.
One way of doing this for complicated situations is to build the entire post body in memory first, set the content length, and then just copy from memory directly to the post body.
Well, you can use File.size(filepath) but it's unlikely that you'll need to - most libraries for making HTTP requests should do that automatically - which library are you using? (Or, what kind of webservice is it?)
Related
I'm trying to learn how to create an API (I use Laravel in the backend and Postman to send requests), but I have a basic doubt when sending data to be processed in the backend.
I see that there are several ways to send data to the backend, but I'm not sure which is the right way to do it.
For example, with Postman I have seen that the sending can be done as parameters through the URI:
www.example.com/api/v1/orders?limit=10&offset=20
I can also do it in the body of the request through the tags
form data
x-www-form-urlencoded
raw
other ...
I understand that I can make the request along with sending data in several ways. I would like to know what should be the correct, standard or optimal way to do it for usual requests such as getting a series of records with a filtering, an order or a pagination.
I would also like to know if the way of sending data should depend on the verb to be used in the request.
My main question/problem is that I would like the way users use the API to be as simple or suitable as possible for them. I'm clear that I want to always return the data (when necessary) in JSON format but I'm not clear on how it should be sent.
Please, could someone clarify these doubts (maybe a link to a page where this kind of doubts are dealt with).
Thank you very much in advance.
It depends:
GET, HEAD and DELETE don't have a request body so all parameters have to be send via URL
POST can be easily sent via form data in Laravel
For PUT/PATCH I prefer application/json because PHP sends it via php://input stream which can have some problems in Laravel sometimes
You can also combine URL parameters and the request body. Compound types (for example models) can only be send as one via request body while it might suffice to send an id via URL parameter.
I guess, nearly more important is the overall format and documentation. The format should be consistent, easy to understand and maybe standardized (for example: https://jsonapi.org/format/#crud).
Keep in mind that forms do two things by default:
Only having methods GET and POST
Only having ectypes application/x-www-form-urlencoded, multipart/form-data and text/plain
If you want to enforce something else, you have to use scripts/libraries to do this.
Nowadays, it appears that JSON content (for POST, PUT, and PATCH) is the most popular and readable. It is well recognizable and clean. Examples in the documentation are easy to read.
I would go for JSON for both, incoming parameters and the outgoing response. This regards parameters related to the business logic of your application.
At the same time, for GET, HEAD, and DELETE methods, you don't have a payload at all. For parameters related to controlling the API (i.e. not strictly related to the business logic of the application, but to the API itself) I'd go for query parameters. This applies to parameters like limit, offset, order_by, etc.
P.S. There is only one caveat related to the JSON format. If your API happens to have file parameters you may face the problem. You can still use JSON format, but in such a case, you should encode your files (e.g. using base64) and put it as string parameters of your JSON. This may be demanding for the consumers of your API ;) This will also enlarge your files and will probably force you to process these files in memory. The alternative is to use multipart/form-data as a request Content-Type - this way you can have both, the form and separate "space" for files. It's worth keeping this case in mind when you decide.
Here is what i am trying to do
Get a file and encode it to base 64
Login to API and get OAUTH token for subsequent API calls (Since Invoke HTTP doesnt take a body so i have to pass a body using GenerateFlow processor before that)
Both Steps 1 and 2 are interchangeable
Now i need the token from the login call as Header value and Base64 encoded file as the body to the next Invoke Processor.
The problem is i am not able to connect the flows 1 and 2 so i can take them to step 3 where i need to invoke another API using the data from the 2. Both GetFile and GenerateFlow are source processors.
Here is what i am thinking, to somehow convert the content to attribute. Since the file is encoded in base64, is there a way i can convert that content to attribute? i am trying to but the content is just random text, i am not sure what param to look for or what i need to put that to. is there a way to say convert whatever is in the content to an attribute say filecontent="all the flow-content"
nifi-flow-snapshot
Base 64 encoded content
First trigger both flow fro mthe same generateflowfie >> updateattribute(create common attribute to be used in merge correlation).
Do your work
Use MergeContent - keep all unique attribute - Set correlaion using the common attribute set in out first step.
This is how i resolved my issue for now. There may be other better ways but this one worked.
Start the flow with
GetFile>>Base64Encode>>ExtractText>>ReplaceText>>Login>>continue
with rest of the flow
The trick that work was to place the ReplaceText so it can grab the entire base64 encoded content and place it in an attribute. then ReplaceText will replace the entire content with the body of the login processor.
Now i have both login token and base 64 encoded file which i can send to my next processors.
Thanks
I am using URLSession 'uploadTask from file'
func uploadTask(with request: URLRequest, fromFile fileURL: URL) -> URLSessionUploadTask
Almost everything works fine, but now our server needs an extra param as 'uploadKey' to be passed as content disposition along with fileName.
This can be done by generating multipart request with content disposition added as we normally do.
I want to add it while using 'uploadTask from file' to avoid memory pressure. Please suggest how to do it.
From reading the question, I suspect that you're subtly misunderstanding what upload tasks do (and unfortunately, Apple's documentation needs some serious improvement in this area, which doesn't help matters). These tasks don't upload a file in the same way that a web browser would if you chose a file in an upload form. Rather, they use the file as the body of an upload request. I think they default to providing a sane Content-Type based on the filename, though I'm not certain, but they do not send data in form encoding.
So assuming I'm fully understanding the question, your options are either:
Keep using multipart encoding. Optionally write the multipart body into a file rather than keeping it in memory, and use the upload task to provide the body from that file instead of from an NSData object.
Upload the file you're trying to send, unencoded, as the entirety of the upload body, and provide whatever additional parameters you need to provide in the form of GET parameters in the URL itself.
Use some other encoding like JSON or protocol buffers.
Either way, the server code will determine which of these approaches is supported. If you can modify the server code, then I would recommend the second approach. It is slightly more efficient than the first approach, much more efficient than JSON, and easier to implement than any of the other approaches.
We have some publicly shared S3 files that we want to make sure won't be indexed by Google. I can't seem to find any documentation on how to do this. Is there a way to set a "noindex" x-robots-tag response header on individual S3 objects?
(We're using the Ruby AWS client)
There does not appear to be a way to do this.
Only certain headers from an S3 PUT object request are documented as being returned when the object is fetched.
http://docs.aws.amazon.com/AmazonS3/latest/API/RESTObjectPUT.html
Anything else you send appears to be simply disregarded, as long as it doesn't actually invalidate the request.
Actually, that's what I thought before researching this, and it's almost true.
The documentation here seems incomplete, and elsewhere suggests the following request headers, if sent with the upload, will appear in the download:
Cache-Control
Content-Disposition
Content-Encoding
Content-Type
x-amz-meta-*
Other headers are listed at the latter link, but some of these like Expect wouldn't make sense on a GET request, so they logically wouldn't appear.
So far, this is all consistent with my experience with S3.
If you send a random but not-invalid header with your request, it's ignored. Example:
X-Foo: bar
S3 seems to accepts this on upload, but discards it (presumably doesn't store it)... downloading the object does not return the X-Foo header.
But X-Robots-Tag appears to be an undocumented exception to this.
Uploading a file with X-Robots-Tag: noindex (for example) does indeed result in the same header and value being returned with the object when you GET it.
Unless somebody can cite the documentation that explains why this works, we're operating in distinctly undocumented territory.
But, if you're interested in going there, the simple answer appears to be, you just add this header to the HTTP PUT request you send to the REST API to upload the object.
"Not so fast," you say, "I'm using the Ruby SDK." Indeed. The AWS Ruby client seems to be too "helpful" to let you get away with this, at least, not easily. The docs there show how to add "metadata" --
:metadata (Hash) — A hash of metadata to be included with the object. These will be sent to S3 as headers prefixed with x-amz-meta. Each name, value pair must conform to US-ASCII.
Well, that's not going to work, because you'd get x-amz-meta-x-robots-tag.
How do you set other headers in the upload? Every other header you'd normally set is an element of the options hash, like :cache_control, which turns into Cache-Control: in the upload request. Unless they're blindly applying the keys from that hash to the upload transaction (which would be terrible design combined with excellent luck) then you may not have a straightforward way to get here from there. I can't be much more specific, because the only I really know about Ruby is the same thing I know about Java -- from what I've seen of it, I don't like it. :)
But X-Robots-Tag does appear to be a custom header S3 supports, to some extent, without clear documentation of that fact. It's, at least, accepted by the REST API.
Failing the above, you can manually add this header to the metadata in the S3 console after uploading the object. (Note, X-Foo: Bar doesn't work from the S3 console, either -- it's silently discarded, with no error -- but X-Robots-Tag: works fine).
You can also, of course, put a publicly-readable robots.txt file (with the appropriate directives in it) in the root of the bucket. Depending on your cobtent mix, path hierarchy, and other factors, that isn't (perhaps) as simple as selectively setting headers, but if the entire bucket is comprised of information you don't want indexed, it should easily accomplish what you want, since content should not be indexed if disallowed in robots.txt, even when a search spider follows a link to it from another site -- every domain (and subdomain)'s robots.txt file stands alone.
#Michael - sqlbot is correct. The SDKs don't support it by default and it won't show in the AWS Console, but if you set it directly with the REST API it works. For those who don't want to figure out the REST API and its authentication method, I was able to modify the node.js aws-sdk to support this feature.
Amazon stores the method params configuration and validation in a large json file: apis/s3-2006-03-01.min.json . I guess that the other SDKs may implement their validation in the same way.
You can go to the "PutObject" command, and under "input.members" you can add a new parameter "XRobotsTag". Configure it as a "header" and set the location to "X-Robots-Tag".
"XRobotsTag": {
"location": "header",
"locationName": "X-Robots-Tag"
}
Your local aws-sdk is now configured to support X-Robots-Tag on your putObject requests. In node.js this would look like this:
s3.putObject({
ACL: "public-read",
Body: "hello world",
Bucket: "my-bucket",
CacheControl: "public, max-age=31536000",
ContentType: "text/plain",
Key: "hello.txt",
XRobotsTag: "noindex, nofollow"
}, function(err, resp){});
I'm attempting to simulate a login call with JMeter 2.11 to a service that uses a binary format. I've created an Http Request with the appropriate settings, except for the body data. I need to POST raw binary data.
According to the docs here, I should be able to set the file path for exactly one file, with no parameter name, and no other content in the Body Data, and have it place the data in the request body.
If it is a POST or PUT or PATCH request and there is a single file whose 'Parameter name' attribute (below) is omitted, then the file is sent as the entire body of the request, i.e. no wrappers are added. This allows arbitrary bodies to be sent. This functionality is present for POST requests after version 2.2, and also for PUT requests after version 2.3.
However, when I run the test, the POST Data is empty.
I have tried the extra set of plugins for JMeter as well, but alas, I'm stuck. The loaded file has 145 bytes of data, and the request shows that the content-length is 0. What am I missing?
The Http Request
The result
Update 1
To clarify, I am NOT attempting to send a file, I'm attempting to send a binary encoded message as raw POST data.
Switch back to Parameters tab not Post body.
See:
http://jmeter.apache.org/usermanual/component_reference.html#HTTP_Request
Yoy could try recoring to see how the resuest look like.
This is my solution,maybe not best, but it works fine:
1st step :
You should write your binary data to a file (assume it's name is
FILENAME);
2nd step :
For your http request sampler,Yout should put ${FILENAME} under file
path in the "Send Files with the request" section (while leaving its
paramter name empty and specifying an encoding (for binary, it is
application/binary)).
Hope it helps.
Refer to this article