I am using URLSession 'uploadTask from file'
func uploadTask(with request: URLRequest, fromFile fileURL: URL) -> URLSessionUploadTask
Almost everything works fine, but now our server needs an extra param as 'uploadKey' to be passed as content disposition along with fileName.
This can be done by generating multipart request with content disposition added as we normally do.
I want to add it while using 'uploadTask from file' to avoid memory pressure. Please suggest how to do it.
From reading the question, I suspect that you're subtly misunderstanding what upload tasks do (and unfortunately, Apple's documentation needs some serious improvement in this area, which doesn't help matters). These tasks don't upload a file in the same way that a web browser would if you chose a file in an upload form. Rather, they use the file as the body of an upload request. I think they default to providing a sane Content-Type based on the filename, though I'm not certain, but they do not send data in form encoding.
So assuming I'm fully understanding the question, your options are either:
Keep using multipart encoding. Optionally write the multipart body into a file rather than keeping it in memory, and use the upload task to provide the body from that file instead of from an NSData object.
Upload the file you're trying to send, unencoded, as the entirety of the upload body, and provide whatever additional parameters you need to provide in the form of GET parameters in the URL itself.
Use some other encoding like JSON or protocol buffers.
Either way, the server code will determine which of these approaches is supported. If you can modify the server code, then I would recommend the second approach. It is slightly more efficient than the first approach, much more efficient than JSON, and easier to implement than any of the other approaches.
Related
I'm trying to learn how to create an API (I use Laravel in the backend and Postman to send requests), but I have a basic doubt when sending data to be processed in the backend.
I see that there are several ways to send data to the backend, but I'm not sure which is the right way to do it.
For example, with Postman I have seen that the sending can be done as parameters through the URI:
www.example.com/api/v1/orders?limit=10&offset=20
I can also do it in the body of the request through the tags
form data
x-www-form-urlencoded
raw
other ...
I understand that I can make the request along with sending data in several ways. I would like to know what should be the correct, standard or optimal way to do it for usual requests such as getting a series of records with a filtering, an order or a pagination.
I would also like to know if the way of sending data should depend on the verb to be used in the request.
My main question/problem is that I would like the way users use the API to be as simple or suitable as possible for them. I'm clear that I want to always return the data (when necessary) in JSON format but I'm not clear on how it should be sent.
Please, could someone clarify these doubts (maybe a link to a page where this kind of doubts are dealt with).
Thank you very much in advance.
It depends:
GET, HEAD and DELETE don't have a request body so all parameters have to be send via URL
POST can be easily sent via form data in Laravel
For PUT/PATCH I prefer application/json because PHP sends it via php://input stream which can have some problems in Laravel sometimes
You can also combine URL parameters and the request body. Compound types (for example models) can only be send as one via request body while it might suffice to send an id via URL parameter.
I guess, nearly more important is the overall format and documentation. The format should be consistent, easy to understand and maybe standardized (for example: https://jsonapi.org/format/#crud).
Keep in mind that forms do two things by default:
Only having methods GET and POST
Only having ectypes application/x-www-form-urlencoded, multipart/form-data and text/plain
If you want to enforce something else, you have to use scripts/libraries to do this.
Nowadays, it appears that JSON content (for POST, PUT, and PATCH) is the most popular and readable. It is well recognizable and clean. Examples in the documentation are easy to read.
I would go for JSON for both, incoming parameters and the outgoing response. This regards parameters related to the business logic of your application.
At the same time, for GET, HEAD, and DELETE methods, you don't have a payload at all. For parameters related to controlling the API (i.e. not strictly related to the business logic of the application, but to the API itself) I'd go for query parameters. This applies to parameters like limit, offset, order_by, etc.
P.S. There is only one caveat related to the JSON format. If your API happens to have file parameters you may face the problem. You can still use JSON format, but in such a case, you should encode your files (e.g. using base64) and put it as string parameters of your JSON. This may be demanding for the consumers of your API ;) This will also enlarge your files and will probably force you to process these files in memory. The alternative is to use multipart/form-data as a request Content-Type - this way you can have both, the form and separate "space" for files. It's worth keeping this case in mind when you decide.
We have some publicly shared S3 files that we want to make sure won't be indexed by Google. I can't seem to find any documentation on how to do this. Is there a way to set a "noindex" x-robots-tag response header on individual S3 objects?
(We're using the Ruby AWS client)
There does not appear to be a way to do this.
Only certain headers from an S3 PUT object request are documented as being returned when the object is fetched.
http://docs.aws.amazon.com/AmazonS3/latest/API/RESTObjectPUT.html
Anything else you send appears to be simply disregarded, as long as it doesn't actually invalidate the request.
Actually, that's what I thought before researching this, and it's almost true.
The documentation here seems incomplete, and elsewhere suggests the following request headers, if sent with the upload, will appear in the download:
Cache-Control
Content-Disposition
Content-Encoding
Content-Type
x-amz-meta-*
Other headers are listed at the latter link, but some of these like Expect wouldn't make sense on a GET request, so they logically wouldn't appear.
So far, this is all consistent with my experience with S3.
If you send a random but not-invalid header with your request, it's ignored. Example:
X-Foo: bar
S3 seems to accepts this on upload, but discards it (presumably doesn't store it)... downloading the object does not return the X-Foo header.
But X-Robots-Tag appears to be an undocumented exception to this.
Uploading a file with X-Robots-Tag: noindex (for example) does indeed result in the same header and value being returned with the object when you GET it.
Unless somebody can cite the documentation that explains why this works, we're operating in distinctly undocumented territory.
But, if you're interested in going there, the simple answer appears to be, you just add this header to the HTTP PUT request you send to the REST API to upload the object.
"Not so fast," you say, "I'm using the Ruby SDK." Indeed. The AWS Ruby client seems to be too "helpful" to let you get away with this, at least, not easily. The docs there show how to add "metadata" --
:metadata (Hash) — A hash of metadata to be included with the object. These will be sent to S3 as headers prefixed with x-amz-meta. Each name, value pair must conform to US-ASCII.
Well, that's not going to work, because you'd get x-amz-meta-x-robots-tag.
How do you set other headers in the upload? Every other header you'd normally set is an element of the options hash, like :cache_control, which turns into Cache-Control: in the upload request. Unless they're blindly applying the keys from that hash to the upload transaction (which would be terrible design combined with excellent luck) then you may not have a straightforward way to get here from there. I can't be much more specific, because the only I really know about Ruby is the same thing I know about Java -- from what I've seen of it, I don't like it. :)
But X-Robots-Tag does appear to be a custom header S3 supports, to some extent, without clear documentation of that fact. It's, at least, accepted by the REST API.
Failing the above, you can manually add this header to the metadata in the S3 console after uploading the object. (Note, X-Foo: Bar doesn't work from the S3 console, either -- it's silently discarded, with no error -- but X-Robots-Tag: works fine).
You can also, of course, put a publicly-readable robots.txt file (with the appropriate directives in it) in the root of the bucket. Depending on your cobtent mix, path hierarchy, and other factors, that isn't (perhaps) as simple as selectively setting headers, but if the entire bucket is comprised of information you don't want indexed, it should easily accomplish what you want, since content should not be indexed if disallowed in robots.txt, even when a search spider follows a link to it from another site -- every domain (and subdomain)'s robots.txt file stands alone.
#Michael - sqlbot is correct. The SDKs don't support it by default and it won't show in the AWS Console, but if you set it directly with the REST API it works. For those who don't want to figure out the REST API and its authentication method, I was able to modify the node.js aws-sdk to support this feature.
Amazon stores the method params configuration and validation in a large json file: apis/s3-2006-03-01.min.json . I guess that the other SDKs may implement their validation in the same way.
You can go to the "PutObject" command, and under "input.members" you can add a new parameter "XRobotsTag". Configure it as a "header" and set the location to "X-Robots-Tag".
"XRobotsTag": {
"location": "header",
"locationName": "X-Robots-Tag"
}
Your local aws-sdk is now configured to support X-Robots-Tag on your putObject requests. In node.js this would look like this:
s3.putObject({
ACL: "public-read",
Body: "hello world",
Bucket: "my-bucket",
CacheControl: "public, max-age=31536000",
ContentType: "text/plain",
Key: "hello.txt",
XRobotsTag: "noindex, nofollow"
}, function(err, resp){});
I need to include the content-length of a image /png file posted in an upload to a webservice.
But how do I calculate the content-length to include in the header?
Thanks.
I am submitting it using rest-client.
The webservice for the upload is Postful: and the documentation has been unclear: http://www.postful.com/developer/guide#uploading_attachments
Because I am writing the payload and headers, seems like I need to input that value.
I am also looking at postalmethods which says that the content-length is the user input:
http://postalmethods.com/method/2009-02-26/UploadFile
The files themselves are .PNG. I am going to attach them to a model using Paperclip, so will have a filepath from that.
The file that I need the content-length to post is stored as an attachment using paperclip, so the specific code generating problems is:
File.size(#postalcard.postalimage.url)
Well, you know how you're reading and posting the data, presumably - so you know how much data you're sending. That's the content length. If you're just sending it directly in binary as the body of the post, it's just the length of the file. If you're base-64 encoding it, then the content length will be the ((file length + 2) / 3) * 4. If it's going in a SOAP envelope etc, you'll need to take account of that.
One way of doing this for complicated situations is to build the entire post body in memory first, set the content length, and then just copy from memory directly to the post body.
Well, you can use File.size(filepath) but it's unlikely that you'll need to - most libraries for making HTTP requests should do that automatically - which library are you using? (Or, what kind of webservice is it?)
What is the difference between GET and POST for Ajax requests?
I don't see any difference between those two, except that when I use GET, the parameters are send in URL, which for me don't really make any difference, since all requests are made on background and user doesn't find any difference.
edit:
What are PUT and DELETE methods used for?
GET is designed for getting data from the server. POST (and lesser-known friends PUT and DELETE) are designed for modifying data on the server.
A GET request should never cause data to be removed from an application. If you have a link you can click on with a GET to remove data, then Google spidering your site could click on all your "Delete" links.
The canonical answer can be found here, which quotes the HTML 2.0 spec:
If the processing of a form is idempotent (i.e. it has no lasting
observable effect on the state of the
world), then the form method should be
GET. Many database searches have no
visible side-effects and make ideal
applications of query forms.
If the service associated with the processing of a form has side effects
(for example, modification of a
database or subscription to a
service), the method should be POST.
In your AJAX call, you need to use whatever method your server supports. You should always design your server so that operations that modify data are called by POST/PUT/DELETE. Other comments have links to REST, which generally maps C/R/U/D to "POST or PUT"(Create)/GET(Read)/PUT(Update)/DELETE(Delete).
If you're sending large amounts of data, or sensitive data over HTTPS, you will want to use POST. If it's just a simple parameter, I would use GET.
GET requests have a limit to the amount of data that can be sent. I forget the exact number, but this can cause issues if you're sending anything substantial.
Basically the difference between GET and POST is that in a GET request, the parameters are passed in the URL where as in a POST, the parameters are included in the message body.
Whether its AJAX or not is irrelevant. Its about the action that you're taking. I'd recommend following the principles of REST. Which have further provisions for updating, deleting, etc...
GET requests are easier to exploit in CSRF (cross site request forgery) attacks. Namely fake POST requests require Javascript to be enabled on the user side, while fake GET requests are still possible just with img, script tags.
Many web servers limit the length of the data that can be passed as part of the URL, so the GET request may break in odd ways that are hard to debug.
Also, most server software logs URLs in the access logs, so if you pass sensitive information (such as passwords) in a GET request, this will in all likelihood be written to disk in plaintext.
From a REST perspective, GET requests should have no side-effects -- they shouldn't modify data. So, if you're just GETting a resource by ID, this makes sense, but if you're committing changes to a resource, you should be using PUT, POST, or UPDATE for the http verb.
Both are used to send some data and receive some response using that data.
GET: Get information store in server. Ie. Search, tweet, Person Information. If you want to send information then get request send request using process.php?name=subroto
So it basically send information through url. Url cannot handle more than 2083 char. So for blog post can you remember it is not possible?
POST: Post do same thing as get. User registration, User login, Big data send, Blog Post.
If you need to send secure information then use post or for big data as it not go through url.
AJAX: $.get() and $.post() contain features that are subsets of $.ajax(). It has much configuration.
$.get () method, which is a kind of shorthand for $.Ajax (). When using $.get (), instead of passing in an object, you pass in arguments. At minimum, you’ll need the first two arguments, which are the URL of the file you want to retrieve (i.e. ‘test.txt’) and a success callback.
Summary:
$.get( url [, data ] [, success ] [, dataType ] )
$.post( url [, data ] [, success ] [, dataType ] ) // for sending secure or Large information
$.ajax( url [, settings ] ) // More Configaration
First, general information. Use GET if you only read data, use POST if you change something on database, txt files etc.
But the problem is, some browsers cache GET results. I had problems with AJAX requests in IE7, but at last I found out that browser caches GET results. I rethought the flow and changes my request to POST.
So, don't use GET if you don't want caching.
(Of course you can disable caching in GET operations. But I didn't prefer it)
About me, i prefer POST. I reserve get to the events i know the sent value is limited to data i have the "control", for example, to retreive an item with an id. Example, "getitem?id=123", "deleteImtem?id=123", ... For the other cases, when i have a form fillable by a user, i prefer POST.
Like Ryan Smith have said, it's better to use POST to send a large amount of data, and less wories in cases of the use in others language/special chars (generally all majors javascript framework should'nt have any problems to deal with that but i think is less wories to use POST).
For the REST perspective, in my opinion, you can use this with a new project (to keep a consistency with the entire project).
Finally, maybee some programs used in a network (URL loguers (ie.: to see if the employees lost their time on non-autorised sites, ...) proxys, ... ) or any other kind of tool can intercept the query. Somes will show in the reports the params you have sent with GET, considering it like a different web page. But in this situation, is could be not your problem it's changes from a project to an other! ;)
The difference is the same between GET and POST whether you're using Ajax, HTML forms, or curl. Here are the relevant definitions:
GET
POST
If you are passing on any arguments with characters that can get messed up in the URL (such as spaces), you use POST. Otherwise you can use GET.
Generally, if you're just passing on a few tiny arguments you would use GET. But for passing on user submitted information such as blog entries, text, etc, its a good practice to use POST.
There are also certain frameworks that rely completely on segment based urls (such as site.com/products/133 rather than site.com/products.php?id=333 and these frameworks unset the GET variables for security. In such cases you would use POST allt the time.
I am using jQuery to call PHP files via the $.get method
function fetchDepartment(company_id)
{
$.get("ajax/fetchDepartment.php?sec=departments&company_id="+company_id, function(data){
$("#department_id").html(data);
});
}
What I am thinking is can I secure the filename even further?
Currently I have a global access check within the .php file that check if the user is logged in, if he can access this data etc.
But I am wondering if there are further steps I can take so a user couldn't see this filename, or what other steps you recommend to take.
Encoded requests
You could make the request details effectively invisible to the casual miscreant by encoding almost all of the URL and then decoding the request details server-side.
The request details would include the action you wish to perform plus the parameters relevant to that action.
All requests would be sent to a single URL, where a server-side process would decode the request details and perform the relevant action as required.
Example Original URL:
/ajax/delete.php?parameter1=foo¶meter2=bar
Request details:
action=delete¶meter1=foo¶meter2=bar
Encoded request details (encoded using base64):
YWN0aW9uPWRlbGV0ZSZwYXJhbWV0ZXIxPWZvbyZwYXJhbWV0ZXIyPWJhcg==
Encoded URL:
/ajax/?request=YWN0aW9uPWRlbGV0ZSZwYXJhbWV0ZXIxPWZvbyZwYXJhbWV0ZXIyPWJhcg==
I don't believe there is native functionality to encode to base64 in JavaScript, but it's far from impossible to find a suitable method or to write your own.
With obfuscated/minified client-side JavaScript it would be quite tricky for someone to determine how to make a request 'by hand'.
Hide implementation details
There are a number of practices you can follow to make your application less susceptible to attack through URL misuse.
Let's start with a URL of: ajax/fetchDepartment.php?sec=departments&company_id=99
There's no need to reveal what server-side technology you're using (PHP) nor, through the query string (sec, company_id), what the query string values actually mean.
Masking the server-side technology
Assuming you have index.php defined as a default, the following URLs are equivalent:
ajax/fetchDepartment.php?sec=departments&company_id=99
ajax/fetchDepartment/index.php?sec=departments&company_id=99
ajax/fetchDepartment/?sec=departments&company_id=99
The third URL does not reveal the server-side technology you're using. This limits the range of possible attacks. It also makes it easier for you to switch over to a different server-side technology without changing your URLs.
Hiding the meaning of request parameters
ajax/fetchDepartment/?sec=departments&company_id=99
ajax/99/departments/
The latter URL still conveys enough information to perform the request without revealing what the information means.
Whilst someone could still change the values, they won't know what they're changing. This will make it more difficult for an attacker to evaluate and understand the result of any URL changes they make.
Pretty much the only way you can obscure the URL for a certain piece of information from the user is by not loading it in through http. Maybe you can load a set of data on the calling page, or another page with a more generic url.