Related
everyone
A few weeks ago I had a problem passing, storing and reading cookies in an app hosted in a different domain of the API. I owned both API and SPA and I could solve the problem by adding 'credentials' both in server and client and allowing CORS for that specific origin.
Although I managed to find a solution, I still don't understand what it means.
What does credentials stand for? What is it meant to solve?
https://developer.mozilla.org/en-US/docs/Web/HTTP/Access_control_CORS
Credentials is a general term that refers to the details about you (or your app) that are used to check that you are who you claim to be when you make a request. A variety of different kinds of credentials could be used, and they could be passed in a variety of ways. For example:
They could be a session token passed as a cookie
They could be a username and password passed using the HTTP BasicAuth mechanism.
They could (in theory) be other things.
In the context if that article, they are referring to all of the different ways that say "I am this person" in a request ... and whether the credentials should be passed around (via the browser requests) between the various hosts / servers involved.
I am working on a web page that uses a lot of AJAX to communicate with the server. The server, in turn, has an extensive REST/JSON API exposing the different operations called by the web client.
This web site is used by both anonymous and authenticated users. As you might expect, the web service calls issued by authenticated users require authentication, and are thus protected from unauthorized users or applications.
However, the web site has a lot of features that require no authentication, and some of these make use of anonymous web services. The only way I am using to prevent outsiders from calling this web services is by using a CSRF token. I know, the CSRF token is not very useful in this regard... with some time in hand, you can figure out how to consume the web services even if they use a CSRF token.
Of course, you can use a CAPTCHA to prevent applications or bots from autonomously using your web service. However, any human will be able to use it.
Sharing a secret key between client and server, on the other side, would be useless. This, because of the ability of any outsider to read it from the web page source code.
I would like to make these web services as difficult to invoke as posible to any 3rd party application. What would you do besides using the CSRF token? It sounds a little stupid, but hey, maybe it is stupid and I am just losing my time.
Note: given this application uses a browser and not an "executable" as the client, this question is irrelevant to the discussion. I cannot use a secret between server and client (not to my knowledge, at least)
I would take a few steps.
Force https on the site. Automatically redirect any incoming http requests to https ones (the RequireHttps attribute is handy for this)
Each page needs to (securely, hence the https) send a one-time use token to the client, to be used for the page. The script running on the client can hold this in the page memory. Any request coming back sends a hashed & salted response, along with the nonce salt. The server can repeat the steps with the saved token + salt and hash to confirm the request. (much like explunit's answer above)
(It's worth noting that the secure request from a client isn't being authenticated from a user account, merely a token sent with the full page.)
The definition for one-time could either be session or page load, depending on your security vs convenience preference. Tokens should be long and expired fairly quickly to frustrate attackers.
The SSL + Hash(token + nonce) should be enough for your needs.
This is interesting. Below is a crazy suggestion. Remember, your question is also equally crazy.
Your website, once opened through a browser, should generate a long polling connection (Comet programing). This will create a unique session between the browser and the server. When ur JS is making the ajax call, send some token (unique token every time) to the server through the long polling thread. Let the AJAX also send the same token. At the server, get the AJAX token and check whether you have a similar token in you long polling session. If yes, fulfill the request. Any coder can break this. But, it won't be easy. Chances are the freeboarders won't even see these second piece of comet code. You can implement the comet code in such a way it is not easy to detect or understand. When they call ur service, send a 'Service Unavailable' message. They will be confused. Also make the comet code https.
You can also check how long that long polling thread is open. If the session was just opened and you get a ajax call right away, you can assume it is a 3rd party call. It depends on ur website flow. If ur Ajax call happens after 1 second of page load, you can check for that pattern on server side.
Anyone coding for your public api, will have 1 to 2 secret checks that they wouldn't even know and even if they know, they might be discouraged by all the extra coding they have to do.
You might have an easier problem than the one described in the linked question since you don't need to distribute a binary to the users. Even if your app is open source, the HMAC/signature key (in the "Request Signatures" part of that answer) can be controlled by an environment/configuration setting.
To summarize:
The secret key is not actually sent between client and server. Rather, it's used to sign the requests
Be sure that the requests include some unique/random element (your CSRF key probably suffices) so that two requests for the same API data are not identical.
Sign the request with the secret key and append the signature to the request. You linked to a PHP question but not clear if what language you're using. In .Net I would use a HMAC class such as HMACSHA256.
On the API server-side use the same HMAC object to verify that the request was signed with the same secret key.
Maybe you could use counters to keep track of conversations. Only the Server and Clients will be able to predict the next iteration in a conversation. This way, I think, you can prevent third party applications to impersonate someone (Just an idea though).
At the beginning, they start talking at some iteration (i=0, for example).
Every time the client requests something, the counter is incremented by some number in both the server side and the client (i=i+some_number).
And, after a few minutes of no communication, they both know they have to reset the counter (i=0).
This is just an idea based on the concept of RSA and also placing Fraud Detection on your system. The Risk from Authorized users is minimal however they can attempt to make anonymous calls to your web-service too.
For UN-Authorised users : For each web-service call , generate a token say using RSA which changes after some time(can be configured say 30 min). This way prediction of code is minimized. I have not heard of RSA collision till now. Send this token back to the user for his browser session. For further security , we might want to attach a session id with RSA token. Since session ids are unique new anonymous calls would require new session id.
Calls can be tracked using Auditing mechanism. Also per-web service there can be a different RSA setup. How the Algorithm for Fraud Detection would work is a challenge by itself.
For Authorized Users :
Every user should be tracked by his IP Address using Header block. The RSA token principle can be applied.
The solution is very vague but worth considering.
Is using sessions in a RESTful API really violating RESTfulness? I have seen many opinions going either direction, but I'm not convinced that sessions are RESTless. From my point of view:
authentication is not prohibited for RESTfulness (otherwise there'd be little use in RESTful services)
authentication is done by sending an authentication token in the request, usually the header
this authentication token needs to be obtained somehow and may be revoked, in which case it needs to be renewed
the authentication token needs to be validated by the server (otherwise it wouldn't be authentication)
So how do sessions violate this?
client-side, sessions are realized using cookies
cookies are simply an extra HTTP header
a session cookie can be obtained and revoked at any time
session cookies can have an infinite life time if need be
the session id (authentication token) is validated server-side
As such, to the client, a session cookie is exactly the same as any other HTTP header based authentication mechanism, except that it uses the Cookie header instead of the Authorization or some other proprietary header. If there was no session attached to the cookie value server-side, why would that make a difference? The server side implementation does not need to concern the client as long as the server behaves RESTful. As such, cookies by themselves should not make an API RESTless, and sessions are simply cookies to the client.
Are my assumptions wrong? What makes session cookies RESTless?
First of all, REST is not a religion and should not be approached as such. While there are advantages to RESTful services, you should only follow the tenets of REST as far as they make sense for your application.
That said, authentication and client side state do not violate REST principles. While REST requires that state transitions be stateless, this is referring to the server itself. At the heart, all of REST is about documents. The idea behind statelessness is that the SERVER is stateless, not the clients. Any client issuing an identical request (same headers, cookies, URI, etc) should be taken to the same place in the application. If the website stored the current location of the user and managed navigation by updating this server side navigation variable, then REST would be violated. Another client with identical request information would be taken to a different location depending on the server-side state.
Google's web services are a fantastic example of a RESTful system. They require an authentication header with the user's authentication key to be passed upon every request. This does violate REST principles slightly, because the server is tracking the state of the authentication key. The state of this key must be maintained and it has some sort of expiration date/time after which it no longer grants access. However, as I mentioned at the top of my post, sacrifices must be made to allow an application to actually work. That said, authentication tokens must be stored in a way that allows all possible clients to continue granting access during their valid times. If one server is managing the state of the authentication key to the point that another load balanced server cannot take over fulfilling requests based on that key, you have started to really violate the principles of REST. Google's services ensure that, at any time, you can take an authentication token you were using on your phone against load balance server A and hit load balance server B from your desktop and still have access to the system and be directed to the same resources if the requests were identical.
What it all boils down to is that you need to make sure your authentication tokens are validated against a backing store of some sort (database, cache, whatever) to ensure that you preserve as many of the REST properties as possible.
I hope all of that made sense. You should also check out the Constraints section of the wikipedia article on Representational State Transfer if you haven't already. It is particularly enlightening with regard to what the tenets of REST are actually arguing for and why.
First, let's define some terms:
RESTful:
One can characterise applications conforming to the REST constraints
described in this section as "RESTful".[15] If a service violates any
of the required constraints, it cannot be considered RESTful.
according to wikipedia.
stateless constraint:
We next add a constraint to the client-server interaction:
communication must be stateless in nature, as in the
client-stateless-server (CSS) style of Section 3.4.3 (Figure 5-3),
such that each request from client to server must contain all of the
information necessary to understand the request, and cannot take
advantage of any stored context on the server. Session state is
therefore kept entirely on the client.
according to the Fielding dissertation.
So server side sessions violate the stateless constraint of REST, and so RESTfulness either.
As such, to the client, a session cookie is exactly the same as any
other HTTP header based authentication mechanism, except that it uses
the Cookie header instead of the Authorization or some other
proprietary header.
By session cookies you store the client state on the server and so your request has a context. Let's try to add a load balancer and another service instance to your system. In this case you have to share the sessions between the service instances. It is hard to maintain and extend such a system, so it scales badly...
In my opinion there is nothing wrong with cookies. The cookie technology is a client side storing mechanism in where the stored data is attached automatically to cookie headers by every request. I don't know of a REST constraint which has problem with that kind of technology. So there is no problem with the technology itself, the problem is with its usage. Fielding wrote a sub-section about why he thinks HTTP cookies are bad.
From my point of view:
authentication is not prohibited for RESTfulness (otherwise there'd be little use in RESTful services)
authentication is done by sending an authentication token in the request, usually the header
this authentication token needs to be obtained somehow and may be revoked, in which case it needs to be renewed
the authentication token needs to be validated by the server (otherwise it wouldn't be authentication)
Your point of view was pretty solid. The only problem was with the concept of creating authentication token on the server. You don't need that part. What you need is storing username and password on the client and send it with every request. You don't need more to do this than HTTP basic auth and an encrypted connection:
Figure 1. - Stateless authentication by trusted clients
You probably need an in-memory auth cache on server side to make things faster, since you have to authenticate every request.
Now this works pretty well by trusted clients written by you, but what about 3rd party clients? They cannot have the username and password and all the permissions of the users. So you have to store separately what permissions a 3rd party client can have by a specific user. So the client developers can register they 3rd party clients, and get an unique API key and the users can allow 3rd party clients to access some part of their permissions. Like reading the name and email address, or listing their friends, etc... After allowing a 3rd party client the server will generate an access token. These access token can be used by the 3rd party client to access the permissions granted by the user, like so:
Figure 2. - Stateless authentication by 3rd party clients
So the 3rd party client can get the access token from a trusted client (or directly from the user). After that it can send a valid request with the API key and access token. This is the most basic 3rd party auth mechanism. You can read more about the implementation details in the documentation of every 3rd party auth system, e.g. OAuth. Of course this can be more complex and more secure, for example you can sign the details of every single request on server side and send the signature along with the request, and so on... The actual solution depends on your application's need.
Cookies are not for authentication. Why reinvent a wheel? HTTP has well-designed authentication mechanisms. If we use cookies, we fall into using HTTP as a transport protocol only, thus we need to create our own signaling system, for example, to tell users that they supplied wrong authentication (using HTTP 401 would be incorrect as we probably wouldn't supply Www-Authenticate to a client, as HTTP specs require :) ). It should also be noted that Set-Cookie is only a recommendation for client. Its contents may be or may not be saved (for example, if cookies are disabled), while Authorization header is sent automatically on every request.
Another point is that, to obtain an authorization cookie, you'll probably want to supply your credentials somewhere first? If so, then wouldn't it be RESTless? Simple example:
You try GET /a without cookie
You get an authorization request somehow
You go and authorize somehow like POST /auth
You get Set-Cookie
You try GET /a with cookie. But does GET /a behave idempotently in this case?
To sum this up, I believe that if we access some resource and we need to authenticate, then we must authenticate on that same resource, not anywhere else.
Actually, RESTfulness only applies to RESOURCES, as indicated by a Universal Resource Identifier. So to even talk about things like headers, cookies, etc. in regards to REST is not really appropriate. REST can work over any protocol, even though it happens to be routinely done over HTTP.
The main determiner is this: if you send a REST call, which is a URI, then once the call makes it successfully to the server, does that URI return the same content, assuming no transitions have been performed (PUT, POST, DELETE)? This test would exclude errors or authentication requests being returned, because in that case, the request has not yet made it to the server, meaning the servlet or application that will return the document corresponding to the given URI.
Likewise, in the case of a POST or PUT, can you send a given URI/payload, and regardless of how many times you send the message, it will always update the same data, so that subsequent GETs will return a consistent result?
REST is about the application data, not about the low-level information required to get that data transferred about.
In the following blog post, Roy Fielding gave a nice summary of the whole REST idea:
http://groups.yahoo.com/neo/groups/rest-discuss/conversations/topics/5841
"A RESTful system progresses from one steady-state to the
next, and each such steady-state is both a potential start-state
and a potential end-state. I.e., a RESTful system is an unknown
number of components obeying a simple set of rules such that they
are always either at REST or transitioning from one RESTful
state to another RESTful state. Each state can be completely
understood by the representation(s) it contains and the set of
transitions that it provides, with the transitions limited to a
uniform set of actions to be understandable. The system may be
a complex state diagram, but each user agent is only able to see
one state at a time (the current steady-state) and thus each
state is simple and can be analyzed independently. A user, OTOH,
is able to create their own transitions at any time (e.g., enter
a URL, select a bookmark, open an editor, etc.)."
Going to the issue of authentication, whether it is accomplished through cookies or headers, as long as the information isn't part of the URI and POST payload, it really has nothing to do with REST at all. So, in regards to being stateless, we are talking about the application data only.
For example, as the user enters data into a GUI screen, the client is keeping track of what fields have been entered, which have not, any required fields that are missing etc. This is all CLIENT CONTEXT, and should not be sent or tracked by the server. What does get sent to the server is the complete set of fields that need to be modified in the IDENTIFIED resource (by the URI), such that a transition occurs in that resource from one RESTful state to another.
So, the client keeps track of what the user is doing, and only sends logically complete state transitions to the server.
As I understand, there are two types of state when we are talking about sessions
Client and Server Interaction State
Resource State
Stateless constraint here refers to the second type in Rest. Using cookies (or local storage) does not violate Rest since it is related to the first.
Fielding says: 'Each request from client to server must contain all of the information necessary to understand the request, and cannot take advantage of any stored context on the server. Session state is therefore kept entirely on the client.'
The thing here is that every request to be fulfilled on the server needs the all necessary data from the client. Then this is considered as stateless. And again, we're not talking about cookies here, we're talking about resources.
HTTP transaction, basic access authentication, is not suitable for RBAC, because basic access authentication uses the encrypted username:password every time to identify, while what is needed in RBAC is the Role the user wants to use for a specific call.
RBAC does not validate permissions on username, but on roles.
You could tric around to concatenate like this: usernameRole:password, but this is bad practice, and it is also inefficient because when a user has more roles, the authentication engine would need to test all roles in concatenation, and that every call again. This would destroy one of the biggest technical advantages of RBAC, namely a very quick authorization-test.
So that problem cannot be solved using basic access authentication.
To solve this problem, session-maintaining is necessary, and that seems, according to some answers, in contradiction with REST.
That is what I like about the answer that REST should not be treated as a religion. In complex business cases, in healthcare, for example, RBAC is absolutely common and necessary. And it would be a pity if they would not be allowed to use REST because all REST-tools designers would treat REST as a religion.
For me there are not many ways to maintain a session over HTTP. One can use cookies, with a sessionId, or a header with a sessionId.
If someone has another idea I will be glad to hear it.
i think token must include all the needed information encoded inside it, which makes authentication by validating the token and decoding the info
https://www.oauth.com/oauth2-servers/access-tokens/self-encoded-access-tokens/
No, using sessions does not necessarily violate RESTfulness. If you adhere to the REST precepts and constraints, then using sessions - to maintain state - will simply be superfluous. After all, RESTfulness requires that the server not maintain state.
Sessions are not RESTless
Do you mean that REST service for http-use only or I got smth wrong? Cookie-based session must be used only for own(!) http-based services! (It could be a problem to work with cookie, e.g. from Mobile/Console/Desktop/etc.)
if you provide RESTful service for 3d party developers, never use cookie-based session, use tokens instead to avoid the problems with security.
I am currently working on the authentication of an AJAX based site, and was wondering if anybody had any reccomendations on best practices for this sort of thing.
My original approach was a cookie based system. Essentially I set a cookie with an auth code, and every data access changed the cookie. As well, whenever there was a failed authentication, all sessions by that user were de-authenticated, to keep hijackers out. To hijack a session, somebody would have to leave themselves logged in, and a hacker would need to have the very last cookie update sent to spoof a session.
Unfortunatley, due to the nature of AJAX, when making multiple requests quickly, they might come back out of order, setting the cookie wrong, and breaking the session, so I need to reimplement.
My ideas were:
A decidedly less secure session based method
using SSL over the whole site (seems like overkill)
Using an iFrame which is ssl authenticated to do secure transactions (I just sorta assume this is possible, with a little bit of jquery hacking)
The issue is not the data being transferred, the only concern is that somebody might get control over an account that is not theirs.
A decidedly less secure session based method
Personally, I have not found using SSL for the entire site (or most of it) to be overkill. Maybe a while ago when speeds and feeds were slower. Now I wouldn't hesitate to put any part of a site under SSL.
If you've decided that using SSL for the entire site is acceptable, you might consider just using the old "Basic Authentication" where the server returns the 401 response which causes the browser to prompt for username/password. If your application can live with this type of login, is works great for AJAX and all other accesses to your site because the browser handles re-submitting requests with appropriate credentials (and it is safe if you use SSL, but only if you use SSL -- don't use Basic auth with plain http!).
SSL is a must, preventing transparent proxy connections that could be used by several users. Then I'd simply check the incoming ip address with the one that got authenticated.
Re-authenticate:
as soon as the ip address changes
on a time out bigger than n seconds without any request
individually on any important transaction
A common solution is to hash the user's session id and pass that in with every request to ensure the request is coming from a valid user (see this slideshow). This is reasonably secure from a CSRF perspective, but if someone was sniffing the data it could be intercepted. Depending on your needs, ssl is always going to be the most secure method.
What if you put a "generated" timestamp on each of the responses from the server and the AJAX application could always use the cookie with the latest timestamp.
Your best bet is using an SSL connection over a previously authenticated connection with something Apache and/or Tomcat. Form based authentication in either one, with a required SSL connection gives you a secure connection. The webapp can then provide security and identity for the session and the client side Ajax doesn't need to be concerned with security.
You might try reading the book Ajax Security,by Billy Hoffman and Bryan Sullivan. I found it changed my way of thinking about security. There are very specific suggestions for each phase of Ajax.
Disclaimer: I'm new to the REST school of thought, and I'm trying to wrap my mind around it.
So, I'm reading this page, Common REST Mistakes, and I've found I'm completely baffled by the section on sessions being irrelevant. This is what the page says:
There should be no need for a client
to "login" or "start a connection."
HTTP authentication is done
automatically on every message. Client
applications are consumers of
resources, not services. Therefore
there is nothing to log in to! Let's
say that you are booking a flight on a
REST web service. You don't create a
new "session" connection to the
service. Rather you ask the "itinerary
creator object" to create you a new
itinerary. You can start filling in
the blanks but then get some totally
different component elsewhere on the
web to fill in some other blanks.
There is no session so there is no
problem of migrating session state
between clients. There is also no
issue of "session affinity" in the
server (though there are still load
balancing issues to continue).
Okay, I get that HTTP authentication is done automatically on every message - but how? Is the username/password sent with every request? Doesn't that just increase attack surface area? I feel like I'm missing part of the puzzle.
Would it be bad to have a REST service, say, /session, that accepts a GET request, where you'd pass in a username/password as part of the request, and returns a session token if the authentication was successful, that could be then passed along with subsequent requests? Does that make sense from a REST point of view, or is that missing the point?
To be RESTful, each HTTP request should carry enough information by itself for its recipient to process it to be in complete harmony with the stateless nature of HTTP.
Okay, I get that HTTP authentication
is done automatically on every message
- but how?
Yes, the username and password is sent with every request. The common methods to do so are basic access authentication and digest access authentication. And yes, an eavesdropper can capture the user's credentials. One would thus encrypt all data sent and received using Transport Layer Security (TLS).
Would it be bad to have a REST
service, say, /session, that accepts a
GET request, where you'd pass in a
username/password as part of the
request, and returns a session token
if the authentication was successful,
that could be then passed along with
subsequent requests? Does that make
sense from a REST point of view, or is
that missing the point?
This would not be RESTful since it carries state but it is however quite common since it's a convenience for users; a user does not have to login each time.
What you describe in a "session token" is commonly referred to as a login cookie. For instance, if you try to login to your Yahoo! account there's a checkbox that says "keep me logged in for 2 weeks". This is essentially saying (in your words) "keep my session token alive for 2 weeks if I login successfully." Web browsers will send such login cookies (and possibly others) with each HTTP request you ask it to make for you.
It is not uncommon for a REST service to require authentication for every HTTP request. For example, Amazon S3 requires that every request have a signature that is derived from the user credentials, the exact request to perform, and the current time. This signature is easy to calculate on the client side, can be quickly verified by the server, and is of limited use to an attacker who intercepts it (since it is based on the current time).
Many people don't understand REST principales very clearly, using a session token doesn't mean always you're stateful, the reason to send username/password with each request is only for authentication and the same for sending a token (generated by login process) just to decide if the client has permission to request data or not, you only violate REST convetions when you use weither username/password or session tokens to decide what data to show !
instead you have to use them only for athentication (to show data or not to show data)
in your case i say YES this is RESTy, but try avoiding using native php sessions in your REST API and start generating your own hashed tokens that expire in determined periode of time!
No, it doesn't miss the point. Google's ClientLogin works in exactly this way with the notable exception that the client is instructed to go to the "/session" using a HTTP 401 response. But this doesn't create a session, it only creates a way for clients to (temporarily) authenticate themselves without passing the credentials in the clear, and for the server to control the validity of these temporary credentials as it sees fit.
Okay, I get that HTTP authentication
is done automatically on every message
- but how?
"Authorization:" HTTP header send by client. Either basic (plain text) or digest.
Would it be bad to have a REST
service, say, /session, that accepts a
GET request, where you'd pass in a
username/password as part of the
request, and returns a session token
if the authentication was successful,
that could be then passed along with
subsequent requests? Does that make
sense from a REST point of view, or is
that missing the point?
The whole idea of session is to make stateful applications using stateless protocol (HTTP) and dumb client (web browser), by maintaining the state on server's side. One of the REST principles is "Every resource is uniquely addressable using a universal syntax for use in hypermedia links". Session variables are something that cannot be accessed via URI. Truly RESTful application would maintain state on client's side, sending all the necessary variables over by HTTP, preferably in the URI.
Example: search with pagination. You'd have URL in form
http://server/search/urlencoded-search-terms/page_num
It's has a lot in common with bookmarkable URLs
I think your suggestion is OK, if you want to control the client session life time. I think that RESTful architecture encourages you to develop stateless applications. As #2pence wrote "each HTTP request should carry enough information by itself for its recipient to process it to be in complete harmony with the stateless nature of HTTP" .
However, not always that is the case, sometimes the application needs to tell when client logs-in or logs-out and to maintain resources such as locks or licenses based on this information. See my follow up question for an example of such case.