Are there any examples of using encryption to encrypt the disk-cache used by OkHttp's HttpResponseCache? Naively, I don't think this is a very hard thing to do, but I'd appreciate any advice or experience to avoid security-pitfalls.
Without too many specifics, here's what I'm trying to achieve: a server that accept user's api-keys (typically 40-character random string) for established service X, and makes many API calls on the users behalf. The server won't persist user's api-keys, but a likely use case is that users will periodically call the server, supplying the api-key each time. Established service X uses reasonable rate-limiting, but supports conditional (ETag, If-Modified-Since) requests, so server-side caching by my server makes sense. The information is private though, and the server will be hosted on Heroku or the like, so I'd like to encrypt the files cached by HttpResponseCache so that if the machine is compromised, they don't yield any information.
My plan would be to create a wrapper around HttpResponseCache that accepts a secret key - which would actually be a hash of half of the api-key string. This would be used to AES-encrypt the cached contents and keys used by HttpResponseCache. Does that sound reasonable?
Very difficult to do with the existing cache code. It's a journaled on-disk datastructure that is not designed to support privacy, and privacy is not a feature you can add on top.
One option is to mount an encrypted disk image and put the cache in there. Similar to Mac OS X's FileVault for example. If you can figure out how to do that, you're golden.
Your other option is to implement your own cache, using the existing cache as a guide. Fair warning: the OkResponseCache is subject to change in the next release!
Related
I am writing a WP7 app and would like to include features to share highscore data using Amazon's AWS as storage service.
As far as I understand WP7 XAP files are (currently) safely encrypted and no known jailbreak for the phone exists. However, given that such a 'safe' encryption can be temporary, I would like to understand if/how this violates best practice.
AWS' dynamoDB uses temporary access tokens that can be generated using given account data and are valid for 36 hours the tokens must be verified using a signature with any request.
I am considering that all access data will be stored in the XAP file, which will also generate the temporary access token and signature. The information will be passed via https requests between the phone and AWS.
I was trying to work out alternative processes including passing the generation of the temporary token calculation to an external webservice, however I cannot think of a way to protect this data which would not be similarly compromised if the XAP file was accessible.
Am I missing the best practice approach completely or am I just overly cautious?
Thanks.
You won't ever be able to prevent users from sending false scores, pretty much for the same reason as unofficial cheating apps exist for every popular game. The best you can do is making it harder.
With a simple approach, the client sends the score directly to the server, without any kind of encryption. Someone can cheat just by running the app on the emulator and capturing the outgoing packets, then opening the same URL on his desktop browser. Estimated time: less than 10 minutes, and it can be done by anyone who knows that he can download XAPs directly from the marketplace, remove the manifest, and deploy it on the emulator.
Then you can add an encryption key on the client. Now someone has to know C# and Reflector to extract it, but it's still easy for someone having those skills.
Next level, you can add an encryption key AND obfuscate the assembly. Knowledge of CIL and Relector are required to extract the key. It'll take 30 minutes to an hour to a highly skilled developper to extract the key, and many hours for most developpers.
Finally, you can add multiple steps to confuse even more the intruder (for instance, downloading a temporary token from a server and using it somehow in the score sending process). Also, you can design the scoring system in a way that some scores are illegal (dumb example: if the minimal scoring action earns 2 points, then if someone sends an odd number as a score you know he's cheating. This one is easy to figure out, but you can make much more complex rules).
In any way, keep in mind that your system will always be vulnerable, it's only a matter of how much time it will take to the attacker to break through it. If it takes many hours or days to a highly skilled developer, then unless you're offering some worthy prize to the best player, you can safely assume that nobody will bother doing that.
I would like to make a LDAP cache with the following goals
Decrease connection attempt to the ldap server
Read local cache if entry is exist and it is valid in the cache
Fetch from ldap if there is no such request before or the entry in the cache is invalid
Current i am using unboundid LDAP SDK to query LDAP and it works.
After doing some research, i found a persistent search example that may works. Updated entry in the ldap server will pass the entry to searchEntryReturned so that cache updating is possible.
https://code.google.com/p/ldap-sample-code/source/browse/trunk/src/main/java/samplecode/PersistentSearchExample.java
http://www.unboundid.com/products/ldapsdk/docs/javadoc/com/unboundid/ldap/sdk/AsyncSearchResultListener.html
But i am not sure how to do this since it is async or is there a better way to implement to cache ? Example and ideas is greatly welcomed.
Ldap server is Apache DS and it supports persistent search.
The program is a JSF2 application.
I believe that Apache DS supports the use of the content synchronization controls as defined in RFC 4533. These controls may be used to implement a kind of replication or data synchronization between systems, and caching is a somewhat common use of that. The UnboundID LDAP SDK supports these controls (http://www.unboundid.com/products/ldap-sdk/docs/javadoc/index.html?com/unboundid/ldap/sdk/controls/ContentSyncRequestControl.html). I'd recommend looking at those controls and the information contained in RFC 4533 to determine whether that might be more appropriate.
Another approach might be to see if Apache DS supports an LDAP changelog (e.g., in the format described in draft-good-ldap-changelog). This allows you to retrieve information about entries that have changed so that they can be updated in your local copy. By periodically polling the changelog to look for new changes, you can consume information about changes at your own pace (including those which might have been made while your application was offline).
Although persistent search may work in your case, there are a few issues that might make it problematic. The first is that you don't get any control over the rate at which updated entries are sent to your client, and if the server can apply changes faster than the client can consume them, then this can overwhelm the client (which has been observed in a number of real-world cases). The second is that a persistent search will let you know what entries were updated, but not what changes were made to them. In the case of a cache, this may not have a huge impact because you'll just replace your copy of the entire entry, but it's less desirable in other cases. Another big problem is that a persistent search will only return information about entries updated while the search was active. If your client is shut down or the connection becomes invalid for some reason, then there's no easy way to get information about any changes while the client was in that state.
Client-side caching is generally a bad thing, for many reasons. It can serve stale data to applications, which has the potential to cause incorrect behavior or in some cases pose a security risk, and it's absolutely a huge security risk if you're using it for authentication. It could also pose a security risk if not all of the clients have the same level of access to the data contained in the cache. Further, implementing a cache for each client application isn't a scalable solution, and if you were to try to share a cache across multiple applications, then you might as well just make it a full directory server instance. It's much better to use a server that can simply handle the desired load without the need for any additional caching.
If I built an application that accessed some of the data from say Gmail, Twitter and Facebook, and I want the user to be able to only have to enter their authentication info once, and it's reset after some days or weeks, what is the best way to do this, dynamically, in Ruby?
I see a lot of people just having a config file of their clients'/users' credentials like so:
gmail_account:
username: myClient
password: myClientsPassword
This seems a) like it's very insecure, and b) it wouldn't work if I wanted to store this kind of information for thousands of users. What is the recommended way to do this?
I would like to be able to build an interface on top of these services, so having to enter credentials every time the user made a transaction isn't feasible.
If you're comforatable with the potential liability when a hacker gets into your database / filesystem, then go for it. And in all fairness, you should also disclose to your users that their passwords will be stored on your system, and let them decide if they want to give your program that level of trust.
But why do this in the first place? Facebook Connect and Twitter & Google using OAuth there's no need for you to store user passwords at all. At some point a user's cookies will expire (or they'll try to access your site from another computer) and they'll have to re-authenticate. You can't prevent re-authentication - instead, you should make it as easy for the end user to handle as possible.
Such services are providing OpenAuth authorization. You are strongly recommended to have a look at it.
Security
I assume your application needs to know the password in plaintext. Then there is no way around storing it in some kind of plain way.
Store in some kind of encoded way eg. Base64, this protects you from knowing password when looking through the database with your eyes, but it does not protect you from anything else.
Ensure that the files are not readable from any other user
Encrypt your harddrive, so nobody can get the passwords from stealing your harddrive. Your computer will require inputung you the password during booting.
Storing
There is nothing wrong with storing much data in your filesystem. For better performance you can do the following
One file for each user, so the filesystem and not ruby needs to search for the data
Make a lot of subdirectorys. Some filessystems performance suffer's if you put to many files into one directory. eg. put the file 'abcd' into 'a/b/c/d'
You could use a database instead of the filesystem
This is the way it works for instance for fetcmailrc which has to be chmod to 600 (readeable and writable only by his owner). And yes, it contains the plain password.
I would strongly suggest you to use OAuth, but if you have to store the passwords (please be absolutely sure that you need to do it) you could use the OpenSSL library to encrypt the passwords. The OpenSSL library is quite poorly documented in Ruby, but as far as I know they are quite similar to the C OpenSSL library. Since I think you should use OAuth, and not storing the passwords I'll let you find the documentation yourself.
However, for the OAuth approach, you want to take a look at the OAuth gem. Google, Twitter (which I recommend you to use the excellent twitter gem for) and facebook (which has two seemingly good alternatives: RFacebook and facebooker)
I was thinking of making a small tool. It is not important what the tool will do. The important thing, is that the tool will need to store some sensitive information on the user's HDD. EDIT: The information that will be stored is USER'S information - I'm not trying to protect my own content, that I distribute with the app.
I understand that I need to encrypt this information. But then, where do I safely store the encryption password? It's some sort of an infinite recursion...
So, is there a way, to encrypt information on windows, and have windows securely manage the passwords? When I say windows I mean Windows XP SP2 or later.
I should also note, that users on the same system must not have access to other users information (even when they are both running my application).
I'm looking for both - .NET 2.0 (C#) and native (C/C++) solutions to this problem.
is there a way, to encrypt information on windows, and have windows securely manage the passwords?
CryptProtectData: http://msdn.microsoft.com/en-us/library/windows/desktop/aa380261(v=vs.85).aspx
Using from .NET: http://msdn.microsoft.com/en-us/library/aa302402.aspx
Historically, Protected Storage (available in XP, read-only in vista+): http://msdn.microsoft.com/en-us/library/bb432403%28VS.85%29.aspx
You should consider using DPAPI for this purpose. It will encrypt your data with a special (internal) symmetric key which is on per-user basis. You don't even need to ask for passwords in this case, because different users on the system will have different keys assigned to them.
The downside of it might be that you can't recover the data if the user is deleted/Windows reinstalled (I believe that this is the case, not quite sure though). In that case encrypt the data with a "self-generated" key derived from the password and store the password in registry/file encrypted using DPAPI.
You can use the native encryption facility. Set the encrypt attribute on your folder or file (from the property page, click on the "advanced" button). Then you can set the users that can access the file (by default this only includes the file creator). The big advantage of this solution is that it is totally transparent from the application and the users points of view.
To do it programmatically: using the Win32 API, call EncryptFile() on the directory where you want to store your sensitive per-user data. From now on all newly created files within this dir will be encrypted and only readable by their creator (that would be the current user of your app). Alternatively you can use the FILE_ATTRIBUTE_ENCRYPTED flag on individual files at creation time. You can check encryption info from the explorer on the file's property page, and see that app-created files are correctly encrypted and restricted to their respective users. There is no password to store or use, everything is transparent.
If you want to hide data from all users then you can create a special app-specific user and impersonate it from your app. This, along with ACLs, is the blessed technique on Windows for system services.
You might want to look at Isolated Storage, which is a way of storing settings and other data on a per-application data automatically.
See an example and MSDN.
This is an alternative to storing normal settings in the registry, a better one in a lot of cases... I'm not sure how the data is stored to file however so you'd need to check, you wouldn't want it to be accessible, even encrypted, to other users. From memory only the app. that created the storage can open it - but that needs checking.
Edit:
From memory when I last used this, a good approach is to write a "Setting" class which handles all the settings etc. in your app. This class then has the equivalent of Serialize and DeSerialize methods which allow it to write all its data to an IsolatedStorage file, or load them back again.
The extra advantage of implementing it in this way is you can use attributes to mark up bits of the source and can then use a Property Grid to quickly give you user-edit control of settings (the Property Grid manipulates class properties at runtime using reflection).
I recommend you look at the Enterprise Library Cryptography Application Block. Check this blog post. Windows has a built in Data Protection API for encrypting data, but the Crypto Application Block makes it more straightforward.
Um, what you're trying to achieve is exactly what DRM tried to achieve. Encrypt something then give the user the keys (however obfuscated) and the crypto. They did it with DVDs. They did it with Blu-Ray. They did it with iTunes.
What you are proposing to do will never be secure. Your average lay person will probably not figure it out, but any sufficiently motivated attacker will work it out and discover the keys, the algorithm and decrypt the data.
If all you're doing is encrypting user data then ask the user for their password. If you're trying to protect your internal data from the user running the application you're S.O.L.
Erm hash the password? You don't need to store the real deal anywhere on the machine just a hashed password (possibly salted too). Then when the user enters their password you perform the same operation on that and compare it to the hashed one you've stored on disk.
Is it possible to store passwords on the local system (Windows XP) that can only be accessed by the application itself?
My instinctive answer would be "no". Even if some kind of hashing or encyption is used I would think that as long as the source code is available then the determined seeker could always use this to retrieve the password.
I'm working on a personal open source hobby project in which I would like to give users the option of storing passwords on disk so that they don't need to type them every time they use the software. One example of a password that could be stored would be the one used to authenticate on their network's proxy server.
There are a few related questions here on Stack Overflow and the most appropriate solution sounds like using an operating system service like DPAPI.
Is the basic premise correct that as long as the password is retrievable by the software without any user input, and the source code is open source, that the password will always be retrievable by a (suitably technically and willfully inclined) passer-by?
You could read about the Pidgin developers' take on it here:Plain Text Passwords.
Using the DPAPI in UserData mode will only allow your account on your machine to access the encrypted data.
It generates a master key based off of your login credentials and uses that for the encryption.
If the password is retrievable by the software without any user input, then the password will always be retrievable by a (suitably technically and willfully inclined) passer-by. Open or closed source only affects how much effort is involved.
Absolutely, you can write a program to store passwords securely.
Using AES, you could have your program generate an AES Key, and have that key stored in an operating system protected area. In WinXP, this is the registry, encrypted with DPAPI. Thus the only way to access the key is to have physical access to the machine.
You need to ensure that when you generate your AES key that you do so in a cryptographically secure manner. Just using RAND won't work, nor will generating a random character string.
Open Source has very little to do with security (in my opinion). Given the level of sophistication in tools for reverse engineering source code, even if you had a closed source solution, people determined to snoop at your code could do so.
Your effort is better spent ensuring that you follow best practice guidelines while using the chosen encryption scheme. I would argue that having your code openly looked at by a larger community would actually make your code more secure; vulnerabilities and threats would likely be identified sooner with a larger audience looking through your code.