First, some info on our system, which is basically an eTendering solution for the construction industry.
So:
List item
Our system has multiple companies
Each company has multiple users
Each company can create multiple auctions
Other companies can then submit their bids for the available auctions. A bid consists of hundreds or thousands of individual items, and we would just need to encrypt the 'price' section of these records.
The problem that we're facing is that our large customers do not want us to ever have access to the bid prices, at least while the bidding is in progress, which is totally understandable. Right now, we are simply encrypting the prices via a symmetric encryption, so even though the prices are effectively encrypted in the database, their concern is that we have the key to decrypt the prices.
We're thus looking at some form of public key encryption system.
Here is our initial thoughts on the solution:
When a company signs up, we create a public/private keypair using OpenSSL for it and we save it in S3 or straight into the database. For this to be really useful, we would enforce the user to use a strong password for the private key, which would of course not be saved in the database.
When a company submits a bid for an auction, we encrypt the prices using the public key of the auction's owner company and we save them into the database.
When the auction bidding period is over and the issuing company wants to generate the report the first time, we ask him to input his password and use that along with his company's private key to decrypt the prices.
To make subsequent traffic faster, we cache the decrypted data (and maybe encrypt it using a simple symetrical encryption system)
So here are the questions (and we're unfortunately not security experts, so sorry if those are stupid questions):
Does this make any sense or is it a totally ludicrous or overkill solution?
Would we generate the keys using OpenSSL, OpenPGP or another solution?
What happens if a user wants to change his password or generate a new key? Would there be no other way but to decrypt/re-encode everything with the new key?
What would some of the pitfalls be with this solution?
Are there any better solutions that you could recommend?
So here's my suggestion, if you want to solve this using encryption...
Each user and each company should generate OpenPGP (or GnuPG) asymmetric public/private key pairs
Each of those public keys should be uploaded to a public key server
Companies could optionally "sign" the public keys of the individual users, to designate a trust relationship with those users (and revoke that signature if that relationship changes)
The auctioneer or non-partisan arbiter would also generate a key pair and push that public key to the public key server
As part of an auction registration process, each user would import the public key of the auctioneer, and the auctioneer would import the public key of each user
A trusted third party, perhaps a SaaS vendor outside of the auctioneer, would host a service through which users and the auctioneer would communicate
Bidding users would create a bid by
signing their bid with their private key
encrypting their proposed price to two keys: their own public key and the auctioneer's public key
submitting their bid to the trusted third party service, which would need to enforce an embargo on any user retrieving bids before the auction expiration
At auction end, and only after auction end, the auctioneer retrieves all of the bids, decrypts them, and verifies signatures
A couple of key points:
It's essential that no user or company private keys are ever shared or stored within the service -- that's the only real "flaw" I see in your proposed methodology in your question. If that's the case, it would be very possible for one user to accuse an administrator of "fraud" or "tampering" with bids, as an administrator of your server would ultimately have access to the private keys of all users, since you've generated them yourself.
Along those same lines, it's essential that any an all communication and "bids" are cryptographically "signed" with the truly private keys by each user. This is how you would know that a bid came from one particular user and only that user, and that the bid could not have been tampered with.
Encrypting the bids to the public keys of the bidding user and the auctioneer ensures that the third party SaaS vendor has no introspection into the bids themselves during the blackout period while the auction is open. I believe this is the most important point to solving your problem as described.
Note that it might actually be preferred to encrypt each bid to a ring of all of the bidding users, if by design you want to make all bids public after the auction is closed. That would be a slight modification to my algorithm as described above.
In the interest of full disclosure and perhaps some subtle marketing, I happen to be the architect and CTO of a company called Gazzang who has implemented a product called zTrustee which operates exactly as described above ;-)
To be clear: I have a hunch that your clients are probably not willing to sacrifice all of the conveniences that come with having your system manage some of the cryptography; you should probably present several options and their weaknesses vs convenience.
General Points
Before anything else, you start start with an explicit threat model, covering every possible attack you can think of. Even if you choose not to address some of the attacks (it's unrealistic to handle everything), you'll ferret out the more obvious attacks, and have at least a basic set of steps for handling others should they occur.
Re: Does this make any sense?
I think the general premise, while overkill on the part of your customers, makes sense from a security perspective. Your clients want a cryptographically secure system; fair enough.
However, some points on your proposed solution:
By allowing the client to pass their password over the wire, an attacker (which your client seems to think could be you) needs to only man in the middle that password to gain access to the pricing data.
SSL helps mitigate this, but an errant log line somewhere along the line could very well expose the client's password by accident.
The only truly cryptographically secure way (as I see it) for the client to ensure that you don't have access to the pricing data is for them to encrypt it, and your system just acts as a broker for the encrypted data. This, in effect, makes you a broker of encrypted packets and public keys, but your system should never see private keys.
The question is: are clients willing to manage their own keys, or is that too burdensome to them? You might be able to automate most of it, at least (client app/website would handle storing the private key locally, and would also be responsible for gathering public keys of the other interested parties in order to decrypt their encrypted bids)
Re: Would we generate the keys using OpenSSL, OpenPGP, or another solution?
Really doesn't matter all that much; each of those options just define the container format for the public/private keys and any metadata. Use whichever one fits your language/platform best.
The main decision point should be in which encryption algorithm and key strength: RSA-2048? RSA-4096? elliptic curve? something else?
Specific to Ruby: You're probably just going to want to use the OpenSSL library, since it's part of the standard library. But to reiterate my point above: It's even better if your servers never even see the private keys (if the clients are ok with the trade off of better security over convenience)
Re: What happens if a user wants to change his password or generate a new key?
Changing a password is trivial: the private key its self is just encrypted w/ some symmetric algorithm. Changing the password involves decrypting the existing key, and re-encrypting it with the new key. If the client were to lose their password, there is no recovery.
Generating a new key is probably safer, but requires more diligence on your part (encrypted payloads will need to identify which key they match, and clients could have multiple keys active at a time). This is a good thing, though; it is a common practice to rotate keys regularly, even if they are not compromised.
When a company signs up, we create a public/private keypair using OpenSSL for it and we save it in S3 or straight into the database. For this to be really useful, we would enforce the user to use a strong password for the private key, which would of course not be saved in the database.
I am a bit sceptical about this step. If you (the developer company) generate both the public and the private key used for the encryption it means you are 50% into being able to break the encryption. The only thing that protects your customer is the password, which you might be able to bruteforce (i am not suggesting that you will but you have the ability to do so)
If you will use PKI (or what you have described) you need to ensure that the key creation does not happen on your system. The client should create the pair on their systems and then provide you with their publoic key which you will use to encrypt the prices. The client then will be able to decrypt using the private key on which they have sole control
What would some of the pitfalls be with this solution?
The pitfall is that you are making a complex solution. Especially if you follow my advice above, then you place your trust on the customer to not "lose" the private key (and/or password) or else they won't be able to decrypt the prices. In addition, if the key leaks from their side, it's difficult to prove that your application is "innocent"
Would we generate the keys using OpenSSL, OpenPGP or another solution?
In order to prevent the pitfall of a customer losing a key, you might want to look into PGP (the commercial version does this for sure) and into the concept of ADK (additional decryption key) and "split keys". The idea is that besides encrypting with the public key of the customer you also encrypt with a "corporate" key which can only be used if y out of x number of people come together (as an example, 10 people can possess parts of the key and if 6 of them come together they can reconstruct the key). The parts can be shared amongst your company, the client, their lawyer, etc
Related
everyone. I'm learning Laravel and I'm in the start of my journey. I was learning about encryption and decryption in Laravel today and then this thought came into my mind. Could be a stupid one but I want to know my answers.
Let's say I make a database which stores sensitive information about users and I encrypt all the data before storing into the database, let's just say using the Encrypt class of Laravel. Now my questions:
If someone steals that database and luckily finds out that this information was encrypted using techniques provided by Laravel or any other technique. Can't that person descript that all data using the same decryption technique that was used to encrypt it. If this can be done, then what's the point of doing this encryption?
If that can be done then how can we make sure that our data is actually encrypted and is safe even if someone steals it?
Thank you guys!
I encrypted my data and then decrypted it and want my answer that how that encrypted data is even safe.
You might want to read up on the basics of encryption.
The common approach is that the technique by which you encrypt should be as open as possible - because the more people look at the algorithm, the less likely there might be bugs.
However, even if the algorithm is public, the key is not. Only people who have the key can decrypt properly encrypted data. This is true of the AES algorithm Laravel uses too.
The mathematics are complicated, but essentially the length of the key determines the amount of computer resources required to break the encryption.
THe real-world example is that everyone knows how door locks work. There are millions of locks that all work in the same way - but only people who have a key can open the door.
So, if an attacker steals your database, they cannot read your content unless they also have the key, as long as the key length is sufficient.
If someone steals that database and luckily finds out that this information was encrypted using techniques provided by Laravel or any other technique. Can't that person descript that all data using the same decryption technique that was used to encrypt it. If this can be done, then what's the point of doing this encryption?
If someone steals that database they will still need a decryption key to decrypt (thats why strong passwords are recommended) so even if they bruteforce it will become almost impossible to decrypt.
The way you’re asking if encrypt and decrypt is easy then i think you’re asking some encryption like base64.
With AES bruteforcing their way in becomes difficult. In laravel encrypt or crypt class it uses AES-256-CBC which is pretty good at that.
Then there is Hash library they are one way encryption techniques which uses bcrypt it can only be verified and not decrypt you have to run all combinations for lines everytime to brute force. Unlike md5 which gives same encryted string every time.
There are many operating system and programs that hash passwords for authentication.
Even though they can encrypt the password in many different ways and save it
why do they save the hash of them?
Is the only reason to that question that encrypting them may cause in breaking and decrypting them or there are other reasons?
Thanks for answering in advance
User credentials (≈passwords) are among the most valuable assets stored in an application. They are a prime target for attackers, and as a developer, you want to protect them the best you can.
The principle of defense in depth (and common sense) indicates that the more layers of protection you can put around something, the more secure it will be. So as you also mentioned, the purpose of hashing passwords is that even if there is a breach, an attacker still can't get hold of actual user credentials.
The problem with encryption is always key management. If passwords were stored encrypted, they would need to be decrypted (or the received password encrypted with the same key) to be able to verify a password. For this, the application would need to have access to the key. But that negates the purpose of encryption, an attacker would also have access to the key in case of a breach. (Public key cryptography could make it somewhat more difficult, but essentially the same problem of key management would still persist.)
So in short, only storing salted hashes with an algorithm that is slow enough to prevent brute-force attacks (like PBKDF2 or Bcrypt) is both the simplest and the most secure. (Also note that plain salted hashes are not good enough anymore.)
Think of the need: You define your new password, and then every time you log-in the entered password is hashed and checked against the stored value. This is the simplest and most secure policy to handle this (since no one will be able to re-construct your password from the stored value). Moreover, imagine that you use the same password in several systems. If Windows would enable (regardless how hard it would be) to re-construct your password from what is stored in a Windows system, there would be (quite many) people that would blame Microsoft for security breach on other system (which could derivate into legal actions).
To summarize, simplicity and commercially logical approach.
Well, actually it's for security reason.
Hash functions are usually not revertibles, so even if someone finds out the hashes it would be really difficult for him to find which password generated that hash value.
Obviously you can try with a dictionary attack or a brute force one, trying to find out the password which generated the hash but it could take a very long time.
Consider that you have a Database with user and their passowrd, take note that a lot of people use the very same password everywhere.
Now immagine if a cracker manages to crack into your DB and finds all the password written clearly. That would be a disaster.
To start, I am trying to encrypt very sensitive information on a public website. Users will be able to update their information, Administrators will need access to this information. I am worried that if the encrypted data is some how compromised, then everyone's information would be as well due to them all using the same salt and key.
So I know using a salt, and key is always preferred. But as mentioned above if they reverse engineer the encrypted data, what use it is.
My solution, is to have the key and salt stored in a DB, with many rows and columns, any of which can be used for the salt or key. I would have an algorithm that will use "something" fixed in the users account that will be used to figure out which salt and key to use. This way statistically speaking no 2 years will have same combo of salt and key.
Is this over kill, or good?
I question the value of this second database that holds keys and salts. Consider:
The "something" in the user's data that identifies the salt and key will necessarily have to be encrypted differently from the rest of the user's data. Otherwise, you wouldn't be able to get it without first already having it.
Statistical analysis of the encrypted user data would almost certainly discover that the "something" is encrypted differently. That will be like waving a red flag at a bull, and an attacker will concentrate on figuring out why that's different.
You can assume that if an attacker can get the database of encrypted user information, he can also get the database of salts and keys.
Given that, there are two possible scenarios:
The encryption of the "something" that identifies the key and salt is unbreakable. That is, it's so good that the attacker's best efforts fail to reveal the connection between that "something" and the key/salt database.
The attacker discovers the encryption of the "something," and therefore is able to decrypt your sensitive data.
If #1 is the case, then you probably should use that encryption algorithm for all of your user data. Why do something in two steps when you can do it just as effectively in one?
If #2 is the case, then all the work you've done just put up a little bump in the road for the attacker.
So, short answer: what you propose looks like either unnecessary work or ineffective road blocking. In either case, it looks to me like a whole lot of work and added complexity for no appreciable gain.
That said, I could have misinterpreted your brief description. If so, please correct me.
This video claims to be able to generate a key pair directly on the Smartcard so the private key never leaves the card. Is this operation common to all smartcards, or do only select smartcards support this?
Most cryptographic smart cards are able to generate keys in the card in addition to storing externally generated plaintext keys, that's the main purpose of cryptographic smart cards.
Some cards don't allow importing plaintext keys, some allow exporting keys generated on the card, some card can do import/export of wrapped (encrypted with another key) private keys, so that the actual private key would never be in clear in an untrusted environment.
Have a look at http://www.opensc-project.org for a further pointers on cards and software.
A smartcard is just a chip on a card, so you probably want to ask about cryptographic smart cards.
I have never worked with cryptographic smart cards but that sounds like a completely reasonable functionality of such smartcards since they are intended usually for authentication and encryption which usally use private keys for that.
The advantage of cryptographic smart cards usually lies in that, that the private key is stored in it. That is intended to prevent the compromise of the keys because even if the computer, that reads the smartcard, is compromised the smartcard is safe and the private key is safe too.
this is called on-board key generation, indeed the card generates itself the keys and the private key will never leave the smartcard, besides of the key marked as not exportable, the card itself is unable to export it, it has no function, no knowledge how to do this.
many smartcards ( of course with a cryptoprocessor ) support this but not all and anyway this is an expensive feature. Most smartcard vendors sell such cards.
I apologize in advance for the incoming Wall-O-Text. This is (at least, to me) a fairly complex issue that I've put quite a bit of thought into. You can read my question and also see a test implementation in Ruby (very hastily built, not database-backed, and probably very ugly) at this GitHub Gist if you are so inclined.
Introduction
Imagine one was required to create a web-based password management system
(over SSL! :) with the following requirements:
Individual users sign in to the system using their own unique pass
phrase.
This pass phrase should be enough to allow the user to use the system
effectively (e.g. from a smartphone, etc.)--the point being that they
should not have to keep a key file with them.
Users can store arbitrary-length bits of data in the system ("entries").
Entries are encrypted in the database in such a way that there is not
enough information in the database or application alone to read the
encrypted entries.
Users should be able to "share" entries with other users of the system
so that the other user(s) can read the contents of the entry.
I'm no expert in cryptography. After thinking about it for a while, I came up
with the following. My question is: is this implementation secure? Am I
missing something? If so, is the above spec even implementable? Or is this
overkill?
Database
The database is set up as such:
+------------------------------------------------------------------------------+
| users |
+---------+--------------+--------------+---------------+----------------------+
| salt | pub_key | enc_priv_key | priv_key_hmac | |
+---------+--------------+--------------+---------------+----------------------+
| entries |
+---------+--------------+--------------+---------------+----------+-----------+
| user_id | parent_entry | enc_sym_key | sym_key_sig | enc_data | data_hmac |
+---------+--------------+--------------+---------------+----------+-----------+
Basic Use Cases
Let's imagine two users of the system, Alice and Bob.
Bob signs up for the site:
Bob enters a password. This password is sent to the server (but not
stored).
The server generates a random salt and stores it in the salt field.
The server generates the SHA-256 hash of Bob's password and salt.
The server generates an RSA key pair. The public key is stored as plain
text in the pub_key field. The private key is encrypted via AES-256
using the hash generated from Bob's password and salt as the key and
stored in the enc_priv_key field.
The server generates a hash-based message authentication code for Bob's
private key using Bob's password and salt as the key and stores this in
the priv_key_hmac field.
Bob stores an entry in the system:
Bob enters some data to be stored as an entry along with his password.
This data is sent to the server.
The server generates a key to be used as a key for AES-256 encryption.
The server uses this key to encrypt the data and stores the result in
the enc_data field.
The server generates a hash-based message authentication code for the
data using the generated key and stores this in the data_hmac field.
The symmetric key used to encrypt the data is encrypted with Bob's public
key and stored in the enc_sym_key field.
The server uses Bob's private key to generate a signature for the
symmetric key.
Bob retrieves his stored entry:
Bob enters his password and the ID of the entry to retrieve.
The server generates the SHA-256 hash of Bob's password and salt.
Bob's encrypted private key is decrypted via AES-256 encryption using the
hash.
The server verifies that Bob's encrypted private key has not been
tampered with by checking the HMAC in priv_key_hmac.
The server decrypts the symmetric key stored in the enc_sym_key field
using Bob's private key.
The server verifies that the encrypted symmetric key has not been tampered
with by verifying the signature in sym_key_sign using Bob's public key.
The server decrypts the data using the symmetric key.
The server verifies that the encrypted data has not been tampered with
by verifying the HMAC stored in the data_hmac field.
The server returns the decrypted data to Bob.
Bob shares an entry with Alice:
Bob wants Alice to have access to an entry he owns. He enters his
password and the ID of the entry to share.
The data for the entry is decrypted using the method in "Bob retrieves
his stored entry."
A new entry is created for Alice in the same fashion as in "Bob stores
an entry in the system," with the following exceptions:
The entry's parent_entry is set to Bob's entry.
The signature for the symmetric key is calculated using Bob's private
key (since Alice's private key is not available to Bob).
When Alice accesses this new entry, the existence of a non-null
parent_entry causes the system to use Bob's public key to verify
the signature (since his private key was used to create it).
Bob changes the data in his shared entry:
Bob decides to change the data in the entry he shared with Alice. Bob
indicates the entry ID to modify and the new data it should contain.
The system overwrites the data created in "Bob stores an entry in the
system."
The system finds every entry with a parent_entry equal to the entry
that was just modified, and for each one overwrites the data created in
"Bob shares an entry with Alice."
Analysis
Advantages:
It is impossible to decrypt any data from the database without the
password of the user that owns the data, as the private key necessary to
decrypt the data is encrypted with the user's password, and that password
(and it's hash) is not stored in the database.
If a user wants to change their password, only their encrypted private
key needs to be regenerated (decrypt the private key with the old
password/hash, then re-encrypt it with the new password/hash).
Shared entries are stored as actual separate records in the database,
so there is no need to share a key between multiple users/groups of users.
Disadvantages/Problems (that I can think of):
If a shared entry is modified, the system must re-encrypt every child
entry; with a large number of users sharing data, this could potentially
be computationally expensive.
Shared entries depend on the parent user's public key for signature
verification. If the user is deleted, or their key changes, the signatures
are invalid.
Repeated from the introduction: my question is: is this implementation
secure? Am I missing something? If so, is the above spec even implementable?
Or is this overkill?
Thanks for sticking it out this long. I'm interested in your opinions! Am I on the right track, or a complete moron? YOU DECIDE! :)
No IV storage? I guess you could use AES-256-ECB, but that only lets users store 32 byte passwords, and you need to make sure that the generated private key is only ever used for one encryption. (Your current design seems safe in this respect, but if you want to allow passwords longer than 32 bytes, or ever think of making this key do double-duty, you'll need to store an IV for every encryption with it.)
I don't see the security value of priv_key_hmac and data_hmac; if either the private key or the encrypted data has been tampered with, then garbage output will result from decrypting with the private key or the symmetric key. Bob will surely be suspicious when he can't figure out how to type the BEL character. :) (Will humans ever see the output? A human will likely realize the returned password is incorrect without needing to be told. A computer couldn't tell the difference, so if automated systems will ever use the resulting passwords, then sure, keep the fields.)
There is no mechanism for "I forgot my password". Make sure your users know that there is no recovering their data if they forget their password. Users are coddled these days, and might expect to be coddled with your service too.
I see no mechanism for users to specify which entry Bob wants decrypted. You should store a name, or, as ssh(1) does in known_hosts, a hashed version of a name, for each entry. Storing a name directly would remove an SHA-256 operation, but a database compromise that reports the cleartext names of services that a user has accounts with might be every bit as damaging. (Perhaps an online escort service, or off-shore bank, or fight club.)
You don't actually need to duplicate anything other than enc_sym_key when you share an entry with Alice - since the symmetric key is never re-used for more than one entry, you only need one copy of the encrypted data.
Why not use certificates for sharing data between users? The use PKCS#12 certificates for holding the PEM and Private keys of users and the PEM per user or per site can sign and encrypt for data verification and security.
A scenario to illustrate.
Bob wants to share with Alice without Eve reading.
Alice gives Bob her public key. Bob adds Alice's public key to his keychain of trusted users. Bob then uses Alice's public key to encrypt a message while using his own PEM to sign the data. Of course this scenario requires that Alice already have a copy of Bob's public key to perform verification of the signature but you get the idea.
Also, why store a salt or iv? Both of these being stored along with at rest data will be accessible in the event of a db compromise.
Best practices...
Use a keyring for each user account for storage of others public keys/PEM certificates
Only use public key encryption for sharing information between accounts
Encrypt data with the users private key that is not to be shared between accounts
Do NOT use AES, RSA or any other reversible encryption for password storage
User specific salts should be used to further enhance hashing algorithm for password and should NOT be stored
Use of AES using a site wide password COULD be used for storage of at rest data to further improve security (but you would run into the problem you have outlined in the CONS section)