NEAR's accounts can have many different key pairs accessing the same account. The keys can also change and rotate. Which means the default way of encrypting messages for specific user with their public key doesn't work.
What is the best pattern to encrypt a message for specific user?
NEAR account keys are not intended for this use case.
Generally, having end-to-end encrypted messages (in the most specific sense a end-to-end encrypted chat, but in general any application that exchanges encrypted messages) with each participant having multiple devices is not trivial. E.g. it is for a reason that in Telegram private chats are attached to a device, and are not available on the other device.
The reason is that generally that would require sharing private keys between devices, doing which securely is a challenge on its own.
Here's a verbatim proposal of how to build a end-to-end encrypted chat with
a) Each participant potentially participating from multiple devices
b) Messages not only shared with someone directly, but also with "groups" of participants.
The design goal was that sending a message should be constant time (not depend on the number of devices target users use / number of people in the group it is sent to), while some operations can be linear.
There's a plan to add is as a library to NEAR, but work on it is not started and is not scheduled to start yet.
Proposal
Problem statement:
We want group chats into which new members can be added, and old members can be removed;
New members being able to see messages posted before they joined is a wish-list feature;
Old members should not be able to see new messages after they left;
Users should be able to use multiple devices and see all the messages in all their group chats from all the devices;
Each message must be stored once (not once per participant of the group);
The proposed solution:
There are three kinds of key pairs in the system: account key (not to be confused with NEAR account keys), device key and a message key.
Each account has exactly one account key. It is generated the first time an account uses the service.
account_keys: PersistentMap
Each device has its own device key generated the first time the chat is accessed from the device (or each time the local storage is erased)
class DeviceKey {
name: string,
device_public_key: PublicKey,
encrypted_account_secret_key: EncryptedSecretKey?,
}
device_keys[account]: PersistentVector
The persistent vector is per account, and each such persistent vector contains device public key (device private key only exists on the device), and the account secret key encrypted with such a public key, or null if the secret key was not encrypted with such public key yet.
There are three methods to manage the device keys:
addDeviceKey(device_public_key: PublicKey, name: string): void
Adds the new key, and associates null as the corresponding encrypted account secret key.
removeDeviceKey(device_public_key: PublicKey): void
Removes the device key
authorizeDeviceKey(device_public_key: PublicKey, encrypted_account_secret_key: EncryptedSecretKey): void
Sets the encrypted account secret key for the device key.
The flow for the user thus will be:
a) Launch the chat from a new device, give it a name.
b) Open chat from some other device that already has the encrypted account key, go to Devices setting and authorize the new device.
All the messages keys are stored in a large persistent vector:
all_message_public_keys: PersistentVector<PublicKey>
And in all other places are referenced using u32 indexes into the vector.
Each user knows some message secret keys:
encrypted_message_secret_keys[account]: PersistentMap<u32, EncryptedSecretKey>
encrypted_mesasge_secret_keys_indexes[account]: PersistentVector<u32>
The map and the vector are per account. The vector is only needed so that when the user changes their account key, we know all the message keys that we need to reencrypt.
The keys are encrypted with the account key.
Each channel has exactly one message key associated with it at each moment, though the keys might change throughout the lifetime of the channel.
channel_public_keys: PersistentMap<u32, u32>
Where the key is the channel id and the value is the message key ID.
Each message has a u32 field that indicates what message key was used to encrypt it. If it is not encrypted, the value is u32::max. Whenever a message is sent to a channel, it is encrypted with the current channel message key.
The flow is then the following:
When a channel is created with the initial set of participants, the creator of the channel creates the message key pair, encrypts the secret key with the account keys of each participant, and calls to
createChannel(channel_name: string,
accounts: AccountId[],
message_public_key: PublicKey,
encrypted_message_secret_keys: EncryptedSecretKey[])
That registers the message key, adds the encrypted secret keys to the corresponding collections, and creates the channel.
If a new user needs to be added, the addUserToChannel(account: AccountId, encrypted_message_secret_key) adds the user to the list of channel users, and grants him access to the latest message access key.
If a user needs to be deleted, the deleteUserFromChallen(account: AccountId) removes the user. In such a case, or if otherwise the channel participant believe their message key was compromised, they call to
updateChannelMessageKey(message_public_key: PublicKey,
encrypted_message_secret_keys: EncryptedSecretKey[])
Note that since each message has the associated key with it, and the channel participants didn’t lose access to the old message keys, the existing channel participants will be able to read all the history, without having to re-encrypt it. However, new users who join the channel will only see the messages since the last time the key was updated.
When a user needs to update the account key, they need to:
a) Encrypt it with all the device keys;
b) Encrypt all their message keys with the new account key;
c) Supply (a) and (b) into a contract method that will update the corresponding collections.
After such a procedure the user will have access to all their old messages from all the devices with the new account key.
Indeed there is no default way to do this.
The easiest way is if specific application, like chat needs to encrypt messages is to require user to "Login with NEAR" - which will create a new key pair on the application's side and authorize this public key in the user's account for the app.
Now any other user can scan recipient's account and find key that authorized for this app and use it for encryption. This will behave similarly to Telegram secret chats, where they only can be decrypted on a single device that started the chat.
To make this work across devices (domains, applications), one can create a key pair, where public key is known and attached to given account. Private key is also stored on chain but encrypted with all the access keys from different devices. When new device / app is added, an existing app needs to authorize this and this will allow to decrypt the private key within this session and re-encrypt with access key of this session.
Is it possible to get some pseudocode for this? Another concern for me is where are these application private keys stored then? Usually I am used to the system where, I have a private key, and I back it up or use a mnemonic. Now when I log in to another device I recover that key.
How can I mirror the private keys on multiple devices?
The other side of this, querying the chain to get the specific public key for a user for an app (maybe with a tag even), makes sense.
Related
Let's say I have a contract function that expects a certain amount of near to be send with a certain transaction, the function is called create_order, create_order takes a couple arguments.
I have my contract setup in the frontend under the name myContract.
I want to call myContract.create_order({...}) but the transaction fails because this method call doesn't have the right amount of NEAR tokens attached.
How do I assign a certain value of deposit to a transaction?
It's possible to use account.functionCall directly (without sugar for RPCs) to either attach amount or specify gas allowance for the call.
See Account#functionCall in nearlib.
Nearlib supports it using account.functionCall(..., amount). But it might not work, because of the design of the access keys with function calls. Default authorized access keys towards applications only allows function calls without attached token deposits (only prepaid gas). It's done this way to prevent apps from automatically using your balance without your explicit approval. Details on access keys are here: https://github.com/nearprotocol/NEPs/blob/master/text/0005-access-keys.md
The way to attach a deposit for the transaction should be done with the explicit approval from the wallet. The app should create a request for the wallet, redirect to the wallet for the approval (or through the popup). Once user approves the transaction, it's signed with full access key from the wallet directly and broadcasted. But I'm afraid we don't have this API on the wallet yet. Issue for this: https://github.com/nearprotocol/near-wallet/issues/56
AFAIK it is not supported at the moment. It will be available after this NEP https://github.com/nearprotocol/NEPs/pull/13 lands.
My query comes from trying to use session store for eg. RedisStore in Golang where the store takes a vararg as last argument which as per document are pairs of authentication and encryption keys.
Most of the examples I see just use a single argument here (for eg. "secret", "mysecret" etc.) and I couldn't get any information about what is achieved by sending multiple pairs of authentication and encryption keys.
Could someone please explain more or direct me to some information about the purpose of the authentication and encryption keys in session management using cookies.
From the docs:
Keys are defined in pairs to allow key rotation, but the common case is to set a single authentication key and optionally an encryption key.
The first pair is used for authentication, encryption and decryption, all other pairs are only used for authentication and decryption (but not encryption).
This is a very common strategy for key rotation. A new key pair is generated regularly and becomes the new key for encryption. Some or all other keys are kept around so that existing data can still be decrypted for some time. The oldest key can be discarded eventually. This limits the impact of a disclosed key because it becomes useless after some time.
Other software that uses this method:
Google's KMS: https://cloud.google.com/kms/docs/key-rotation
Amazon's KMS: https://docs.aws.amazon.com/kms/latest/developerguide/rotate-keys.html
HashiCorp's Vault: https://www.vaultproject.io/api/secret/transit/index.html#rotate-key
I'm in an infosec class and I stumbled upon this concept online and it intrigued me. I've also looked at a few websites and wikipedia that explain the concept, as well as a few posts on stackoverflow, but I'm still getting confused. From what I understand is in a typical HTTPS public key exchange, a browser and a server come together with keys to create a session key...if someone ever obtained a private key that derived the session key, they could see all the data that was sent between this connection, even in the past.
My understanding is that with PFS, the 'session key' is never sent , even in encrypted form. It is kept secret so that even if someone found a private key, they wouldn't be able to access encrypted recorded information from the past. Is this correct?
I also was wondering, If I am partaking in a PFS exchange call me "A", with a server "B", PFS is supposed to work with the fact that if my key becomes compromised, A and B's conversation wont become compromised because they don't know the session key. But how does "B" authenticate me as "A", if my key has in fact became compromised...e.g. how would it know the difference between me (A) or another user (C) using my key attempting to access the data.
I really like the answer on Quora given by Robert Love: http://www.quora.com/What-is-perfect-forward-secrecy-PFS-as-used-in-SSL
Let's look at how key exchange works in the common non-ephemeral case.
Instead of giving a practical example using, say, Diffie-Hellman, I'll
give a generalized example where the math is simple:
Alice (client) wants to talk to Bob (server).
Bob has a private key X and a public key Y. X is secret, Y is public.
Alice generates a large, random integer M.
Alice encrypts M using Y and sends Y(M) to Bob.
Bob decrypts Y(M) using X, yielding M.
Both Alice and Bob now have M and use it as the key to whatever cipher they agreed to use for the SSL session—for example, AES.
Pretty simple, right? The problem, of course, is that if anyone ever finds out X, every single communication is compromised: X lets an attacker decrypt Y(M), yielding M. Let's look at the PFS version of this scenario:
Alice (client) wants to talk to Bob (server).
Bob generates a new set of public and private keys, Y' and X'.
Bob sends Y' to Alice.
Alice generates a large, random integer M.
Alice encrypts M using Y' and sends Y'(M) to Bob.
Bob decrypts Y'(M) using X', yielding M.
Both Alice and Bob now have M and use it as the key to whatever cipher they agreed to use for the SSL session—for example, AES.
(X and Y are still used to validate identity; I'm leaving that out.)
In this second example, X isn't used to create the shared secret, so even if X becomes compromised, M is undiscoverable. But you've just pushed the problem to X', you might say. What if X' becomes known? But that's the genius, I say. Assuming X' is never reused and never stored, the only way to obtain X' is if the adversary has access to the host's memory at the time of the communication. If your adversary has such physical access, then encryption of any sort isn't going to do you any good. Moreover, even if X' were somehow compromised, it would only reveal this particular communication.
That's PFS.
In a non-PFS session the browser decides on the session key (or rather secret from which it is derived) and encrypts it using RSA, with the RSA public key obtained from a certificate that belongs to the server. The certificate is also used to authenticate the server. The server then uses its private key (what you call master key) to decrypt the session key.
All connections to the server use different session keys, but if you possess the master key you can figure them all out, the way the server does.
In PFS you use algorithms such as Diffie-Hellman, where the master key is not used. In such connection the master key is used to authenticate the parameters for the algorithm. After the parameters are agreed on, the key exchange takes place using those parameters, and a secret of both parties. The parameters are not secret, and the secrets the parties used are discarder after the session key is established (ephemeral). This way if you discover the master key you cant discover the session key. However you can pose as the server if you get the key, and the certificate is not invalidated.
To find out more read about Diffie-Hellman.
You generate a new public key for every message, and use the real permanent public key only for authentication
This was mentioned in other answers, but I just want to give a more brain parseable and contextual version of it.
There are two things you can do with someone's public key:
verify that a message was written by them, AKA verify a message signature AKA authenticate a message. This is needed to prevent a man in the middle attack.
encrypt a message that only they can decrypt
In many ways, authentication is the more critical/costly step, because to know that a given public key belongs to someone while avoiding a man in the middle attack, you need to take steps such as:
meet them in real life and share the public key (leave your home???)
talk to them over video (deepfakes???)
trusted signature providers (centralization!!!)
Generating new keys is however comparatively cheap.
So once you have done this costly initial key validation step, you can now just:
ask the receiver to generate a new temporary public key for every message you want to send them
the receiver sends you the temporary public key back to you, signed by their permanent public key. Nothing ever gets encrypted by the permanent key, only signed. No need to encrypt public keys being sent!
you verify the message signature with the permanent public key to avoid MITM, and you then use that temporary key encrypt your message
After the message is received and read, they then immediately delete that temporary private key and the decrypted message.
So now if their computer gets hacked and the permanent private key leaks, none of the old encrypted messages that the attacker captured over the wire can be decrypted, because the temporary key was used to encrypt them, and that has been long since deleted.
Future messages would be susceptible to MITM however if they don't notice and change their permanent key after the leak.
The public key and private key pairs are created on the client side via a java script algorithm and the public key is then transferred over to the Server.
A copy of the persons private key was stored on the users computer in the form of a java script variable.
When User A sends a message to User B
The server encrypts the message with User B's public key.
User B picks up the message and decrypts (algorithm written in java script) it with User B's private key which is private and kept in a java script variable.
At no point in the time the User B's private key is disclosed over the network what so ever.
Would that be secure???
'public' and 'private' are just names given to the two keys. It doesn't matter WHICH of the two keys is public and which is private, as long as you never ever mix up the usage. Once both keys are available to someone at the same time, the security of the messaging system is utterly destroyed.
Technically, since you say the keys are stored in javascript variables, you're implying that the variables were sent IN THE CLEAR embedded in some browser-based html/javascript. That further implies that there's no security - since both keys are exposed to the network.
To decide if something is "secure", you have to know what the security requirements are. Your case satisfies a few likely requirements, but there are several likely requirements that it does not satisfy. For example:
A plaintext copy of the message is apparently transmitted over the network from User A to the server, so anyone can eavesdrop on it at that point. (This is likely to be a serious problem.)
You don't explain how the public key is transmitted to the server. If it's not transmitted in an authenticated fashion, then a man-in-the-middle can generate his own public-private key pair, and give his public-key to the server. (This could be a serious problem.)
User B cannot verify the authenticity of the message (s)he receives. The message may have come from the server (and ultimately from User A), or it may have come from anyone else with a copy of the public key. (This may or may not be a serious problem, depending on the application.)
So overall, I would not consider this design to be "secure".
It wouldn't be too secure since:
The private key of any user (say user B) can be leaked out of your app via injected JS code, or a bad browser addon
Once this is done, any one who gets access to any of the messages directed at User B, will be able to decrypt it and make sense of it
Of course the above wont happen if you are the only one using the app - since you'll likely have other users with different browser setups/addons/browsing-behaviors etc, it is entirely possible
When user A tries to send something to user B, you said the server will encrypt the message using user B's public key - Now, this request made via JS can be interpreted by a middle man. Once done, this middle man can initiate any request to any user by manipulating the sender, referrer etc. This can lead to impersonation and so..
You also mentioned that after generation, you intend to send the public key over to the server. This call made from JS can easily be interpreted, which means the public key can be leaked.
I just wanted to make sure I am on the right track
Sender generates private key, encrypts message
Sender requests receivers public key
"Secret" is then encrypted with this public key
Data sent to receiver for decryption
I assume logically speaking the "Secret" can only be encrypted with the receivers public key, although is the encrypted secret attached to the encrypted file / document?
Public key encryption: An easy way to visualize this is as follows:
I send someone an unlocked box that they can put stuff in. Then they close it and it locks. They can ship me the box and I have the only key that will open the lock.
The locked box is the public key encryption; the key is my private key.
Symmetric key encryption works like this:
We buy a padlock at Home Depot and it comes with two keys. You take one and move to Boston; I take one and move to San Jose. We can padlock our box and ship it back and forth, only we have the keys.
p1. Sender generates session symmetric key.
p3. session key is encrypted using this public key.
Yes, the encrypted secret is attached to the encrypted file. There's no risk in this as this is a session (one-time) key.
Sender generates session symmetric key.
Symmetric meaning both parties will use the same key.
Sender encrypts data with session key.
Sender requests receivers public key.
Receiver sends public key while keeping the matching private key secret.
Session key is encrypted using receivers pubic key.
Now only the receivers private key can decrypt the session key.
Encrypted Data and encrypted session key are sent to receiver.
Receiver decrypts senders session key using receivers secret private key.
Receiver decrypts data using decrypted session key.
Sticking with John Browne's metaphor:
I lock a document in a box with a key.
I ask a friend to send me a box to which only they have a key too.
I place a copy of my document-box-key in my friends box and lock it.
I send the document box and my friends box (which contains the key to the document box) to my friend.
My friend opens their box with their secret key, and retrieves the document-box-key from inside.
My friend then uses that key to open the document box.