I was using the secrets/master.key that was there when the password was encrypted and stored to credentials.xml. But restoring the the same set of credentials.xml and master.key is not working in a new jenkins setup. I even tried to restore secret.key but that too is not working.
I also noticed the ecrypted string in is credentials.xml is also not same for same string.
I am trying to automate the jenkins setup. Is there a way I can get the encrypted password that the jenkins produce from bash?
Jenkins and its plugins usually encrypt strings using the Secret class, which (AFAICT) stores the key under ${JENKINS_HOME}/secrets/hudson.util.Secret.
I don't know of any easy standalone solution, but you can use the Jenkins Script Console (or the groovy CLI command) to attempt to decrypt secret values that you have:
import hudson.util.Secret
Secret a = Secret.fromString('my secret value')
String ciphertext = a.getEncryptedValue()
println ciphertext
// '{AQAAABAAAAAQdIQUuG2AhKoV7mCIcd3PXBdw8ItgchIrvQrQ=}'
// or similar; will change with each new secret object
Secret b = Secret.decrypt(ciphertext)
String plaintext = b.getPlainText()
println plaintext
// 'my secret value'
host=http://$JENKINS_USERNAME:$JENKINS_PASSWORD#localhost:8080
CRUMB=$(curl -s "$host"'/crumbIssuer/api/xml?xpath=concat(//crumbRequestField,":",//crumb)')
encrypted_passphrase=$(curl -H "$CRUMB" -d "script=println(hudson.util.Secret.fromString('password').getEncryptedValue())" -X POST $host/scriptText)
Related
What mechanism does Ansible Vault use to detect wrong vault passwords? In other word, if a user inputs wrong vault password then Ansible shows error message below. How?
Decryption failed (no vault secrets were found that could decrypt)
Is there any section in Vault Payload that Ansible uses to detect wrong passwords?
The code for ansible-vault with the relevant section can be found here: https://github.com/ansible/ansible/blob/devel/lib/ansible/parsing/vault/init.py#L736
Summarised, it uses the specified password and vault ID to decrypt the file. So it will look for the vault ID in the vault file and will then try to decrypt the password. The crytpo part will only return a byte string when the decryption was successful and the expected format (PKCS7) is returned:
So first, the content of the vault is parsed (hex format is converted to actual bytes):
b_ciphertext, b_salt, b_crypted_hmac = parse_vaulttext(b_vaulttext)
Then, the relevant keys are generated from the salt and the password:
b_password = secret.bytes
b_key1, b_key2, b_iv = cls._gen_key_initctr(b_password, b_salt)
As you note correctly, the first thing that the _decrypt_cryptography function does is to check if the HMAC is correct, using one of the keys derived from the password above:
hmac = HMAC(b_key2, hashes.SHA256(), CRYPTOGRAPHY_BACKEND)
hmac.update(b_ciphertext)
try:
hmac.verify(_unhexlify(b_crypted_hmac))
except InvalidSignature as e:
raise AnsibleVaultError('HMAC verification failed: %s' % e)
Then, the actual decryption happens:
cipher = C_Cipher(algorithms.AES(b_key1), modes.CTR(b_iv), CRYPTOGRAPHY_BACKEND)
decryptor = cipher.decryptor()
unpadder = padding.PKCS7(128).unpadder()
b_plaintext = unpadder.update(
decryptor.update(b_ciphertext) + decryptor.finalize()
) + unpadder.finalize()
The b_plaintext is then returned.
So when you use the wrong password, the crypto function will return non-PKCS7 data and this then leads to the message above.
I created a pair of *.pub and *.sec files using the instructions and code given here:
https://www.gnupg.org/documentation/manuals/gnupg/Unattended-GPG-key-generation.html
(I am using this documentation because the ultimate application I have in
mind is an automated encryption/decryption pipeline.)
Q1: How can I use gpg2 and the *.pub file to encrypt another file?
Q2: How can I use gpg2 and the companion *.sec to decrypt a file encrypted using the companion *.pub file?
Important: I am interested only in answers that are suitable for programmatic implementation of an unsupervised operation. Please do not post answers that can only be carried out interactively. I am particularly interested in solutions that can be implemented in Python.
Please include precise pointers to the relevant documentation.
Some information about what you said:
I created a pair of *.pub and *.sec files using the instructions
Perfect to share the public key(s) with people you are exchanging information, but technically, when you are working programmatically, you don't need to use these files directly.
To be noted:
when you encrypt data, you will specify the recipient corresponding to the key to use to encrypt
when you decrypt data, you will first import the owner's public key, and then you will be able to decrypt data without specifying recipient, because it is embedded in the encrypted data
Actually, I am somewhat confused on this question. I have read conflicting information [...]
I agree it's quite confusing. In this situation, I think it is better to use version 1 for which there is more experience, and for which you find third party library to use.
In this answer, I tried:
python-gnupg (for GnuPG v1) which is a well known Python library and match perfectly your needs
cryptorito (for GnuPG v2) for which I didn't find enough documentation
With the first library, you can simply install it in your system:
sudo pip install python-gnupg
And then write a Python script to perform all the operations you want.
I wrote a simple one to answer your question.
#!/bin/python
import gnupg
GPG_DIR='/home/bsquare/.gnupg'
FILE_TO_ENCRYPT='/tmp/myFileToEncrypt'
ENCRYPTED_FILE='/tmp/encryptedFile'
DECRYPTED_FILE='/tmp/decryptedFile'
SENDER_USER='Bsquare'
TARGET_USER='Kjo'
gpg = gnupg.GPG(gnupghome=GPG_DIR)
print("Listing keys ...")
print(gpg.list_keys())
# On SENDER_USER side ... encrypt the file for TARGET_USER, with his public key (would match the kjo.pub if the key was exported).
print("Encrypting file " + FILE_TO_ENCRYPT + " for " + TARGET_USER + " ...")
with open(FILE_TO_ENCRYPT, "rb") as sourceFile:
encrypted_ascii_data = gpg.encrypt_file(sourceFile, TARGET_USER)
# print(encrypted_ascii_data)
with open(ENCRYPTED_FILE, "w+") as targetFile:
print("encrypted_ascii_data", targetFile)
# On TARGET_USER side ... decrypt the file with his private key (would match the kjo.sec if the key was exported).
print("Decrypting file " + ENCRYPTED_FILE + " for " + TARGET_USER + " ...")
with open(ENCRYPTED_FILE, "rb") as sourceFile:
decrypted_ascii_data = gpg.decrypt_file(sourceFile)
# print(decrypted_ascii_data)
with open(DECRYPTED_FILE, "w+") as targetFile:
print(decrypted_ascii_data, targetFile)
To be noted my keyring contains pub/sec pair for my Bsquare user, and the pub key of Kjo user.
when looking at encrypting and decrypting documents
this hints for pexpect; while I can provide regular expect scripts:
this is not directly a Python solution, but it should be easy to port.
as the tagline reads:
Pexpect makes Python a better tool for controlling other applications.
Encryption:
gpg --output doc.gpg --encrypt --recipient blake#cyb.org doc
as expect script; usage ./encrypt.exp doc blake#cyb.org 1234 (notice the space after the :):
#!/usr/bin/expect -f
set filename [lindex $argv 0]
set recipient [lindex $argv 1]
set passphrase [lindex $argv 2]
spawn gpg --output $filename.gpg --encrypt --recipient $recipient $filename
expect -exact "Enter pass phrase: "
send -- "$passphrase\r"
expect eof
Decryption:
gpg --output doc --decrypt doc.gpg
as expect script; usage: ./decrypt.exp doc 1234:
#!/usr/bin/expect -f
set filename [lindex $argv 0]
set passphrase [lindex $argv 1]
spawn gpg --output $filename --decrypt $filename.gpg
expect -exact "Enter pass phrase: "
send -- "$passphrase\r"
expect eof
Import:
keys can be imported into either key-chain with:
gpg --import somekey.sec
gpg --list-secret-keys
gpg --import somekey.pub
gpg --list-keys
there barely is anything to automate; however setting an imported key as "trusted" would require expect for automation. found this cheat-sheet, which has all commands on one page; and it also hints for: If you have multiple secret keys, it'll choose the correct one, or output an error if the correct one doesn't exist (which should confirm my comment below).
file ~/.gnupg/options is a user's options file; where one can eg. define the default key-server.
Since version 2.1.14, GPG supports the --recipient-file option, which lets you specify the public key to encrypt with without using the keyring. To quote the developer:
It is now possible to bypass the keyring and take the public key
directly from a file. That file may be a binary or an ascii armored
key and only the first keyblock from that file is used. A key
specified with this option is always fully trusted.
This option may be mixed with the standard -r options.
--hidden-recipient-file (or -F) is also available.
To futher assist some use cases the option
--no-keyring
has also been implemented. This is similar to
--no-default-keyring --keyring /dev/null
but portable to Windows and also ignores any keyring specified
(command line or config file).
So to encrypt, you would do:
gpg --output myfileenc --encrypt --recipient-file key.pub myfile
To automate, in addition to using expect or Python as explained in the other answers, you can also use the --batch option. (You will need to see which of the offered answers works best on your system).
No such option, however, is available for the secret key, and, as a matter of fact, the same version of PGP (2.1) deprecated the secring option in the --generate-key command, so this file is not even available any more. The generated key will need to be added to the keyring to be used for decryption.
I am using the ruby EC2 SDK, Version 2. The private key material of a key generated with EC2 is stored in a string. I am trying to generate the public key material that is necessary to import the key into EC2 using OpenSSL::PKey::RSA
After that I am trying to import the key pair.
It looks like this:
kk=OpenSSL::PKey::RSA.new my_private_key_material
pub=kk.public_key
ec2.import_key_pair({key_name: "my_key", public_key_material: pub.export})
The API is throwing this error:
*** Aws::EC2::Errors::InvalidKeyFormat Exception: Key is not in valid OpenSSH public key format
I am not sure what is wrong and how to generate the public key material correctly. I already tried to Base64 encode the public key string without success.
Edit
I tried a couple of new things.
I generated a new key using the EC2 web console from scratch and then geneerated the public one the way Raphael points out below with
openssl rsa -in mykey.pem -outform PEM -pubout -out mykey.pub
The key is not encrypted.
Whey trying to import the public key, either with the web console or by code, I get the same error.
Edit 2
I found this.
When generating the public key with a different command, it works:
ssh-keygen -y
The generated public key looks different. It starts with
ssh-rsa AAAAB3NzaC1yc2EAAAADA....
While the first generated one starts with
-----BEGIN PUBLIC KEY-----
MIIBIjANBgkqhkiG....
Now the question is how to generate the first format in ruby. I also found this post on different formats.
OK, I solved it by following this post.
It turned out the public key had to be generated in a different way
kk=OpenSSL::PKey::RSA.new my_private_key_material
key=kk.public_key
type = key.ssh_type
data = [ key.to_blob ].pack('m0')
openssh_format = "#{type} #{data}"
ec2.import_key_pair({key_name: "my_key", public_key_material: openssh_format})
The documentation suggests that key contents must be encoded in Base64 client-side, however this is not the case: The SSH key contents should be provided as-is, in the format "ssh-rsa XXXXX....".
Rory is right. In NodeJS below code worked.
let keypair = fs.readFileSync(homedir + '/.ssh/id_rsa.pub');
result = await ec2.importKeyPair({
KeyName: 'KeyPairName',
PublicKeyMaterial: keypair,
}).promise();
I m using Ruby SDK V3 and this is work for me to import key to AWS
#!/usr/bin/env ruby
require 'rubygems'
require 'aws-sdk-ec2'
key_content = File.read('/home/xxx/keys/sshkey.pub')
ec2_client = Aws::EC2::Client.new()
resp = ec2_client.import_key_pair({
dry_run: false,
key_name: "test-ruby-key",
public_key_material: key_content,
})
Hope it helpful!
I am using Chef with kitchen (1.5.0) and vagrant (1.8.1) to manage a user consistently with a new server. My user recipe looks like this:
include_recipe "users"
group 'sudo'
password_secret = Chef::EncryptedDataBagItem.load_secret(node['enterprise_sp']['secret_file'])
jays_password = Chef::EncryptedDataBagItem.load('user_secrets','jgodse', password_secret)['password']
shadow_password = `openssl passwd -1 -salt xyz #{jays_password}`.strip
user 'jgodse' do
action :create
group 'sudo'
system true
shell '/bin/bash'
home '/home/jgodse'
manage_home true
password shadow_password #added to /etc/shadow when chef runs
end
The unencrypted data bag was where I configured my password in the clear. I then encrypted the data bag with a knife command.
This works, but this seems like a really dirty way around the problem of setting my password. I had to do that because the password directive of the user block only takes the shadow password, and that can only be generated by shelling out to an openssl command.
Is there a cleaner way of getting the shadow password without shelling out to an openssl command which generates the password?
You should not be storing the password at all, just hash it beforehand and put the hash in the data bag in the first place. Also using encrypted data bags like this is scary-level unsafe, please take some time to familiarize yourself with the threat model of Chef's encryption tools, this ain't it.
At least pre-calculate the password hash and put that into the data bag.
See https://github.com/chef-cookbooks/users for inspiration.
This is a long shot, but I'm trying to add an ssh key to a Heroku for its use in connecting to another server through SFTP:
Net::SFTP.start(HOST, USER, password: PASSWORD, keys: ['yada.pem']) do |sftp|
#sftp = sftp
end
My original solution was to push a .ssh directory to the repo and store yada.pem there. keys would include the path to this file.
A safer solution I've been told would be to store the key in an environment variable on Heroku. Problem is, this would store the key as a string, which I couldn't really pass to SFTP.start.
I could solve the problem in a couple ways:
Is there a way to pass the key as a string with Ruby net/sftp?
Is there a way to add a public key to Heroku so that net/sftp would use it when trying to connect to the remote server?
Thanks
You can pass keys as strings in the option hash under the key :key_data (should be an array of strings, each element of which containing a key in PEM format).
Net::SFTP.start(HOST, USER, password: PASSWORD, key_data: ['PEM key as string']) do |sftp|
#sftp = sftp
end
See Net::SSH#start (to which Net::SFTP#start defers).