Update the content of an encrypted file using gpg2 and bash scripting - bash

Say there's an encrypted file file.txt
In order to update its content the file must be decrypted first, then run through the desired processing and afterwards it must be encrypted again.
(1) What would be the most straight forward way to do so in bash scripting using gpg2? The operation should request the user only once for a password to decrypt. It should use the same password for the final encryption afterwards.
Here is a most likely extremely unsecure, but working example of what I try to archieve:
function update-encrypted-file() {
read pass_tmp;
local pass=$pass_tmp;
unset pass_tmp;
local file="file.txt";
local tmp_result=$(cat $file | gpg2 --batch --passphrase $pass | update);
echo $tmp_result | gpg2 -c --batch --passphrase $pass > $file;
}
whereas update might be sth. like this: alias update="tr -d X" (delete all X's)
(2) What exactly could make the above example insecure? I guess using read itself is a no-go, but it would be interesting to see why. Not having the variable set locally causes the password to be inside the global space for some short time. Could that possibly be fetched? I could not figure out how to use pinentry-tty in this case (see this post)
(3) Apart from that, the gpg2 documentation remarks regarding the --passphrase option: "Obviously, this is of very questionable security on a multi-user system. Don't use this option if you can avoid it."
Is this only the case when manually used inside a terminal, since the commands are logged? Or would there also be concerns when using e.g. inside a function with the password having saved in only this function scope.

I'm not answering your individual questions directly, but have a slight discussion of password management and GnuPG, but for the third question: the command line of all processes from all users running on the machine are available to anybody. To confirm, just run a simple ps ax as an unprivileged user. Never pass secrets as parameters!
Obviously, the most secure option will be to never get hold of the passphrase at all. If you don't have it, you can't mess with it. With GnuPG 2.1, this was even applied to the actual GnuPG binary (gpg/gpg2): The most critical secret key operations (involving handling the passphrase) are performed by the small gpg-agent (thus having smaller attack surface), the rather large and complicated GnuPG binary neither gets direcct access to the key nor the passphrase.
This is also what I'd go for: instead of handling the passphrase, rely on gpg-agent doing so instead. It is available (and since GnuPG 2.1, also required) anyway. If configured properly (and this is the default), gpg-agent will cache the passphrase for a while. If the user configured something else, he decided he does not want cached passphrases, which you also should respect with your application.
As soon as required, the gpg-agent will query the user for the passphrase through the configured method -- if you're running a graphical user interface, this will likely be a window popping up.
If you mess with the GnuPG configuration (for example, your own configuration files, starting your own gpg-agent, ...), it's your job of course to take care of this. To start your own instance of gpg-agent to have full control over caching and other options, take advantage of the --options, --homedir and --no-use-standard-socket as required by your individual use case.
Finally, you're storing an intermediate result of the contents, which you echo:
local tmp_result=$(cat $file | gpg2 --batch --passphrase $pass | update);
echo $tmp_result | gpg2 -c --batch --passphrase $pass > $file;
Don't do this for the same reasons discussed for passphrases! Instead, directly pipe the result into the encryption process (and there is no need for cat here):
< $file gpg2 --batch --passphrase $pass | update | gpg2 -c --batch --passphrase $pass > $file;

Related

Automating PGP Decryption

I will be receiving PGP encrypted files from a customer through sFTP - I already have a process to automate pulling of files. Once I receive the encrypted file, I'd like to automate decryption.
I created a key pair with GoAnywhere OpenPGP Studio (public key will go to customer). I want to use the private key along with the secret passphrase in a batch file script that will run as a scheduled task in Windows Task Scheduler. This is my script:
gpg --keyring "C:\UserFolder\.openpgpstudio\keys\pubring.pkr" --secret-keyring "C:\UserFolder\.openpgpstudio\keys\secring.skr" --batch --yes --passphrase-fd "secretPassPhrase" -o "D:\FilePath\testPGP.txt" -d "D:\FilePath\testPGP.txt.pgp"
exit
When I try to run my script, there are still some manual steps needed to decrypt files - there are a couple issues I faced:
When executing the batch file, a command prompt window opens with the same script I have in my batch file. For decryption to occur and output a text file, I have to hit Enter. This is not automatic and would cause the scheduled task to get stuck.
Even though I am using the --passphrase-fd option, there are times when I am still prompted for the passphrase - this passphrase popup would also cause the scheduled task to get stuck running.
Is there a way to bypass these two manual steps so that my script is fully automated?
Follow up question: Does the -d option accept wildcard characters so that I can just decrypt any found file with a .pgp extension, and am I able to use the -o option to output a .txt file of the same name as the .pgp file?
--passphrase-fd option should be used with file descriptor (i.e. number like 3, 4, 5 and so on), not the password string. You should use --passphrase option, adding --pinentry-mode=loopback. Currently most likely it works since password is asked via popup and cached.
-d with wildcards doesn't work, however you may use simple script to iterate over all files with pgp extension

How to enter gnupg-agent key passhprase from CLI?

I do sign commits with git, and it's a big problem that I can't enter gpg key passphprase from anywhere except CLI. If I'll do commit in vscode for example, it will fail. So I've came up with idea to just simply input passphrase from CLI, and cache it for some period of time.
I'm using fish shell in here so here's a config:
set -x GPG_TTY (tty)
eval (gpg-agent --daemon --allow-preset-passphrase --default-cache-ttl 43200)
As I understand I need to enable to preset the passhprase for the agent. So now what's next?
I've tried to preset a key like this, but it fails:
$ echo mypassphrase | /usr/lib/gnupg2/gpg-preset-passphrase -c E2AB66331DA5CA780B7B1FA5D4BF11DA1E39EDFF
gpg-preset-passphrase: caching passphrase failed: Not supported
I've googled everything I could, but no one is answering this question anywhere. Would be nice to have something like ssh-add, you just add a private key, and enter password, wonder why gpg-agent haven't adopted this nice design.
Seems to be solved. There's a need to add couple of things to ~/.gnupg/gpg-agent.conf:
default-cache-ttl 46000
pinentry-program /usr/bin/pinentry-gtk-2
allow-preset-passphrase
Reload gpg-agent
gpg-connect-agent reloadagent /bye

How to circumvent "apt-key output should not be parsed"?

I'm automating my Docker installation. Something like this:
if apt-key fingerprint 0EBFCD88 | grep "Key fingerprint = 9DC8 5822 9FC7 DD38 854A E2D8 8D81 803C 0EBF CD88" > /dev/null
then
# proceed
fi
This worked fine in older versions of apt-key, but recent versions have two issues:
A different output format: I can hack around that
A warning:
Warning: apt-key output should not be parsed (stdout is not a terminal)
Clearly, I can hack around this as well, just redirect stderr to /dev/null. It just made me curious:
How do these fine folks suggest I verify my key fingerprints? Or am I getting this fundamentally wrong by wanting to automate it, does that defeat the point? (I think not, since I still manually lifted the expected fingerprint from the website, but feel free to tell me otherwise...)
From apt-key sources, you can set APT_KEY_DONT_WARN_ON_DANGEROUS_USAGE to a non-empty value to disable this warning.
You can also use "grep -q" so you don't need to redirect stdout in /dev/null either.
To reply to the part "am I getting this fundamentally wrong":
I believe that possibly yes. That's the reason of the warning.
You don't write what you do exactly, but one thing to realize is:
The keys do have some expiration so after some time the fingerprint in your script will become obsolete and possibly it will not behave as expected anymore.
Once the repository and its key is installed so the system somehow upgrades the key automatically but when the repository is initially added so an up-to-date key has to be provided. In my scripts automating the installation I do not test whether a key was already added but I test whether the repository was already added. If the repository was not added, so I add it together with an up-to-date key that I download always from its URL.
I'm modifying the command itself to use batch mode so it will not complain about stdout. For the bigger apt-key this will work, quickly tested on debian:
sed -i "s%{GPG_EXE}\")' --%{GPG_EXE}\")' --batch --%g" /usr/bin/apt-key
While for the smaller apt-key this could work (untested as I can't recall exactly where I have seen this simpler variant):
sed -i 's%GPG_CMD="gpg %GPG_CMD="gpg --batch %g' /usr/bin/apt-key
You need privileges to write to the /usr/bin/apt-key so either run as root or use sudo
This works
apt-key exportall > test.key 2>/dev/null
or
apt-key exportall 2>&1 | grep -v '^Warning' > test.key

To send encrypted messages with gpg in php 7 it ask for password

We recently switched over to Ubuntu 16.04 and there for php 7.0
No i run in to problem with one of our php scripts, this script updates ripe using the mail interface of ripe and send GPG signed mail to them, witch they use for authentication. But now the php cli starts to prompt for a password to use with the key. How can i prevent this passphrase question i tryed to remove the passphrase from the key but this didn't help.
my php code looks like this:
$gpgkey = "some thubprint of a key";
$gpgpwd = "Secret";
$res = gnupg_init();
gnupg_addsignkey($res,$gpgkey,$gpgpwd);
$updatemessages = "test";
$signedupdatemessages = gnupg_sign($res, $updatemessages);
Hope some knows the sollution
With kind regards,
Bas van den Dikkenberg
Ubuntu 16.04 ships a rather recent version of GnuPG, which does not allow setting a passphrase without pinentry without further hassles. This is because the rather large (and thus prone to security issues) GnuPG application should not be able to access private keys and passphrases, thus these operations are extracted to the smaller gpg-agent (which is now querying for the passphrase).
Removing the passphrase/setting it to an empty one should work out well, be sure to set the passphrase for the proper user (GnuPG has per-system-user keystores).
The "GnuPG-2.0-way" of setting a passphrase is using the gpg-preset-passphrase utility, which is usually installed in /usr/lib/gnupg2/gpg-preset-passphrase for Debian and derived distributions. You need to allow this operation by adding allow-preset-passphrase to ~/.gnupg/gpg-agent.conf, and run
/usr/lib/gnupg2/gpg-preset-passphrase -c [keygrip]
while providing the passphrase on STDIN (do not use echo or the -P option, using both ways all other system users can read the passphrase!). The keygrip can be obtained by running gpg2 --with-keygrip --list-secret-keys.

How to use Gnupg's passphrase-fd argument?

I would like to use GnuPG´s decrypt command without any user interation. The script's --passphrase-fd argument seems exactly what I need. But I don't know how it works - haven't found examples.
Could anyone give me an example of such a command, on both Windows and UNIX environments?
(FYI, I'm using GnuPG 2).
Thanks already :)
In order to use the gpg option --passphrase-fd in GnuPG v2, you must specify the --batch parameter. I will first explain how --passphrase-fd works, and then get to the examples.
--passphrase-fd tells GnuPG which file descriptor (-fd) to expect the passphrase to come from. The standard file descriptors are STDIN (0), STDOUT (1) and STDERR (2). For the context of this question, you would normally only be concerned about STDIN (0).
You didn't specify where you want the passphrase to come from, so I will demonstrate the usage of STDIN (standard in) in a variety of ways.
--passphrase-fd 0 tells GnuPG to retrieve the passphrase from input into the current shell; so for example if you want GnuPG to get the passphrase data in the very next line of console input, the command and output would be like so:
gpg2 --batch --passphrase-fd 0 --armor --decrypt /path/to/encrypted_file.pgp
<next line of input is passphrase followed by hitting enter>
gpg: encrypted with 1024-bit RSA key, ID EC18C175, created 2013-10-26
"testkey4321 (4321) <test#4321.com>"
this is a test... this is only a test...
In the above example, the passphrase was provided via file descriptor 0 (STDIN) - which we provided by entering it on the shells current standard input.
In the next example, we will tell GnuPG to retrieve the passphrase from input into the current shell that is actually the output of another command (echo, in this case, which merely "echos" what you tell it to):
echo "mypassphrase" | gpg2 --batch --passphrase-fd 0 --armor --decrypt /path/to/encrypted_file.pgp
gpg: encrypted with 1024-bit RSA key, ID EC18C175, created 2013-10-26
"testkey4321 (4321) <test#4321.com>"
this is a test... this is only a test...
Another example that dumps the contents of a file that contains the passphrase to STDIN -
cat /path/to/file_with_passphrase | gpg2 --batch --passphrase-fd 0 --armor --decrypt /path/to/encrypted_file.pgp
gpg: encrypted with 1024-bit RSA key, ID EC18C175, created 2013-10-26
"testkey4321 (4321) <test#4321.com>"
this is a test... this is only a test...
In summary, --passphrase-fd just tells GnuPG that you want to feed it the requisite passphrase via a standard file descriptor; the difference between GnuPG v2 and GnuPG is merely the --batch parameter.
The above examples should work the same in Windows and *nix environments, with the only difference being that in Windows - depending on your configuration and version - you will have to replace cat with type in order to dump the contents of a file to STDIN.
kylehuff's answer still wouldn't work for me, with gpupg still popping up a password prompt.
According to https://wiki.archlinux.org/index.php/GnuPG#Unattended_passphrase with gnupg version 2.1.0 and higher, you need to do additional steps to support --passphrase-fd
First, edit the gpg-agent configuration to allow loopback pinentry mode:
~/.gnupg/gpg-agent.conf
allow-loopback-pinentry
Restart the gpg-agent process if it is running to let the change take effect.
Second, either the application needs to be updated to include a commandline parameter to use loopback mode like so:
$ gpg --pinentry-mode loopback ...
Using GPG4win/gpg 2.2.3: to use the passphrase-fd 0 and bypass the prompt, I can confirm that the following works:
--pinentry-mode loopback
As I've had to recently figure this out myself I thought it might be worth chiming in.
The answer by kylehuff is very good if you're decryping files, however, if you've need of input/output redirection, such as piping, here's an example of using a non-0 file descriptor to pass the passphrase.
#!/usr/bin/env bash
# Set some variables for easy modding
Var_fd='9'
Var_pass_location="/path/to/passphrase.file"
Var_gpg_decrypt_opts="--passphrase-fd ${Var_fd} --decrypt"
Var_output_location="out.txt"
Arr_string=( "$#" )
# Open file descriptor and shove the passphrase file into it
exec ${Var_fd}<${Var_pass_location}
# Pipe input array though gpg and append to output file
cat <<<"${Arr_string[*]}" | $(which gpg) ${Var_gpg_decrypt_opts} >> ${Var_output_location}
# Do not forget to close the file descriptor
exec ${Var_fd}>&-
Do be warned, outside of special use cases, that saving your private keys passphrase is generally seen as a bad idea or bad security practice. -Also please don't forget to close the descriptor when finished so that your passphrase isn't accessible via that method anymore.- Often I've seen advised in these use cases to use specifically non-passphrase protected keys but that's totally your choose. If you like the above code then you may want to also checkout the script I debugged for key generation either unattended or attended because it covers even less commonly used gpg file descriptor options.
Edits/updates
So I've been debugging the bulk decryption operations and have evidence to show that file descriptors seem to close automatically or perhaps it's auto closed by GnuPG. Check build 152 all the way at the bottom of the raw logs, just before diff checks, you'll find that the first block of encrypted data ate the passphrase leaving the next two blocks of data without a valid passphrase. The related scripts in this operation are ; first the script_decrypt.sh build script sets the test key's passphrase to file descriptor 9 as shown in above examples, then the Helper script is called such that it'll make use of that file descriptor... it's a funky use case but the moral of the story seems to be that what ever bulk decryption operations you plan to implement with GnuPG file descriptors will likely need to follow the steps outlined above as a whole function to properly have the file descriptors reopened. I'll be rewriting the helper script over the next few pushes so check the Travis-CI build logs greater than 152 to find if I've a solution to where file descriptors get closed...
... so that only took two tries to get things working, see the difference in build 154 both the encrypted file and raw input log match. As hypothesised the file descriptors get dumped after first usage by either GnuPG or a sub shell, thus the passphrase needs to be assigned before every decrypt command for bulk decryption to happen.
Hope this was valuable to y'all.

Resources