I am writing a bash script on a machine without expect.
The script automates the build of a helm (v2.9.1) chart.
I've got to the part in the script where it runs a
helm package --sign --key 'mykey' --keyring 'path/to/keyring' -d chartdir chart
and because helm prompts for the passphrase of the gpg key I created in order to use provenance files (a requirement), I cannot script around the soliciting of the passphrase, which I would like to prompt for separately as the script is part of a CI/CD build chain.
I have tried using
yes $promptedPassPhrase | helm package --sign...
and I got
xrealloc: cannot allocate 18446744071562067968 bytes (237568 bytes allocated)
I also tried plain old
echo $promptedPassPhrase | helm package --sign...
and I got
Error: inappropriate ioctl for device
I also tried script and got the same response. As I do not have expect on the server, I cannot expect my way round it, so I'm stumped as to how to automate the helm package command and am not going to use a key without a passphrase as it is bad practice.
I don't know how long this has been the case, or exactly which versions this works on, but this doc page says:
If your PGP private key has a passphrase, [...] You can set the HELM_KEY_PASSPHRASE environment variable to that passphrase in case you don't want to be prompted to enter the passphrase.
This works for me on v2.13.1. It looks like it was added in October of 2018, so my guess is that it was first available in v2.12.0-rc.1.
Although this doesn't directly answer the OP's question (since they asked about v2.9.1), it will hopefully help anyone who ends up here because they (like me the first time round) missed that line in the doc.
Related
Edit: As of 01/31/2023 the scripts that I am using below ARE working. Any patterns of inconsistencies that I find I will report here. Would like to leave this open in case others have findings/advice they are interested in sharing in relation to bash scripting/LastPass CLI/WSL
I am looking to use the LastPass CLI to make some changes to Shared Sites within our LastPass enterprise. I was able to write the scripts (fortunately with some help from others on here), however I am unable to get the commands to work properly within a script.
One of the commands that I WAS having troubles with was lpass share create. This command worked directly from the command line, but I was unable to run this command within a script successfully. I have a very simple script, similar to the one below:
#!/bin/bash
folderpath=$1
lpassCreateStoreFolder(){
lpass share create "$folderpath"
}
lpassLogin(){
echo 'testPWD' | LPASS_DISABLE_PINENTRY=1 lpass login --trust --force tester#test.com
}
lpassLogin
lpassCreateStoreFolder
I've been invoking my script through the PowerShell command line like so:
wsl "path/to/script" "Shared-00 Test LastPass CLI"
Sometimes this command works within the script and other times it does not. When I tried running this script around mid December, I had no success at all. The script would run through all the way, the CLI would even give me a response
Folder Shared-00 Test LastPass CLI created.
and the LastPass Admin Console logs show me a report of "Create Shared Folder". The problem is when I go to my LastPass Vault, the Shared Folder was rarely/if ever created. Running the command without a script, directly from the command line worked almost 100% of the time. I initially chalked this up to inconsistencies on their end, but now I am experiencing these same problems with a different command.
Similarly I have been using the lpass edit command to make edits to sites within our LastPass vault. Once again, I have a relatively simple script to make the edit to the site:
#!/bin/bash
lpassId=$1
lpassSetNotes(){
printf "Notes:\n What are your notes?\nThese are my notes" | lpass edit --non-interactive --sync=now "$lpassId"
}
lpassLogin(){
echo 'testPWD' | LPASS_DISABLE_PINENTRY=1 lpass login --trust --force test#test.com
}
lpassLogin
lpassSetNotes
and have been invoking this script through Powershell like so:
wsl "path/to/script" "000LastPassID000"
like the lpass share create command, running the script does not produce the desired output. The script runs all the way through and my changes are reflected in the logs, but when I go to the vault the site itself is never changed. The command DOES however work when I run it from the command line directly within WSL.
I am relatively new to writing Bash scripts/the Linux operating system, so I'm not entirely sure if this something wrong on my end or just the vendor's tool that I am utilizing producing inconsistencies. Any help would be appreciated, I know this issue might be hard to replicate without a LastPass account
Example LastPass CLI calls that work directly from command line in WSL
lpass share create "Shared-00 Testing LastPass CLI"
printf "Notes:\n What are your notes?\nThese are my notes" | lpass edit --non-interactive --sync=now "$lpassId"
References
LastPass CLI
CLI Manual
CLI GitHub
I will be receiving PGP encrypted files from a customer through sFTP - I already have a process to automate pulling of files. Once I receive the encrypted file, I'd like to automate decryption.
I created a key pair with GoAnywhere OpenPGP Studio (public key will go to customer). I want to use the private key along with the secret passphrase in a batch file script that will run as a scheduled task in Windows Task Scheduler. This is my script:
gpg --keyring "C:\UserFolder\.openpgpstudio\keys\pubring.pkr" --secret-keyring "C:\UserFolder\.openpgpstudio\keys\secring.skr" --batch --yes --passphrase-fd "secretPassPhrase" -o "D:\FilePath\testPGP.txt" -d "D:\FilePath\testPGP.txt.pgp"
exit
When I try to run my script, there are still some manual steps needed to decrypt files - there are a couple issues I faced:
When executing the batch file, a command prompt window opens with the same script I have in my batch file. For decryption to occur and output a text file, I have to hit Enter. This is not automatic and would cause the scheduled task to get stuck.
Even though I am using the --passphrase-fd option, there are times when I am still prompted for the passphrase - this passphrase popup would also cause the scheduled task to get stuck running.
Is there a way to bypass these two manual steps so that my script is fully automated?
Follow up question: Does the -d option accept wildcard characters so that I can just decrypt any found file with a .pgp extension, and am I able to use the -o option to output a .txt file of the same name as the .pgp file?
--passphrase-fd option should be used with file descriptor (i.e. number like 3, 4, 5 and so on), not the password string. You should use --passphrase option, adding --pinentry-mode=loopback. Currently most likely it works since password is asked via popup and cached.
-d with wildcards doesn't work, however you may use simple script to iterate over all files with pgp extension
I do sign commits with git, and it's a big problem that I can't enter gpg key passphprase from anywhere except CLI. If I'll do commit in vscode for example, it will fail. So I've came up with idea to just simply input passphrase from CLI, and cache it for some period of time.
I'm using fish shell in here so here's a config:
set -x GPG_TTY (tty)
eval (gpg-agent --daemon --allow-preset-passphrase --default-cache-ttl 43200)
As I understand I need to enable to preset the passhprase for the agent. So now what's next?
I've tried to preset a key like this, but it fails:
$ echo mypassphrase | /usr/lib/gnupg2/gpg-preset-passphrase -c E2AB66331DA5CA780B7B1FA5D4BF11DA1E39EDFF
gpg-preset-passphrase: caching passphrase failed: Not supported
I've googled everything I could, but no one is answering this question anywhere. Would be nice to have something like ssh-add, you just add a private key, and enter password, wonder why gpg-agent haven't adopted this nice design.
Seems to be solved. There's a need to add couple of things to ~/.gnupg/gpg-agent.conf:
default-cache-ttl 46000
pinentry-program /usr/bin/pinentry-gtk-2
allow-preset-passphrase
Reload gpg-agent
gpg-connect-agent reloadagent /bye
I would like to use GnuPG´s decrypt command without any user interation. The script's --passphrase-fd argument seems exactly what I need. But I don't know how it works - haven't found examples.
Could anyone give me an example of such a command, on both Windows and UNIX environments?
(FYI, I'm using GnuPG 2).
Thanks already :)
In order to use the gpg option --passphrase-fd in GnuPG v2, you must specify the --batch parameter. I will first explain how --passphrase-fd works, and then get to the examples.
--passphrase-fd tells GnuPG which file descriptor (-fd) to expect the passphrase to come from. The standard file descriptors are STDIN (0), STDOUT (1) and STDERR (2). For the context of this question, you would normally only be concerned about STDIN (0).
You didn't specify where you want the passphrase to come from, so I will demonstrate the usage of STDIN (standard in) in a variety of ways.
--passphrase-fd 0 tells GnuPG to retrieve the passphrase from input into the current shell; so for example if you want GnuPG to get the passphrase data in the very next line of console input, the command and output would be like so:
gpg2 --batch --passphrase-fd 0 --armor --decrypt /path/to/encrypted_file.pgp
<next line of input is passphrase followed by hitting enter>
gpg: encrypted with 1024-bit RSA key, ID EC18C175, created 2013-10-26
"testkey4321 (4321) <test#4321.com>"
this is a test... this is only a test...
In the above example, the passphrase was provided via file descriptor 0 (STDIN) - which we provided by entering it on the shells current standard input.
In the next example, we will tell GnuPG to retrieve the passphrase from input into the current shell that is actually the output of another command (echo, in this case, which merely "echos" what you tell it to):
echo "mypassphrase" | gpg2 --batch --passphrase-fd 0 --armor --decrypt /path/to/encrypted_file.pgp
gpg: encrypted with 1024-bit RSA key, ID EC18C175, created 2013-10-26
"testkey4321 (4321) <test#4321.com>"
this is a test... this is only a test...
Another example that dumps the contents of a file that contains the passphrase to STDIN -
cat /path/to/file_with_passphrase | gpg2 --batch --passphrase-fd 0 --armor --decrypt /path/to/encrypted_file.pgp
gpg: encrypted with 1024-bit RSA key, ID EC18C175, created 2013-10-26
"testkey4321 (4321) <test#4321.com>"
this is a test... this is only a test...
In summary, --passphrase-fd just tells GnuPG that you want to feed it the requisite passphrase via a standard file descriptor; the difference between GnuPG v2 and GnuPG is merely the --batch parameter.
The above examples should work the same in Windows and *nix environments, with the only difference being that in Windows - depending on your configuration and version - you will have to replace cat with type in order to dump the contents of a file to STDIN.
kylehuff's answer still wouldn't work for me, with gpupg still popping up a password prompt.
According to https://wiki.archlinux.org/index.php/GnuPG#Unattended_passphrase with gnupg version 2.1.0 and higher, you need to do additional steps to support --passphrase-fd
First, edit the gpg-agent configuration to allow loopback pinentry mode:
~/.gnupg/gpg-agent.conf
allow-loopback-pinentry
Restart the gpg-agent process if it is running to let the change take effect.
Second, either the application needs to be updated to include a commandline parameter to use loopback mode like so:
$ gpg --pinentry-mode loopback ...
Using GPG4win/gpg 2.2.3: to use the passphrase-fd 0 and bypass the prompt, I can confirm that the following works:
--pinentry-mode loopback
As I've had to recently figure this out myself I thought it might be worth chiming in.
The answer by kylehuff is very good if you're decryping files, however, if you've need of input/output redirection, such as piping, here's an example of using a non-0 file descriptor to pass the passphrase.
#!/usr/bin/env bash
# Set some variables for easy modding
Var_fd='9'
Var_pass_location="/path/to/passphrase.file"
Var_gpg_decrypt_opts="--passphrase-fd ${Var_fd} --decrypt"
Var_output_location="out.txt"
Arr_string=( "$#" )
# Open file descriptor and shove the passphrase file into it
exec ${Var_fd}<${Var_pass_location}
# Pipe input array though gpg and append to output file
cat <<<"${Arr_string[*]}" | $(which gpg) ${Var_gpg_decrypt_opts} >> ${Var_output_location}
# Do not forget to close the file descriptor
exec ${Var_fd}>&-
Do be warned, outside of special use cases, that saving your private keys passphrase is generally seen as a bad idea or bad security practice. -Also please don't forget to close the descriptor when finished so that your passphrase isn't accessible via that method anymore.- Often I've seen advised in these use cases to use specifically non-passphrase protected keys but that's totally your choose. If you like the above code then you may want to also checkout the script I debugged for key generation either unattended or attended because it covers even less commonly used gpg file descriptor options.
Edits/updates
So I've been debugging the bulk decryption operations and have evidence to show that file descriptors seem to close automatically or perhaps it's auto closed by GnuPG. Check build 152 all the way at the bottom of the raw logs, just before diff checks, you'll find that the first block of encrypted data ate the passphrase leaving the next two blocks of data without a valid passphrase. The related scripts in this operation are ; first the script_decrypt.sh build script sets the test key's passphrase to file descriptor 9 as shown in above examples, then the Helper script is called such that it'll make use of that file descriptor... it's a funky use case but the moral of the story seems to be that what ever bulk decryption operations you plan to implement with GnuPG file descriptors will likely need to follow the steps outlined above as a whole function to properly have the file descriptors reopened. I'll be rewriting the helper script over the next few pushes so check the Travis-CI build logs greater than 152 to find if I've a solution to where file descriptors get closed...
... so that only took two tries to get things working, see the difference in build 154 both the encrypted file and raw input log match. As hypothesised the file descriptors get dumped after first usage by either GnuPG or a sub shell, thus the passphrase needs to be assigned before every decrypt command for bulk decryption to happen.
Hope this was valuable to y'all.
Alright, here's what I'm trying to do. I'm attempting to write a quick build script in bash that will check out a private repository from GitHub on a remote server. To do this a "hands off" as possible, I want to generate a local RSA key set on the remote server and add the public key as a Deploy Key for that particular repository. I know how to do this using GitHub's API, but I'm having trouble building the JSON payload using Bash.
So far, I have this particular process included below:
#!/bin/bash
ssh-keygen -t rsa -N '' -f ~/.ssh/keyname -q
public_key=`cat ~/.ssh/keyname.pub`
curl -u 'username:password' -d '{"title":"Test Deploy Key", "key":"'$public_key'"}' -i https://api.github.com/repos/username/repository/keys
It's just not properly building the payload. I'm not an expert when it comes to string manipulation in Bash, so I could seriously use some assistance. Thanks!
It's not certain, but it may help to quote where you use public_key, i.e.
curl -u 'username:password' \
-d '{"title":"Test Deploy Key", "key":"'"$public_key"'"}' \
-i https://api.github.com/repos/username/repository/keys
Otherwise it will be much easier to debug if you use the shell's debugging options set -vx near the top of your bash script.
You'll see each line of code (or block (for, while, etc) as it is in your file. Then you see each line of code with the variables expanded to their values.
If you're still stuck, edit your post to show the expanded values of variables for the problem line in your script. What you have looks reasonable at first glance.