I'm automating my Docker installation. Something like this:
if apt-key fingerprint 0EBFCD88 | grep "Key fingerprint = 9DC8 5822 9FC7 DD38 854A E2D8 8D81 803C 0EBF CD88" > /dev/null
then
# proceed
fi
This worked fine in older versions of apt-key, but recent versions have two issues:
A different output format: I can hack around that
A warning:
Warning: apt-key output should not be parsed (stdout is not a terminal)
Clearly, I can hack around this as well, just redirect stderr to /dev/null. It just made me curious:
How do these fine folks suggest I verify my key fingerprints? Or am I getting this fundamentally wrong by wanting to automate it, does that defeat the point? (I think not, since I still manually lifted the expected fingerprint from the website, but feel free to tell me otherwise...)
From apt-key sources, you can set APT_KEY_DONT_WARN_ON_DANGEROUS_USAGE to a non-empty value to disable this warning.
You can also use "grep -q" so you don't need to redirect stdout in /dev/null either.
To reply to the part "am I getting this fundamentally wrong":
I believe that possibly yes. That's the reason of the warning.
You don't write what you do exactly, but one thing to realize is:
The keys do have some expiration so after some time the fingerprint in your script will become obsolete and possibly it will not behave as expected anymore.
Once the repository and its key is installed so the system somehow upgrades the key automatically but when the repository is initially added so an up-to-date key has to be provided. In my scripts automating the installation I do not test whether a key was already added but I test whether the repository was already added. If the repository was not added, so I add it together with an up-to-date key that I download always from its URL.
I'm modifying the command itself to use batch mode so it will not complain about stdout. For the bigger apt-key this will work, quickly tested on debian:
sed -i "s%{GPG_EXE}\")' --%{GPG_EXE}\")' --batch --%g" /usr/bin/apt-key
While for the smaller apt-key this could work (untested as I can't recall exactly where I have seen this simpler variant):
sed -i 's%GPG_CMD="gpg %GPG_CMD="gpg --batch %g' /usr/bin/apt-key
You need privileges to write to the /usr/bin/apt-key so either run as root or use sudo
This works
apt-key exportall > test.key 2>/dev/null
or
apt-key exportall 2>&1 | grep -v '^Warning' > test.key
Related
I have been following the steps of the courses pre-work, including:
checking for, generating, copy/paste, and
saving the SSH keys to GitHub.
But when I am instructed to check the matching fingerprints using "ssh -T git#github.com", the prints don't match.
I've even started from the beginning clear through, but they still don't match.
Thought I'd reach out here before using my 1 tutoring.
Hopefully the screenshot showing what I see helps(link).
EDIT- I understand there's some stuff in there that shouldn't be, I was just trying things for diff results. I would just like to know where I went wrong and how to avoid it.
What you ssh is the remote site SSH key fingerprint, not you registered SSH key fingerprint.
You see (or should see if you are contacting the correct github.com) the fingerprints exposed with api.github.com/meta as explained here.
Using jq, you can add them to your ~/.ssh/known_hosts with:
curl --silent https://api.github.com/meta \
| jq --raw-output '"github.com "+.ssh_keys[]' >> ~/.ssh/known_hosts
From there, you can test your connection with ssh -Tv github.com, and check if you see a welcome message:
Hi username!
You've successfully authenticated, but GitHub does not provide shell access
I do sign commits with git, and it's a big problem that I can't enter gpg key passphprase from anywhere except CLI. If I'll do commit in vscode for example, it will fail. So I've came up with idea to just simply input passphrase from CLI, and cache it for some period of time.
I'm using fish shell in here so here's a config:
set -x GPG_TTY (tty)
eval (gpg-agent --daemon --allow-preset-passphrase --default-cache-ttl 43200)
As I understand I need to enable to preset the passhprase for the agent. So now what's next?
I've tried to preset a key like this, but it fails:
$ echo mypassphrase | /usr/lib/gnupg2/gpg-preset-passphrase -c E2AB66331DA5CA780B7B1FA5D4BF11DA1E39EDFF
gpg-preset-passphrase: caching passphrase failed: Not supported
I've googled everything I could, but no one is answering this question anywhere. Would be nice to have something like ssh-add, you just add a private key, and enter password, wonder why gpg-agent haven't adopted this nice design.
Seems to be solved. There's a need to add couple of things to ~/.gnupg/gpg-agent.conf:
default-cache-ttl 46000
pinentry-program /usr/bin/pinentry-gtk-2
allow-preset-passphrase
Reload gpg-agent
gpg-connect-agent reloadagent /bye
Say there's an encrypted file file.txt
In order to update its content the file must be decrypted first, then run through the desired processing and afterwards it must be encrypted again.
(1) What would be the most straight forward way to do so in bash scripting using gpg2? The operation should request the user only once for a password to decrypt. It should use the same password for the final encryption afterwards.
Here is a most likely extremely unsecure, but working example of what I try to archieve:
function update-encrypted-file() {
read pass_tmp;
local pass=$pass_tmp;
unset pass_tmp;
local file="file.txt";
local tmp_result=$(cat $file | gpg2 --batch --passphrase $pass | update);
echo $tmp_result | gpg2 -c --batch --passphrase $pass > $file;
}
whereas update might be sth. like this: alias update="tr -d X" (delete all X's)
(2) What exactly could make the above example insecure? I guess using read itself is a no-go, but it would be interesting to see why. Not having the variable set locally causes the password to be inside the global space for some short time. Could that possibly be fetched? I could not figure out how to use pinentry-tty in this case (see this post)
(3) Apart from that, the gpg2 documentation remarks regarding the --passphrase option: "Obviously, this is of very questionable security on a multi-user system. Don't use this option if you can avoid it."
Is this only the case when manually used inside a terminal, since the commands are logged? Or would there also be concerns when using e.g. inside a function with the password having saved in only this function scope.
I'm not answering your individual questions directly, but have a slight discussion of password management and GnuPG, but for the third question: the command line of all processes from all users running on the machine are available to anybody. To confirm, just run a simple ps ax as an unprivileged user. Never pass secrets as parameters!
Obviously, the most secure option will be to never get hold of the passphrase at all. If you don't have it, you can't mess with it. With GnuPG 2.1, this was even applied to the actual GnuPG binary (gpg/gpg2): The most critical secret key operations (involving handling the passphrase) are performed by the small gpg-agent (thus having smaller attack surface), the rather large and complicated GnuPG binary neither gets direcct access to the key nor the passphrase.
This is also what I'd go for: instead of handling the passphrase, rely on gpg-agent doing so instead. It is available (and since GnuPG 2.1, also required) anyway. If configured properly (and this is the default), gpg-agent will cache the passphrase for a while. If the user configured something else, he decided he does not want cached passphrases, which you also should respect with your application.
As soon as required, the gpg-agent will query the user for the passphrase through the configured method -- if you're running a graphical user interface, this will likely be a window popping up.
If you mess with the GnuPG configuration (for example, your own configuration files, starting your own gpg-agent, ...), it's your job of course to take care of this. To start your own instance of gpg-agent to have full control over caching and other options, take advantage of the --options, --homedir and --no-use-standard-socket as required by your individual use case.
Finally, you're storing an intermediate result of the contents, which you echo:
local tmp_result=$(cat $file | gpg2 --batch --passphrase $pass | update);
echo $tmp_result | gpg2 -c --batch --passphrase $pass > $file;
Don't do this for the same reasons discussed for passphrases! Instead, directly pipe the result into the encryption process (and there is no need for cat here):
< $file gpg2 --batch --passphrase $pass | update | gpg2 -c --batch --passphrase $pass > $file;
(kubuntu, but trying to remain platform independent in my approaches.)
Either of conceptual explanations or necessary steps are appreciated.
I'm reading through the documentation, and tried to get gpg-agent to work once, but am getting discouraged as each attempt will require a reboot, and I don't really understand how it works in the first place (just cutting and pasting code).
I want to use gpg-agent so that I can try out GNUS without entering passphrase on .authinfo.gpg every time I open GNUS.
I'd like to understand more about the process before I dig into the problem more.
Is a gpg --gen-key necessary before using gpg-agent for .authinfo.gpg?
Are the config lines mentioning SSH necessary for my minimal use of gpg?
Is .authinfo.gpg to take position on some keychain? Does that keychain itself need a master key?
Please correct me on the imagined flow of the process. (Is an absent key needed?). I regret that it is very sparse.
After boot, initialization turns on the gpg daemon, and other related settings are made.
When GNUS accesses .authinfo.gpg, it caches something with gpg-agent.
Here are some more details I consider.
The following variables are set
GPG_AGENT_INFO ${HOME}/.gpg-agent-info
GPG_TTY $(TTY)
In ~/.gnupg/gpg.conf, we need to see use-agent. I leave the dummy #default-key ******** commented out, as I believe it's only needed if you have more than one key.
In ~/.gnupg/gpg-agent.conf, we need to see
pinentry-program /usr/bin/pinentry-qt4
no-grab
default-cache-ttl 1800
(Though, pinentry-qt4 might be replaced with other present versions, such as pinentry-curses.)
There may be some redundancies, but I've also seen lines for I assume .bashrc, such as eval $(gpg-agent --daemon), or
gpg-agent --daemon \
--write-env-file "${HOME}/.gpg-agent-info"
(I've omitted lines regarding SSH, as I assume I don't need this for now.)
I've seen a troubleshooting command
echo "test" | gpg -ase -r 0xMYKEYID | gpg
but since I wasn't sure if I needed a key in the first place, I didn't go further with this.
Here is my very amateur tutorial in reply to my own question.
To check if gpg-agent is already enabled, try
ps aux | grep gpg
I find
iam#heeere:/e$ ps aux | grep gpg
iam 1490 0.0 0.0 16728 900 ? Ss 17:25 0:00 gpg-agent --daemon --sh
iam 2611 0.0 0.0 11748 912 pts/0 S+ 17:33 0:00 grep --color=auto gpg
This together with
(setq epg-gpg-program "/usr/bin/gpg2")
seemed to solve my problem. As Jens Erat pointed out, gpg-agent is associated with gpg2, not gpg.
In fact, I deleted all the modifications I had made from
GnuPG and EasyPG Assistant Configuration - Emacs auth-source Library,
and it still worked perfectly. That is, no eval $(gpg-agent --daemon), no gpg-agent.conf needed. Though, I may add some back, for instance default-cache-ttl. And while pinentry-curses looks appealing, I think greater minds than mine have struggled with preventing gpg-agent using pop-up from Emacs, so I'll leave that alone.
I wrote a simple script which mails out svn activity logs nightly to our developers. Until now, I've run it on the same machine as the svn repository, so I didn't have to worry about authentication, I could just use svn's file:/// address style.
Now I'm running the script on a home computer, accessing a remote repository, so I had to change to svn+ssh:// paths. With ssh-key nicely set up, I don't ever have to enter passwords for accessing the svn repository under normal circumstances.
However, crontab did not have access to my ssh-keys / ssh-agent. I've read about this problem a few places on the web, and it's also alluded to here, without resolution:
Why ssh fails from crontab but succedes when executed from a command line?
My solution was to add this to the top of the script:
### TOTAL HACK TO MAKE SSH-KEYS WORK ###
eval `ssh-agent -s`
This seems to work under MacOSX 10.6.
My question is, how terrible is this, and is there a better way?
In addition...
If your key have a passhphrase, keychain will ask you once (valid until you reboot the machine or kill the ssh-agent).
keychain is what you need! Just install it and add the follow code in your .bash_profile:
keychain ~/.ssh/id_dsa
So use the code below in your script to load the ssh-agent environment variables:
. ~/.keychain/$HOSTNAME-sh
Note: keychain also generates code to csh and fish shells.
Copied answer from https://serverfault.com/questions/92683/execute-rsync-command-over-ssh-with-an-ssh-agent-via-crontab
When you run ssh-agent -s, it launches a background process that you'll need to kill later. So, the minimum is to change your hack to something like:
eval `ssh-agent -s`
svn stuff
kill $SSH_AGENT_PID
However, I don't understand how this hack is working. Simply running an agent without also running ssh-add will not load any keys. Perhaps MacOS' ssh-agent is behaving differently than its manual page says it does.
I had a similar problem. My script (that relied upon ssh keys) worked when I ran it manually but failed when run with crontab.
Manually defining the appropriate key with
ssh -i /path/to/key
didn't work.
But eventually I found out that the SSH_AUTH_SOCK was empty when the crontab was running SSH. I wasn't exactly sure why, but I just
env | grep SSH
copied the returned value and added this definition to the head of my crontab.
SSH_AUTH_SOCK="/tmp/value-you-get-from-above-command"
I'm out of depth as to what's happening here, but it fixed my problem. The crontab runs smoothly now.
One way to recover the pid and socket of running ssh-agent would be.
SSH_AGENT_PID=`pgrep -U $USER ssh-agent`
for PID in $SSH_AGENT_PID; do
let "FPID = $PID - 1"
FILE=`find /tmp -path "*ssh*" -type s -iname "agent.$FPID"`
export SSH_AGENT_PID="$PID"
export SSH_AUTH_SOCK="$FILE"
done
This of course presumes that you have pgrep installed in the system and there is only one ssh-agent running or in case of multiple ones it will take the one which pgrep finds last.
My solution - based on pra's - slightly improved to kill process even on script failure:
eval `ssh-agent`
function cleanup {
/bin/kill $SSH_AGENT_PID
}
trap cleanup EXIT
ssh-add
svn-stuff
Note that I must call ssh-add on my machine (scientific linux 6).
To set up automated processes without automated password/passphrase hacks,
I use a separate IdentityFile that has no passphrase, and restrict the target machines' authorized_keys entries prefixed with from="automated.machine.com" ... etc..
I created a public-private keyset for the sending machine without a passphrase:
ssh-keygen -f .ssh/id_localAuto
(Hit return when prompted for a passphrase)
I set up a remoteAuto Host entry in .ssh/config:
Host remoteAuto
HostName remote.machine.edu
IdentityFile ~/.ssh/id_localAuto
and the remote.machine.edu:.ssh/authorized_keys with:
...
from="192.168.1.777" ssh-rsa ABCDEFGabcdefg....
...
Then ssh doesn't need the externally authenticated authorization provided by ssh-agent or keychain, so you can use commands like:
scp -p remoteAuto:watchdog ./watchdog_remote
rsync -Ca remoteAuto/stuff/* remote_mirror
svn svn+ssh://remoteAuto/path
svn update
...
Assuming that you already configured SSH settings and that script works fine from terminal, using the keychain is definitely the easiest way to ensure that script works fine in crontab as well.
Since keychain is not included in most of Unix/Linux derivations, here is the step by step procedure.
1. Download the appropriate rpm package depending on your OS version from http://pkgs.repoforge.org/keychain/. Example for CentOS 6:
wget http://pkgs.repoforge.org/keychain/keychain-2.7.0-1.el6.rf.noarch.rpm
2. Install the package:
sudo rpm -Uvh keychain-2.7.0-1.el6.rf.noarch.rpm
3. Generate keychain files for your SSH key, they will be located in ~/.keychain directory. Example for id_rsa:
keychain ~/.ssh/id_rsa
4. Add the following line to your script anywhere before the first command that is using SSH authentication:
source ~/.keychain/$HOSTNAME-sh
I personally tried to avoid to use additional programs for this, but everything else I tried didn't work. And this worked just fine.
Inspired by some of the other answers here (particularly vpk's) I came up with the following crontab entry, which doesn't require an external script:
PATH=/usr/bin:/bin:/usr/sbin:/sbin
* * * * * SSH_AUTH_SOCK=$(lsof -a -p $(pgrep ssh-agent) -U -F n | sed -n 's/^n//p') ssh hostname remote-command-here
Here is a solution that will work if you can't use keychain and if you can't start an ssh-agent from your script (for example, because your key is passphrase-protected).
Run this once:
nohup ssh-agent > .ssh-agent-file &
. ssh-agent-file
ssh-add # you'd enter your passphrase here
In the script you are running from cron:
# start of script
. ${HOME}/.ssh-agent-file
# now your key is available
Of course this allows anyone who can read '~/.ssh-agent-file' and the corresponding socket to use your ssh credentials, so use with caution in any multi-user environment.
Your solution works but it will spawn a new agent process every time as already indicated by some other answer.
I faced similar issues and I found this blogpost useful as well as the shell script by Wayne Walker mentioned in the blog on github.
Good luck!
Not enough reputation to comment on #markshep's answer, just wanted to add a simpler solution. lsof was not listing the socket for me without sudo, but find is enough:
* * * * * SSH_AUTH_SOCK="$(find /tmp/ -type s -path '/tmp/ssh-*/agent.*' -user $(whoami) 2>/dev/null)" ssh-command
The find command searches the /tmp directory for sockets whose full path name matches that of ssh agent socket files and are owned by the current user. It redirects stderr to /dev/null to ignore the many permission denied errors that will usually be produced by running find on directories that it doesn't have access to.
The solution assumes only one socket will be found for that user.
The target and path match might need modification for other distributions/ssh versions/configurations, should be straightforward though.