I'm trying to use Blackbox to encrypt sensitive files in a repository I'm working with, but I am unable to make the GitLab CI pipeline load the private GPG key required to decrypt the files.
At the , you can see the output from the GitLab CI runner where it is evident that:
My private key, stored as a Secret Variable for the runner, is available.
The secret key is properly written to a file that can be read back.
The GPG key is properly formatted and contains a complete and proper key.
GPG exists and can be properly initiated.
However, when I run the command gpg -v --import /tmp/server.gpg, I keep getting gpg: no valid OpenPGP data found.. This makes no sense.
I have followed all posts I could find on this, but they are either about verifying file contents rather than actually importing a proper key, or that the key is actually malformed, which mine evidently is not.
I'd appreciate any help I can get on this matter as I am making no progress whatsoever.
Thank you!
Output from the GitLab CI runner:
Running with gitlab-ci-multi-runner 9.1.0 (0118d89) on docker-auto-scale (e11ae361)
Using Docker executor with image golang:1.8.1 ...
Using docker image sha256:3858bc6d4732445082339c9ccccfe56bf599d1fe7d9e850fb67ceec76807ed8d for predefined container...
Pulling docker image golang:1.8.1 ...
Using docker image golang:1.8.1 ID=sha256:6d0bfafa0452c6398be979f89614b5e3cb5d10e853ccd4f5791c4971a88065e0 for build container...
Running on runner-e11ae361-project-3172553-concurrent-0 via runner-e11ae361-machine-1493644095-36064084-digital-ocean-2gb...
Cloning repository...
Cloning into '/builds/project/repo'...
Checking out 7013b30a as feature/blackbox...
Skipping Git submodules setup
$ git clone https://github.com/StackExchange/blackbox
Cloning into 'blackbox'...
$ cd blackbox
$ make manual-install
Symlinking files from ./bin to /usr/local/bin
Done.
$ echo $GPG_PRIVATE_KEY > /tmp/server.gpg
$ chmod 400 /tmp/server.gpg
$ cat /tmp/server.gpg
-----BEGIN PGP PRIVATE KEY BLOCK-----
Version: GnuPG v2
[redacted]
-----END PGP PRIVATE KEY BLOCK-----
$ gpg -v --list-keys
gpg: directory `/root/.gnupg' created
gpg: new configuration file `/root/.gnupg/gpg.conf' created
gpg: WARNING: options in `/root/.gnupg/gpg.conf' are not yet active during
this run
gpg: keyring `/root/.gnupg/pubring.gpg' created
gpg: /root/.gnupg/trustdb.gpg: trustdb created
gpg: using PGP trust model
$ gpg -v --import /tmp/server.gpg
gpg: keyring `/root/.gnupg/secring.gpg' created
gpg: no valid OpenPGP data found.
gpg: Total number processed: 0
ERROR: Job failed: exit code 1
I found the answer. In my .gitlab-ci.yml, I changed my GPG read implementation from:
- echo $GPG_PRIVATE_KEY > /tmp/server.gpg
- gpg -v --import /tmp/server.gpg
to:
- gpg -v --import <(echo "$GPG_PRIVATE_KEY")
This solved the issue.
Change the variable to type "file" instead of "variable", in the repository settings.
Then import the GPG key normally as a file:
gpg -v --import $GPG_PRIVATE_KEY
Ref: https://docs.gitlab.com/ee/ci/variables/#create-a-custom-variable-in-the-ui
Related
I've set up a repository in the Maven Central Repository via Sonatype, and now I'm trying to create a GitHub action that will automatically publish an updated version when I create a new release on GitHub.
I know how to create the trigger using the on: clause and run the gradlew publish task, but I don't know how to supply GitHub with the GPG encryption key needed for it.
How do I do it?
My progress so far is here.
Your issue seems similar to this one
Summarizing:
GitHub Actions is basically a container that runs commands, you can define your key as a secret on your project and then importing it in your Github Action workflow
Here are the steps that could be used on a project to publish the generated artifacts to Sonatype's repository:
On a terminal window, you can search for the key ID by e-mail: gpg --list-secret-keys user#example.com
Export your key as Base64: gpg --export-secret-keys YOUR_ID_HERE | base64 > private.key
In your Github project, create a new Secret named GPG_SECRET_KEY (for example) and paste the Base64 content of your key (here is a reference how to do it)
In your workflow .yml file, include a step to import the key from your just defined secret like the example below:
- name: Configure GPG Key
run: |
mkdir -p ~/.gnupg/
printf "$GPG_SIGNING_KEY" | base64 --decode > ~/.gnupg/private.key
gpg --import ~/.gnupg/private.key
env:
GPG_SECRET_KEY: ${{ secrets.GPG_SECRET_KEY }}
Note: Your GPG Key can't be protected by a password.
I'm getting this error when trying to sign a commit:
git commit -S -m "test"
gpg: skipped "EF617ACA9EC3XXXX": No secret key
gpg: signing failed: No secret key
error: gpg failed to sign the data
fatal: failed to write commit object
This is the output of gpg --list-secret-keys --keyid-format LONG
The key is present there
sec rsa4096/EF617ACA9EC3XXXX 2020-05-17 [SC] [expires: 2022-05-17]
AD68154000A712DCD161D826EF617ACA9EC3XXXX
uid [ultimate] name <email#gmail.com>
And this is git config with the same key
user.signingkey=EF617ACA9EC3XXXX
user.email=email#gmail.com
Any idea what's wrong?
git config --global gpg.program "c:/Program Files (x86)/GnuPG/bin/gpg.exe"
I installed with Kleopatra and generated my key within that. I was unable to create a commit until I ran the command above.
Check first the git config gpg.program to see if this is gpg or gpg2 (as in here).
And type where gpg nd where gpg2 to check which path is considered for the GPG program.
I suggested to set gpg.program to gpg2, and copy your gpg.exe (assuming its version is a 2.x) to gpg2.exe
That should force Git/GPG to act as gpg2.
In case anyone is a dufus like me I was getting this error because I had the gitkey wrong while directly editing the file via: git config --global -e or code ~/.gitconfig or whatever flavor of editor you enjoy.
singingkey 🎶🔑 instead of
signingkey 🤦♂️
Was facing the same issue in windows 10 git bash. Doing this solved my problem. You could find this path by running where gpg
$ git config --global gpg.program "C:\Program Files\Git\usr\bin\gpg.exe"
Another potential issue/resolution:
Make sure the email you set in Github is verified and that it exactly matches the email you used when you generated your GPG key.
Had this same problem on Windows 11 when trying to get GitHub Desktop to work.
The answer given here solved the issue for me:
git config --global gpg.program "C:\Program Files\Git\usr\bin\gpg.exe"
i got a Puppet Enterprise Master Server 2018.1.3 which should get the Code with Code Manager from a git-Repository via https, where the server certificate of the git server is signed by a third party CA.
after getting everything afaik correctly configured, i get following:
> puppet-code deploy --dry-run
Dry-run deploying all environments.
Errors while collecting a list of environments to deploy (exit code: 1).
ERROR -> Unable to determine current branches for Git source 'puppet'
(/etc/puppetlabs/code-staging/environments)
Original exception:
The SSL certificate is invalid
executing directly r10k produces a similar error. which makes sense, since i have not installed the third party CA certificate anywhere yet.
so i thought, r10k most likely runs jruby which runs java (i do not any idea about ruby), so i will install the certificate in the jvm:
keytool -import -file gitCA.cer -alias gitCA -keystore /opt/puppetlabs/server/apps/java/lib/jvm/java/jre/lib/security/cacerts -storepass changeit
but i am still getting the same error, also after a system restart, so ok, it means r10k does not use jruby but ruby, so i will install also the certificate in the OS, put the certificate under /etc/pki/trust/anchors and called update-ca-certificates (on SLES12). After that, i can access the git-Repo-URL with wget without getting any certificate error, so the certificate is installed in the OS correctly, but still, even after a system restart, i am getting the same error with r10k.
after lot of goggling for certificate stores and ruby i found out that
export SSL_CERT_FILE=<path_to_cert>
fixes the direct call of r10k:
> r10k deploy display --fetch ---
:sources:
- :name: :puppet
:basedir: "/etc/puppetlabs/code/environments"
:remote: https://xxx#git.xxx/git/puppet
:environments:
- develop
- master
- production
- puppet_test
but puppet-code still not working with same error message. but i thought, obviously i am right now root and puppet-code is executed by user pe-puppet, so i put the export command in the /etc/profile.local file, so it is available to all users.
still not working. even after system restart and deleting /opt/puppetlabs/server/data/puppetserver/r10k/ that was created with user root while directly calling r10k.
first question: why does r10k works, but puppet-code not?
second question: where is the correct place for that certificate?
many thanks
Michael
UPDATE: 27.AUG.2018
i tried this:
sudo -H -u pe-puppet bash -c '/opt/puppetlabs/puppet/bin/r10k deploy display --fetch'
which did not work, despite i am setting the SSL_CERT_FILE variable in the /etc/profile.local file.
but i got it working by setting the variable in the /etc/environment file.
but puppet code still not working. why?
for those looking for a solution to this problem checkout this post on the Puppet Support Base.
Simply put you have two options:
Use a Git source instead of an HTTPS source to refer to your repository in your Puppetfile. This option requires adding SSH keys to your Puppet master and your repository.
Add a certificate authority (CA) cert for the repository to the list of trusted CAs in /opt/puppetlabs/puppet/ssl/cert.pem.
Option one: Use a Git source instead of an HTTPS source
To deploy code from your repository using a Git source, configure a private SSH key on your Puppet master and a public SSH key on your repository:
In your Puppetfile, change references to your Git repository from an HTTPS source to a Git source:
For example, change:
mod 'site_date', :git: 'https://example.com/user/site_data.git',
to:
mod 'site_data', :git: 'ssh://user#example.com:22/user/site_data.git',
Configure your SSH keys. Configure the private key using our documentation on how to Declare module or data content with SSH private key authentication for PE 2018.1.
Note: Use the version selector to choose the right version of our documentation for your deployment.
The details of configuring your public key depend on how your Git repository is configured. Talk to your Git repository administrator.
Option two: Add a trusted CA cert
If you are unable to specify a Git s
ource, add your repository to the list of CAs trusted by Code Manager by adding a CA cert to the file /opt/puppetlabs/puppet/ssl/cert.pem.
Transfer the cert (ca.pem) file to your CA node.
On the CA node, add the cert to the list of CAs trusted by Code Manager: cat ca.pem >> /opt/puppetlabs/puppet/ssl/cert.pem
Agent runs won't revert changes made to cert.pem because the file isn't managed by PE, but upgrades to PE will overwrite the file. After you upgrade PE, you must add the CA cert to cert.pem again.
so, i got it working, but not happy with the solution.
i turned on debug logging on /etc/puppetlabs/puppetserver/logback.xml, confirming that puppet-code is indeed calling r10k:
2018-08-27T14:54:24.149+02:00 DEBUG [qtp462609859-78] [p.c.core] Invoking shell:
/opt/puppetlabs/bin/r10k deploy --config /opt/puppetlabs/server/data/code-manager/r10k.yaml --verbose warn display --format=json --fetch
2018-08-27T14:54:24.913+02:00 ERROR [qtp462609859-78] [p.c.app] Errors while collecting a list of environments to deploy (exit code: 1).
ERROR -> Unable to determine current branches for Git source 'puppet' (/etc/puppetlabs/code-staging/environments)
Original exception:
The SSL certificate is invalid
so i did it the very quick and dirty way:
cd /opt/puppetlabs/puppet/bin/
mv r10k r10k-bin
touch r10k
chmod +x r10k
vi r10k
and
#!/bin/bash
export SSL_CERT_FILE=<new_cert_path>
/opt/puppetlabs/puppet/bin/r10k-bin "$#"
now it is working:
puppet:~ # puppet-code deploy --dry-run
Dry-run deploying all environments.
Found 5 environments.
but not happy, any better idea?
I try to deploy my opensource project to Nexus Repository (https://oss.sonatype.org) using the travis ci, but unfortunately travis doesn't found the secret key for gpg signed step.
I follow all steps on https://github.com/making/travis-ci-maven-deploy-skelton but the release deploy continuous doesn't work. At my workspace all works correctly and I can deploy releases to Nexus Repository.
I'm using one script to deploy the project:
#!/usr/bin/env bash
echo "Checking the current branch..."
if [ "$TRAVIS_BRANCH" = 'master' ] && [ "$TRAVIS_PULL_REQUEST" == 'false' ]; then
echo "The current branch is: master"
echo "Run maven deploy parameter using sign and build-extras profiles..."
mvn deploy -P sign,build-extras --settings setting-maven.xml
fi
Such issues commonly occur if the service is running under another user than the developer's account. GnuPG has per-user "GnuPG home directories" in ~/.gnupg. Make sure to import the keys under the service's user (run this command from your developer account):
gpg --export-secret-keys [key-id] | sudo -u [service user] gpg --import
Alternatively, you could use gpg's --homedir option to change to GnuPG home directory location, but be aware GnuPG is very picky about properly set, tight permissions by default (which is a good thing).
The solution in https://github.com/making/travis-ci-maven-deploy-skelton relies on symmetrically encrypted keyrings in your $GPG_DIR. In the sample that would be the folder deploy.
To create these keyrings, you do this (copied):
$ export ENCRYPTION_PASSWORD=<password to encrypt>
$ openssl aes-256-cbc -pass pass:$ENCRYPTION_PASSWORD -in ~/.gnupg/secring.gpg -out deploy/secring.gpg.enc
$ openssl aes-256-cbc -pass pass:$ENCRYPTION_PASSWORD -in ~/.gnupg/pubring.gpg -out deploy/pubring.gpg.enc
This creates the encrypted keyrings in the folder deploy. You probably need to create the folder before you run the openssl commands.
Both encrypted keyrings need to be checked in, so that they are available as part of the project at build-time.
At build-time you need to decrypt the keyrings. You do this by adding something like this to your .travis.yml file:
before_install:
- openssl aes-256-cbc -pass pass:$ENCRYPTION_PASSWORD -in $GPG_DIR/pubring.gpg.enc -out $GPG_DIR/pubring.gpg -d
- openssl aes-256-cbc -pass pass:$ENCRYPTION_PASSWORD -in $GPG_DIR/secring.gpg.enc -out $GPG_DIR/secring.gpg -d
Note, how the openssl command uses $GPG_DIR? That's basically your deploy directory. To make sure Travis knows about $GPG_DIR, define it, e.g. like this:
env:
global:
- GPG_DIR="`pwd`/deploy"
So basically Travis now knows how to decrypt your GPG keyrings and puts them into a defined location. Now you still have to tell GPG how to pick it up. For that you have two options:
Properties in your pom.xml or
properties in your settings.xml
The project https://github.com/making/travis-ci-maven-deploy-skelton uses the first option (pom.xml). The essential part is:
<profiles>
<profile>
<id>ossrh</id>
<properties>
<gpg.executable>gpg</gpg.executable>
<gpg.keyname>${env.GPG_KEYNAME}</gpg.keyname>
<gpg.passphrase>${env.GPG_PASSPHRASE}</gpg.passphrase>
<!-- tell gpg to NOT use the default keyring from the current user's home -->
<gpg.defaultKeyring>false</gpg.defaultKeyring>
<!-- instead tell gpg to use the keyrings from your GPG_DIR -->
<gpg.publicKeyring>${env.GPG_DIR}/pubring.gpg</gpg.publicKeyring>
<gpg.secretKeyring>${env.GPG_DIR}/secring.gpg</gpg.secretKeyring>
</properties>
[...]
With these properties you define parameters for the gpg executable. This works, because you have set up $GPG_DIR in your .travis.yml file.
In essence you tell gpg to not use the standard keyrings in the current user's home directory, but instead the keyrings you've just decrypted and stuck into your $GPG_DIR.
You may ask yourself where the other <properties>/env variables come from. They are appended to your .travis.yml when running the following commands:
$ travis encrypt --add -r <username>/<repository> SONATYPE_USERNAME=<sonatype username>
$ travis encrypt --add -r <username>/<repository> SONATYPE_PASSWORD=<sonatype password>
$ travis encrypt --add -r <username>/<repository> ENCRYPTION_PASSWORD=<password to encrypt>
$ travis encrypt --add -r <username>/<repository> GPG_KEYNAME=<gpg keyname (ex. 1C06698F)>
$ travis encrypt --add -r <username>/<repository> GPG_PASSPHRASE=<gpg passphrase>
<username> is your Github username and <repository> your Github repo.
After running the travis encrypt --add commands, your .travis.yml file will have appended entries like this:
env:
global:
- GPG_DIR="`pwd`/deploy"
- secure: VYxU+0zMoKExcopJ8z74Pd5KE6TnoP72hZchnpy+gxLVrt4d5lBJ042xT2D/4qebG8stHpq5DtYO8EQaZVMKVQl48fXwQk4aWiY0OWNY2Pz63Y9IFDGX0n/B1NPxbPToCoXHsddGvAVOlRXbDTfkF+yc3nheaLLnjhxFAM9X0/e1/bqnFyrwqrJmenG7RaGclsscjLPLExTAy+jIbj59loZkclVfKpMS98Ol605Xpmn6VTxr8Z7k3FEQ4mt3VI350QKBbmMsiEWpVAbPVPWsUkEYpM5VuU7Pi1W5fbaJBxIlOAdKjDtYfUvyY63iQK79787dBrGM5T2FDUV05UXpi4NvKnrcdkhuOFlXB4Io3qroen1lrh5igBdIlYc4kBpvDMpnewIuM7F+5fPS9XgBZIlOkiAEPqInr0sonlj4c+vkd92PeujDYiKUCA2uVLEzLYAAu7oC6O5K18JBzLurKNAda+9f+XuQrc1t140u8jic9YF7oM7fUdiu1MJ6j9WMiu3Syh9yjAOC+5RaBxy/ZcDUmYazH/oQNe3d55AMKYOdsryF51W/WfSrHoHtKUGsy9RsDvY690GU6XZ+Zev79nRKs9uVSqqlcGv+YPoB3zlDjmks51fm0HovgWWsCDbDgP4/FXPFKzr0Ht6qnYjJ=
- secure: Wl1/oERtbz739uq+cfHQdpXGC/ZIX1l9HBihyTSt0qta7HlqQeCHtCQfpbq92BYj518CZjNl3ijXlYaXOoW4Z1L2VGzJwxNVdiG2XVkUrXfTO6i711Q/f6ezINlDhRhH+Sn1GhFPB8x7i5vnlqSvMqG19x2mfPsD50yi/58elU7t3zUg5HnBpHfyCdrlaa1pI/sHYIog4Y/Nm3H6/9WDu5ErnhmSKT9LCHdXDXn8AO8UfQXP/eMHUAMdnn8LP/+HtGXmI1Jij9UFaB1PTMyKRCMiVizMDgMqtjXhzBqg8Wqy6pp2yicSEn4JVgBM26vsNQwHXgz3kut4FwlY7Aph8Mx61jU8OvVh/vD6y1gm7r7PW4lcy1PT3pTtfL2XLH3p0/cl+WqHJIfupyOZg/z0dEd0JKJAxJ7XR3y6Z0QVTKe0QTSOO8O5g+EfuyoJFC4d28G8gM+Zc1OctpXOMrU0l4x3PDrb8xoxugsUMpYfUIQl7L9Dxr6PqHbDIgNM/5N5L3ZwWiI12fKtIqfurVJ2jsA9ahzCskzRSK745lwIPrpw6NVPjN8CzbTWZjyR9aMuxpHO+ptMmXxjo3asA7tJQDBtfbAWWz0FGro429UK3IWa5dgtVQpP2GG4/VWtUM1CUhG9x74FpojIHa4EzpLji=
[...]
These secure entries are translated to environment variables by Travis at build-time.
A couple more remarks:
By default Travis uses gpg version 1. Because of issues with entering passphrases via loopback in gpg 2.0, version 1 is preferred. Once Travis supports gpg 2.2, this may change.
If you haven't done so yet, you need to publish your key to
http://keys.gnupg.net:11371
http://keyserver.ubuntu.com:11371
http://pool.sks-keyservers.net:11371
Good luck.
I have a BitBucket account. Despite setting up the ssh key on the computer and then adding it on my bitbucket account, I fail at connecting:
$ git clone git#bitbucket.org:my-account/myrepo.git
Cloning into 'myrepo'...
conq: repository does not exist.
fatal: Could not read from remote repository.
Please make sure you have the correct access rights
and the repository exists.
I did the following:
$ cd ~/.ssh
$ ssh-keygen -f ~/.ssh/my-key -N ''
$ vim config
Host bitbucket.org
HostName bitbucket.org
IdentityFile ~/.ssh/my-key
$ ls -l
-rw------- 1 me ... 229 Jul 8 15:50 config
-rw------- 1 me ... 1679 Jul 8 14:07 my-key
-rw-r--r-- 1 me ... 411 Jul 8 14:07 my-key.pub
Then I added the content of my-key.pub in my bitbucket account's ssh keys. Then I retried cloning my repository:
$ git clone git#bitbucket.org:my-account/myrepo.git
Cloning into 'myrepo'...
conq: repository does not exist.
fatal: Could not read from remote repository.
Please make sure you have the correct access rights
and the repository exists.
I repeated the identical procedure on another machine, it worked:
$ git clone git#bitbucket.org:my-account/myrepo.git
Initialized empty Git repository in /home/me/.ssh/myrepo/.git/
remote: Counting objects: 3, done.
remote: Total 3 (delta 0), reused 0 (delta 0)
Receiving objects: 100% (3/3), done.
I need to do the same on my first machine now.
As a last resort, I followed some advises on the net using the ssh-agent:
$ eval `ssh-agent -s`
$ ssh-add ~/.ssh/my-key
But still no luck:
$ git clone git#bitbucket.org:my-account/myrepo.git
Cloning into 'myrepo'...
conq: repository does not exist.
fatal: Could not read from remote repository.
Please make sure you have the correct access rights
and the repository exists.
I have no idea what is different on the first machine, why does it reject my request?
EDIT 1
Of course I tried doing it the very basic way, as suggested by D.Samchuk:
Generating the id_rsa
$ ssh-keygen -t rsa
enterx3
Copied the exact content of the key
$ cat ~/.ssh/id_rsa.pub
And I added id_rsa.pub's content in my BitBucket account's ssh keys.
Git clone will still not allow me to access the repository, whit the same output error message as above.
EDIT 2
I have no idea maybe it is related to mac configurations? Since The other machine is a redhat linux I just wondered but couldn't find any reported similar issue.
just do following:
$ ssh-keygen -t rsa
name: id_rsa
passphrase: just press enter twice to avoid print keyword everytime you connect
$ cat ~/.ssh/id_rsa.pub
copy output
open bitbucket --> profile --> bitbucket settings --> add ssh key --> paste your output
If you are not using a passphrase for the ssh key you shouldn't need the ssh-agent
https://help.github.com/articles/working-with-ssh-key-passphrases/
Solved. As I delved deeper in the issue I found it is absolutely not related to ssh keys but to missing repository as the error message suggested. It was all about experimenting the global config using git config --global that messed up my git configuration on that computer. I detailed my steps in another thread here: https://unix.stackexchange.com/questions/294780/git-retains-remote-information-from-a-deleted-repository/294809#294809.