Unable to push image to GCR from Jenkins Pipeline - jenkins-pipeline

I am running a VM in google cloud that runs a Jenkins server (within a docker container). I am trying to build a Docker image for my application and push it out to Google Container Registry using Jenkins pipeline.
I installed all the required Jenkins plugins:
Google OAuth Credentials Plugin,
Docker Pipeline Plugin,
Google Container Registry Auth Plugin
Created a service account + key with Storage Admin and Object Viewer roles. Downloaded the json file.
Created a credential in Jenkins using the google project name as the id and the json key.
My pipeline code for build looks like this:
stage('Build Image') {
app = docker.build("<gcp-project-id>/<myproject>")
}
My pipeline code for build looks like this:
stage('Push Image') {
docker.withRegistry('https://us.gcr.io', 'gcr:<gcp-project-id>') {
app.push("${commit_id}")
app.push("latest")
}
}
However, the build fails at the last step with this error:
unauthorized: You don't have the needed permissions to perform this operation, and you may have invalid credentials. To authenticate your request, follow the steps in: https://cloud.google.com/container-registry/docs/advanced-authentication
I have spent several hours trying to figure this out. Any help would be greatly appreciated!

Create a service account in GCP with permission to push image and then copy the credential json fie and save it as credentials inside Jenkins; call in the credentials id inside your pipeline like below and it should push images to gcr
withCredentials([file(credentialsId: 'gcr', variable: 'GC_KEY')]){
sh "cat '$GC_KEY' | docker login -u _json_key --password-stdin https://eu.gcr.io"
sh "gcloud auth activate-service-account --key-file='$GC_KEY'"
sh "gcloud auth configure-docker"
GLOUD_AUTH = sh (
script: 'gcloud auth print-access-token',
returnStdout: true
).trim()
echo "Pushing image To GCR"
sh "docker push eu.gcr.io/${google_projectname}/${image_name}:${image-tag}"
}
Additionally i have defined some variables used above

I have an identical problem. I found out that Jenkins doesn't seem to use those credentials: Under usage it says 'This credential has not been recorded as used anywhere.' . When used with gcloud util, the service account and key work fine, so the problem is somewhere in Jenkins.

Related

JIB is not able to detect docker credentials

I am building a generated app whith Jhipster.
I run the command to build the images and run the app containerized. I started Docker Desktop on windows 11.
To remind, this is the command: ./gradlew -Pprod bootJar jib
The output after a while is :
Execution failed for task ':jib'.
> com.google.cloud.tools.jib.plugins.common.BuildStepsExecutionException: Build image failed, perhaps you should make sure your credentials for 'registr
y-1.docker.io/library/app2' are set up correctly. See https://github.com/GoogleContainerTools/jib/blob/master/docs/faq.md#what-should-i-do-when-the-registry-responds-with-unauthorized for help
I tried multiple times to log in on docker:
docker login registry-1.docker.io
The login is successful and the config.json of docker content is:
{
"auths": {
"https://index.docker.io/v1/": {},
"registry-1.docker.io": {}
},
"credsStore": "desktop"
}
I'm sure that this is where JIB, by default looks for docker creds, but I can not see any creds here. It looks like the credentials are stored somewhere else, here is the version of Docker: Docker version 20.10.17, build 100c701
Maybe try building offline first, probably a permission issue on the remote repository, which will need to be fixed on hub.docker.com

Gitlab Runner registration return (401 Unauthorized)

We have installed gitlab runner on redhat server but we can't register it with our gitlab instance as it is protected (we need to login before it shows HTTPS).
When we clone the project repository:
git clone https://domain\.com/project.git
It asks for username and password and if we store credentials in a file or cache like
git config --global credential.helper 'store --file ~/.my-credentials'
it will ask the first time only for authentication when cloning => my-credentials will contains the raw username/password but it solves the problem of being asked for authentification each time.
How can we do the same things with the runner in order to register it to our gitlab :
when we run
gitlab-runner register --url http://domain\.com/gitlab/ -r 1TqfdYFsJsdsdLNc-7J
we got immediately the below error without asking us for username/password :
ERROR: Registering runner... failed runner=1TqfdYFs status=401 Unauthorized
PANIC: Failed to register the runner. You may be having network problems.
we know that the runnercan't access our gitlab because of the authentification needed but we don't know how to solve it we tried many things like saving credentials in git cache but it seems that they are not connected or use the same things.
Any ideas please ?
I see that you are trying to register your runner with http method. Try registering with https instead. This resolved the issue for me.
gitlab-runner register --url https://domain\.com/gitlab/ -r 1TqfdYFsJsdsdLNc-7J

Copy docker image to heroko registry through Azure devops

I have a build pipeline(in Azure DevOps) that pushes an image to the docker hub. I would also like to push the same image to the Heroku hub.
I tried to follow the Heroku document. But it asks for a login. I didn't find any way to login to Heroku through the Azure pipeline. Is there any way to login to Heroku using a token? Is there any other way through which I can push the docker image to Heroku?
Azure pipeline: https://dev.azure.com/abhishekgoenkapublic/github-projects/_build?definitionId=3
Docker image: https://hub.docker.com/r/abhishek1950/mean
GitHub Project: https://github.com/abhishekgoenka/mean
Heroku provides an environment variable to add an access token to execute its commands.
The HEROKU_API_KEY variable is used to assign access token. It is possible to generate the token through the panel in Heroku:
https://devcenter.heroku.com/articles/authentication
In order to pass the token to our agent job, we have to configure this variable in our pipeline. For this, in the Variables tab we will create a new key as shown in the image below.
Having configured the environment variable, we were able to add the necessary steps and commands.
Heroku Container Login: Log in via the heroku CLI .
Docker Push: Push the docker image to Heroku.

Jenkins: Can't fetch project's source code from GitLab

I run my Jenkins server on the local machine, and I met problem with fetching project from GitLab repository. I don't know exactly what I have to done.
ERROR: Error cloning remote repo 'origin'
hudson.plugins.git.GitException: Command "C:\Program Files\Git\bin\git.exe fetch --tags --progress http://repository.vrpconsulting.com/roman.skaskevich/koshcheck-copy.git +refs/heads/*:refs/remotes/origin/*" returned status code 128:
stdout:
stderr: Logon failed, use ctrl+c to cancel basic credential prompt.
remote: HTTP Basic: Access denied
fatal: Authentication failed for 'http://repository.vrpconsulting.com/roman.skaskevich/koshcheck-copy.git/'
GITLAB SETTINGS: here
EDIT #1
When I run git fetch --tags --progress http://repository.vrpconsulting.com/roman.skaskevich/koshcheck-copy.git +refs/heads/*:refs/remotes/origin/* on local machine, it performs successfully and dialog for entering login/password is showed.
But what I have to do that jenkins job performed this command successfully?
EDIT #2
For now, Credentials Binding Plugin helps me implements to checking credentials in Jenkinsfile.
EDIT #3
I run Jenkins server on another machine and have the same problem. I don't want to use login/password, so adding SSH key solved this problem.
Thanks in advance!
Seems like your repo is private. Private git repos requires authentication. Reconfigure your git client plugin check if you are able to connect to remote repo via password based authentication , if not , try SSHing to the repo.
Looks like this ins private repo if you want to clone that repo to jenkins workspace you need to provide your gitlab credentials in jenins.
for that, you need to install jenkins plugin called "Gitlab Authentication plugin". once you completed the installation please do a restart.
Add Gitlab credentials in "Jenkins Credentials Provider: Jenkins".
later you can add your gitlab credentials in Source Code Management
Source Code Management
in credentials select the saved gitlab credentials.
Steps:
Install plugins:
Gitlab
Gitlab authentication
Configure global credentials in jenkins:
go to credentials >> system >> global creds >> username & password
ID: git
username: your gitlab username
password: your gitlab password
Pipeline:
stage('SCM') {
steps {
git credentialsId: 'git', url: 'your gitlab url'
}
}
I got this error too. In my case, the git repo is a private one.
I added the git link and then supplied the git credentials to the particular Jenkins job. The issue is gone and able to build project from Jenkins.

Automate Heroku CLI login

I'm developing a bash script to automatic clone some projects and another task in dev VM's, but we have one project in Heroku and repository is in it. In my .sh file I have:
> heroku login
And this prompt to enter credentials, I read the "help" guide included on binary and documentation but I can't found anything to automatic insert username and password, I want something like this:
> heroku login -u someUser -p mySecurePassword
Exist any way similar to it?
The Heroku CLI only uses your username and password to retrieve your API key, which it stores in your ~/.netrc file ($HOME\_netrc on Windows).
You can manually retrieve your API key and add it to your ~/.netrc file:
Log into the Heroku web interface
Navigate to your Account settings page
Scroll down to the API Key section and click the Reveal button
Copy your API key
Open your ~/.netrc file, or create it, with your favourite text editor
Add the following content:
machine api.heroku.com
login <your-email#address>
password <your-api-key>
machine git.heroku.com
login <your-email#address>
password <your-api-key>
Replace <your-email#address> with the email address registered with Heroku, and <your-api-key> with the API key you copied from Heroku.
This should manually accomplish what heroku login does automatically. However, I don't recommend this. Running heroku login does the same thing more easily and with fewer opportunities to make a mistake.
If you decide to copy ~/.netrc files between machines or accounts you should be aware of two major caveats:
This file is used by many other programs; be careful to only copy the configuration stanzas you want.
Your API key offers full programmatic access to your account. You should protect it as strongly as you protect your password.
Please be very careful if you intend to log into Heroku using any mechanism other than heroku login.
You can generate a non-expiring OAuth token then pass it to the CLI via an environment variable. This is useful if you need to run Heroku CLI commands indefinitely from a scheduler and you don't want the login to expire. Do it like this (these are not actual Tokens and IDs, BTW):
$ heroku authorizations:create
Creating OAuth Authorization... done
Client: <none>
ID: 80fad839-876b-4ea0-a41e-6a9a2fb0cf97
Description: Long-lived user authorization
Scope: global
Token: ddf4a0e5-9294-4c5f-8820-b51c52fce4f9
Updated at: Fri Aug 02 2019 21:26:09 GMT+0100 (British Summer Time) (less than a minute ago)
Get the token (not the ID) from that authorization and pass it to your CLI:
$ HEROKU_API_KEY='ddf4a0e5-9294-4c5f-8820-b51c52fce4f9' heroku run ls --app my-app
Running ls on ⬢ my-app... up, run.2962 (Hobby)
<some file names>
$
By the way this also solves the problem of how to use the Heroku CLI when you have MFA enabled on your Heroku account but your machine doesn't have a web browser e.g., if you are working on an EC2 box via SSH:
$ heroku run ls --app my-app
heroku: Press any key to open up the browser to login or q to exit:
› Error: quit
$ HEROKU_API_KEY='ddf4a0e5-9299-4c5f-8820-b51c52fce4f9' heroku run ls --app my-app
Running ls on ⬢ my-app... up, run.5029 (Hobby)
<some file names>
$
EDIT: For Windows Machines
After you run heroku authorizations:create, copy the "Token", and run the following commands:
set HEROKU_API_KEY=ddf4a0e5-9299-4c5f-8820-b51c52fce4f9
heroku run ls --app my-app
If your goal is just to get the source code, you could use a simple git client. You just need the api key.
Steps to get api key
Log into the Heroku web interface
Navigate to your Account settings page
Scroll down to the API Key section and click the Reveal button
Copy your API key
Download source code using git
Use this url template for git clone
https://my_user:my_password#git.heroku.com/name_of_your_app.git
In my case the user value was my email without domain.
Example :
if mail is **duke#gmail.com**
user for heroku auth will be **duke**
Finally just clone it like any other git repositories:
git clone https://duke:my_password#git.heroku.com/name_of_your_app.git
I agree that Heroku should have by now provided a way to do this with their higher level CLI tool.
You can avoid extreme solutions (and you should, just like Chris mentioned in his answer) by simply using curl and the Heroku API. Heroku allow you to use your API Token (obtainable through your user settings / profile page on the Heroku dashboard).
You can then use the API to achieve whatever it is you wanted to do with their command line tool.
For example, if I wanted to get all config vars for an app I would write a script that did something like the following:
-H "Accept: application/vnd.heroku+json; version=3" \
-H "Authorization: Bearer YOUR_TOKEN```
If *YOUR_APP_NAME* had only one config variable called *my_var* the response of the above call would be
{
"my_var": some_value
}
I've found using this all the time in CI tools that need access to *Heroku* information / resources.

Resources