I have an AWX installation with a project. The project is fetched perfectly from the private git repo.
The project has a requirements file which is executed by AWX. The requirements file contains private repositories like this:
- src: git+http://gitlab.test.com/ansible/test.git
version: master
name: test
This however fails with the following error:
test was NOT installed successfully: - command git clone failed in directory.
How can I configure AWX to use the git credentials that are in place already in awx?
It seems like AWX works 'smarter' than I thought it would.
In this case, AWX used the same credentials that it used to fetch the project in the first place. I had two options to resolve this:
Change the links to work via git
Update the same git credential and also add the username and password of that git user.
Related
I published a private package to my organizations github repository. When I try to deploy a lambda through github actions that uses this package, I get an error saying
"fatal: could not read Username for 'https://github.com': terminal prompts disabled
Confirm the import path was entered correctly."
Is there an extra line that I need to add to my yml file or would this permission need to be given in the package itself?
I think you should Configure git for private modules in the GitHub action, adding one simple step in your workflow, like:
- name: Configure git for private modules
env:
GITHUB_API_TOKEN: ${{ secrets.GH_API_TOKEN }}
run: git config --global url."https://x:${GITHUB_API_TOKEN}#github.com".insteadOf "https://github.com"
And add the GH_API_TOKEN secrets in the repo in order to be able to download the go modules during build step, like the go mod tidy command.
In my package.json I have a depedency to antoher git repository:
"my-dependeny":"git+https://somehost.com/my-dependency.git#<commit-hash>"
After running npm install this works fine while developing because git is using my personal stored credentials.
But executing npm install in a jenkins-pipline will fail, because there are not stored credentials. So I first tried to generate a deploy token in gitlab and edit the package.json:
"my-dependeny":"git+https://<deploy-token>:<password>#somehost.com/my-dependency.git#<commit-hash>"
Now it will work on jenkins!
But the problem with this approach is, that Windows autoamtically updates the credential for the host (somehost.com) in the Credentials Manager:
So when any developer is running npm install and is afterwards trying to access another project within the same host it will fail, because it will use the deploy-token for the autenthication.
Of course you could add new user to gitlab which has access to all relevant projects. Then you could store this user on jenkins, but this doesn't seem to be an optimal or even good solution.
Maybe there could also be a way to stop the Credential Manager from updating git credentials. But this also doesn't sound well, because every developer would have to prevent this, which is not very intuitive.
My Question
Is there a good way to access npm git-depencies from jenkins without requiring extra steps that need to be done by each developer?
I was finally able to find a solution by myself with the help of git-credential-store.
The following steps apply for linux systems, because in my case the Jenkins is running on Linux. But there should be a similiar solution for Windows systems.
Create empty ~/.git-credentials file. If you use a Dockerfile to copy the .git-credentials file you most likly have to use /root/git-credentials as path (e.g. COPY .git-credentials /root/)
Insert your credentials like the following to your ~/.git-credentials file:
<protocol>://<deploy-token>:<password>#<hostname>/<project>
Example:
https://gitlab+deploy-10:mYSecReTPaSSWord#somehost.com/my-dependency.git
or multiple credentials (follow 4. step below to enable this feature):
https://gitlab+deploy-10:mYSecReTPaSSWord#somehost.de/some-path/some-project.git
https://gitlab+deploy-11:otHerPaSSWord#somehost.de/some-path/some-other-project.git
NOTE:
The line-endings in the ~/.git-credentials have to be LF. If
they are CR;LF unexpected errors can occur.
git config --global credential.helper store
This will tell git that you wanna use your .git-credentials file for authentication.
(~/.git-credentials is the default location)
optional: git config --global credential.useHttpPath true
This is needed if you want to use mutliple Deploy Tokens for multiple projects within the same hostname. Without credential.useHttpPath true in the example above this would mean, that gitlab+deploy-10:mYSecReTPaSSWord would be used in every request to somehost.de. The HTTP path would be ignored completly.
The following is how I solved this in my jenkins file:
withCredentials([usernamePassword(credentialsId: 'my-git-username-and-api-token', passwordVariable: 'GIT_PASSWORD', usernameVariable: 'GIT_USERNAME')]) {
sh '''
git config --global credential.username ${GIT_USERNAME}
git config --global credential.helper "!echo password=${GIT_PASSWORD}; echo"
npm install
npm run build
'''
}
As you can see it uses GitHub username and API token from the credentials store. It also runs git config every time. For me this is necessary as I'm running my builds in a docker node. This solves the problem we encountered where git+https dependencies in npm were not being resolved in our jenkins pipeline.
I run my Jenkins server on the local machine, and I met problem with fetching project from GitLab repository. I don't know exactly what I have to done.
ERROR: Error cloning remote repo 'origin'
hudson.plugins.git.GitException: Command "C:\Program Files\Git\bin\git.exe fetch --tags --progress http://repository.vrpconsulting.com/roman.skaskevich/koshcheck-copy.git +refs/heads/*:refs/remotes/origin/*" returned status code 128:
stdout:
stderr: Logon failed, use ctrl+c to cancel basic credential prompt.
remote: HTTP Basic: Access denied
fatal: Authentication failed for 'http://repository.vrpconsulting.com/roman.skaskevich/koshcheck-copy.git/'
GITLAB SETTINGS: here
EDIT #1
When I run git fetch --tags --progress http://repository.vrpconsulting.com/roman.skaskevich/koshcheck-copy.git +refs/heads/*:refs/remotes/origin/* on local machine, it performs successfully and dialog for entering login/password is showed.
But what I have to do that jenkins job performed this command successfully?
EDIT #2
For now, Credentials Binding Plugin helps me implements to checking credentials in Jenkinsfile.
EDIT #3
I run Jenkins server on another machine and have the same problem. I don't want to use login/password, so adding SSH key solved this problem.
Thanks in advance!
Seems like your repo is private. Private git repos requires authentication. Reconfigure your git client plugin check if you are able to connect to remote repo via password based authentication , if not , try SSHing to the repo.
Looks like this ins private repo if you want to clone that repo to jenkins workspace you need to provide your gitlab credentials in jenins.
for that, you need to install jenkins plugin called "Gitlab Authentication plugin". once you completed the installation please do a restart.
Add Gitlab credentials in "Jenkins Credentials Provider: Jenkins".
later you can add your gitlab credentials in Source Code Management
Source Code Management
in credentials select the saved gitlab credentials.
Steps:
Install plugins:
Gitlab
Gitlab authentication
Configure global credentials in jenkins:
go to credentials >> system >> global creds >> username & password
ID: git
username: your gitlab username
password: your gitlab password
Pipeline:
stage('SCM') {
steps {
git credentialsId: 'git', url: 'your gitlab url'
}
}
I got this error too. In my case, the git repo is a private one.
I added the git link and then supplied the git credentials to the particular Jenkins job. The issue is gone and able to build project from Jenkins.
I have a Laravel project with some dependencies to private packages, secured through ssh, working on windows 10 with Laragon.
On composer install I get a permission denied (public key), however, if I clone the repo directly I get my regular prompt to type the ssh-key password for authentication and the clone works with no problems (using git#gitlab...., so no https).
The output looks like this:
Failed to execute git clone ...
Cloning into 'project/path/foo/bar'...
Permission denied (public key).
fatal: Could not read from remote repository.
Please make sure you have the correct access rights
and the repository exists.
I already added the host (a private GitLab server) to my git config file, including the path to my ssh key, which seems to work due to git clone succeeding.
I expected that composer install would prompt for the password, but it just stops.
Any ideas how to solve this problem? I normally work on Ubuntu and didn't have such a problem before.
Edit:
Tried out the same commands with git bash (Git for windows) and it worked. But if I use Powershell or ConEmu(through Laragon) it won't work, any ideas?
I get this error when configuring git repo in a new jenkins project:
Failed to connect to repository : Error performing command: ls-remote -h file:///c:\Dev\git\rx HEAD
I have a local git repo on my windows machine and cloning the same path from command prompt works on the same machine. I don't know what could be the problem (git.exe is on system path)
Windows installs Jenkins as a service. But it runs the Jenkins service under different user permissions that don't have access to GIT. You can follow this link to find the user that Jenkins runs under. http://antagonisticpleiotropy.blogspot.com/2012/08/running-jenkins-in-windows-with-regular.html
For example. My local user name on my windows is "nathandrewsire", exclude the quotes. Open windows services manager (either cmd and c:\services.msc, OR use windows key + R). Enable Log On > This account:. Then type in .\username (e.g. .\nathandrewsire). Then restart the service and you will have access to GIT from Jenkins builds.
I also found a snippet for windows shell command that will output the user to verify it.
echo %PATH%
echo %USERDOMAIN%\%USERNAME%
http://antagonisticpleiotropy.blogspot.com/2012/08/running-jenkins-in-windows-with-regular.html
Linux
If you're using Linux, this error can also be caused from not enabling share on your Git repo. Linux jenkins user won't be able to access Git report under another user unless....
git --bare init --shared=group
Also, your jenkins user and Git repo user must belong to the same group for file permission access. There's other alternatives to that like messing with umasks and ACL's but setting up a linux group for your two users is the easiest way.
Make sure NOT TO OVERWRITE your Git repo with the bare init command. If you already have a Git repo, you'll need to setup/move to a new one with the shared option. I didn't see a way to convert existing repos to shared because I am guessing the files would all need to be updated?
NOTE***
I used VMWare Workstation shared folders to build on a windows guest. Jenkins needs the following format when entering the Git repo.
//vmware-host/Shared Folders/vm-share/ridcyDevelopment.git/
Unable to get Jenkins and Git to play nice