pip and private repositories - pip

I'm deploying a python application and I'm trying to create a pipeline with Drone.io
In my main application, I have multiple dependencies that are located in private Github repositories requirements.txt:
git+ssh://git#hostname_A/org/repo_A.git
git+ssh://git#hostname_B/org/repo_B.git
For each repository, I have a Github Deploy Key.
So my .drone.yml contains the multiple deploy keys store in drone secrets.
Inside my container, I'm copying a .ssh/config file:
Host hostname_A
HostName github.com
User git
IdentityFile /root/.ssh/repo_A_rsa
Host hostname_B
HostName github.com
User git
IdentityFile /root/.ssh/repo_B_rsa
The issue is that the repository/package repo_B also has a dependency with the private repository repo_A.
And I don't know how to link this private repository, I have tried to add install_requires and dependency_links inside the setup.py of repo_B:
setuptools.setup(
name="repo_B",
install_requires=[
"repo_A_alias # git+ssh://git#hostname_A/org/repo_A.git"
],
)
But I still get the error:
collecting repo_B # git+ssh://git#github.com/org/repo_B.git
Cloning ssh://****#github.com/org/repo_B.git to /tmp/pip-install-naymvvlo/repo_B_67066986e78c4124b8cc99242aeaa673
ERROR: Command errored out with exit status 128: git clone -q 'ssh://****#github.com/org/repo_B.git' /tmp/pip-install-naymvvlo/repo_B_67066986e78c4124b8cc99242aeaa673 Check the logs for full command output.
Or is there a way to access the log file?

Related

Installing golang package from private gerrit repository

I'm trying to use one of projects hosted in gerrit as a dependency in golang.
I installed go-import plugin in gerrit
I added this to .gitconfig:
[url "ssh://myuser#gerrit.example.com:29418"]
insteadOf = https://gerrit.example.com
Now I run:
go get gerrit.example.com/myproject
but it fails with:
go get: module gerrit.example.com/myproject: git ls-remote -q origin in /home/me/go/pkg/mod/cache/vcs/513f491cb527a8cec5b684e8d77254c851f76499e1f725440f98d4e9ad8bbf4f: exit status 128:
fatal: project a/myproject not found
fatal: Could not read from remote repository.
Make sure that global git configuration is not setup with something along the lines:
git config --global url."git#gerrit.example.com:".insteadOf "https://gerrit.example.com"
.gitconfig
[url "git#gerrit.example.com"]
insteadOf = https://gerrit.example.com
For regular use of the git tool, it's convient to have a ~/.ssh/config file set up. In order to do this, run the following commands:
mkdir ~/.ssh
chmod 700 ~/.ssh
touch ~/.ssh/config
chmod 600 ~/.ssh/config
Please note the permissions on the files and directory above are essentail for SSH to work in it's default configuration on most Linux systems.
Then, edit the ~/.ssh/config file to match the following:
Host gerrit.example.com
HostName gerrit.example.com
Port 29418
User USERNAME_HERE
IdentityFile ~/.ssh/id_gerrit_example_com
Please note the spacing in the above file matters and will invalidate the file if it is incorrect.
Where USERNAME_HERE is your gerrit username and ~/.ssh/id_rsa is the path to your SSH private key in your file system. You've already uploaded its public key to gerrit.
Note: I have created this configuration just on assumption since i do not have gerrit setup to test on.

AWX Failing to install requirements from private repo

I have an AWX installation with a project. The project is fetched perfectly from the private git repo.
The project has a requirements file which is executed by AWX. The requirements file contains private repositories like this:
- src: git+http://gitlab.test.com/ansible/test.git
version: master
name: test
This however fails with the following error:
test was NOT installed successfully: - command git clone failed in directory.
How can I configure AWX to use the git credentials that are in place already in awx?
It seems like AWX works 'smarter' than I thought it would.
In this case, AWX used the same credentials that it used to fetch the project in the first place. I had two options to resolve this:
Change the links to work via git
Update the same git credential and also add the username and password of that git user.

composer install does not use ssh key on private repo (permission denied public key)

I have a Laravel project with some dependencies to private packages, secured through ssh, working on windows 10 with Laragon.
On composer install I get a permission denied (public key), however, if I clone the repo directly I get my regular prompt to type the ssh-key password for authentication and the clone works with no problems (using git#gitlab...., so no https).
The output looks like this:
Failed to execute git clone ...
Cloning into 'project/path/foo/bar'...
Permission denied (public key).
fatal: Could not read from remote repository.
Please make sure you have the correct access rights
and the repository exists.
I already added the host (a private GitLab server) to my git config file, including the path to my ssh key, which seems to work due to git clone succeeding.
I expected that composer install would prompt for the password, but it just stops.
Any ideas how to solve this problem? I normally work on Ubuntu and didn't have such a problem before.
Edit:
Tried out the same commands with git bash (Git for windows) and it worked. But if I use Powershell or ConEmu(through Laragon) it won't work, any ideas?

Gitolite, can't clone new repo

I've just installed gitolite on my Debian server.
Then I cloned gitolite-admin repo and and new public key + following lines to conf file:
repo wallr_common
RW+ = wall
New repo was created after push.
At first I go to authorized_keys and saw that key for new user not exists, then I run ~/.gitolite/keydir$ gl-setup, and the key appears in authorized_keys.
Now I'm trying to clone it but I'm getting error:
git.exe clone --progress -v "ssh://wall#192.168.1.110:/wallr_common.git" "D:\wallr_common"
Cloning into 'D:\wallr_common'...
fatal: Could not read from remote repository.
Please make sure you have the correct access rights and the repository exists.
git did not exit cleanly (exit code 128) (5834 ms # 04.07.2013 0:27:46)
What can be the root cause of this?
You must use the git or gitolite account for your ssh session. Not the user account wall that you registered in gitolite.
That means ssh://git#..., instead of ssh://wall#....
Replace 'git' with the account you used to install gitolite.
See more at "How do programs like gitolite work?".
This is a similar mistake as in this question.

Jenkins SSH issue on Mac OS X installed by homebrew

I installed Jenkins CI from homebrew (brew install jenkins). So, it runs under my own user. Everything works fine except for SSH access to my Git repository. I do not want to create a separate user (e.g. jenkins) to run Jenkins and create separate SSH keys for it. I just want to use my own SSH keys. Accessing my remote git repo on the command-line works fine with my ssh keys, but in Jenkins it does not work. It gives a 'Permission denied (publickey)' error (username and project are obfuscated (<user> and <project>)):
Started by user anonymous
Checkout:workspace / /Users/<user>/.jenkins/jobs/<project>_run_tests/workspace - hudson.remoting.LocalChannel#456d3d51
Using strategy: Default
Last Built Revision: Revision 093f1641f26632afd1a74c14276ec544356c1d08 (origin/dev)
Checkout:workspace / /Users/<user>/.jenkins/jobs/<project>_run_tests/workspace - hudson.remoting.LocalChannel#456d3d51
Fetching changes from 1 remote Git repository
Fetching upstream changes from ssh://git#bitbucket.org/<user>/<project>.git
ERROR: Problem fetching from origin / origin - could be unavailable. Continuing anyway
ERROR: (Underlying report) : Error performing command: /usr/local/bin/git fetch -t ssh://git#bitbucket.org/<user>/<project>.git +refs/heads/*:refs/remotes/origin/*
Command "/usr/local/bin/git fetch -t ssh://git#bitbucket.org/<user>/<project>.git +refs/heads/*:refs/remotes/origin/*" returned status code 128: Permission denied (publickey).
fatal: The remote end hung up unexpectedly
ERROR: Could not fetch from any repository
FATAL: Could not fetch from any repository
hudson.plugins.git.GitException: Could not fetch from any repository
at hudson.plugins.git.GitSCM$2.invoke(GitSCM.java:1012)
at hudson.plugins.git.GitSCM$2.invoke(GitSCM.java:972)
at hudson.FilePath.act(FilePath.java:783)
at hudson.FilePath.act(FilePath.java:765)
at hudson.plugins.git.GitSCM.checkout(GitSCM.java:972)
at hudson.model.AbstractProject.checkout(AbstractProject.java:1195)
at hudson.model.AbstractBuild$AbstractRunner.checkout(AbstractBuild.java:568)
at hudson.model.AbstractBuild$AbstractRunner.run(AbstractBuild.java:457)
at hudson.model.Run.run(Run.java:1404)
at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:46)
at hudson.model.ResourceController.execute(ResourceController.java:88)
at hudson.model.Executor.run(Executor.java:230)
It seems it is not using my ssh keys to connect to the git server. I tried adding a UserName key to the jenkins launchagent plist, did a restart, but no effect.
How can I make jenkins use my SSH keys to connect to my remote git repo?
I do not know how homebrew sets up Jenkins, but apparently it is running in a context where it has no access to your ssh agent and your ssh key is passphrase-protected.
When you run it in a Terminal window, you have SSH_AUTH_SOCK variable in your environment which ssh client uses to authenticate. If you cannot use the ssh agent, you need to remove the passphrase from the ssh key.

Resources