SSH access to github repo on codeship - bash

I am attempting to push to github from a container on Codeship. After getting a Permission denied (publickey) error, I followed the suggestion here:
https://documentation.codeship.com/pro/builds-and-configuration/setting-ssh-private-key/
I created a service called publish to and some steps to try to recreate the article's suggestion.
My codeship_services.yml file:
# codeship_services.yml
publish:
build:
image: codeship/setting-ssh-key-test
dockerfile: Dockerfile.publish
encrypted_env_file: codeship.env.encrypted
volumes:
- ./.ssh:/root/.ssh
My codeship_steps.yml file:
- name: temp publish service
service: publish
command: /bin/bash -c "echo -e $PRIVATE_SSH_KEY >> /root/.ssh/id_rsa"
- name: chmod id_rsa
service: publish
command: chmod 600 /root/.ssh/id_rsa
- name: add server to list of known hosts
service: publish
command: /bin/bash -c "ssh-keyscan -H github.com >> /root/.ssh/known_hosts"
- name: confirm ssh connection to server, authenticating with generated public ssh key
service: publish
command: /bin/bash -c "ssh -T git#github.com"
When running jet steps, however, I still get the Permission denied (publickey) error:
(step: temp_publish_service) success ✔
(step: chmod_id_rsa)
(step: chmod_id_rsa) success ✔
(step: add_server_to_list_of_known_hosts)
(service: publish) (step: add_server_to_list_of_known_hosts) # github.com:22 SSH-2.0-babeld-80573d3e
(service: publish) (step: add_server_to_list_of_known_hosts) # github.com:22 SSH-2.0-babeld-80573d3e
(service: publish) (step: add_server_to_list_of_known_hosts) # github.com:22 SSH-2.0-babeld-80573d3e
(step: add_server_to_list_of_known_hosts) success ✔
(step: confirm_ssh_connection_to_server,_authenticating_with_generated_public_ssh_key)
(service: publish) (step: confirm_ssh_connection_to_server,_authenticating_with_generated_public_ssh_key) Permission denied (publickey).
(step: confirm_ssh_connection_to_server,_authenticating_with_generated_public_ssh_key) error ✗
(step: confirm_ssh_connection_to_server,_authenticating_with_generated_public_ssh_key) container exited with a 255 code
I have generated the keys as instructed in the article and added the encrypted private key to codeship.env.encryped.
Is there something I am missing?

The only missing step would be to register the public Key on your GitHub account itself
Only then an SSH connection using that same public key would have a chance to succeed.
If not, try at least an ssh -Tvv git#github.com in your last step, in order to get more clues.

Related

Deploy to EC2 instance from Github Actions

Good night guys, I've been trying to find out this but it's the first time I'm doing it.
With Github OpenID connect I could to deploy the code to a S3 bucket, but now I need to pass these files from s3 to an EC2 instance in the deploy.yml file, but I can't use
access-key-id: ${{ secrets.AWS_ACCESS_KEY_ID }}
secret-access-key: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
Because the client doesn't want to give access to secrets.
There's any other way to do this?
I'm completely lost and I don't have administrator access or the enough permissions in AWS to work.
Thanks.
I have already the files in a S3 bucket with this code:
- name: Copy to S3 bucket
run: |
aws s3 sync --delete ./my_path s3://${{ env.BUCKET_NAME }}/${{ steps.extract_branch.outputs.branch }}/${{steps.extract_hash.outputs.commit_hash }}
And I've been trying to do something like this:
- name: Deploy to EC2 instance
uses: easingthemes/ssh-deploy#v2.1.5
env:
SSH_PRIVATE_KEY: "ALL PRIVATE KEY CODE PASTED"
SOURCE: "./"
REMOTE_HOST: "my host"
REMOTE_USER: "my user"
TARGET: "/path/to/copy/"
EXCLUDE: "/dist/, /node_modules/, /venv/"
This is the error:
Run easingthemes/ssh-deploy#v2.1.5
[general] GITHUB_WORKSPACE: /home/runner/work/project
[SSH] Creating /home/runner/.ssh dir in /home/runner/work/project
✅ [SSH] dir created.
[SSH] Creating /home/runner/.ssh/known_hosts file in /home/runner/work/project
✅ [SSH] file created.
✅ Ssh key added to `.ssh` dir /home/runner/.ssh/deploy_key
[Rsync] Starting Rsync Action: /home/runner/work/project/docker/ to ubuntu#ec2-server.com:/home/ubuntu/
⚠️ [Rsync] error: rsync exited with code 255
⚠️ [Rsync] stderr: Warning: Permanently added 'ec2-server.com' (ED25519) to the list of known hosts.
Load key "/home/runner/.ssh/deploy_key": error in libcrypto
ubuntu#ec2-server.com: Permission denied (publickey).
rsync: connection unexpectedly closed (0 bytes received so far) [sender]
rsync error: unexplained error (code 255) at io.c(228) [sender=3.2.3]
⚠️ [Rsync] stdout:
⚠️ [Rsync] cmd: rsync /home/runner/work/project/docker/ ubuntu#ec2-server.com:/home/ubuntu/ --rsh "ssh -p 22 -i /home/runner/.ssh/deploy_key -o StrictHostKeyChecking=no" --recursive -rltgoDzvO
1: 0xa1a640 node::Abort() [/home/runner/runners/2.299.1/externals/node12/bin/node]
2: 0xa90649 [/home/runner/runners/2.299.1/externals/node12/bin/node]
3: 0xc06599 [/home/runner/runners/2.299.1/externals/node12/bin/node]
4: 0xc08387 v8::internal::Builtin_HandleApiCall(int, unsigned long*, v8::internal::Isolate*) [/home/runner/runners/2.299.1/externals/node12/bin/node]
5: 0x140dd19 [/home/runner/runners/2.299.1/externals/node12/bin/node]

Cannot connect to the Docker daemon at tcp://xxx.xxx.xx.xxx:2375. Is the docker daemon running?

I'm using Gitlab and Docker to get continuous integration to my spring boot application and I'm getting this error:
Cannot connect to the Docker daemon at tcp://xxx.xxx.xx.xxx:2375. Is the docker daemon running?
.development.env:
export SPRING_ACTIVE_PROFILE='development'
export DOCKER_REPO='DOCKER_HUB_ID/app_name:dev'
export APP_NAME='app_name_dev'
export PORT='8080'
export SERVER_IP='xxx.xxx.xx.xxx' #SERVER_IP
export SERVER_SSH_KEY="$DEV_SSH_PRIVATE_KEY"
export DOCKER_HOST='tcp://xxx.xxx.xx.xxx:2375' #SERVER_IP
.gitlab-ci.yml
services:
- docker:19.03.7-dind
stages:
- build and push docker image
docker build:
image: docker:stable
stage: build and push docker image
before_script:
- source .${CI_COMMIT_REF_NAME}.env #.development.env
script:
- docker build --build-arg SPRING_ACTIVE_PROFILE=$SPRING_ACTIVE_PROFILE -t $DOCKER_REPO .
- docker login -u $DOCKER_USER -p $DOCKER_PASSWORD docker.io
- docker push $DOCKER_REPO
This is the whole logs from gitlab:
Running with gitlab-runner 13.5.0 (ece86343)
on gitlab-server JuhWVkPJ
Preparing the "docker" executor
00:38
Using Docker executor with image docker:stable ...
Starting service docker:19.03.7-dind ...
Pulling docker image docker:19.03.7-dind ...
Using docker image sha256:14af3ba31e635475ec8f7fbe17470424514777621e627a91c41bbbe028dbae16 for docker:19.03.7-dind with digest docker#sha256:2683fcdf7480ea101415833f7793fb058c5f20227890a953b0a70bfc350af5bc ...
Waiting for services to be up and running...
*** WARNING: Service runner-juhwvkpj-project-13-concurrent-0-7c99eb8ace2e2ae6-docker-0 probably didn't start properly.
Health check error:
service "runner-juhwvkpj-project-13-concurrent-0-7c99eb8ace2e2ae6-docker-0-wait-for-service" timeout
Health check container logs:
Service container logs:
2020-12-30T03:14:07.879506461Z Generating RSA private key, 4096 bit long modulus (2 primes)
2020-12-30T03:14:08.459745140Z ..............................................++++
2020-12-30T03:14:08.673203110Z ..................++++
2020-12-30T03:14:08.673231544Z e is 65537 (0x010001)
2020-12-30T03:14:08.713960023Z Generating RSA private key, 4096 bit long modulus (2 primes)
2020-12-30T03:14:08.851463609Z ..............++++
2020-12-30T03:14:09.403244538Z .....................................................++++
2020-12-30T03:14:09.403286293Z e is 65537 (0x010001)
2020-12-30T03:14:09.516423752Z Signature ok
2020-12-30T03:14:09.516463300Z subject=CN = docker:dind server
2020-12-30T03:14:09.516471290Z Getting CA Private Key
2020-12-30T03:14:09.536975767Z /certs/server/cert.pem: OK
2020-12-30T03:14:09.553642146Z Generating RSA private key, 4096 bit long modulus (2 primes)
2020-12-30T03:14:09.927078677Z ...................................................++++
2020-12-30T03:14:10.107451624Z ...................++++
2020-12-30T03:14:10.108457646Z e is 65537 (0x010001)
2020-12-30T03:14:10.156096074Z Signature ok
2020-12-30T03:14:10.156125397Z subject=CN = docker:dind client
2020-12-30T03:14:10.156302268Z Getting CA Private Key
2020-12-30T03:14:10.178703934Z /certs/client/cert.pem: OK
2020-12-30T03:14:10.194290163Z mount: permission denied (are you root?)
2020-12-30T03:14:10.194438175Z Could not mount /sys/kernel/security.
2020-12-30T03:14:10.194456604Z AppArmor detection and --privileged mode might break.
2020-12-30T03:14:10.195933829Z mount: permission denied (are you root?)
*********
Pulling docker image docker:stable ...
Using docker image sha256:b0757c55a1fdbb59c378fd34dde3e12bd25f68094dd69546cf5ca00ddbaa7a33 for docker:stable with digest docker#sha256:fd4d028713fd05a1fb896412805daed82c4a0cc84331d8dad00cb596d7ce3e3a ...
Preparing environment
00:01
Running on runner-juhwvkpj-project-13-concurrent-0 via gitlab-server...
Getting source from Git repository
00:03
Fetching changes with git depth set to 50...
Reinitialized existing Git repository in /builds/abdallah/harvis/.git/
Checking out 5568bbc9 as DM_Module...
Skipping Git submodules setup
Executing "step_script" stage of the job script
00:36
$ source .${CI_COMMIT_REF_NAME}.env
$ whoami
root
$ docker build --build-arg SPRING_ACTIVE_PROFILE=$SPRING_ACTIVE_PROFILE -t $DOCKER_REPO .
Cannot connect to the Docker daemon at tcp://xxx.xxx.xx.xxx:2375. Is the docker daemon running?
Cleaning up file based variables
00:01
ERROR: Job failed: exit code 1
I think the error is in DOCKER_HOST, what should I assign it?
I'd appreciate any help or suggestion.
The Docker daemon can listen for Docker Engine API requests via three different types of Socket: unix, tcp, and fd
your docker daemon is trying to connect via tcp (2375), and it seems not enabled.
you need to start docker daemon with -H tcp://<ip>:2375
or put it in /etc/docker/daemon.json
"hosts": ["tcp://<ip>:2375", "unix:///var/run/docker.sock"],
EDIT
Binding to 0.0.0.0 is dangerous as David pointed out.

Deploying to FTP server in Github Actions does not work

As the title says, deploying to FTP server isn't working for me from a Github Action. I've tried using a couple of actions to accomplish this (FTP-Deploy and ftp-action), but FTP-Deploy just kept running with sporadic
curl: (7) Failed to connect to ftpservername.com port 21: Connection timed out
messages and ftp-action kept running without any output. Note: The server is available, I connected and transferred some files using Filezilla without any issues.
After that I tried using lftp, this is the command I used on a local Ubuntu machine
lftp -c "open -u username,password ftpservername.com; mirror -R locfolder remote/remotefolder"
and the file transfer worked, but when used in a Github Action it produced this output:
---- Connecting to ftpservername.com (123.456.789.123) port 21
mkdir `remote/remotefolder' [Connecting...]
**** Socket error (Connection timed out) - reconnecting
---- Closing control socket
---- Connecting to ftpservername.com (123.456.789.123) port 21
I tried setting both ftp:ssl-allow and ssl:verify-certificate to false, but this did not produce any results. Also, I do not have access to the server, so I can't check the server logs.
This is the workflow file:
name: Test
on:
push:
branches: [master]
jobs:
build:
runs-on: ubuntu-latest
steps:
- name: Checkout repo
uses: actions/checkout#v2
- name: Setup Python
uses: actions/setup-python#v2
with:
python-version: '3.x'
- name: Install pip
run: python -m pip install --upgrade pip
- name: Install packages
run: |
sudo apt install lftp
sudo apt install expect
.
.
.
- name: FTP Deploy
run: |
echo Starting...
unbuffer lftp -c "debug; set ftp:ssl-allow false; set ssl:verify-certificate false; open -u username,${{ secrets.PASSWORD }} ftpservername.com; mirror -R -v locfolder remote/remotefolder"
echo Done transferring files.
Any help is appreciated, thank you!
Found the issue, the hosting service was blocking the IP address (as it was an IP address outside of the country). After setting up a self-hosted runner and whitelisting the IP of the runner everything works fine.

Bitbucket fatal: Can't access remote

I tried to set up sftp connection between Bitbucket and Runcloud server. Runcloud only uses sftp connection. Bitbucket config:
image: php:7.3
pipelines:
branches:
master:
- step:
name: Deploy to production
deployment: production
script:
- apt-get update
- apt-get -qq install git-ftp
- git ftp init --user $SFTP_username --passwd $FTP_password sftp://runcloud#1.111.111.11/home/runcloud/webapps/mywebsite/wp-content/themes/mywebsiteTheme
Connection always fails with error fatal: Can't access remote 'sftp://1.111.111.11', exiting...
I tried a different sftp Path combination but the result always the same.
sftp://1.111.111.11/home/runcloud/webapps/mywebsite/wp-content/themes/mywebsiteTheme
sftp://mywebsite/home/runcloud/webapps/mywebsite/wp-content/themes/mywebsiteTheme
My website
Root Path: /home/runcloud/webapps/mywebsite
Public Path: /home/runcloud/webapps/mywebsite
Runcload have different as "normal" set up for ftp. For example to conect with FileZila HOST is my server ip. And to get to my website i have to navigate /webapps/mywebsite
Not sure what I doing wrong is my sftp path incorrect?

Capistrano deploy - Permission denied

I'm trying to deploy my application with capistrano but I'm having some problems. My machine is a ec2 amazon and I have the .pem locally. I can do ssh and run commands with no problem, but for cap production deploy I get the following error:
DEBUG [4f4633f7] Command: ( export GIT_ASKPASS="/bin/echo" GIT_SSH="/tmp/git-ssh-hybrazil-production-ronanlopes.sh" ; /usr/bin/env git ls-remote --heads git#git#github.com:fneto/hybrazil.git )
DEBUG [4f4633f7] Permission denied (publickey).
DEBUG [4f4633f7]
DEBUG [4f4633f7] fatal: Could not read from remote repository.
DEBUG [4f4633f7]
DEBUG [4f4633f7]
DEBUG [4f4633f7] Please make sure you have the correct access rights
DEBUG [4f4633f7]
and the repository exists.
DEBUG [4f4633f7]
On my production/deploy.rb, I have the config like this:
set :ssh_options, {
keys: %w(/home/ronanlopes/Pems/hybrazil-impulso.pem ~/.ssh/id_rsa),
forward_agent: true,
auth_methods: %w(publickey)
}
any ideas? Thanks in advance!
You can add your key to agent, use command:
ssh-add ~/.ssh/id_rsa
In your code you should use full path to ssh key, without pem:
keys: %w(/home/user_name/.ssh/id_rsa)

Resources