Gitlab remote server push on the deploy step does not push new files - continuous-integration

I have the below ci/cd script, (removed the test steps from it which works fine)
Our repo has two branches ready_4_release and master(prod), any feature branch gets merged into ready_4_release and at the end of the week we push the changes into master.
Any feature branch that gets merged into ready_4_release gets deployed into the remote airflow servers,
problem:
When a new MR gets merged into ready_4_release branch the changes are not getting deployed into remote servers but the changes on the feature branch gets merged into the ready_4_release branch, i am not able to figure out why?
When we rerun the deploy step the changes does appear on the remote server(Airflow),
OR when a new MR is submitted the previous changes gets deployed into remote servers.
Below is how the branches are setup in gitlab.
This is how the merge request settings are on the repo
stages:
- test
- deploy
.ssh-connection: &ssh-connection
- 'which git || ( apt-get update -y && apt-get install git -y )'
- 'which ssh-agent || ( apt-get update -y && apt-get install openssh-client -y )'
- 'which rsync || ( apt-get update -y && apt-get install rsync -y )'
- eval $(ssh-agent -s)
- echo "$LOTBOT_SSH_PRIVATE_KEY" | tr -d '\r' | ssh-add -
- mkdir -p ~/.ssh
- chmod 700 ~/.ssh
- '[[ -f /.dockerenv ]] && echo -e "Host *\n\tStrictHostKeyChecking no\n\n" > ~/.ssh /config'
- ssh-keyscan ip-10-0-1-24.ec2.internal >> ~/.ssh/known_hosts
- chmod 644 ~/.ssh/known_hosts
deploy-dev:
stage: deploy
tags:
- prd-runner
image: ${CI_DEPENDENCY_PROXY_GROUP_IMAGE_PREFIX}/ubuntu
before_script:
- echo "Deploying repo dev-data-analytics to Airkube EFS"
- *ssh-connection
script:
- ssh lotbot#10.0.1.24 "sudo rm -r /mnt/airflow_dags/airkube/external_repos/dev- -data-analytics/"
- echo "${PWD}"
- cd ..
- mkdir dev-lotlinx-data-analytics && cp -R "${PWD}"/data-analytics/* "${PWD}"/dev-data-analytics
- cd dev-data-analytics
# - echo "Git commit id"
# - git show -s --format=%h
- echo "${PWD}"
- ssh lotbot#10.0.1.24 "sudo mkdir -p /mnt/airflow_dags/airkube/external_repos/dev- data-analytics"
- ssh lotbot#10.0.1.24 "sudo chown -R lotbot:sudo /mnt/airflow_dags/airkube/external_repos/dev-lotlinx-data-analytics/ && sudo chmod -R g+w /mnt/airflow_dags/airkube/external_repos/dev-data-analytics/"
- rsync -avu --delete --exclude "*__pycache__" "${PWD}"/ lotbot#10.0.1.24:/mnt/airflow_dags/airkube/external_repos/lotlinx-data-analytics;
- ssh lotbot#10.0.1.24 "sudo sleep 3s"
- ssh lotbot#10.0.1.24 "sudo chown -R lotbot:sudo /mnt/airflow_dags/airkube/external_repos/dev-lotlinx-data-analytics/ && sudo chmod -R g+w /mnt/airflow_dags/airkube/external_repos/lotlinx-data-analytics/"
rules:
- if: '$CI_COMMIT_REF_NAME == "ready_for_release" && $CI_PIPELINE_SOURCE == "push"'

Related

Gitlab CI check if directory exists before pulling origin

I'm trying to deploy my flask application to AWS EC2 instance using gitlab ci runner.
.gitlab.ci.yml
stages:
- test
- deploy
test_app:
image: python:latest
stage: test
before_script:
- python -V
- pip install virtualenv
- virtualenv env
- source env/bin/activate
- pip install flask
script:
- cd flask-ci-cd
- python test.py
prod-deploy:
stage: deploy
only:
- master # Run this job only on changes for stage branch
before_script:
- mkdir -p ~/.ssh
- echo -e "$RSA_KEY" > ~/.ssh/id_rsa
- chmod 600 ~/.ssh/id_rsa
- '[[ -f /.dockerenv ]] && echo -e "Host *\n\tStrictHostKeyChecking no\n\n" > ~/.ssh/config'
script:
- bash .gitlab-deploy-prod.sh
environment:
name: deploy
.gitlab-deploy-prod.sh
#!/bin/bash
# Get servers list
set -f
# access server terminal
shell="ssh -o StrictHostKeyChecking=no ${SERVER_URL}"
git_token=$DEPLOY_TOKEN
echo "Deploy project on server ${SERVER_URL}"
if [ ${shell} -d "/flask-ci-cd" ] # check if directory exists
then
eval "${shell} cd flask-ci-cd && git clone https://sbhusal123:${git_token}#gitlab.com/sbhusal123/flask-ci-cd.git master && cd flask-ci-cd"
else
eval "${shell} git pull https://sbhusal123:${git_token}#gitlab.com/sbhusal123/flask-ci-cd.git master && cd flask-ci-cd && cd flask-ci-cd"
fi
Error: .gitlab-deploy-prod.sh: line 7: -o: command not found
How can i check if directory existing??
What i've tried.
#!/bin/bash
# Get servers list
set -f
# access server terminal
shell="ssh -o StrictHostKeyChecking=no ${SERVER_URL}"
git_token=$DEPLOY_TOKEN
eval "${shell}" # i thought gitlab would provide me with shell access
echo "Deploy project on server ${SERVER_URL}"
if [-d "/flask-ci-cd" ] # check if directory exists
then
eval "cd flask-ci-cd && git clone https://sbhusal123:${git_token}#gitlab.com/sbhusal123/flask-ci-cd.git master && cd flask-ci-cd"
else
eval "git pull https://sbhusal123:${git_token}#gitlab.com/sbhusal123/flask-ci-cd.git master && cd flask-ci-cd && cd flask-ci-cd"
fi
I've tried to log into the ssh shell before executing the scripts inside if else. But it doesn't works the way intended.
Your script has some errors.
Do not use eval. No, eval does not work that way. eval is evil
When storing a command to a variable, do not use normal variables. Use bash arrays instead to preserve "words".
Commands passed via ssh are double escaped. I would advise to prefer to use here documents, they're simpler to get the quoting right. Note the difference in expansion when the here document delimiter is quoted or not.
i thought gitlab would provide me with shell access No, without open standard input the remote shell will just terminate, as it will read EOF from input. No, it doesn't work that way.
Instead of doing many remote connection, just transfer the execution to remote side once and do all the work there.
Take your time and research how quoting and word splitting works in shell.
git_token=$DEPLOY_TOKEN No, set variables are not exported to remote shell. Either pass them manually or expand them before calling the remote side. (Or you could also use ssh -o SendEnv=git_token and configure remote ssh with AcceptEnv=git_token I think, never tried it).
Read documentation for the utilities you use.
No, git clone doesn't take branch name after url. You can specify branch with --branch or -b option. After url it takes directory name. See git clone --help. Same for git pull.
How can i check if directory existing??
Use bash arrays to store the command. Check if the directory exists just by executing the test command on the remote side.
shell=(ssh -o StrictHostKeyChecking=no "${SERVER_URL}")
if "${shell[#]}" [ -d "/flask-ci-cd" ]; then
...
In case of directory name with spaces I would go with:
if "${shell[#]}" sh <<'EOF'
[ -d "/directory with spaces" ]
EOF
then
Pass set -x to sh to see what's happening also on the remote side.
For your script, try rather to move the execution to remote side - there is little logic in making 3 separate connections. I say just
echo "Deploy project on server ${SERVER_URL}"
ssh -o StrictHostKeyChecking=no "${SERVER_URL}" bash <<EOF
if [ ! -d /flask-ci-cd ]; then
# Note: git_token is expanded on host side
git clone https://sbhusal123:${git_token}#gitlab.com/sbhusal123/flask-ci-cd.git /flask-ci-cd
fi
cd /flask-ci-cd
git pull
EOF
But instead of getting the quoting in some cases right, use declare -p and declare -f to transfer properly quoted stuff to remote side. That way you do not need case about proper quoting - it will work naturally:
echo "Deploy project on server ${SERVER_URL}"
work() {
if [ ! -d /flask-ci-cd ]; then
# Note: git_token is expanded on host side
git clone https://sbhusal123:"${git_token}"#gitlab.com/sbhusal123/flask-ci-cd.git /flask-ci-cd
fi
cd /flask-ci-cd
git pull
{
ssh -o StrictHostKeyChecking=no "${SERVER_URL}" bash <<EOF
$(declare -p git_token) # transfer variables you need
$(declare -f work) # transfer function you need
work # call the function.
EOF
Updated answer for future reads.
.gitlab-ci.yml
stages:
- test
- deploy
test_app:
image: python:latest
stage: test
before_script:
- python -V
- pip install virtualenv
- virtualenv env
- source env/bin/activate
- pip install flask
script:
- cd flask-ci-cd
- python test.py
prod-deploy:
stage: deploy
only:
- master
before_script:
- mkdir -p ~/.ssh
- echo -e "$RSA_KEY" > ~/.ssh/id_rsa
- chmod 600 ~/.ssh/id_rsa
- '[[ -f /.dockerenv ]] && echo -e "Host *\n\tStrictHostKeyChecking no\n\n" > ~/.ssh/config'
script:
- bash .gitlab-deploy-prod.sh
environment:
name: deploy
.gitlab-deploy-prod.sh
#!/bin/bash
# Get servers list
set -f
shell=(ssh -o StrictHostKeyChecking=no "${SERVER_URL}")
git_token=$DEPLOY_TOKEN
echo "Deploy project on server ${SERVER_URL}"
ssh -o StrictHostKeyChecking=no "${SERVER_URL}" bash <<EOF
if [ ! -d flask-ci-cd ]; then
echo "\n Cloning into remote repo..."
git clone https://sbhusal123:${git_token}#gitlab.com/sbhusal123/flask-ci-cd.git
# Create and activate virtualenv
echo "\n Creating virtual env"
python3 -m venv env
else
echo "Pulling remote repo origin..."
cd flask-ci-cd
git pull
cd ..
fi
# Activate virtual env
echo "\n Activating virtual env..."
source env/bin/activate
# Install packages
cd flask-ci-cd/
echo "\n Installing dependencies..."
pip install -r requirements.txt
EOF
There is a test command which is explicit about checking files and directories:
test -d "/flask-ci-cd" && eval $then_commands || eval $else_commands
Depending on the AWS instance I'd expect "test" to be available. I'd recommend putting the commands in variables. (e.g. eval $then_commands)

Gitlab runner CI/CD do not checkout and pull last commit

I want to have a CI/CD with gitlab-runner and docker swarm. I have problem when i deploy the commit will not checkout or checkout without changes, I wonder to know if problem is gitlab or docker or docker build. my .gitlab-ci.yml, look like:
stages:
- build
- deploy
build_image:
stage: build
image: docker:git
services:
- docker:dind
variables:
DOCKER_DRIVER: overlay
script:
- docker login -u gitlab-ci-token -p $CI_BUILD_TOKEN registry.gitlab.com
- docker build -t registry.gitlab.com/insuretech1/backend:${CI_COMMIT_REF_SLUG} ./DockerFiles/Worker
- docker push registry.gitlab.com/insuretech1/backend:${CI_COMMIT_REF_SLUG}
only:
- branches
deploy_staging:
stage: deploy
image: rastasheep/ubuntu-sshd:latest
script:
# add the server as a known host
- ssh-keyscan 46.4.151.121 >> ~/.ssh/known_hosts
- chmod 600 ~/.ssh/known_hosts
# add ssh key stored in SSH_PRIVATE_KEY variable to the agent store
- eval $(ssh-agent -s)
- touch key.txt
- echo "$SSH_PRIVATE_KEY" >> key.txt
- chmod 600 key.txt
- ssh-add key.txt
# log into Docker registry
- ssh alireza#46.4.151.121 "docker login -u gitlab-ci-token -p $CI_BUILD_TOKEN registry.gitlab.com"
# stop container, remove image.
- ssh alireza#46.4.151.121 "docker stop dockergitlab_${CI_COMMIT_REF_SLUG}" || true
- ssh alireza#46.4.151.121 "docker rm dockergitlab_${CI_COMMIT_REF_SLUG}" || true
- ssh alireza#46.4.151.121 "docker rmi registry.gitlab.com/insuretech1/backend:${CI_COMMIT_REF_SLUG}" || true
# start new container
- ssh alireza#46.4.151.121 "docker run --name dockergitlab_${CI_COMMIT_REF_SLUG} -d registry.gitlab.com/insuretech1/backend:${CI_COMMIT_REF_SLUG}"
only:
- branches
except:
- master
and I also I put my pipeline log below, that might help to describe more:
$ eval "$CI_PRE_CLONE_SCRIPT"
00:02
Fetching changes with git depth set to 50...
Initialized empty Git repository in /builds/insuretech1/backend/.git/
Created fresh repository.
From https://gitlab.com/insuretech1/backend
* [new ref] refs/pipelines/124187268 -> refs/pipelines/124187268
* [new branch] develop -> origin/develop
Checking out 735209a2 as develop...
Skipping Git submodules setup
$ docker login -u gitlab-ci-token -p $CI_BUILD_TOKEN registry.gitlab.com
03:43
WARNING! Using --password via the CLI is insecure. Use --password-stdin.
WARNING! Your password will be stored unencrypted in /root/.docker/config.json.
Configure a credential helper to remove this warning. See
https://docs.docker.com/engine/reference/commandline/login/#credentials-store
Login Succeeded
the content of dockerfile which I use for my build
FROM debian:buster
MAINTAINER Alireza Rahmani Khalili "alirezarahmani#live.com"
ENV TERM xterm
RUN apt-get update --fix-missing && apt-get install -y --force-yes curl sudo vim
RUN apt-get install -y --force-yes wget apt-transport-https lsb-release ca-certificates
RUN wget -O /etc/apt/trusted.gpg.d/php.gpg https://packages.sury.org/php/apt.gpg
RUN echo "deb https://packages.sury.org/php/ $(lsb_release -sc) main" > /etc/apt/sources.list.d/php.list
RUN echo "deb http://ftp.uk.debian.org/debian buster-backports main" >> /etc/apt/sources.list
RUN wget -O /etc/apt/trusted.gpg.d/php.gpg https://packages.sury.org/php/apt.gpg
RUN echo "deb https://packages.sury.org/php/ $(lsb_release -sc) main" > /etc/apt/sources.list.d/php.list
RUN DEBIAN_FRONTEND="noninteractive" apt-get update && apt-get install -y --force-yes \
nginx \
php7.3 \
php7.3-cli \
php7.3-fpm \
php7.3-curl \
php7.3-json \
php7.3-mysql \
php7.3-sqlite \
php7.3-xml \
php7.3-intl \
php7.3-mbstring \
php7.3-xdebug \
php-memcached \
git \
openssh-server \
php7.3-gd \
zip \
php7.3-zip
# configure php-fpm
RUN sed -i 's/^;*clear_env = .*/clear_env = no/' /etc/php/7.3/fpm/pool.d/www.conf
RUN curl -sS https://getcomposer.org/installer | php && \
mv composer.phar /usr/local/bin/composer && chmod +x /usr/local/bin/composer
RUN mkdir /var/run/sshd
RUN echo 'root:root' | chpasswd
RUN sed -i 's/PermitRootLogin without-password/PermitRootLogin yes/' /etc/ssh/sshd_config
RUN sed 's#session\s*required\s*pam_loginuid.so#session optional pam_loginuid.so#g' -i /etc/pam.d/sshd
RUN echo "UseDNS no" >> /etc/ssh/sshd_config
RUN echo "KexAlgorithms diffie-hellman-group1-sha1" >> /etc/ssh/sshd_config
RUN echo "fastcgi_param PATH_TRANSLATED \$document_root\$fastcgi_script_name;" >> /etc/nginx/fastcgi_params
RUN mkdir /etc/nginx/ssl
RUN openssl ecparam -out /etc/nginx/ssl/nginx.key -name prime256v1 -genkey
RUN openssl req -new -batch -key /etc/nginx/ssl/nginx.key -out /etc/nginx/ssl/csr.pem
RUN openssl req -x509 -nodes -days 365 -key /etc/nginx/ssl/nginx.key -in /etc/nginx/ssl/csr.pem -out /etc/nginx/ssl/nginx.pem
RUN chmod 600 /etc/nginx/ssl/*
RUN ln -sf /dev/stdout /var/log/nginx/access.log
RUN ln -sf /dev/stderr /var/log/nginx/error.log
ADD docker-entrypoint.sh /usr/bin/docker-entrypoint
RUN chmod +x /usr/bin/docker-entrypoint
RUN sed -i 's/^user nginx;/user www-data;/' /etc/nginx/nginx.conf
RUN echo "apc.enable_cli=1" >> /etc/php/7.3/cli/php.ini
RUN echo "apc.shm_size=128M" >> /etc/php/7.3/fpm/conf.d/20-apcu.ini
RUN sed -i "s/\(max_execution_time *= *\).*/\1180/" /etc/php/7.3/fpm/php.ini
RUN sed -i "s/\(upload_max_filesize *= *\).*/\1100M/" /etc/php/7.3/fpm/php.ini
RUN sed -i "s/\(post_max_size *= *\).*/\1100M/" /etc/php/7.3/fpm/php.ini
RUN sed -i "s/\(^.*max_input_vars *= *\).*/max_input_vars = 10000/" /etc/php/7.3/fpm/php.ini
RUN sed -i "s/\(pm.max_children = 5\).*/\pm.max_children = 50/" /etc/php/7.3/fpm/pool.d/www.conf
RUN sed -i "s/\(pm.max_spare_servers = 3\).*/\pm.max_spare_servers = 10/" /etc/php/7.3/fpm/pool.d/www.conf
RUN echo "xdebug.default_enable=1" >> /etc/php/7.3/fpm/conf.d/20-xdebug.ini
RUN echo "xdebug.profiler_output_dir=/var/www/cachegrind/" >> /etc/php/7.3/fpm/conf.d/20-xdebug.ini
RUN echo "xdebug.profiler_enable_trigger=1" >> /etc/php/7.3/fpm/conf.d/20-xdebug.ini
RUN echo "xdebug.profiler_output_name= cachegrind.out" >> /etc/php/7.3/fpm/conf.d/20-xdebug.ini
RUN mkdir /root/.ssh/
ADD default.conf /etc/nginx/conf.d/default.conf
ADD default.conf /etc/nginx/sites-enabled/default
ADD default.conf /etc/nginx/sites-available/default
EXPOSE 22 443 80
WORKDIR /var/www/
ENTRYPOINT ["docker-entrypoint"]
CMD ["nginx", "-g", "daemon off;"]
and also content of my docker compose file which i use when I build in my ci/cd:
version: '3'
services:
worker:
image: registry.gitlab.com/insuretech1/backend:develop
ports:
- 0.0.0.0:80:80
depends_on:
- mysql
deploy:
mode: replicated
replicas: 3
# service resource management
resources:
# Hard limit - Docker does not allow to allocate more
limits:
cpus: '0.25'
memory: 512M
# Soft limit - Docker makes best effort to return to it
reservations:
cpus: '0.25'
memory: 256M
# service restart policy
restart_policy:
condition: on-failure
delay: 5s
max_attempts: 3
window: 120s
# service update configuration
update_config:
parallelism: 1
delay: 10s
failure_action: continue
monitor: 60s
max_failure_ratio: 0.3
volumes:
- /var/www/backend:/var/www
mysql:
image: mariadb:10.4
ports:
- 0.0.0.0:3306:3306
environment:
MYSQL_ROOT_PASSWORD: root
volumes:
- /opt/mysql_data:/var/lib/mysql
deploy:
placement:
constraints: [node.role == manager]
redis:
image: redis
deploy:
placement:
constraints: [node.role == manager]
the issue is I can not see my last changes of my commit in my server (I mean i should manually git pull to fetch last changes), is there anything wrong?
first of all in your Dockerfile you should copy content of directory into docker container. that will help you keep git changes with your container, for example:
COPY . /var/www/
and other problem is in your docker compose file you have:
volumes:
- /var/www/backend:/var/www
this will override changes that git made on your container and that is why you are not able to see git changes.

Trigger after merge successful?

I have this .gitlab-ci.yml script --- it triggers if I push directly on the master. How can I trigger this if there's a successful merge on the master branch?
before_script:
- apt-get update -qq
- apt-get install -qq git
# Setup SSH deploy keys
- 'which ssh-agent || ( apt-get install -qq openssh-client )'
- eval $(ssh-agent -s)
- echo "$SSH_PRIVATE_KEY" | tr -d '\r' | ssh-add - > /dev/null
- mkdir -p ~/.ssh
- '[[ -f /.dockerenv ]] && echo -e "Host *\n\tStrictHostKeyChecking no\n\n" > ~/.ssh/config'
deploy_staging:
type: deploy
environment:
name: staging
url: domain.com
script:
- ssh user#domain.com "cd /server/directory && git fetch && git pull origin master && exit"
only:
- master
Like the reference notes
test:
stage: test
script: ./test
only:
- merge_requests
see https://docs.gitlab.com/ee/ci/merge_request_pipelines/index.html

EC2 userdata codecommit clone fails

I'm starting an ec2 instance from the userdata and I need to clone a repo with my ansible playbooks but it fails to clone. See details below. Can anyone help me figure this out. when I ssh to the instance after bootstrap, then clone works but not while bootstrapping.
#!/usr/bin/env bash
set -x
exec > >(tee /var/log/user-data.log|logger -t user-data -s 2>/dev/console) 2>&1
cd /home/ec2-user
mkdir -p .ssh
ssh-keygen -b 2048 -t rsa -f /home/ec2-user/.ssh/codecommit -q -N ""
KEY_ID=`aws iam upload-ssh-public-key --user-name ${user_id} --ssh-public-key-body "$(cat /home/ec2-user/.ssh/codecommit.pub)" \
--query 'SSHPublicKey.SSHPublicKeyId' --output text`
echo -e "
Host git-codecommit.*.amazonaws.com
User $KEY_ID
IdentityFile /home/ec2-user/.ssh/codecommit
" >> /home/ec2-user/.ssh/config
ssh-keyscan -t rsa git-codecommit.us-east-2.amazonaws.com >> /home/ec2-user/.ssh/known_hosts
sudo chown -R ec2-user:ec2-user /home/ec2-user/.ssh
sudo chmod 700 /home/ec2-user/.ssh
sudo chmod 644 /home/ec2-user/.ssh/*
sudo chmod 600 /home/ec2-user/.ssh/codecommit*
eval "$(ssh-agent -s)"
export GIT_SSH_COMMAND="ssh -v -F /home/ec2-user/.ssh/config -o StrictHostKeyChecking=no"
export GIT_TRACE_PACKET=true
export GIT_TRACE=2
export GIT_CURL_VERBOSE=1
**sleep 60s**
git clone ssh://git-codecommit.us-east-2.amazonaws.com/v1/repos/ansible
Adding a sleep of 60 seconds before the git clone command did the trick. It seems like SSH Key uploads take a bit of time before becoming active.
sleep 60s
git clone ssh://git-codecommit.us-east-2.amazonaws.com/v1/repos/ansible
OR
for i in {1..30}; do
git clone ssh://git-codecommit.us-east-2.amazonaws.com/v1/repos/ansible
[ $? == 0 ] && break || sleep 2s; echo "keep trying ..."
done

GitLab Pipeline: Works in YML, Fails in Extracted SH

I followed the GitLab Docs to enable my project's CI to clone other private dependencies. Once it was working, I extracted from .gitlab-ci.yml:
before_script:
- 'which ssh-agent || ( apt-get update -y && apt-get install openssh-client -y )'
- eval $(ssh-agent -s)
- ssh-add <(echo "$SSH_PRIVATE_KEY")
- mkdir -p ~/.ssh
- '[[ -f /.dockerenv ]] && echo -e "Host *\n\tStrictHostKeyChecking no\n\n" > ~/.ssh/config'
into a separate shell script setup.sh as follows:
which ssh-agent || ( apt-get update -y && apt-get install openssh-client -y )
eval $(ssh-agent -s)
ssh-add <(echo "$SSH_PRIVATE_KEY")
mkdir -p ~/.ssh
[[ -f /.dockerenv ]] && echo -e "Host *\n\tStrictHostKeyChecking no\n\n" > ~/.ssh/config
leaving only:
before_script:
- chmod 700 ./setup.sh
- ./setup.sh
I then began getting:
Cloning into '/root/Repositories/DependentProject'...
Warning: Permanently added 'gitlab.com,52.167.219.168' (ECDSA) to the list of known hosts.
Permission denied (publickey).
fatal: Could not read from remote repository.
Please make sure you have the correct access rights
and the repository exists.
How do I replicate the original behavior in the extracted script?
When running ssh-add either use source or . so that the script runs within the same shell, in your case it would be:
before_script:
- chmod 700 ./setup.sh
- . ./setup.sh
or
before_script:
- chmod 700 ./setup.sh
- source ./setup.sh
For a better explanation as to why this needs to run in the same shell as the rest take a look at this answer to a related question here.

Resources