For some reason when I use an if statement it will not execute the configure for it. Here is my full travis.yml below.
travis.yml:
language: php
php:
- '5.6.32'
- '7.0.26'
- '7.1.12'
- '7.2.0'
os:
- windows
- linux
git:
depth: 1
matrix:
fast_finish: true
sudo: false
before_install:
- if [[ ${TRAVIS_PHP_VERSION:0:3} == "7.2.0" ]]; then git clone -b stable https://github.com/jedisct1/libsodium.git; fi
- if [[ ${TRAVIS_PHP_VERSION:0:3} == "7.2.0" ]]; then cd libsodium && sudo ./configure && sudo make check && sudo make install && cd ..; fi
install:
- if [[ ${TRAVIS_PHP_VERSION:0:3} == "7.2.0" ]]; then pecl install libsodium; fi
- if [[ ${TRAVIS_PHP_VERSION:0:3} == "7.2.0" ]]; then echo "extension=sodium.so" >> ~/.phpenv/versions/$(phpenv version-name)/etc/php.ini; fi
- travis_retry composer install --no-interaction
- wget -c -nc --retry-connrefused --tries=0 https://github.com/satooshi/php-coveralls/releases/download/v1.0.1/coveralls.phar
- chmod +x coveralls.phar
- php coveralls.phar --version
before_script:
- mkdir -p build/logs
- ls -al
script:
- ./vendor/bin/phpunit --coverage-clover build/logs/clover.xml
after_success:
- travis_retry php coveralls.phar -v
branches:
only: master
cache:
directories:
- vendor
- $HOME/.cache/composer
So for some reason
- if [[ ${TRAVIS_PHP_VERSION:0:3} == "7.2.0" ]]; then
export git clone -b stable https://github.com/jedisct1/libsodium.git;
fi
does not work and I got this solution from this
My goal is to execute certain lines on certain PHP version.
Is there anything I missed?
EDIT:
you extracted the first 3 characters of the TRAVIS_PHP_VERSIONand compared it with 5 characters.. of course that doesn't work. you can either try:
if [[ ${TRAVIS_PHP_VERSION:0:5} == "7.2.0" ]]
or
if [[ ${TRAVIS_PHP_VERSION:0:3} == "7.2" ]]
END EDIT
I had some problems running scripts inside the Travis config file, since i wanted blank lines and comments and Travis got confused with that. So in general i would recommend to do the scripting in a separate bash file.
The easiest way is to do all the bash scripting in a script file and use the script files in your travis.yml
before_install: ./travis-scripts/before_install.sh
Now you can write your scripts with the bash syntax and they work right away.
If you still want to write the scripts inside the travis file, try that (not everything worked for me everytime):
install: >
if [[ ${TRAVIS_PHP_VERSION:0:3} == "7.2" ]]; then pecl install libsodium; fi;
if [[ ${TRAVIS_PHP_VERSION:0:3} == "7.2" ]]; then echo "extension=sodium.so" >> ~/.phpenv/versions/$(phpenv version-name)/etc/php.ini; fi;
travis_retry composer install --no-interaction;
wget -c -nc --retry-connrefused --tries=0 https://github.com/satooshi/php-coveralls/releases/download/v1.0.1/coveralls.phar;
chmod +x coveralls.phar;
php coveralls.phar --version;
Or leave it as it is and only put the if statements in quotes.
Related
I'm trying to deploy my flask application to AWS EC2 instance using gitlab ci runner.
.gitlab.ci.yml
stages:
- test
- deploy
test_app:
image: python:latest
stage: test
before_script:
- python -V
- pip install virtualenv
- virtualenv env
- source env/bin/activate
- pip install flask
script:
- cd flask-ci-cd
- python test.py
prod-deploy:
stage: deploy
only:
- master # Run this job only on changes for stage branch
before_script:
- mkdir -p ~/.ssh
- echo -e "$RSA_KEY" > ~/.ssh/id_rsa
- chmod 600 ~/.ssh/id_rsa
- '[[ -f /.dockerenv ]] && echo -e "Host *\n\tStrictHostKeyChecking no\n\n" > ~/.ssh/config'
script:
- bash .gitlab-deploy-prod.sh
environment:
name: deploy
.gitlab-deploy-prod.sh
#!/bin/bash
# Get servers list
set -f
# access server terminal
shell="ssh -o StrictHostKeyChecking=no ${SERVER_URL}"
git_token=$DEPLOY_TOKEN
echo "Deploy project on server ${SERVER_URL}"
if [ ${shell} -d "/flask-ci-cd" ] # check if directory exists
then
eval "${shell} cd flask-ci-cd && git clone https://sbhusal123:${git_token}#gitlab.com/sbhusal123/flask-ci-cd.git master && cd flask-ci-cd"
else
eval "${shell} git pull https://sbhusal123:${git_token}#gitlab.com/sbhusal123/flask-ci-cd.git master && cd flask-ci-cd && cd flask-ci-cd"
fi
Error: .gitlab-deploy-prod.sh: line 7: -o: command not found
How can i check if directory existing??
What i've tried.
#!/bin/bash
# Get servers list
set -f
# access server terminal
shell="ssh -o StrictHostKeyChecking=no ${SERVER_URL}"
git_token=$DEPLOY_TOKEN
eval "${shell}" # i thought gitlab would provide me with shell access
echo "Deploy project on server ${SERVER_URL}"
if [-d "/flask-ci-cd" ] # check if directory exists
then
eval "cd flask-ci-cd && git clone https://sbhusal123:${git_token}#gitlab.com/sbhusal123/flask-ci-cd.git master && cd flask-ci-cd"
else
eval "git pull https://sbhusal123:${git_token}#gitlab.com/sbhusal123/flask-ci-cd.git master && cd flask-ci-cd && cd flask-ci-cd"
fi
I've tried to log into the ssh shell before executing the scripts inside if else. But it doesn't works the way intended.
Your script has some errors.
Do not use eval. No, eval does not work that way. eval is evil
When storing a command to a variable, do not use normal variables. Use bash arrays instead to preserve "words".
Commands passed via ssh are double escaped. I would advise to prefer to use here documents, they're simpler to get the quoting right. Note the difference in expansion when the here document delimiter is quoted or not.
i thought gitlab would provide me with shell access No, without open standard input the remote shell will just terminate, as it will read EOF from input. No, it doesn't work that way.
Instead of doing many remote connection, just transfer the execution to remote side once and do all the work there.
Take your time and research how quoting and word splitting works in shell.
git_token=$DEPLOY_TOKEN No, set variables are not exported to remote shell. Either pass them manually or expand them before calling the remote side. (Or you could also use ssh -o SendEnv=git_token and configure remote ssh with AcceptEnv=git_token I think, never tried it).
Read documentation for the utilities you use.
No, git clone doesn't take branch name after url. You can specify branch with --branch or -b option. After url it takes directory name. See git clone --help. Same for git pull.
How can i check if directory existing??
Use bash arrays to store the command. Check if the directory exists just by executing the test command on the remote side.
shell=(ssh -o StrictHostKeyChecking=no "${SERVER_URL}")
if "${shell[#]}" [ -d "/flask-ci-cd" ]; then
...
In case of directory name with spaces I would go with:
if "${shell[#]}" sh <<'EOF'
[ -d "/directory with spaces" ]
EOF
then
Pass set -x to sh to see what's happening also on the remote side.
For your script, try rather to move the execution to remote side - there is little logic in making 3 separate connections. I say just
echo "Deploy project on server ${SERVER_URL}"
ssh -o StrictHostKeyChecking=no "${SERVER_URL}" bash <<EOF
if [ ! -d /flask-ci-cd ]; then
# Note: git_token is expanded on host side
git clone https://sbhusal123:${git_token}#gitlab.com/sbhusal123/flask-ci-cd.git /flask-ci-cd
fi
cd /flask-ci-cd
git pull
EOF
But instead of getting the quoting in some cases right, use declare -p and declare -f to transfer properly quoted stuff to remote side. That way you do not need case about proper quoting - it will work naturally:
echo "Deploy project on server ${SERVER_URL}"
work() {
if [ ! -d /flask-ci-cd ]; then
# Note: git_token is expanded on host side
git clone https://sbhusal123:"${git_token}"#gitlab.com/sbhusal123/flask-ci-cd.git /flask-ci-cd
fi
cd /flask-ci-cd
git pull
{
ssh -o StrictHostKeyChecking=no "${SERVER_URL}" bash <<EOF
$(declare -p git_token) # transfer variables you need
$(declare -f work) # transfer function you need
work # call the function.
EOF
Updated answer for future reads.
.gitlab-ci.yml
stages:
- test
- deploy
test_app:
image: python:latest
stage: test
before_script:
- python -V
- pip install virtualenv
- virtualenv env
- source env/bin/activate
- pip install flask
script:
- cd flask-ci-cd
- python test.py
prod-deploy:
stage: deploy
only:
- master
before_script:
- mkdir -p ~/.ssh
- echo -e "$RSA_KEY" > ~/.ssh/id_rsa
- chmod 600 ~/.ssh/id_rsa
- '[[ -f /.dockerenv ]] && echo -e "Host *\n\tStrictHostKeyChecking no\n\n" > ~/.ssh/config'
script:
- bash .gitlab-deploy-prod.sh
environment:
name: deploy
.gitlab-deploy-prod.sh
#!/bin/bash
# Get servers list
set -f
shell=(ssh -o StrictHostKeyChecking=no "${SERVER_URL}")
git_token=$DEPLOY_TOKEN
echo "Deploy project on server ${SERVER_URL}"
ssh -o StrictHostKeyChecking=no "${SERVER_URL}" bash <<EOF
if [ ! -d flask-ci-cd ]; then
echo "\n Cloning into remote repo..."
git clone https://sbhusal123:${git_token}#gitlab.com/sbhusal123/flask-ci-cd.git
# Create and activate virtualenv
echo "\n Creating virtual env"
python3 -m venv env
else
echo "Pulling remote repo origin..."
cd flask-ci-cd
git pull
cd ..
fi
# Activate virtual env
echo "\n Activating virtual env..."
source env/bin/activate
# Install packages
cd flask-ci-cd/
echo "\n Installing dependencies..."
pip install -r requirements.txt
EOF
There is a test command which is explicit about checking files and directories:
test -d "/flask-ci-cd" && eval $then_commands || eval $else_commands
Depending on the AWS instance I'd expect "test" to be available. I'd recommend putting the commands in variables. (e.g. eval $then_commands)
I am trying to create a script I can provide to those who will use an academic game I am making. I am trying to do the following:
Verify that Apache2 is installed and Running
If Apache2 is installed, move a folder containing the website files to /var/www/html while backing up apache's original index.html
The code is as follows:
#!/bin/sh
acm=[]
cnn=[]
gnu=[]
ieee=[]
if [pgrep -x "apache2" > /dev/null]; then
echo "Apache 2 Installed and Running!"
if [ -d "$HOME/acmDL" ]; then
sudo mkdir /var/www/bak
sudo mv /var/www/html/index.html /var/www/bak
sudo mv acmDL /var/www/html/
cd /var/www/html/
sudo mv acmDL/* .
exit
elif [ -d "$HOME/cnnDL" ]; then
sudo mkdir /var/www/bak
sudo mv /var/www/html/index.html /var/www/bak
sudo mv cnnDL /var/www/html/
cd /var/www/html/
sudo mv cnnDL/* .
exit
elif [ -d "$HOME/gnuDL" ]; then
sudo mkdir /var/www/bak
sudo mv /var/www/html/index.html /var/www/bak
sudo mv gnuDL /var/www/html/
cd /var/www/html/
sudo mv gnuDL/* .
exit
elif [ -d "$HOME/ieeeDL" ]; then
sudo mkdir /var/www/bak
sudo mv /var/www/html/index.html /var/www/bak
sudo mv ieeeDL /var/www/html/
cd /var/www/html/
sudo mv ieeeDL/* .
exit
else
echo "Provided websites not found... Are you using a custom website?"
fi
else
echo "Please check apache2... It may not have installed correctly"
fi
The error I keep getting is syntax error near unexpected token `elif' on line 15.
As you can see, I even tried moving the boolean expression [ - d "$HOME/site" ] to their own variables, but then the error becomes : -d: command not found and the error on line 15.
Is what I am trying to do impossible, or am I missing something undocumented and yet completely obvious (like a handful of my previous posts)?
This is being run on a minimal installation of Ubuntu 18 on a Virtual Machine. The site directories are shared by Filezilla. Script written in Notepad++ on Windows 7 x64.
First of all, can you rewrite it like this?
Please tell me the execution result.
This is wrong.
if [pgrep -x "apache2" > /dev/null]; then
This is correct.
pgrep -x "apache2" > /dev/null
if [ $? -eq 0 ]; then
I am trying to run a custom egg through the Pterodactyl panel however, I get the error "/entrypoint.sh: line 30: syntax error: unexpected end of file"
My Docker image is as followed;
FROM ubuntu:18.04
MAINTAINER Amelia, <me#amelia.fun>
RUN apt-get update -y
RUN DEBIAN_FRONTEND=noninteractive apt-get install -y dos2unix curl gnupg2 git-core zlib1g-dev libssl-dev libreadline-dev libyaml-dev libsqlite3-dev sqlite3 libxml2-dev libxslt1-dev libcurl4-openssl-dev libffi-dev yarn build-essential gpg-agent zip unzip software-properties-common git default-jre python3-pip python-minimal python-pip ffmpeg libopus-dev libsodium-dev libpython2.7 libpython2.7-dev wget php7.2 php7.2-common php7.2-cli php7.2-fpm
RUN curl -sL https://deb.nodesource.com/setup_10.x -o nodesource_setup.sh
RUN bash nodesource_setup.sh
RUN apt-get install -y nodejs
RUN rm -rf nodesource_setup.sh
RUN adduser -D -h /home/container container
USER container
ENV USER=container HOME=/home/container
WORKDIR /home/container
COPY ./entrypoint.sh /entrypoint.sh
CMD ["/bin/bash", "/entrypoint.sh"]
and my entrypoint.sh is as followed;
#!/bin/bash
cd /home/container
MODIFIED_STARTUP=`eval echo $(echo ${STARTUP_PARAMETERS} | sed -e 's/{{/${/g' -e 's/}}/}/g')`
rm -rf *
git clone ${REPO_PARAMETERS}
cd */
if grep -q 'Java' AppType
then
${STARTUP_PARAMETERS}
if grep -q 'PHP' AppType
then
${STARTUP_PARAMETERS}
elif grep -q 'Python2' AppType
then
[ -f "requirements.txt" ] && pip2 install -r requirements.txt ${STARTUP_PARAMETERS} || ${STARTUP_PARAMETERS}
elif grep -q 'Python3' AppType
then
[ -f "requirements.txt" ] && pip3 install -r requirements.txt ${STARTUP_PARAMETERS} || ${STARTUP_PARAMETERS}
elif grep -q 'NodeJS' AppType
then
npm install
${STARTUP_PARAMETERS}
else
echo "Application not supported"
fi
echo "${MODIFIED_STARTUP}"
the Bash file is nowhere near 30 lines long so I'm not really sure.
The guide I used can also be found here
The immediate problem is that you have two if statements, but only one of them is closed with fi; it looks to me like the second one should be elif. But there are a number of other things that look like bad ideas to me:
cd commands in scripts should (almost) always have error tests -- for example, if cd /home/container fails for some reason, the rest of the script (including rm -rf *) will run in an unexpected location. Now, a self-destroying Docker environment may not be as big a deal as a self-destroying real system, but it's still not a good thing. I'd use something like this instead:
cd /home/container || {
echo "Error -- can't move to /home/container, something rotten in Denmark." >&2
exit 1
}
A similar comment applies to cd */.
The next line, that sets MODIFIED_STARTUP, is a mishmash of bad ideas. I'm not familiar with what's going to be in $STARTUP_PARAMETERS, but in general: Use $( ) instead of backticks (and not a weird mix of both). echo $(somecommand) is pretty much a no-op, just run the command directly. Also, variable references (and similar expansions like $( )) should almost always be in double-quotes (exception: on the right side of an assignment). And eval is generally dangerous, and should be avoided if possible. I you give me an example of what $STARTUP_PARAMETERS looks like, I could probably give a cleaned-up version of this.
The big if ... elif... etc has several conditions that do the same thing, e.g.
elif grep -q 'Python2' AppType
then
[ -f "requirements.txt" ] && pip2 install -r requirements.txt ${STARTUP_PARAMETERS} || ${STARTUP_PARAMETERS}
elif grep -q 'Python3' AppType
then
[ -f "requirements.txt" ] && pip3 install -r requirements.txt ${STARTUP_PARAMETERS} || ${STARTUP_PARAMETERS}
On the DRY principle (Don't Repeat Yourself), it'd be better to have a single test for all equivalent situations, like this:
elif grep -q 'Python2' AppType || grep -q 'Python3' AppType
then
[ -f "requirements.txt" ] && pip2 install -r requirements.txt ${STARTUP_PARAMETERS} || ${STARTUP_PARAMETERS}
or even:
elif grep -q 'Python[23]' AppType
then
[ -f "requirements.txt" ] && pip2 install -r requirements.txt ${STARTUP_PARAMETERS} || ${STARTUP_PARAMETERS}
BTW, the use of ${STARTUP_PARAMETERS} without quotes is setting off warning bells for me here, but may be inevitable -- again, I don't know its format. And the && ... || construction isn't always a safe replacement for if then else fi, since it can run both branches. In this script, if requirements.txt exists but the pip2 install command fails, it'll go ahead and run ${STARTUP_PARAMETERS} as well. Is that intentional? If not, I'd use a proper if statement instead.
I followed the GitLab Docs to enable my project's CI to clone other private dependencies. Once it was working, I extracted from .gitlab-ci.yml:
before_script:
- 'which ssh-agent || ( apt-get update -y && apt-get install openssh-client -y )'
- eval $(ssh-agent -s)
- ssh-add <(echo "$SSH_PRIVATE_KEY")
- mkdir -p ~/.ssh
- '[[ -f /.dockerenv ]] && echo -e "Host *\n\tStrictHostKeyChecking no\n\n" > ~/.ssh/config'
into a separate shell script setup.sh as follows:
which ssh-agent || ( apt-get update -y && apt-get install openssh-client -y )
eval $(ssh-agent -s)
ssh-add <(echo "$SSH_PRIVATE_KEY")
mkdir -p ~/.ssh
[[ -f /.dockerenv ]] && echo -e "Host *\n\tStrictHostKeyChecking no\n\n" > ~/.ssh/config
leaving only:
before_script:
- chmod 700 ./setup.sh
- ./setup.sh
I then began getting:
Cloning into '/root/Repositories/DependentProject'...
Warning: Permanently added 'gitlab.com,52.167.219.168' (ECDSA) to the list of known hosts.
Permission denied (publickey).
fatal: Could not read from remote repository.
Please make sure you have the correct access rights
and the repository exists.
How do I replicate the original behavior in the extracted script?
When running ssh-add either use source or . so that the script runs within the same shell, in your case it would be:
before_script:
- chmod 700 ./setup.sh
- . ./setup.sh
or
before_script:
- chmod 700 ./setup.sh
- source ./setup.sh
For a better explanation as to why this needs to run in the same shell as the rest take a look at this answer to a related question here.
I added another condition to my if, elseif condition for my bash shell script. I am new to shell scripting, can you guys review my code if my syntax for if conditions are right especially the "fi" implementation. Much appreciated.
if [ -f /etc/system-release ] && grep Amazon /etc/system-release > /dev/null; then
cd /tmp
sudo yum install -y https://s3.amazonaws.com/ec2-downloads-windows/SSMAgent/latest/linux_amd64/amazon-ssm-agent.rpm
else
# we're either RedHat or Ubuntu
DISTRIBUTOR=`lsb_release -is`
DISTRIBUTOR2=`lsb_release -cs`
if [ "trusty" == $DISTRIBUTOR2 ]; then
cd /tmp
wget https://s3.amazonaws.com/ec2-downloads-windows/SSMAgent/latest/debian_amd64/amazon-ssm-agent.deb
sudo dpkg -i amazon-ssm-agent.deb
sudo start amazon-ssm-agent
elif [ "RedHatEnterpriseServer" == $DISTRIBUTOR ]; then
cd /tmp
sudo yum install -y https://s3.amazonaws.com/ec2-downloads-windows/SSMAgent/latest/linux_amd64/amazon-ssm-agent.rpm
elif [ "xenial" == $DISTRIBUTOR2 ]; then
cd /tmp
wget https://s3.amazonaws.com/ec2-downloads-windows/SSMAgent/latest/debian_amd64/amazon-ssm-agent.deb
sudo dpkg -i amazon-ssm-agent.deb
sudo systemctl enable amazon-ssm-agent
fi
fi
sleep 10
Looks basically ok, but https://www.shellcheck.net/ has a couple of comments that you should probably address.