Setup Robot Framework pipeline with GitLab CI / CD - continuous-integration

So i have written my automated Robot Framework tests and they are in a GitLab repo. I want to run these automatically once a day.
Is this possible?
Do I need a .gitlab-ci.yml file for it? (if yes what do I put in it?)

Yes you can totally run the robot tests in gitlab ci.
so answer
Yes its very much possible , infact that is how you execute pipeline tests . You just need to build a Dockerfile that has the things you need to execute the framework inside docker. Here's the sample dockerfile. I would suggest you wrap the .robot script to run from bash script (like robot -d *.robot).
FROM ubuntu:18.04
ENV LANG=C.UTF-8 LC_ALL=C.UTF-8
RUN apt-get update --fix-missing && \
apt-get install -y python3-setuptools wget git bzip2 ca-certificates curl bash chromium-browser chromium-chromedriver firefox python3.8 python3-pip nano && \
apt-get clean && \
rm -rf /var/lib/apt/lists/*
RUN wget https://github.com/mozilla/geckodriver/releases/download/v0.27.0/geckodriver-v0.27.0-linux64.tar.gz
RUN tar xvf geckodriver*
RUN chmod +x geckodriver
RUN mv geckodriver /usr/bin
RUN update-alternatives --install /usr/bin/python3 python3 /usr/bin/python3.6 1
RUN update-alternatives --install /usr/bin/python3 python3 /usr/bin/python3.8 2
RUN pip3 install --upgrade pip
RUN ln -s /usr/bin/python3 /usr/bin/python
RUN ln -s /usr/bin/pip3 /usr/bin/pip
RUN pip install rpaframework
COPY . /usr/src/
ADD robot.sh /usr/local/bin/robot.sh
RUN chmod +x /usr/local/bin/robot.sh
WORKDIR /usr/src
Now you need .gitlab-ci.yml in your repository to have a content like this.
stages:
- build
- run
variables:
ARTIFACT_REPORT_PATH: "${CI_PROJECT_DIR}/reports"
build_image:
stage: build
variables:
DOCKER_IMAGE_TAG: ${CI_REGISTRY_IMAGE}:${CI_COMMIT_REF_NAME}
script:
- docker build -t ${DOCKER_IMAGE_TAG} .
after_script:
- docker push ${DOCKER_IMAGE_TAG}
- docker logout
robot_tests:
stage: run
variables:
DOCKER_IMAGE_TAG: ${CI_REGISTRY_IMAGE}:${CI_COMMIT_TAG}
image: ${DOCKER_IMAGE_TAG}
script:
- robot-test.sh
artifacts:
paths:
- $ARTIFACT_REPORT_PATH
when: always
That should be it and once the job finishes you would see the output in the job at the path location in the repository.

Related

gem command not found when install rbenv in debian

I'm creating a Dockerfile to run truffleruby. I'm getting an error when trying to install bundler and foreman. The error is /bin/sh: 1: gem: not found
Dockerfile
FROM debian:buster-slim
# Install packages for building ruby
RUN apt update -y && apt install -y git curl libssl-dev libpq-dev libreadline-dev zlib1g-dev \
autoconf bison build-essential libyaml-dev \
libreadline-dev libncurses5-dev libffi-dev libgdbm-dev
RUN apt clean
# Install rbenv and ruby-build
RUN git clone https://github.com/sstephenson/rbenv.git /root/.rbenv
RUN git clone https://github.com/sstephenson/ruby-build.git /root/.rbenv/plugins/ruby-build
RUN /root/.rbenv/plugins/ruby-build/install.sh
ENV PATH /root/.rbenv/bin:$PATH
RUN echo 'eval "$(rbenv init -)"' >> /etc/profile.d/rbenv.sh # or /etc/profile
RUN echo 'eval "$(rbenv init -)"' >> .bashrc
RUN . ~/.bashrc
RUN rbenv install truffleruby-20.3.0
RUN rbenv global truffleruby-20.3.0
RUN rbenv rehash
ENV BUNDLER_VERSION=2.2.4 NODE_ENV=production RAILS_ENV=production RAILS_SERVE_STATIC_FILES=true RAILS_LOG_TO_STDOUT=true PORT=3000
ENV CONFIGURE_OPTS --disable-install-doc
RUN apt-get install -y curl -sL https://deb.nodesource.com/setup_14.x | bash - && \
apt-get update && apt-get install -y nodejs && \
apt-get clean
RUN rbenv versions
RUN gem install bundler:2.2.4 foreman
RUN mkdir /app
WORKDIR /app
COPY Gemfile Gemfile.lock ./
RUN bundle config set --local deployment 'true'
RUN bundle config set --local without 'development test'
RUN bundle install
COPY . .
EXPOSE 3000
CMD ["foreman", "start"]
tail of build
Removing intermediate container 1a445fde7fc0
---> 43c3d72b7eb6
Step 17/27 : RUN rbenv versions
---> Running in feb5bb9361cc
* truffleruby-20.3.0 (set by /root/.rbenv/version)
Removing intermediate container feb5bb9361cc
---> c7d1a5826af5
Step 18/27 : RUN gem install bundler:2.2.4 foreman
---> Running in 998461afc89c
/bin/sh: 1: gem: not found
The command '/bin/sh -c gem install bundler:2.2.4 foreman' returned a non-zero code: 127
You don't generally use version managers like rbenv in Docker. There are a couple of reasons for this. One is that an image usually only contains a single application and its single runtime, so you'd never have more than one Ruby in an image and therefore there's no need to switch. A second is that most common paths of running containers (including docker run and the Dockerfile RUN directive) don't look at shell dotfiles like .bashrc or /etc/profile, so the version manager setup will never get run.
TruffleRuby is distributed (among other ways) as a standalone tar file so you can just install that in your Dockerfile. I'd make the Dockerfile look roughly like:
FROM debian:buster-slim
# Install the specific dependency packages TruffleRuby recommends
# (build-essential is much larger but might actually be necessary)
RUN apt-get update \
&& DEBIAN_FRONTEND=noninteractive \
apt-get install --no-install-recommends --assume-yes \
curl \
gcc \
libssl-dev \
libz-dev \
make
# Download and unpack TruffleRuby
ARG TRUFFLERUBY_VERSION=20.3.0
ENV PATH /opt/truffleruby-$TRUFFLERUBY_VERSION-linux-amd64/bin:$PATH
RUN cd /opt \
&& curl -L https://github.com/oracle/truffleruby/releases/download/vm-$TRUFFLERUBY_VERSION/truffleruby-$TRUFFLERUBY_VERSION-linux-amd64.tar.gz | tar xz \
&& /opt/truffleruby-$TRUFFLERUBY_VERSION-linux-amd64/lib/truffle/post_install_hook.sh
# Now build and install your application
RUN gem install bundler:2.2.4
WORKDIR /app
COPY Gemfile Gemfile.lock ./
RUN bundle config set --local deployment 'true'
RUN bundle config set --local without 'development test'
RUN bundle install
COPY . .
ENTRYPOINT ["bundle", "exec"]
EXPOSE 3000
CMD ["rails", "start"]
You can reasonably split this into two separate Dockerfiles. End the first one before the "build and install your application" comment, and build it with docker build -t myname/truffleruby:20.3.0 -f Dockerfile.truffleruby .. Then the second one can begin with FROM myname/truffleruby:20.3.0 in the same way as the standard Docker Hub ruby image.
Does the same work with CRuby?
I'd suspect RUN doesn't read shell files, so PATH needs to be modified.
RUN git clone https://github.com/sstephenson/rbenv.git /root/.rbenv
...
RUN /root/.rbenv/plugins/ruby-build/install.sh
ENV PATH /root/.rbenv/bin:$PATH
seems a bit weird, is rbenv cloned and installed in the same place?
I would skip rbenv in Docker, and instead just:
RUN ruby-build truffleruby ~/.rubies/truffleruby
ENV PATH $HOME/.rubies/truffleruby/bin:$PATH

mix release on Amazon Linux 2

Goal: to be able to run a phoenix, mix release in an ec2 instance (this machine: https://hub.docker.com/_/amazonlinux/)
Problem: Running my release produces the following error:
/my_app/erts-11.0.3/bin/beam.smp: /lib64/libtinfo.so.6: no version information available (required by /my_app/erts-11.0.3/bin/beam.smp)
2020-09-08 13:17:33.469328
args: [load_failed,"Failed to load NIF library /my_app/lib/crypto-4.7/priv/lib/crypto: 'libcrypto.so.1.1: cannot open shared object file: No such file or directory'","OpenSSL might not format: label: be installed on this system.\n"]
"Unable to load crypto library. Failed with error:~n\"~p, ~s\"~n~s"
{error_logger,error_msg}
but I have openssl installed in each scenario (OpenSSL 1.0.2k-fips 26 Jan 2017).
Setup:
I create a new phoenix project with:
yes | mix phx.new my_app --no-webpack --no-ecto --no-dashboard --no-gettext
cd my_app
and uncomment the config :my_app, MyAppWeb.Endpoint, server: true line in config/prod.secret.exs to start the server when running the app.
I create the following Dockerfile to build my project:
FROM debian:buster
# Install essential build packages
RUN apt-get update
RUN apt-get install -y wget git locales curl gnupg-agent
# Set locale
RUN sed -i -e 's/# en_US.UTF-8 UTF-8/en_US.UTF-8 UTF-8/' /etc/locale.gen && \
dpkg-reconfigure --frontend=noninteractive locales && \
update-locale LANG=en_US.UTF-8
ENV LANG en_US.UTF-8
ENV HOME=/opt/app
WORKDIR /opt/app
# Install erlang and elixir
RUN wget https://packages.erlang-solutions.com/erlang-solutions_2.0_all.deb
RUN dpkg -i erlang-solutions_2.0_all.deb
RUN apt-get update
RUN apt-get install -y esl-erlang
RUN apt-get install -y elixir
# Install hex and rebar
RUN mix do local.hex --force, local.rebar --force
# Install phoenix
RUN mix archive.install hex phx_new 1.5.4 --force
COPY mix.exs mix.lock ./
COPY config config
COPY priv priv
COPY lib lib
RUN mix deps.get
ENV SECRET_KEY_BASE='secretExampleQrzdplBPdbHHhr2bpELjiGVGVqmjvFl2JEXdkyla8l6+b2CCcvs'
ENV MIX_ENV=prod
RUN mix phx.digest
RUN mix compile
RUN mix release
CMD ["_build/prod/rel/my_app/bin/my_app", "start"]
and build the image with:
docker build . -t my_app
We can check that everything is running as expected with:
docker run -p 4000:4000 -i my_app:latest
and visiting localhost:4000.
I copy the _build/prod/rel/my_app directory from the built docker container (as this is all I'll be transferring across to my ec2 instance).
# list all containers
docker container ls -a
# locate the container with image tag: my_app:latest. It should look like:
# f9c46df97e55 my_app:latest "_build/prod/rel/my_…"
# note the container_id, and copy across the build release
docker cp f9c46df97e55:/opt/app/_build/prod/rel/my_app .
We create an instance.Dockerfile to run the commands of our ec2 instance:
FROM amazonlinux:latest
COPY my_app my_app
CMD ["my_app/bin/my_app", "start"]
and run it:
docker build . -f instance.Dockerfile -t my_app_instance && docker run -p 4000:4000 -i my_app_instance:latest
This fails to run, with the error:
[load_failed,"Failed to 2020-09-08 13:27:49.980715
args: load NIF library /my_app/lib/crypto-4.7/priv/lib/crypt format: label: 2020-09-08 13:27:49.981847 supervisor_report o: 'libcrypto.so.1.1: cannot open shared object file: No such file or directory'","OpenSSL might not be installed on this system.\n"]
"Unable to load crypto library. Failed with error:~n\"~p, ~s\"~n~s"
{error_logger,error_msg}
Note:
I am able to replicate the error on a debian:buster machine with the above docker build ... && docker run ... command, but with this instance.Dockerfile:
FROM debian:buster
RUN apt-get update
RUN apt-get install -y locales
# Set locale
RUN sed -i -e 's/# en_US.UTF-8 UTF-8/en_US.UTF-8 UTF-8/' /etc/locale.gen && \
dpkg-reconfigure --frontend=noninteractive locales && \
update-locale LANG=en_US.UTF-8
ENV LANG en_US.UTF-8
COPY my_app my_app
CMD ["my_app/bin/my_app", "start"]
and fix the error by changing: RUN apt-get install -y locales to RUN apt-get install -y locales curl.
I have tried yum install curl and yum install openssl on the amazonlinux:latest machine, but still experience the same error.
Question:
Where should I look to make progress on this? It seems to be an erlang/otp requirement issue, but the above is hardly a sscce, so difficult to raise.
I have struggled with finding what crypto or openssl library the apt-get curl package installs which causes the error to be fixed.
Any pointers to a particular forum to ask for help, or what to try next would be greatly appreciated.
Thanks in advance for the help.
Thanks to the suggestion to build it on CentOs from #VenkatakumarSrinivasan,
I managed to get it working on an amazonlinux machine with the following Dockerfiles.
Building the release:
FROM centos:7
RUN yum update -y
RUN yum clean -y all
RUN echo 'LC_ALL="en_US.UTF-8"' >> /etc/locale.conf
ENV LC_ALL="en_US.UTF-8"
RUN yum install -y epel-release
RUN yum install -y gcc gcc-c++ glibc-devel make ncurses-devel openssl-devel \
autoconf java-1.8.0-openjdk-devel git wget wxBase.x86_64
WORKDIR /opt
RUN wget http://packages.erlang-solutions.com/erlang-solutions-1.0-1.noarch.rpm
RUN rpm -Uvh erlang-solutions-1.0-1.noarch.rpm
RUN yum update -y
RUN yum install -y erlang
WORKDIR /opt/elixir
RUN git clone https://github.com/elixir-lang/elixir.git /opt/elixir
RUN make clean test
ENV PATH=/opt/elixir/bin:${PATH}
RUN mix do local.hex --force, local.rebar --force
RUN mix archive.install hex phx_new 1.5.4 --force
WORKDIR /opt/app
COPY mix.exs mix.lock ./
COPY config config
COPY priv priv
COPY lib lib
RUN mix deps.get
ENV SECRET_KEY_BASE='secretExampleQrzdplBPdbHHhr2bpELjiGVGVqmjvFl2JEXdkyla8l6+b2CCcvs'
ENV MIX_ENV=prod
RUN mix phx.digest
RUN mix compile
RUN mix release
CMD ["_build/prod/rel/my_app/bin/my_app", "start"]
Running the release:
FROM amazonlinux:latest
RUN yum -y update
ENV LANG="en_US.UTF-8"
ENV LC_ALL="en_US.UTF-8"
RUN ln -s /usr/lib64/libtinfo.so.{6,5}
COPY my_app my_app
CMD ["my_app/bin/my_app", "start"]
I'm not sure if this is a satisfying solution though, I'm open to a more elegant solution.

Why dotnet command is not found in dockerfile?

I tried to run a dotnet command located in a shell file which will be called by dockerfile during the docker build process.
Here is the dockerfile snippet:
FROM ubuntu:16.04
FROM microsoft/dotnet:2.2-sdk as build-env
# .net core
RUN apt-get update -y && apt-get install -y wget apt-transport-https
RUN wget -q https://packages.microsoft.com/config/ubuntu/16.04/packages-microsoft-prod.deb -O packages-microsoft-prod.deb && dpkg -i packages-microsoft-prod.deb
RUN apt-get update -y && apt-get install -y aspnetcore-runtime-2.2=2.2.1-1
# dotnet tool command
RUN apt-get update -y && apt-get install dotnet-sdk-2.2 -y
# for dot net tool #https://stackoverflow.com/questions/51977474/install-dotnet-core-tool-dockerfile
ENV PATH="${PATH}:/root/.dotnet/tools"
# Supervisor
RUN apt-get update -y && apt-get install -y supervisor && mkdir -p /etc/supervisor
# main script as defacult command when docker container runs
# Run the main sh script to run script in each xxx/*/db-migrate.sh.
CMD ["/xxx/main-migrate.sh"]
# Microservice files
ADD xxx /xxx
# install the xxx deploy tool
WORKDIR /xxx
RUN for d in /xxx/*/ ; do cd "$d"; if [ -f "./install.sh" ]; then sh ./install.sh; fi; done
In the install.sh, here is the code:
dotnet tool install -g xxx.DEPLOY --version [$(cat version)] --add-source /xxx/
When I run docker build -t xxx:v0 ., I get an error message saying:
./install.sh: 1: ./install.sh: dotnet: not found
I have added FROM microsoft/dotnet:2.2-sdk as build-env & RUN apt-get update -y && apt-get install dotnet-sdk-2.2 -y, but why Docker could not find the dotnet command during build?
How do I call the dotnet command located in the shell script file during the docker build process?
Thank you
FROM ubuntu:16.04
FROM microsoft/dotnet:2.2-sdk as build-env
In the above lines FROM ubuntu:16.04 will be totally ignored as there should be only one base image, so the last FROM will be considered as a base image which is FROM microsoft/dotnet:2.2-sdk not the ubuntu.
So if your base image is FROM microsoft/dotnet:2.2-sdk as build-env then why to bother to run these complex script to install dotnet?
You are good to go to check version of dotnet.
FROM microsoft/dotnet:2.2-sdk as build-env
RUN dotnet --version
output
Step 1/6 : FROM microsoft/dotnet:2.2-sdk as build-env
---> f13ac9d68148
Step 2/6 : RUN dotnet --version
---> Running in f1d34507c7f2
> 2.2.402
Removing intermediate container f1d34507c7f2
---> 7fde8596c331

How to Recreate Travis CI Environment inside GitLab CI

I'm attempting to migrate a project from Travis CI to GitLab CI. I believe that the bash scripts shouldn't need to change - aside from swapping a few env variables provided by default. But, I've been unable to recreate the environment inside the GitLab yaml file.
sudo: required
services:
- docker
env:
DOCKER_COMPOSE_VERSION: 1.21.1
before_install:
- sudo rm /usr/local/bin/docker-compose
- curl -L https://github.com/docker/compose/releases/download/${DOCKER_COMPOSE_VERSION}/docker-compose-`uname -s`-`uname -m` > docker-compose
- chmod +x docker-compose
- sudo mv docker-compose /usr/local/bin
script:
- bash test-ci.sh $TRAVIS_BRANCH
after_success:
- bash ./docker-push.sh
- bash ./docker-deploy-stage.sh
- bash ./docker-deploy-prod.sh
Here's my most recent failed attempt:
image: ubuntu:14.04
services:
- docker:dind
variables:
DOCKER_COMPOSE_VERSION: 1.21.1
before_script:
- apt-get update -qq && apt-get install -y -qq apt-transport-https ca-certificates curl software-properties-common unzip python3 python3-pip docker.io libcgroup1
- pip3 install awscli
- curl -L https://github.com/docker/compose/releases/download/${DOCKER_COMPOSE_VERSION}/docker-compose-`uname -s`-`uname -m` > docker-compose
- chmod +x docker-compose
- sudo mv docker-compose /usr/local/bin
stages:
- build
build:
stage: build
script:
- bash ./docker-push.sh
- bash ./docker-deploy-stage.sh
- bash ./docker-deploy-prod.sh
This is my first attempt to setup CI. Anyone know what I'm missing?

Running a cucumber test on firefox in a docker container

I'm trying to run my cucumber tests in a docker container using firefox. I'm getting this error when my code just tries to visit 'google.com'
Net::ReadTimeout (Net::ReadTimeout)
Here is my Dockerfile
FROM ruby:2.2.2
RUN mkdir /app
WORKDIR /app
RUN gem update
ADD Gemfile /app/Gemfile
ADD Gemfile.lock /app/Gemfile.lock
RUN bundle install
RUN apt-get update && apt-get install -y --fix-missing iceweasel xvfb unzip
ENV GECKODRIVER_VERSION v0.13.0
RUN echo $GECKODRIVER_VERSION
RUN mkdir -p /opt/geckodriver_folder
RUN wget -O /tmp/geckodriver_linux64.tar.gz https://github.com/mozilla/geckodriver/releases/download/$GECKODRIVER_VERSION/geckodriver-$GECKODRIVER_VERSION-linux64.tar.gz
RUN tar xf /tmp/geckodriver_linux64.tar.gz -C /opt/geckodriver_folder
RUN rm /tmp/geckodriver_linux64.tar.gz
RUN chmod +x /opt/geckodriver_folder/geckodriver
RUN ln -fs /opt/geckodriver_folder/geckodriver /usr/local/bin/geckodriver
ADD features /app/features
I've tried increasing my read_timeout to 120 with no effect.
When i bash into the container and run firefox it says "no display specified"
Any suggestions ?
I was having the same issue. This worked for me: https://github.com/juandelgado/docker-headless-firefox-cucumber/blob/master/Dockerfile
FROM ruby:2.4-slim
MAINTAINER Juan Delgado <juan#ustwo.com>
RUN mkdir /home/ustwo
WORKDIR /home/ustwo
RUN apt-get update && \
apt-get install -y xvfb build-essential libffi-dev wget firefox-esr && \
wget https://github.com/mozilla/geckodriver/releases/download/v0.17.0/geckodriver-v0.17.0-linux64.tar.gz && \
tar -zxvf geckodriver-v0.17.0-linux64.tar.gz && \
chmod +x geckodriver && \
mv geckodriver /usr/local/bin && \
rm geckodriver-v0.17.0-linux64.tar.gz
COPY Gemfile Gemfile
COPY Gemfile.lock Gemfile.lock
RUN bundle
RUN ["cucumber", "--version"]

Resources