VMSS Custom script not fully executing - bash

We are currently experimenting with new VMSS build agents for our devops environment, which requires some components for each build pipeline.
So to make sure don't need to add this to every build pipeline, we created a startup script which is executed every time the machine is created (standard custom script extension) with the following contents
#install kubectl
echo "Installing KubeCTL"
curl -LO "https://dl.k8s.io/release/$(curl -L -s https://dl.k8s.io/release/stable.txt)/bin/linux/amd64/kubectl"
sudo install -o root -g root -m 0755 kubectl /usr/local/bin/kubectl
echo "Installing DotNet SDK"
#install dotnet runtime (5) for dotnet tool install
sudo apt-get install -y dotnet-sdk-5.0
echo "Installing DotNet DotNet Runtime"
#install dotnet runtime (5) for FluentMigrator
sudo apt-get install -y dotnet-runtime-5.0
echo "Installing DotNet AzureCLI"
#install AzureCLI
curl -sL https://aka.ms/InstallAzureCLIDeb | sudo bash
echo "Installing Powershell"
#install powershell
# Update the list of packages
sudo apt-get update
# Install pre-requisite packages.
sudo apt-get install -y wget apt-transport-https software-properties-common
# Download the Microsoft repository GPG keys
wget -q "https://packages.microsoft.com/config/ubuntu/$(lsb_release -rs)/packages-microsoft-prod.deb"
# Register the Microsoft repository GPG keys
sudo dpkg -i packages-microsoft-prod.deb
# Update the list of packages after we added packages.microsoft.com
sudo apt-get update
# Install PowerShell
sudo apt-get install -y powershell
echo "Installing FluentMigrator"
#install fluent migrator
dotnet tool install -g FluentMigrator.DotNet.Cli
echo "Installing OpenJDK"
#install openJDK
sudo apt-get install -y openjdk-11-jre
Now, everything in this script executes and installs without any issues, the build pipeline runs correctly after a new agent is booted up.
However, during release time, we require fluentmigrator, which is not installed even though it is included in the script.
If we add the same install line dotnet tool install -g FluentMigrator.DotNet.Cli as a build step or as a step during release it gets installed correctly. To do this we run a custom bash with the command.
However, i would very much prefer to have this run within the boot-up of the machine instead of adding a custom bash script to 20 release pipelines. Anyone has any idea why the tool is not installing correctly within this script?

Related

Cannot locate a 64-bit Oracle Client library: "libclntsh.so: cannot open shared object file: No such file or directory"

I am trying to establish a connection from Azure pipelines to oracle using cx_oracle.
I'm using a Dockerfile:
FROM python:3.9-slim-buster
RUN apt-get update && apt-get install -y git && apt-get install telnet
RUN apt-get install -y gcc
RUN apt-get -y install wget
RUN apt -y install unzip
RUN wget https://download.oracle.com/otn_software/linux/instantclient/215000/instantclient-basic-linux.x64-21.5.0.0.0dbru.zip
RUN unzip instantclient-basic-linux.x64-21.5.0.0.0dbru.zip
RUN apt-get -y install libaio1
RUN sh -c "echo /instantclient_21_5 > /etc/ld.so.conf.d/oracle-instantclient.conf"
RUN ldconfig
RUN apt -y remove unzip
RUN rm instantclient-basic-linux.x64-21.5.0.0.0dbru.zip
CMD ["/bin/sh"]
Im doing automated tests using robotframework, i just updated to cx-Oracle 8.3.0. I still get the error
DatabaseError: DPI-1047:
Cannot locate a 64-bit Oracle Client library:
"libclntsh.so: cannot open shared object file:
No such file or directory".

Installing Oracle Client Tools - Docker, Roundhouse, Oracle

I am trying to get Roundhouse running from a Docker image where I have some Oracle scripts, but currently when I execute the rh command against my Docker image I get the following error:
A type could not be created from the object you passed. "roundhouse.databases.oracle.OracleDatabase, roundhouse.databases.oracle" resolves to null.
I think this a result of not having the Oracle Client tools configured correctly in my Docker Image(https://github.com/chucknorris/roundhouse/wiki/Oracle).
I have downloaded the rpm files from https://www.oracle.com/database/technologies/instant-client/linux-x86-64-downloads.html and I have managed to get the Oracle Client Tools installed but I carry on getting the above error so I assume that this is because I haven't configured the client tools correctly?
Can anyone advise what I am doing wrong?
FROM mcr.microsoft.com/dotnet/core/sdk:2.1
ENV PATH="$PATH:/root/.dotnet/tools"
ADD *.* /
RUN apt-get update && \
apt-get install sudo
#RUN apk --update add libaio bc net-tools
RUN sudo apt-get install unzip
RUN sudo apt-get install wget
RUN sudo apt-get install alien -y
RUN sudo alien -i oracle-instantclient*-basic-*.rpm
RUN sudo alien -i oracle-instantclient*-devel-*.rpm
RUN sudo alien -i oracle-instantclient*-sqlplus-*.rpm
RUN echo /usr/lib/oracle/19.1/client/lib > /etc/ld.so.conf.d/oracle.conf
RUN sudo ldconfig
RUN dotnet tool install --global dotnet-roundhouse --version 1.2.1
ENTRYPOINT [ "rh"]
I had the same error when I tried to use the MS repo.
This worked for me:
Install the oficial oracle instant client container as
oracle/instantclient:19. Instruction here.
On top of that image build this Dockerfile
from oracle/instantclient:19
RUN microdnf install git RUN git clone https://github.com/chucknorris/roundhouse.git
RUN microdnf install dotnet-sdk-2.1
RUN cd roundhouse && dotnet tool install --global dotnet-roundhouse --version 1.2.1
ENTRYPOINT [ "rh"]

Dockerfile for windows container-Java-maven-chrome

I am new to docker and I have installed docker for windows server 2016 and it is working fine. However, I am struggling to write a Dockerfile for windows. Can someone translate the below dockerfile to be able to run in windows?? I am trying to build an image that has java8, maven and chrome and chrome driver installed. However, windows based docker daemon has issues with apt-get, wget etc etc
Dockerfile
FROM openjdk
#Install Maven
RUN apt-get update && \apt-get install -y maven && \apt-get clean
# Install Chrome
RUN wget https://dl.google.com/linux/direct/google-chrome-stable_current_amd64.deb
RUN dpkg -i google-chrome-stable_current_amd64.deb; apt-get -fy install
# install chromedriver
#RUN apt-get install -yqq unzip
RUN wget -O /tmp/chromedriver.zip http://chromedriver.storage.googleapis.com/`curl -sS chromedriver.storage.googleapis.com/LATEST_RELEASE`/chromedriver_linux64.zip
RUN unzip /tmp/chromedriver.zip chromedriver -d /usr/local/bin/
copy ..

Why dotnet command is not found in dockerfile?

I tried to run a dotnet command located in a shell file which will be called by dockerfile during the docker build process.
Here is the dockerfile snippet:
FROM ubuntu:16.04
FROM microsoft/dotnet:2.2-sdk as build-env
# .net core
RUN apt-get update -y && apt-get install -y wget apt-transport-https
RUN wget -q https://packages.microsoft.com/config/ubuntu/16.04/packages-microsoft-prod.deb -O packages-microsoft-prod.deb && dpkg -i packages-microsoft-prod.deb
RUN apt-get update -y && apt-get install -y aspnetcore-runtime-2.2=2.2.1-1
# dotnet tool command
RUN apt-get update -y && apt-get install dotnet-sdk-2.2 -y
# for dot net tool #https://stackoverflow.com/questions/51977474/install-dotnet-core-tool-dockerfile
ENV PATH="${PATH}:/root/.dotnet/tools"
# Supervisor
RUN apt-get update -y && apt-get install -y supervisor && mkdir -p /etc/supervisor
# main script as defacult command when docker container runs
# Run the main sh script to run script in each xxx/*/db-migrate.sh.
CMD ["/xxx/main-migrate.sh"]
# Microservice files
ADD xxx /xxx
# install the xxx deploy tool
WORKDIR /xxx
RUN for d in /xxx/*/ ; do cd "$d"; if [ -f "./install.sh" ]; then sh ./install.sh; fi; done
In the install.sh, here is the code:
dotnet tool install -g xxx.DEPLOY --version [$(cat version)] --add-source /xxx/
When I run docker build -t xxx:v0 ., I get an error message saying:
./install.sh: 1: ./install.sh: dotnet: not found
I have added FROM microsoft/dotnet:2.2-sdk as build-env & RUN apt-get update -y && apt-get install dotnet-sdk-2.2 -y, but why Docker could not find the dotnet command during build?
How do I call the dotnet command located in the shell script file during the docker build process?
Thank you
FROM ubuntu:16.04
FROM microsoft/dotnet:2.2-sdk as build-env
In the above lines FROM ubuntu:16.04 will be totally ignored as there should be only one base image, so the last FROM will be considered as a base image which is FROM microsoft/dotnet:2.2-sdk not the ubuntu.
So if your base image is FROM microsoft/dotnet:2.2-sdk as build-env then why to bother to run these complex script to install dotnet?
You are good to go to check version of dotnet.
FROM microsoft/dotnet:2.2-sdk as build-env
RUN dotnet --version
output
Step 1/6 : FROM microsoft/dotnet:2.2-sdk as build-env
---> f13ac9d68148
Step 2/6 : RUN dotnet --version
---> Running in f1d34507c7f2
> 2.2.402
Removing intermediate container f1d34507c7f2
---> 7fde8596c331

Pipeline Shell Script Permission Issue on .NET Build Attempt

I am trying to build an ASP.NET5 application via Bluemix Pipeline using a shell script to configure a runtime that supports .NET builds with DNVM. When building the application we need to get dependencies from Mono 4.0 (such as kestrel) but the latest Mono available via apt-get is 3.2. I tried to resolve this by adding the Mono deb repository in /etc/apt/sources.list so that an apt-get update would fetch the latest Mono package but due to a permission error we are not allowed to alter sources.list nor add or alter any files in /etc/apt/sources.list.d/*.
For example, running:
sudo apt-key adv --keyserver hkp://keyserver.ubuntu.com:80 --recv-keys 3FA7E0328081BFF6A14DA29AA6A19B38D3D831EF
sudo echo "deb http://download.mono-project.com/repo/debian wheezy main" | sudo -i tee /etc/apt/sources.list.d/mono-xamarin.list
Will result in:
sudo: no tty present and no askpass program specified
Not using sudo will give a permission issue and I think we have exhausted all possible workarounds such as ssh -t -t and forth.
Does anyone have any suggestions on a workaround for this or an alternative method to run a shell script where a .NET build with DNVM and all dependencies would be supported? Using another language or cf push in this case is not an option, we really want to push .NET through pipeline at any cost.
When experimenting with the pipeline I wasn't able to get it working with Mono either, but if you can get away with just the CoreCLR on Linux then you should be able to. Kestrel, for example, doesn't require Mono anymore.
This was a build script from the beta7 timeframe but it should be close to what's needed to use RC1 now:
#!/bin/bash
sudo apt-get update
sudo apt-get -y install libunwind8 gettext libssl-dev libcurl3-dev zlib1g
curl -sSL https://raw.githubusercontent.com/aspnet/Home/dev/dnvminstall.sh | DNX_BRANCH=dev sh && source ~/.dnx/dnvm/dnvm.sh
dnvm install 1.0.0-beta7 -r coreclr -a x64
cd src/dotnetstarter
dnu restore
dnu build
cd ../../test/dotnetstarter.tests
dnu restore
dnu build
dnx test
cd ../../src/dotnetstarter
dnu publish --runtime ~/.dnx/runtimes/dnx-coreclr-linux-x64.1.0.0-beta7
The app was https://github.com/IBM-Bluemix/asp.net5-helloworld and I added the dotnetstarter.tests project which I was trying to run in the pipeline (the dnx test step). The last publish step isn't required but is included to show it was working.
Thanks to opiethehokie, this is the working script:
#!/bin/bash
echo --- UPDATING DEPENDENCIES! ---
sudo apt-get update
echo --- DOWNLOADING PACKAGES! ---
sudo apt-get -y install libunwind8 gettext libssl-dev libcurl3-dev zlib1g libcurl4-openssl-dev libicu-dev uuid-dev
echo --- DOWNLOADING DNVM! ---
curl -sSL https://raw.githubusercontent.com/aspnet/Home/dev/dnvminstall.sh | DNX_BRANCH=dev sh && source ~/.dnx/dnvm/dnvm.sh
echo --- INSTALLING DNVM! ---
dnvm install 1.0.0-rc1-final -r coreclr -a x64
echo --- EXECUTING RESTORE! ---
cd /path-to-project-folder
dnu restore
echo --- EXECUTING BUILD! ---
dnu build
echo --- PUBLISH BUILD (OPTIONAL)! ---
dnu publish --runtime ~/.dnx/runtimes/dnx-coreclr-linux-x64.1.0.0-*`

Resources