I activated a Web App (linux) on Azure.
Then i imported a Laravel 5.8 fresh installation from local to the app trough FTP.
In local the app works instead on azure i get 500 error when i try to reach the homepage.
Laravel 5.8 has already a web.config file into the public folder so i guess that is not the missing of that file, but should be something else related to the fact that the server is IIS and not Apache.
Any help will be (very) appreciated.
I'm going to assume you've already logged into the Azure SSH Terminal and installed Composer and Node.js already.
From the Microsoft documentation :
Step 1:
Login to the Azure CLI
Step 2:
Configure a deployment user
az webapp deployment user set --user-name <username> --password <password>
Step 3:
Create a resource group
az group create --name myResourceGroup --location "West Europe"
Step 4:
Create an Azure App Service Plan
az appservice plan create --name myAppServicePlan --resource-group myResourceGroup --sku B1 --is-linux
Step 5:
Create a web app
# Bash
az webapp create --resource-group myResourceGroup --plan myAppServicePlan --name <app_name> --runtime "PHP|7.0" --deployment-local-git
# PowerShell
az --% webapp create --resource-group myResourceGroup --plan myAppServicePlan --name <app_name> --runtime "PHP|7.0" --deployment-local-git
Step 6:
Place the following inside an .htcaccess file inside your Laravel root directory
This part isn't in the documentation, but it's what solved the issue for me
<IfModule mod_rewrite.c>
RewriteEngine on
RewriteCond %{REQUEST_URI} !^public
RewriteRule ^(.*)$ public/$1 [L]
</IfModule>
Step 7:
Head over and check out projectkudu over at Github.
Basically two files, .deployment and deploy.sh are added to your Laravel root directory.
Contents of .deployment:
[config]
command = bash deploy.sh
Contents of deploy.sh
#!/bin/bash
# ----------------------
# KUDU Deployment Script
# Version: 0.2.2
# ----------------------
# Helpers
# -------
exitWithMessageOnError () {
if [ ! $? -eq 0 ]; then
echo "An error has occurred during web site deployment."
echo $1
exit 1
fi
}
# Prerequisites
# -------------
# Verify node.js installed
hash node 2>/dev/null
exitWithMessageOnError "Missing node.js executable, please install node.js, if already installed make sure it can be reached from current environment."
# Setup
# -----
SCRIPT_DIR="${BASH_SOURCE[0]%\\*}"
SCRIPT_DIR="${SCRIPT_DIR%/*}"
ARTIFACTS=$SCRIPT_DIR/../artifacts
KUDU_SYNC_CMD=${KUDU_SYNC_CMD//\"}
if [[ ! -n "$DEPLOYMENT_SOURCE" ]]; then
DEPLOYMENT_SOURCE=$SCRIPT_DIR
fi
if [[ ! -n "$NEXT_MANIFEST_PATH" ]]; then
NEXT_MANIFEST_PATH=$ARTIFACTS/manifest
if [[ ! -n "$PREVIOUS_MANIFEST_PATH" ]]; then
PREVIOUS_MANIFEST_PATH=$NEXT_MANIFEST_PATH
fi
fi
if [[ ! -n "$DEPLOYMENT_TARGET" ]]; then
DEPLOYMENT_TARGET=$ARTIFACTS/wwwroot
else
KUDU_SERVICE=true
fi
if [[ ! -n "$KUDU_SYNC_CMD" ]]; then
# Install kudu sync
echo Installing Kudu Sync
npm install kudusync -g --silent
exitWithMessageOnError "npm failed"
if [[ ! -n "$KUDU_SERVICE" ]]; then
# In case we are running locally this is the correct location of kuduSync
KUDU_SYNC_CMD=kuduSync
else
# In case we are running on kudu service this is the correct location of kuduSync
KUDU_SYNC_CMD=$APPDATA/npm/node_modules/kuduSync/bin/kuduSync
fi
fi
# Node Helpers
# ------------
selectNodeVersion () {
if [[ -n "$KUDU_SELECT_NODE_VERSION_CMD" ]]; then
SELECT_NODE_VERSION="$KUDU_SELECT_NODE_VERSION_CMD \"$DEPLOYMENT_SOURCE\" \"$DEPLOYMENT_TARGET\" \"$DEPLOYMENT_TEMP\""
eval $SELECT_NODE_VERSION
exitWithMessageOnError "select node version failed"
if [[ -e "$DEPLOYMENT_TEMP/__nodeVersion.tmp" ]]; then
NODE_EXE=`cat "$DEPLOYMENT_TEMP/__nodeVersion.tmp"`
exitWithMessageOnError "getting node version failed"
fi
if [[ -e "$DEPLOYMENT_TEMP/.tmp" ]]; then
NPM_JS_PATH=`cat "$DEPLOYMENT_TEMP/__npmVersion.tmp"`
exitWithMessageOnError "getting npm version failed"
fi
if [[ ! -n "$NODE_EXE" ]]; then
NODE_EXE=node
fi
NPM_CMD="\"$NODE_EXE\" \"$NPM_JS_PATH\""
else
NPM_CMD=npm
NODE_EXE=node
fi
}
##################################################################################################################################
# Deployment
# ----------
echo Handling node.js deployment.
# 1. KuduSync
if [[ "$IN_PLACE_DEPLOYMENT" -ne "1" ]]; then
"$KUDU_SYNC_CMD" -v 50 -f "$DEPLOYMENT_SOURCE" -t "$DEPLOYMENT_TARGET" -n "$NEXT_MANIFEST_PATH" -p "$PREVIOUS_MANIFEST_PATH" -i ".git;.hg;.deployment;deploy.sh"
exitWithMessageOnError "Kudu Sync failed"
fi
# 2. Select node version
selectNodeVersion
# 3. Install NPM packages
if [ -e "$DEPLOYMENT_TARGET/package.json" ]; then
cd "$DEPLOYMENT_TARGET"
eval $NPM_CMD install --production
exitWithMessageOnError "npm failed"
cd - > /dev/null
fi
# 4. Install Bower modules
if [ -e "$DEPLOYMENT_TARGET/bower.json" ]; then
cd "$DEPLOYMENT_TARGET"
eval ./node_modules/.bin/bower install
exitWithMessageOnError "bower failed"
cd - > /dev/null
fi
# 5. Install Composer modules
if [ -e "$DEPLOYMENT_TARGET/composer.json" ]; then
cd "$DEPLOYMENT_TARGET"
eval php composer.phar install
exitWithMessageOnError "composer failed"
cd - > /dev/null
fi
##################################################################################################################################
# Post deployment stub
if [[ -n "$POST_DEPLOYMENT_ACTION" ]]; then
POST_DEPLOYMENT_ACTION=${POST_DEPLOYMENT_ACTION//\"}
cd "${POST_DEPLOYMENT_ACTION_DIR%\\*}"
"$POST_DEPLOYMENT_ACTION"
exitWithMessageOnError "post deployment action failed"
fi
echo "Finished successfully."
Step 8:
Assuming you're using Git, (if you're not - why not??), push to Azure
git remote add azure <deploymentLocalGitUrl-from-create-step>
git push azure master
You didn't mention interacting with a DB specifically, but you should this is a good read for that.
Hopefully, this will help someone using Azure tutorial app : https://learn.microsoft.com/en-us/azure/app-service/tutorial-php-mysql-app?pivots=platform-linux
I got the same error 500. Problem is that when you push the code, the deploy.sh runs and it errors on line 111 where error happens because php is not installed.
I commented out the line in .deploy file to run the bash deploy.sh and pushed code again and then it ran fine, by installing php, then the dependencies for laravel etc.
I have managed to make it work (sorry if its too late, started with Laravel this year).
After uploading all files make sure to use Kudu console to access the CMD terminal and copy the .env.example to the root of your app. Once done it is important to set up .env correctly and run:
php artisan key:generate
This will write the key on your env file.
After that it should run.
Do not forget to map the routes to your public folder.
Related
I'm trying to deploy my flask application to AWS EC2 instance using gitlab ci runner.
.gitlab.ci.yml
stages:
- test
- deploy
test_app:
image: python:latest
stage: test
before_script:
- python -V
- pip install virtualenv
- virtualenv env
- source env/bin/activate
- pip install flask
script:
- cd flask-ci-cd
- python test.py
prod-deploy:
stage: deploy
only:
- master # Run this job only on changes for stage branch
before_script:
- mkdir -p ~/.ssh
- echo -e "$RSA_KEY" > ~/.ssh/id_rsa
- chmod 600 ~/.ssh/id_rsa
- '[[ -f /.dockerenv ]] && echo -e "Host *\n\tStrictHostKeyChecking no\n\n" > ~/.ssh/config'
script:
- bash .gitlab-deploy-prod.sh
environment:
name: deploy
.gitlab-deploy-prod.sh
#!/bin/bash
# Get servers list
set -f
# access server terminal
shell="ssh -o StrictHostKeyChecking=no ${SERVER_URL}"
git_token=$DEPLOY_TOKEN
echo "Deploy project on server ${SERVER_URL}"
if [ ${shell} -d "/flask-ci-cd" ] # check if directory exists
then
eval "${shell} cd flask-ci-cd && git clone https://sbhusal123:${git_token}#gitlab.com/sbhusal123/flask-ci-cd.git master && cd flask-ci-cd"
else
eval "${shell} git pull https://sbhusal123:${git_token}#gitlab.com/sbhusal123/flask-ci-cd.git master && cd flask-ci-cd && cd flask-ci-cd"
fi
Error: .gitlab-deploy-prod.sh: line 7: -o: command not found
How can i check if directory existing??
What i've tried.
#!/bin/bash
# Get servers list
set -f
# access server terminal
shell="ssh -o StrictHostKeyChecking=no ${SERVER_URL}"
git_token=$DEPLOY_TOKEN
eval "${shell}" # i thought gitlab would provide me with shell access
echo "Deploy project on server ${SERVER_URL}"
if [-d "/flask-ci-cd" ] # check if directory exists
then
eval "cd flask-ci-cd && git clone https://sbhusal123:${git_token}#gitlab.com/sbhusal123/flask-ci-cd.git master && cd flask-ci-cd"
else
eval "git pull https://sbhusal123:${git_token}#gitlab.com/sbhusal123/flask-ci-cd.git master && cd flask-ci-cd && cd flask-ci-cd"
fi
I've tried to log into the ssh shell before executing the scripts inside if else. But it doesn't works the way intended.
Your script has some errors.
Do not use eval. No, eval does not work that way. eval is evil
When storing a command to a variable, do not use normal variables. Use bash arrays instead to preserve "words".
Commands passed via ssh are double escaped. I would advise to prefer to use here documents, they're simpler to get the quoting right. Note the difference in expansion when the here document delimiter is quoted or not.
i thought gitlab would provide me with shell access No, without open standard input the remote shell will just terminate, as it will read EOF from input. No, it doesn't work that way.
Instead of doing many remote connection, just transfer the execution to remote side once and do all the work there.
Take your time and research how quoting and word splitting works in shell.
git_token=$DEPLOY_TOKEN No, set variables are not exported to remote shell. Either pass them manually or expand them before calling the remote side. (Or you could also use ssh -o SendEnv=git_token and configure remote ssh with AcceptEnv=git_token I think, never tried it).
Read documentation for the utilities you use.
No, git clone doesn't take branch name after url. You can specify branch with --branch or -b option. After url it takes directory name. See git clone --help. Same for git pull.
How can i check if directory existing??
Use bash arrays to store the command. Check if the directory exists just by executing the test command on the remote side.
shell=(ssh -o StrictHostKeyChecking=no "${SERVER_URL}")
if "${shell[#]}" [ -d "/flask-ci-cd" ]; then
...
In case of directory name with spaces I would go with:
if "${shell[#]}" sh <<'EOF'
[ -d "/directory with spaces" ]
EOF
then
Pass set -x to sh to see what's happening also on the remote side.
For your script, try rather to move the execution to remote side - there is little logic in making 3 separate connections. I say just
echo "Deploy project on server ${SERVER_URL}"
ssh -o StrictHostKeyChecking=no "${SERVER_URL}" bash <<EOF
if [ ! -d /flask-ci-cd ]; then
# Note: git_token is expanded on host side
git clone https://sbhusal123:${git_token}#gitlab.com/sbhusal123/flask-ci-cd.git /flask-ci-cd
fi
cd /flask-ci-cd
git pull
EOF
But instead of getting the quoting in some cases right, use declare -p and declare -f to transfer properly quoted stuff to remote side. That way you do not need case about proper quoting - it will work naturally:
echo "Deploy project on server ${SERVER_URL}"
work() {
if [ ! -d /flask-ci-cd ]; then
# Note: git_token is expanded on host side
git clone https://sbhusal123:"${git_token}"#gitlab.com/sbhusal123/flask-ci-cd.git /flask-ci-cd
fi
cd /flask-ci-cd
git pull
{
ssh -o StrictHostKeyChecking=no "${SERVER_URL}" bash <<EOF
$(declare -p git_token) # transfer variables you need
$(declare -f work) # transfer function you need
work # call the function.
EOF
Updated answer for future reads.
.gitlab-ci.yml
stages:
- test
- deploy
test_app:
image: python:latest
stage: test
before_script:
- python -V
- pip install virtualenv
- virtualenv env
- source env/bin/activate
- pip install flask
script:
- cd flask-ci-cd
- python test.py
prod-deploy:
stage: deploy
only:
- master
before_script:
- mkdir -p ~/.ssh
- echo -e "$RSA_KEY" > ~/.ssh/id_rsa
- chmod 600 ~/.ssh/id_rsa
- '[[ -f /.dockerenv ]] && echo -e "Host *\n\tStrictHostKeyChecking no\n\n" > ~/.ssh/config'
script:
- bash .gitlab-deploy-prod.sh
environment:
name: deploy
.gitlab-deploy-prod.sh
#!/bin/bash
# Get servers list
set -f
shell=(ssh -o StrictHostKeyChecking=no "${SERVER_URL}")
git_token=$DEPLOY_TOKEN
echo "Deploy project on server ${SERVER_URL}"
ssh -o StrictHostKeyChecking=no "${SERVER_URL}" bash <<EOF
if [ ! -d flask-ci-cd ]; then
echo "\n Cloning into remote repo..."
git clone https://sbhusal123:${git_token}#gitlab.com/sbhusal123/flask-ci-cd.git
# Create and activate virtualenv
echo "\n Creating virtual env"
python3 -m venv env
else
echo "Pulling remote repo origin..."
cd flask-ci-cd
git pull
cd ..
fi
# Activate virtual env
echo "\n Activating virtual env..."
source env/bin/activate
# Install packages
cd flask-ci-cd/
echo "\n Installing dependencies..."
pip install -r requirements.txt
EOF
There is a test command which is explicit about checking files and directories:
test -d "/flask-ci-cd" && eval $then_commands || eval $else_commands
Depending on the AWS instance I'd expect "test" to be available. I'd recommend putting the commands in variables. (e.g. eval $then_commands)
I am trying to create a script I can provide to those who will use an academic game I am making. I am trying to do the following:
Verify that Apache2 is installed and Running
If Apache2 is installed, move a folder containing the website files to /var/www/html while backing up apache's original index.html
The code is as follows:
#!/bin/sh
acm=[]
cnn=[]
gnu=[]
ieee=[]
if [pgrep -x "apache2" > /dev/null]; then
echo "Apache 2 Installed and Running!"
if [ -d "$HOME/acmDL" ]; then
sudo mkdir /var/www/bak
sudo mv /var/www/html/index.html /var/www/bak
sudo mv acmDL /var/www/html/
cd /var/www/html/
sudo mv acmDL/* .
exit
elif [ -d "$HOME/cnnDL" ]; then
sudo mkdir /var/www/bak
sudo mv /var/www/html/index.html /var/www/bak
sudo mv cnnDL /var/www/html/
cd /var/www/html/
sudo mv cnnDL/* .
exit
elif [ -d "$HOME/gnuDL" ]; then
sudo mkdir /var/www/bak
sudo mv /var/www/html/index.html /var/www/bak
sudo mv gnuDL /var/www/html/
cd /var/www/html/
sudo mv gnuDL/* .
exit
elif [ -d "$HOME/ieeeDL" ]; then
sudo mkdir /var/www/bak
sudo mv /var/www/html/index.html /var/www/bak
sudo mv ieeeDL /var/www/html/
cd /var/www/html/
sudo mv ieeeDL/* .
exit
else
echo "Provided websites not found... Are you using a custom website?"
fi
else
echo "Please check apache2... It may not have installed correctly"
fi
The error I keep getting is syntax error near unexpected token `elif' on line 15.
As you can see, I even tried moving the boolean expression [ - d "$HOME/site" ] to their own variables, but then the error becomes : -d: command not found and the error on line 15.
Is what I am trying to do impossible, or am I missing something undocumented and yet completely obvious (like a handful of my previous posts)?
This is being run on a minimal installation of Ubuntu 18 on a Virtual Machine. The site directories are shared by Filezilla. Script written in Notepad++ on Windows 7 x64.
First of all, can you rewrite it like this?
Please tell me the execution result.
This is wrong.
if [pgrep -x "apache2" > /dev/null]; then
This is correct.
pgrep -x "apache2" > /dev/null
if [ $? -eq 0 ]; then
I have a deploy script I only want to run if my test is successful but believe there is an issue with my conditional statement if [ "$VALID" ]
#!/bin/bash
# install dependencies
echo 'INSTALLING YARN'
npm install yarn -g
echo "INSTALLING DEPENDENCIES"
yarn install
echo "TESTING"
VALID="$(npm test)"
if [ "$VALID" ]
then
# ZIP up the code
echo 'INSTALLING ZIP'
apt-get update
echo "y" | apt-get install zip
echo 'ZIPPING'
zip -r ./Lambda-Image-Compression.zip index.js node_modules
# install aws cli so we can deploy code
echo 'INSTALLING PIP'
# echo "y" | apt-get install python-pip
echo "y" | apt-get install python-pip python-dev build-essential
echo "y" | pip install --upgrade pip
# echo "y" | sudo pip install --upgrade virtualenv
echo 'INSTALLING AWSCLI'
pip install awscli
# Copy config file to root so AWS config & credentials are set
echo 'MAKING AWS CREDENTIALS'
CREDENTIALS="[default]
aws_access_key_id = $AWS_ACCESS_KEY_ID
aws_secret_access_key = $AWS_SECRET_ACCESS_KEY"
touch .aws/credentials
"$CREDENTIALS" > .aws/credentials
echo "MOVING AWS CONFIG"
cp -R .aws ~/.
# Upload to AWS
echo 'UPDATING LAMBDA FUNCTION'
aws lambda update-function-code \
--function-name resizeHandler \
--zip-file fileb://Lambda-Image-Compression.zip \
--region ap-southeast-2
fi
My test result if successful looks like:
> Lambda-Image-Compression#1.0.0 test
> mocha
myLambda
RUNNING OPTIMSATION
download
downloadImage: 69.381ms
End of step null
✓ Should move testImage.png from srcBucket to dstBucket and return true (286ms)
1 passing (299ms)
Failure shows the following:
TEST RESULT:
> Lambda-Image-Compression#1.0.0 test
> mocha
myLambda
RUNNING OPTIMSATION
download
1) Should move testImage.png from srcBucket to dstBucket and return true
0 passing (22ms)
1 failing
1) myLambda Should move testImage.png from srcBucket to dstBucket and return true:
ReferenceError: s3 is not defined
at download (index.js:32:4)
at nextTask (node_modules/async/dist/async.js:5273:14)
at Object.waterfall (node_modules/async/dist/async.js:5283:5)
at exports.handler (index.js:24:8)
at error (node_modules/lambda-tester/lib/runner.js:151:25)
at Promise.resolve.then (node_modules/lambda-tester/lib/runner.js:138:24)
You could do something along the lines of:
VALID="$(npm test | grep -o 'failing')"
And then:
if [[ $VALID != "failing" ]] ...
Another way would be to pick up on " 0 passing":
VALID="$(npm test | grep -o ' 0 passing')"
And for the conditional:
if [[ $VALID != " 0 passing" ]] ...
In either case if the word(s) "failing" or " 0 passing" are found in the returned string it would indicate the test was a failure. The key is to find something unique to the passing/failure output.
NOTE: Study the output carefully; The "0 passing" string is not fool proof, since you technically could have 10 tests... grep would
see the "10 passing" and mistake it for a failure. Maybe you don't
have that many tests, but definitely be aware of it, or include the
exact number of spaces that are leading up to it.
Though the question is fairly old and has been answered already, I would like to share how I did it since the "correct" solutions seems a bit error-prone.
Npm - Prebuild:
Npm allows you to chain commands in the package.json file, allowing tests to run before your build:
{
"scripts": {
"build": "<build>",
"prebuild": "npm run test",
"test": "<test>",
}
}
so when your tests fail, the build command won't be executed. If your build command is called e.g. deploy, prebuild must be renamed to predeploy. (Check npm documentation here)
Check if build was successful
in your shell script now add the following logic:
delete build folder
build
check if build folder has been created
which could look something like this:
#!/bin/bash
BUILD_DIR="./build"
rm -r $BUILD_DIR
npm run build
if [ -d "$BUILD_DIR" ]; then
echo "Build was successful! Starting Deployment";
fi
Again, if your build folder is called differently, BUILD_DIR must be adjusted accordingly.
As part of a bash script, I want to check if a particularly docker image:tag combination exists on docker hub. Also, it will be a private repository.
i.e. the pseudocode would be like:
tag = something
if image:tag already exists on docker hub:
Do nothing
else
Build and push docker image with that tag
Update: Docker-Free solution see below
Using Docker
This is the solution I use with gitlab using the docker:stable image.
Login
docker login -u $USER -p $PASSWORD $REGISTRY
Check whether it's there:
docker manifest inspect $IMGNAME:$IMGTAG > /dev/null ; echo $?
docker will return 0 on success or 1 on failure.
If you get a warning: Update Docker or enable experimental client-features:
Set the environment variable DOCKER_CLI_EXPERIMENTAL to enabled (See Matěj's answer below)
Alternatively adjust the config (original answer):
echo '{"experimental": "enabled"}' > ~/.docker/config.json
This will also overwrite your config. If that is not an option you need to do that manually or use jq, sed or whatever you have available.
Testing without Docker
If you don't have access to a docker-daemon you can give Skopeo or crane a try.
Please try this one
function docker_tag_exists() {
curl --silent -f -lSL https://index.docker.io/v1/repositories/$1/tags/$2 > /dev/null
}
if docker_tag_exists library/nginx 1.7.5; then
echo exist
else
echo not exists
fi
Update:
In case of usage Docker Registry v2 (based on this):
# set username and password
UNAME="user"
UPASS="password"
function docker_tag_exists() {
TOKEN=$(curl -s -H "Content-Type: application/json" -X POST -d '{"username": "'${UNAME}'", "password": "'${UPASS}'"}' https://hub.docker.com/v2/users/login/ | jq -r .token)
curl --silent -f --head -lL https://hub.docker.com/v2/repositories/$1/tags/$2/ > /dev/null
}
if docker_tag_exists library/nginx 1.7.5; then
echo exist
else
echo not exists
fi
To build on morty's answer, notice that docker supports setting the experimental flag using environment variable:
DOCKER_CLI_EXPERIMENTAL Enable experimental features for the cli (e.g. enabled or disabled)
The snippet therefore becomes:
tag=something
if DOCKER_CLI_EXPERIMENTAL=enabled docker manifest inspect $image:$tag >/dev/null; then
Do nothing
else
Build and push docker image with that tag
fi
Easiest:
docker pull alpine:invalid > /dev/null && echo "success" || echo "failed"
Pulls & prints success if image exists, or prints failed if it doesn't:
You can even export it in a var if using in bash script:
Note that this will pull the image if it exists. Beware of the overhead cost before using this solution.
I have a docker private repo stood up on my LAN using registry:2, private CA, and basic auth.
I just looked at the official docker API docs (https://docs.docker.com/registry/spec/api/) and came up with this solution which seems pretty elegant, easy to debug, customize, and is CICD/scripting friendly.
curl --silent -i -u "demoadmin":"demopassword" https://mydockerrepo.local:5000/v2/rancher/pause/manifests/3.1 | grep "200 OK"
--silient gets rid of some extra text
-i is what makes the return code "200 OK" show up
if it exists return code is 0, if doesn't exist return code is 1
you can verify that using
Bash# echo $?
Here's a Bash function that will help:
docker_image_exists() {
local image_full_name="$1"; shift
local wait_time="${1:-5}"
local search_term='Pulling|is up to date|not found'
local result="$((timeout --preserve-status "$wait_time" docker 2>&1 pull "$image_full_name" &) | grep -v 'Pulling repository' | egrep -o "$search_term")"
test "$result" || { echo "Timed out too soon. Try using a wait_time greater than $wait_time..."; return 1 ;}
echo $result | grep -vq 'not found'
}
Usage example:
docker_image_exists elifarley/docker-dev-env:alpine-sshd && \
echo EXISTS || \
echo "Image does not exist"
Just a small improvement of Evgeny Oskin's solution. When it comes to a user repo that hasn't been created yet, jq says that it "Cannot iterate over null". To overcome it. one can skip not present blocks with ?
Here is a modification to above mentioned solution that is applicable to a public repo in particular:
#!/usr/bin/env bash
function docker_image_tag_exists() {
EXISTS=$(curl -s https://hub.docker.com/v2/repositories/$1/tags/?page_size=10000 | jq -r "[.results? | .[]? | .name == \"$2\"] | any")
test ${EXISTS} = true
}
if docker_image_tag_exists $1 $2; then
echo "true"
else
echo "false"
fi
I was struggling getting this to work for a private docker hub repository and finally decided to write a ruby script instead, which works as of today. Feel free to use!
#!/usr/bin/env ruby
require 'base64'
require 'net/http'
require 'uri'
def docker_tag_exists? repo, tag
auth_string = Base64.strict_encode64 "#{ENV['DOCKER_USER']}:#{ENV['DOCKER_PASSWORD']}"
uri = URI.parse("https://registry.hub.docker.com/v1/repositories/#{repo}/tags/#{tag}")
request = Net::HTTP::Get.new(uri)
request['Authorization'] = "Basic #{auth_string}"
request['Accept'] = 'application/json'
request['Content-Type'] = 'application/json'
response = Net::HTTP.start(request.uri.hostname, request.uri.port, use_ssl: true) do |http|
http.request(request)
end
(response.body == 'Tag not found') ? 0 : 1
end
exit docker_tag_exists? ARGV[0], ARGV[1]
Note: you need to specify DOCKER_USER and DOCKER_PASSWORD when calling this like...
DOCKER_USER=XXX DOCKER_PASSWORD=XXX config/docker/docker_hub.rb "NAMESPACE/REPO" "TAG" && echo 'latest'
This line would print out 'latest', if authentication is successful and the specified tag does not exists! I was using this in my Vagrantfile when trying to fetch a tag based on the current git branch:
git rev-parse --symbolic-full-name --abbrev-ref HEAD
All of the options above assume that you can authenticate using username/password. There are a lot of cases where this is inconvenient, for example when using Google Container Registry, for which one would run gcloud auth configure-docker gcr.io first. That command installs an authentication helper for Docker, and you wouldn't want to manage that token yourself.
One tool that supports these docker authentication helpers, and also allows getting a manifest - like experimental Docker - is crane.
Example using crane:
# you would have done this already
gcloud auth configure-docker gcr.io;
# ensure we have crane installed
which crane || (echo 'installing crane' && GO111MODULE=on go get -u github.com/google/go-containerregistry/cmd/crane)
# check manifest
crane manifest ubuntu || echo "does not exist"
If you are querying Hub for the existence of a tag, make sure you use a HEAD rather than a GET request. The GET request count against your rate limit. A script that does this specific to Docker Hub, only supports the Docker media types, and anonymous logins that are rate limited by your requesting IP, looks like:
$ more ~/data/docker/registry-api/manifest-v2-head.sh
#!/bin/sh
ref="${1:-library/ubuntu:latest}"
sha="${ref#*#}"
if [ "$sha" = "$ref" ]; then
sha=""
fi
wosha="${ref%%#*}"
repo="${wosha%:*}"
tag="${wosha##*:}"
if [ "$tag" = "$wosha" ]; then
tag="latest"
fi
api="application/vnd.docker.distribution.manifest.v2+json"
apil="application/vnd.docker.distribution.manifest.list.v2+json"
token=$(curl -s "https://auth.docker.io/token?service=registry.docker.io&scope=repository:${repo}:pull"
\
| jq -r '.token')
curl -H "Accept: ${api}" -H "Accept: ${apil}" \
-H "Authorization: Bearer $token" \
-I -s "https://registry-1.docker.io/v2/${repo}/manifests/${sha:-$tag}"
To work on other registries, handle more media types (like the OCI types), and handle logins, use a tool like crane, skopeo, or my own regclient:
# the "image digest" command uses a HEAD instead of a GET
if regctl image digest registry.example.com/repo:tag >/dev/null 2>&1; then
echo tag exists
else
echo tag does not exist
fi
For the local docker registry, you can try this:
function docker_tag_exists() {
curl --silent -f -lSL http://localhost:5000/v2/$1/manifests/$2 > /dev/null
}
if docker_tag_exists image_on_local latest; then
echo exists
else
echo not exists
fi
I like solutions based on docker.
This oneliner is what I use in our CI:
docker run --rm anoxis/registry-cli -l user:password -r registry-url -i docker-name | grep -q docker-tag || echo do something if not found
Have you tried something like that, simply trying to pull the tag and deciding to push or not according to the return code?
#! /bin/bash
if docker pull hello-world:linux > /dev/null; then
echo "already exist -> do not push"
else
echo "does not exist -> push"
fi
I installed docker-compose by following the official documentation: http://docs.docker.com/compose/install/
Now I want to uninstall docker-compose.
$ docker-compose -h
offers no command for uninstalling, nor does the official documentation offer any instructions.
I deleted my docker-compose.yml file and /usr/local/bin/docker-compose, but I want to make sure that's everything.
I'm using OSX Yosemite 10.10.3 on a MacbookPro.
EDIT: Regarding the installation instructions I followed, I didn't use pip. I used the documented curl command:
$ curl -L https://github.com/docker/compose/releases/download/1.3.1/docker-compose-`uname -s`-`uname -m` > /usr/local/bin/docker-compose
$ chmod +x /usr/local/bin/docker-compose
Documenation
Please note that this is now in the docs.
Coupled Installation and Removal
Note: on Mac Docker now installs Docker Compose. So the strategy for removal has changed a bit. If you uninstall Docker, and you want to uninstall both, then you also uninstall Docker Compose.
Individual Removal if Installed Using curl
It is commonly installed to /usr/local/bin/docker-compose on macs. However, you can run which docker-compose to find the exact location.
Run the following command (*nix systems) to remove:
rm $(which docker-compose)
If you get a permission denied error then you will need to prepend sudo:
sudo rm $(which docker-compose)
To verify that it was successful, run the following command which should return nothing:
which docker-compose
It should say that the command wasn't found.
Individual Removal if Installed Using PIP
If you've installed Docker Compose using PIP then you can run:
pip uninstall docker-compose
You may have to use sudo if you get a permission denied error:
sudo pip uninstall docker-compose
first get docker path by:
which docker-compose
then it will return path like:/usr/bin/docker-compose
then remove it:
sudo rm -rf /usr/bin/docker-compose
Nowadays docker-compose is part of the docker toolbox.
If you want to remove everything that comes with the Docker Toolbox (including Docker itself).
You can execute this shell script:
#!/bin/bash
# Uninstall Script
if [ "${USER}" != "root" ]; then
echo "$0 must be run as root!"
exit 2
fi
while true; do
read -p "Remove all Docker Machine VMs? (Y/N): " yn
case $yn in
[Yy]* ) docker-machine rm -f $(docker-machine ls -q); break;;
[Nn]* ) break;;
* ) echo "Please answer yes or no."; exit 1;;
esac
done
echo "Removing Applications..."
rm -rf /Applications/Docker
echo "Removing docker binaries..."
rm -f /usr/local/bin/docker
rm -f /usr/local/bin/docker-machine
rm -r /usr/local/bin/docker-machine-driver*
rm -f /usr/local/bin/docker-compose
echo "Removing boot2docker.iso"
rm -rf /usr/local/share/boot2docker
echo "All Done!"
If you still have the depreciated Boot2docker and you want to get rid of it as well.
You can uninstall it by executing the following shell script:
#!/bin/bash
# Uninstall Script
if [ "$(which boot2docker)" == "" ]; then
echo "boot2docker does not exist on your machine!"
exit 1
fi
if [ "${USER}" != "root" ]; then
echo "$0 must be run as root!"
exit 2
fi
echo "Stopping boot2docker processes..."
boot2docker stop && boot2docker delete
echo "Removing boot2docker executable..."
rm -f /usr/local/bin/boot2docker
echo "Removing boot2docker ISO and socket files..."
rm -rf ~/.boot2docker
rm -rf /usr/local/share/boot2docker
echo "Removing boot2docker SSH keys..."
rm -f ~/.ssh/id_boot2docker*
echo "Removing boot2docker OSX files..."
rm -f /private/var/db/receipts/io.boot2docker.*
rm -f /private/var/db/receipts/io.boot2dockeriso.*
echo "Removing Docker executable..."
rm -f /usr/local/bin/docker
echo "All Done!"
You can also just:
sudo yum remove docker-compose-plugin
on RPM-based like Centos
according to: https://docs.docker.com/compose/install/uninstall/
I would use pip uninstall docker-compose