I wrote this simple Makefile to illustrate my problem.
$make target
calls the dep as a dependency and pulls the image
But the subsequent check for docker image list -q $(IMG) does not find my image.
What is happening here and how should I fix this?
IMG := hello-world
.PHONY: target
target: dep
ifeq ($(shell docker image list -q $(IMG)),)
echo "docker image list did not recognize the pull"
endif
.PHONY: dep
dep:
#docker pull $(IMG)
That test isn't subsequent. It's substituted into the Makefile when it's read, before any rules are executed.
You probably want to perform that test in the commands of the target rule:
target: dep
if test -z "$$(docker image list -q $(IMG))"; then \
echo "docker image list did not recognize the pull" >&2; \
false; \
fi
We could change the command to just run docker image inspect - that will return a true status if the image exists, and false otherwise:
target: dep
if ! docker image inspect "$(IMG))" >/dev/null 2>&1; then \
echo "docker image list did not recognize the pull" >&2; \
false; \
fi
Related
So I found a link that shows I should use the following, but perhaps my logic is wrong within the Makefile. I need to use the Makefile for testing purposes to work on both Mac and Windows. The image is fine and the docker container works, I am just trying to make use of the fact that in Linux/Mac \ can be used to shorten the commands, whereas in Windows you have to use the backtick (`).
Example:
.PHONY: validate-lookml
validate-lookml:
UNAME_S=$(shell uname -s)
ifeq ($(UNAME), Linux)
docker run --rm -it -e LOOKER_BASE_URL=${LOOKER_BASE_URL} \
-e LOOKER_CLIENTID=${LOOKER_CLIENTID} \
-e LOOKER_CLIENT_SECRET=${LOOKER_CLIENTSECRET} mirantis/mirantis_spectacles \
lookml \
--base-url ${LOOKER_BASE_URL} \
--client-id ${LOOKER_CLIENTID} \
--client-secret ${LOOKER_CLIENTSECRET} \
--project ${PROJECT} \
--branch ${BRANCH}
endif
ifeq ($(UNAME), Darwin)
docker run --rm -it -e LOOKER_BASE_URL=${LOOKER_BASE_URL} \
-e LOOKER_CLIENTID=${LOOKER_CLIENTID} \
-e LOOKER_CLIENT_SECRET=${LOOKER_CLIENTSECRET} mirantis/mirantis_spectacles \
lookml \
--base-url ${LOOKER_BASE_URL} \
--client-id ${LOOKER_CLIENTID} \
--client-secret ${LOOKER_CLIENTSECRET} \
--project ${PROJECT} \
--branch ${BRANCH}
endif
ifeq ($(UNAME), Windows_NT)
docker run --rm -it -e LOOKER_BASE_URL=${LOOKER_BASE_URL} `
-e LOOKER_CLIENTID=${LOOKER_CLIENTID} `
-e LOOKER_CLIENT_SECRET=${LOOKER_CLIENTSECRET} mirantis/mirantis_spectacles `
lookml `
--base-url ${LOOKER_BASE_URL} `
--client-id ${LOOKER_CLIENTID} `
--client-secret ${LOOKER_CLIENTSECRET} `
--project ${PROJECT} `
--branch ${BRANCH}
endif
Unfortunately, it doesn't work on Windows, and I need my Makefile to support analysts on Windows Laptops:
Error:
C:\Users\richa\Git\Mirantis\dataops-looker [main ≡ +0 ~1 -0 !]> make -s validate-lookml
process_begin: CreateProcess(NULL, uname -s, ...) failed.
Makefile:9: pipe: No error
process_begin: CreateProcess(NULL, uname -s, ...) failed.
Makefile:12: pipe: No error
process_begin: CreateProcess(NULL, uname -s, ...) failed.
Makefile:15: pipe: No error
usage: spectacles lookml [-h] [--config-file CONFIG_FILE] --base-url BASE_URL
--client-id CLIENT_ID --client-secret CLIENT_SECRET
[--port PORT] [--api-version API_VERSION] [-v]
[--log-dir LOG_DIR] [--do-not-track]
[--severity {success,info,warning,error,fatal}]
--project PROJECT [--branch BRANCH]
[--remote-reset | --commit-ref COMMIT_REF | --pin-imports PIN_IMPORTS [PIN_IMPORTS ...]]
spectacles lookml: error: argument --base-url: expected one argument
make: *** [Makefile:40: validate-lookml] Error 2
Much confusion here.
A makefile consists of lines written in two completely different languages: one is the make language, and the other is the shell. You cannot send make operations to the shell, and you cannot run (directly) shell commands in make.
Make tells the difference between these two by use of the TAB character. Lines that are not indented with TAB are parsed by make, and lines that are indented with TAB are given to the shell. So, in your makefile:
validate-lookml:
UNAME_S=$(shell uname -s)
ifeq ($(UNAME), Linux)
docker run --rm -it -e LOOKER_BASE_URL=${LOOKER_BASE_URL} \
this is not right because the first two indented lines here are make commands, and the third is a shell command. You should write this like:
UNAME_S := $(shell uname -s)
validate-lookml:
ifeq ($(UNAME), Linux)
docker run --rm -it -e LOOKER_BASE_URL=${LOOKER_BASE_URL} \
...
endif
ifeq ($(UNAME), Darwin)
...
etc.
But, there is no uname command on Windows so when you run this it won't work, that's why you're getting the error process_begin: CreateProcess(NULL, uname -s, ...) failed. If you have GNU make 4.0 or better I recommend that you look at the MAKE_HOST variable and use that instead of trying to run uname.
Finally, you don't have to worry about the backslash difference, because make will parse all the backslashes and remove them on its own BEFORE it starts the shell. So just use backslashes to continue all the lines in your recipe and it will work the same way on all different platforms.
I have a simple if statement inside of Makefile to say that delete docker image if it exists but it doesn't work. Notice in the log file image exists but when I do make clean if statement fails.
NAME=program1
all: .docker-build
docker run ${NAME} make all
.docker-build:
docker build . -t ${NAME}
#echo "" > .docker-build
run-%: .docker-build
docker run ${NAME} make $*
clean:
ifeq ($(docker images -q ${NAME} 2> /dev/null), "")
docker image rm -f ${NAME}
endif
#rm -f .docker-build
Log (notice image exist but if statement inside of make clean fails):
➜ dockerfile-test docker images -q program1 2> /dev/null
5ee4797b91ad
➜ dockerfile-test make clean
rm -f .docker-build
This:
ifeq ($(docker images -q ${NAME} 2> /dev/null), "")
expands the make variable named docker images -q program1 2> /dev/null, which doesn't exist and so expands to the empty string, then it compares it to the two-character string "" and it never matches.
You probably meant:
ifeq ($(shell docker images -q ${NAME} 2> /dev/null),)
What is the meaning of this make target:
.PHONY: docker.%
docker.%:
go mod tidy
docker build -t $* .
docker run -p 5751:5751 -v $$(pwd)/:/work $* /work/config.yaml
In particular, what is the meaning of % in docker.% and how is it being utilized here?
Cloud-build is not showing build failure status
I created my own remote-builder which scp all files from /workspace to my Instance and running build on using gcloud compute ssh -- COMMAND
remote-builder
#!/bin/bash
USERNAME=${USERNAME:-admin}
REMOTE_WORKSPACE=${REMOTE_WORKSPACE:-/home/${USERNAME}/workspace/}
GCLOUD=${GCLOUD:-gcloud}
KEYNAME=builder-key
ssh-keygen -t rsa -N "" -f ${KEYNAME} -C ${USERNAME} || true
chmod 400 ${KEYNAME}*
cat > ssh-keys <<EOF
${USERNAME}:$(cat ${KEYNAME}.pub)
EOF
${GCLOUD} compute scp --compress --recurse \
$(pwd)/ ${USERNAME}#${INSTANCE_NAME}:${REMOTE_WORKSPACE} \
--ssh-key-file=${KEYNAME}
${GCLOUD} compute ssh --ssh-key-file=${KEYNAME} \
${USERNAME}#${INSTANCE_NAME} -- ${COMMAND}
below is the example of the code to run build(cloudbuild.yaml)
steps:
- name: gcr.io/$PROJECT_ID/remote-builder
env:
- COMMAND="docker build -t [image_name]:[tagname] -f Dockerfile ."
During docker build inside Dockerfile it got failure and show errors in log but status showing SUCCESS
can any help me how to resolve it.
Thanks in advance.
try adding
|| exit 1
at the end of your docker command... alternatively, you might just need to change the entrypoint to 'bash' and run the script manually
To confirm -- the first part was the run-on.sh script, and the second part was your cloudbuild.yaml right? I assume you trigger the build manually via UI and/or REST API?
I wrote all docker commands on bash script and add below error handling code to it.
handle_error() {
echo "FAILED: line $1, exit code $2"
exit 1
}
trap 'handle_error $LINENO $?' ERR
It works!
Code
Consider the following makefile snippet:
COMMIT := $(shell git rev-parse HEAD)
build:
docker build -f Dockerfile --no-cache=false -t $(COMMIT) .
rebuild:
docker build -f Dockerfile --no-cache=true -t $(COMMIT) .
The problem
The only difference between build and rebuild is the value of the --no-cache parameter. Obviously, rewriting the same command with a slight change is a bad practice; it breaks the DRY principle, and if I ever need to change something else in the command - for example, the value of -t - I would have to change it across all relevant targets.
I had something like this in mind:
COMMIT := $(shell git rev-parse HEAD)
NO_CACHE := false
build:
docker build -f Dockerfile --no-cache=$(NO_CACHE) -t $(COMMIT) .
rebuild:
NO-CACHE = true
make build
I tried playing with the variables, with no luck.
My question
What would be an elegant way to write the docker build command once, and have each target alter its parameter?
You can use constructed variable names:
COMMIT := $(shell git rev-parse HEAD)
build_NOCACHE = false
rebuild_NOCACHE = true
build rebuild:
docker build -f Dockerfile --no-cache=$($#_NOCACHE) -t $(COMMIT) .
Or you can use target-specific variables:
COMMIT := $(shell git rev-parse HEAD)
build: NOCACHE = false
rebuild: NOCACHE = true
build rebuild:
docker build -f Dockerfile --no-cache=$(NOCACHE) -t $(COMMIT) .
Use the call function
Positional arguments are specified from 1 to n and used in the command definition as $(1), $(2), $(n).
COMMIT := $(shell git rev-parse HEAD)
DOCKER_BUILD_CMD = docker build -f Dockerfile --no-cache=$(1) -t $(COMMIT) .
build:
$(call DOCKER_BUILD_CMD, false)
rebuild:
$(call DOCKER_BUILD_CMD, true)