I have a makefile in which I want to do checkout on a file if this file isn't already checkedout:
VAR=$(shell cleartool ls $(HOME)/all_files.tgz | grep CHECKEDOUT)
build:
#if ["$(VAR)" == ""]; then \
cleartool co -unres -nc $(HOME)/all_files.tgz;\
fi
# tar czf $(HOME)/all_files.tgz $(OUT)/*.log
I get the following error if all_files.tgz is checked out:
/bin/sh: [/home/ge/prj/all_files.tgz##/main/10/CHECKEDOUT from /main/10 Rule: CHECKEDOUT: not found
When you use the brackets in the shell you MUST include whitespace around them. This is because [ is actually a separate command: if you use ["$(VAR)" that expands to the string you quote above ([/home/ge/prj/all_files.tgz##/main/10/CHECKEDOUT from /main/10) and that is not a valid command name. Similarly for the final ]: must have whitespace around it.
Use:
VAR=$(shell cleartool ls $(HOME)/all_files.tgz | grep CHECKEDOUT)
build:
#if [ "$(VAR)" == "" ]; then \
cleartool co -unres -nc $(HOME)/all_files.tgz;\
fi
# tar czf $(HOME)/all_files.tgz $(OUT)/*.log
This is a kind of odd rule though. Since VAR is just a shell function, and you're using it in the shell, why even bother to use $(shell ...)?
Related
I would like to convert and execute
if egrep -r 'my_pattern' ./template_builder
then exit 1
elif egrep -r 'my_second_pattern' ./template_builder
then exit 1
fi
in a Makefile, without success for now.
To build this:
cd /tmp;
mkdir template_builder;
echo "not_pattern" >> ./template_builder/test.txt
# Do the command at the top, nothing happens
echo "my_pattern" >> ./template_builder/test.txt
# Do the command at the top, terminal stops
touch Makefile
In a Makefile, I thought this would work :
check:
if egrep -r 'my_pattern' ./template_builder
then exit 1
elif egrep -r 'my_second_pattern' ./template_builder
then exit 1
fi
make check
if egrep -r 'my_pattern' ./template_builder
/bin/sh: -c: line 1: syntax error: unexpected end of file
make: *** [template] Error 2
How can I fix this?
Your attempt was not far from working!
Add backslashes at the end of every line, and ;s as explicit command separators (and of course use real tabs instead of the 8-space indents below):
check:
if egrep -r 'my_pattern' ./template_builder; \
then exit 1; \
elif egrep -r 'my_second_pattern' ./template_builder; \
then exit 1; \
fi
If I understand you correctly, if the directory template_builder located in /tmp does not contain a file matching the string 'my_pattern' or 'my_second_pattern', you want to exit from make with an error code.
You can achieve this with this rule in Makefile:
check:
egrep -r -v 'my_pattern' /tmp/template_builder || egrep -r -v 'my_second_pattern' /tmp/template_builder
Explanation: the first egrep is going to return an error in the case he finds a match. Due to the presence of the || operator, the second egrep will be invoked. The result of this second command will be the result that make will see. If it returns an error, the execution of make is aborted, which seems to be the behaviour you are expecting.
Caution: I edited my answer. The right boolean operator is || and not &&.
As others have already noted, make runs each separate line in a recipe in a new shell subprocess. (For the record, it uses sh out of the box, not Bash.) The trivial fix is to add a backslash to escape the newline at the end of each line which should be executed in the same shell as the next one. (You need to add a semicolon as well in some places, like before then and else and fi.) But you really want to refactor to use the facilities and idioms of make.
The default logic of make is to terminate the recipe if any line fails. So, your code can be reduced to simply
check: template_builder
! egrep -r 'my_pattern' $<
! egrep -r 'my_second_pattern' $<
The explicit exit 1 is not necessary here (negating a zero exit code produces exactly that); but if you wanted to force a particular exit code, you could do that with
egrep -r 'my_pattern' $< && exit 123 || true
Modern POSIX prefers grep -E over legacy egrep; of course, with these simple patterns, you can use just grep, or even grep -F (née fgrep).
Moreover, if you want to search for both patterns in the same set of files, it's much more efficient to search for them both at once.
check: template_builder
! egrep -e 'my_pattern' -e 'my_second_pattern' -r $<
... or combine them into a single regex my_(second_)?pattern (which requires egrep / grep -E).
Notice also how I factored out the dependency into $< and made it explicit; but you probably want to make this recipe .PHONY anyway, so that it gets executed even if nothing has changed.
(You can't directly copy/paste this code, because Stack Overflow stupidly renders the literal tabs in the markdown source as spaces.)
In my Makefile I want to execute multiple steps as a part of a single target. Those steps should be done sequentially, because they depend on one another. This is the simplified case:
target:
git archive --remote=some-remote master --format zip -o ./zipfile.zip
echo "$(VARIABLE_IN_MAKE):$(shell unzip -z ./zipfilezip | tail -1)" > ./textfile
$(shell cat ./textfile)
The problem here is that the shell command - $(shell unzip -z ./zipfilezip | tail -1) is executed as soon as the rule is "loaded", i.e. before the zipfile even exists. That returns errors. The cat command is also expanded too early.
What is the correct way to execute a subshell not before, but only after all the steps above have finished? Do I have to wrap all the commands in a bash -c call? Or chain them via &&?
Get rid of these $(shell...). Each line of a make recipe is already a shell script:
target:
git archive --remote=some-remote master --format zip -o ./zipfile.zip
echo "$(VARIABLE_IN_MAKE):$$(unzip -z ./zipfilezip | tail -1)" > ./textfile
cat ./textfile
Note that, in the second line, $$(unzip -z ./zipfilezip | tail -1) is expanded twice: a first time by make before passing the recipe to the shell, leading to $(unzip -z ./zipfilezip | tail -1), and a second time by the shell that treats it as a command substitution. This is why the $$ is needed: to escape the first expansion by make. If you were using $(unzip -z ./zipfilezip | tail -1) directly, make would expand it as the empty string (unless you have a make variable which name is unzip -z ./zipfilezip | tail -1, but this is very unlikely).
I am trying to move files from a directory if it is not empty. This script is added in cronjob but it is always executing regardless of files are present or not? What is wrong in this thing?
#!/bin/bash
logFolder="/dstDir/`date '+%Y-%m-%d/%H-%M'`"
tempLogFolder="sourceDir"
if [ -z "$(ls -A $tempLogFolder | grep *.log)" ]; then
mkdir -p $logFolder
mv $tempLogFolder/*.log $logFolder/
fi
Don't parse the output of ls. You don't need any external commands to count files in a directory. The shell can do it itself.
Try creating a helper function that checks how many arguments are passed to it. Then have the shell expand the glob "$tempLogFolder"/*.log. No need for ls or grep. The only trick is enabling the nullglob option so if no files exist the glob expands to nothing.
files_exist() { (($# > 0)); }
shopt -s nullglob
if files_exist "$tempLogFolder"/*.log; then
mkdir -p "$logFolder"
mv "$tempLogFolder"/*.log "$logFolder"/
fi
grep accepts regex, not glob. If you want to use glob like *.log you put it in your ls something like ls .... *.log (using ls output is not 100% safe).
If you want to use grep, please use regex, I guess what you meant is \.log$.
If you want to check if grep has found matches, you should check the return code ($?) of grep, instead of using -z test.
If you put it in crontab, the script will always be executed.
I think you want,
if [ -n "$(ls -A $tempLogFolder | grep *.log)" ]; then
...
because you are only interested in whether the string returned has a non-zero length.
The script is located here: https://github.com/docker-library/ghost/blob/master/docker-entrypoint.sh
#!/bin/bash
set -e
if [[ "$*" == npm*start* ]]; then
baseDir="$GHOST_SOURCE/content"
for dir in "$baseDir"/*/ "$baseDir"/themes/*/; do
targetDir="$GHOST_CONTENT/${dir#$baseDir/}"
mkdir -p "$targetDir"
if [ -z "$(ls -A "$targetDir")" ]; then
tar -c --one-file-system -C "$dir" . | tar xC "$targetDir"
fi
done
if [ ! -e "$GHOST_CONTENT/config.js" ]; then
sed -r '
s/127\.0\.0\.1/0.0.0.0/g;
s!path.join\(__dirname, (.)/content!path.join(process.env.GHOST_CONTENT, \1!g;
' "$GHOST_SOURCE/config.example.js" > "$GHOST_CONTENT/config.js"
fi
ln -sf "$GHOST_CONTENT/config.js" "$GHOST_SOURCE/config.js"
chown -R user "$GHOST_CONTENT"
set -- gosu user "$#"
fi
exec "$#"
From what I know, it says that if you use some variation of npm start to move some files around from $GHOST_SOURCE to $GHOST_CONTENT, do something to the config.js file, link the config file, set ownership of the content files, and then execute npm start as the user user. Otherwise, it just runs your commands normally.
The specifics are what are hard for me to understand because there are a lot of things from bash that I've never seen before. So I have a lot of questions.
for dir in "$baseDir"/*/ "$baseDir"/themes/*/; do
In the above, why do they specify both /*/ and /themes/*/? Shouldn't /*/ contain themes? Is * not a wildcard for some reason?
targetDir="$GHOST_CONTENT/${dir#$baseDir/}"
In the above, what is the point of # in the variable expansion?
tar -c --one-file-system -C "$dir" . | tar xC "$targetDir"
In the above, does this somehow save time? Why not use something like rsync? I understand the point of -C, but why -c and --one-file-system?
sed -r '
s/127\.0\.0\.1/0.0.0.0/g;
s!path.join\(__dirname, (.)/content!path.join(process.env.GHOST_CONTENT, \1!g;
' "$GHOST_SOURCE/config.example.js" > "$GHOST_CONTENT/config.js"
What does this sed command do? I know it's a replacement, but why the "$GHOST_SOURCE/config.example.js" > "$GHOST_CONTENT/config.js" as the end?
ln -sf "$GHOST_CONTENT/config.js" "$GHOST_SOURCE/config.js"
In the above, what is the point of this symlink? Why try to link them to each other if both files already exist?
set -- gosu user "$#"
In the above what does calling set with no args do?
I hope that's not too much. I felt making a separate question for each of these would be too much especially since it's all related to each other.
for dir in "$baseDir"/*/ "$baseDir"/themes/*/; do
In the above, why do they specify both /*/ and /themes/*/? Shouldn't
/*/ contain themes? Is * not a wildcard for some reason?
themes/ is in the first match, but themes/*/ is not, so you need the second entry to include the contents of themes.
targetDir="$GHOST_CONTENT/${dir#$baseDir/}"
In the above, what is the point of # in the variable expansion?
It removes the $baseDir prefix from $dir. So for example:
bash$ dir=/home/bmitch/data/docker
bash$ echo $dir
/home/bmitch/data/docker
bash$ echo ${dir#/home/bmitch}
/data/docker
tar -c --one-file-system -C "$dir" . | tar xC "$targetDir"
In the above, does this somehow save time? Why not use something like
rsync? I understand the point of -C, but why -c and --one-file-system?
rsync may not be installed on every machine by default, tar is fairly universal. The -c is to create, vs extract, and --one-file-system avoids tar continuing to an outside mount point (nfs, symlink to root, etc).
sed -r '
s/127\.0\.0\.1/0.0.0.0/g;
s!path.join\(__dirname, (.)/content!path.join(process.env.GHOST_CONTENT, \1!g;
' "$GHOST_SOURCE/config.example.js" > "$GHOST_CONTENT/config.js"
What does this sed command do? I know it's a replacement, but why the
"$GHOST_SOURCE/config.example.js" > "$GHOST_CONTENT/config.js" as the
end?
config.example.js is the input (last arg to the sed), config.js is the output (after the >). So it takes the config.example.js, change the ip address from 127.0.0.1 to 0.0.0.0, effectively listening on all interfaces/ip's instead of just internally on the loopback. The second half of the sed is changing the path.join arguments from __dirname to process.env.GHOST_CONTENT.
ln -sf "$GHOST_CONTENT/config.js" "$GHOST_SOURCE/config.js"
In the above, what is the point of this symlink? Why try to link them
to each other if both files already exist?
The $GHOST_SOURCE/config.js is replaced (-f) with a link to $GHOST_CONTENT/config.js. Symbolic links give a file name reference to another actual file, so there will be two names, but one copy of the data, which means you will only have a single configuration in this situation.
set -- gosu user "$#"
In the above what does calling set with no args do?
This changes the values of $1, $2, ... $n to be $1=gosu, $2=user, $3=the old $1, $4=the old $2..., essentially adding the gosu and user to the beginning of the passed parameters to the script. The -- makes sure that set doesn't interpret any values from $# as a flag for itself.
I have an objective in my makefile named "cambios" that makes a cvs commit on each file of the project (by separate) and shows the last revision.
Now, I have an auxiliar shellscript that do that, but I'd like to know how I can do it in the makefile. I've created the objective cambios2 that do the same without the auxiliar shellscript, but it has some syntax problems.
makefile:
(...)
TODO= makefile cambiosaux.sh lib/libreria.cc include/libreria.h src/principal.cc
(...)
cambios:
#./cambiosaux.sh "$(TODO)"
cambios2:
#for dir in $(TODO); do \
A = $(cvs commit -m "Incorporando cambios automáticamente." $$dir) \
ifneq ($(A),)
echo $dir ; \
echo "Última revisión:"$(echo $(A) | sed 's/.*new revision: //' | sed 's/;.*//') ; \
endif ; \
done
cambiosaux.sh :
for dir in $1
do
A=$(cvs commit -m "Incorporando cambios automáticamente." $dir)
if [ "$A" != "" ]; then
echo $dir
echo "Última revisión:"$(echo $A | sed 's/.*new revision: //' | sed 's/;.*//')
fi
done
There are some syntax problems in the objective cambios2, but I'm really new on doing makefiles and I really don't know how to solve that problems.
Thanks!
You forgot to escape dollars that are parts of Bash command command substitution, and Make tries to perform variable expansion: $(cvs commit ...), $(echo $(A) ...).
Also you can't assign a Make variable inside a recipe. A = $(cvs commit ...) is illegal, it won't be treated neither as Make assignment nor as Bash. Try to run make with --warn-undefined-variables, I guess it will say lots of interesting details.
Finally ifneq conditional is part of Make language, and it gets interpreted at the very early stage of reading Makefile. Thus you must not indent ifneq and endif with tabs. How Make reads a Makefile chapter gives a good explanation.
To conclude, I would recommend you to leave a separate sh as is and just invoke it from your Makefile. It is not good practice to mix code in two different languages.
Okay, I found the way it works:
CVS: $(TODO)
#for dir in $?; do \
echo $$dir ; \
echo "Última revisión:" $$(cvs commit -m "Incorporando cambios automáticamente." $$dir | grep "new revision" | sed 's/.*new revision: //' | sed 's/;.*//') ; \
done
cambios: CVS