Let me first write a quick Makefile as a showcase:
#!/bin/make -f
folders := $(shell find -mindepth 1 -maxdepth 1 -type d -print)
make_dir:
#mkdir -p "test0"
pwd_test:
#cd "test0" && pwd
#pwd
pwd_all:
#for f in $(folders); do \
cd "$${f}" && pwd; \
pwd; \
cd ..; \
done
First do make make_dir and then see the different results:
➜ so make pwd_test
/data/cache/tmp/so/test0
/data/cache/tmp/so
➜ so make pwd_all
/data/cache/tmp/so/test0
/data/cache/tmp/so/test0
You see that in the for loop it is necessary to do cd ... Apparently, now there is no child process spawn for the cd X && pwd command, while that is normally the case. Is this behaviour specific to make or specific to my shell?
Make spawns a new process for each command in the rule. Since the for loop is one command you get only one process.
Take a look at Recipe Execution
Edit:
Each line in a makefile gets it own subshell. Commands that have
\ tells make that the next line should be part of the current line.
The reason the for loop get its own subshell is because make see the line as
#for f in $(folders); do cd "$${f}" && pwd; pwd; cd ..; done
MadScientist explains it fairly well. Any command that you can type in your
shell in one line will be executed by make in one subshell or process.
If you were to run this in ksh, ksh would be passed
for f in $(folders); do cd "$${f}" && pwd; pwd; cd ..; done and it would be
run in that one subshell. If ksh did not have a for loop implemented this
probably would error and make would say the command returned some error code.
Explanation of pwd_test
pwd_test:
#cd "test0" && pwd
#pwd
#cd "test0" && pwd is seen as one line so the subshell updates its current
working directory and then prints out what the current working is.
#pwd At this line make spawns a new subshell that contains the old working
directory (or the directory make was called form) and pwd prints that
directory.
Related
I'm sure this has been asked but my search has been fruitless.
I want to run 3 bash commands in order with both the second and third only running if the first succeeded.
Example:
## Setup
mkdir so_test_tmp && cd so_test_tmp
echo "This is something" > something
cd ..
## Example commands
cd so_test_tmp ??? cat nothing ??? cd .. # 0.
cd so_test_tmp ??? cat something ??? cd .. # 1.
cd does_not_exist ??? cat nothing ??? cd .. # 2.
These three commands should always end in PWD. In 0. the first cd is run, then the last. In 1. all three commands successfully run. In 2. the first command fails so the second and third are not run.
What about?
pushd .; cmd1 && cmd2 && ... cmdn; popd
pushd . saves your current dir.
Then you execute your commands; you use && so that, if one fails, the others are not executed.
popd goes back to your initial dir.
EDIT: regarding your comment below, yes, this pushd .; popd construct is quite silly; it lets you forget about how the execution of each set of commands went.
pushd .; cd so_test_tmp && cat nothing; popd; # 0.
pushd .; cd so_test_tmp && cat something; popd; # 1.
pushd .; cd does_not_exist && cat nothing; popd; # 2.
You finish at your original dir after running the three sets of commands.
Within each set of commands, whenever a command fails, it shortcuts the execution of the others behind (see they are separated by &&).
If you need to know if each set of commands succeeded or not, you can always test the result of the execution (and go to your initial dir and save it again before running the following set of commands):
pushd .;
cd so_test_tmp && cat nothing && cd .. ; # 0.
test $? -eq 0 || (popd; pushd .) ;
cd so_test_tmp && cat something && cd ..; # 1.
test $? -eq 0 || (popd; pushd .) ;
cd does_not_exist && cat nothing && cd ..; # 2.
test $? -eq 0 || (popd; pushd .) ;
Specifically for cd somewhere && somecommand && cd ..
The cd .. is only necessary because you're doing cd so_test_tmp inside your parent shell, as opposed to the subshell that's fork()ed off to then be replaced with a copy of /bin/cat.
By creating an explicit subshell with ( ... ), you can scope the cd to its contents. By using exec for the last command in the subshell, you can consume it, balancing out the performance overhead of that subshell's creation.
(cd so_test_tmp && exec cat nothing) # 0.
(cd so_test_tmp && exec cat something) # 1.
(cd does_not_exist && exec cat nothing) # 2.
Note that this applies only when the command you're running in a subdirectory doesn't change the state of the shell that started it (like setting a variable). If you need to set a variable, you might instead want something like output=$(cd /to/directory && exec some_command).
Answering the more general question
Use && to connect the first command to a group with the second and third commands, and use ; to combine those 2nd and 3rd commands, if your goal is to ensure that both 2nd and 3rd run if-and-only-if the 1st succeeds.
cd so_test_tmp && { cat nothing; cd ..; } # 0.
cd so_test_tmp && { cat something; cd ..; } # 1.
cd does_not_exist && { cat nothing; cd ..; } # 2.
Setup:
$ cd /tmp
$ mkdir so_test_tmp
$ echo "This is something" > so_test_tmp/something
Wrapping an if/then/fi around OPs current examples:
$ if cd so_test_tmp; then cat nothing; cd ..; fi ; pwd
cat: nothing: No such file or directory
/tmp
$ if cd so_test_tmp; then cat something; cd ..; fi ; pwd
This is something
/tmp
$ if cd does_not_exist; then cat something; cd ..; fi ; pwd
-bash: cd: does_not_exist: No such file or directory
/tmp
I have a small script that I use to organizes files in directories, and now I am attempting to run it on a folder or directories. MasterDir/dir1, MasterDir/dir2, etc.
Running the following code on MasterDir results in an error as the $dir is only a relative path, but I can't figure out how to get the full path of $dir in this case
for dir in */; do
echo $dir
cd $dir
cwd="$PWD"
mkdir -p "VSI"
mv -v *.vsi "$cwd/VSI"
mv -v _*_ "$cwd/VSI"
done
I'd suggest using parentheses to run the loop body in a subshell:
for dir in */; do
(
echo $dir
cd $dir
cwd="$PWD"
mkdir -p "VSI"
mv -v *.vsi "$cwd/VSI"
mv -v _*_ "$cwd/VSI"
)
done
Since that's running in a subshell, the cd command won't affect the parent shell executing the script.
The problem you are having is that after you cd "$dir" the first time, you are one directory below where you generated your list of directories with for dir in */. So the next time you call cd "$dir" it fails because you are still in the first subdirectory you cd'ed into and the next "$dir" in your list is one level above.
There are several ways to handle this. One simple one is to use pushd to change to the directory instead of cd, so you can popd and return to your original directory. (though in this case you could simply add cd .. to change back to the parent directory since you are only one-level deep)
Using pushd/popd you could do:
for dir in */; do
echo $dir
pushd "$dir" &>/dev/null || {
printf "error: failed to change to %s\n" "$dir" >&2
continue
}
cwd="$PWD"
mkdir -p "VSI" || {
printf "error: failed to create %s\n" "$cwd/VSI" >&2
continue
}
mv -v *.vsi "$cwd/VSI"
mv -v _*_ "$cwd/VSI"
popd &>/dev/null || {
printf "error: failed to return to parent dir\n" >&2
break
}
done
(note: the || tests validate the return of pushd, mkdir, popd causing the loop to either continue to the next dir or break the loop if you can't return to the original directory. Also note the &>/dev/null just suppresses the normal output of pushd/popd, and redirection of output to >&2 sends the output to stderr instead of stdout)
As mentioned in the comment, you can always use readlink -f "$dir" to generate the absolute path to "$dir" -- though it's not really needed here.
This is one of the reasons I tend to avoid using cd in shell scripts -- all relative file paths change meaning when you cd, and if you aren't very careful about that you can get into trouble. The other is that cd can fail (e.g. because of a permissions error), in which case you'd better have an error check & handler in place, or something even weirder will happen.
IMO it's much safer to just use explicit paths to refer to files in other directories. That is, instead of cd somedir; mkdir -p "VSI", use `mkdir -p "somedir/VSI". Here's a rewrite of your loop using this approach:
for dir in */; do
echo $dir
mkdir -p "${dir}/VSI"
mv -v "${dir}"/*.vsi "${dir}/VSI"
mv -v "${dir}"/_*_ "${dir}/VSI"
done
Note: the values of $dir will end with a slash, so using e.g. ${dir}/VSI will give results like "somedir//VSI". The extra slash is redundant, but shouldn't cause trouble; I prefer to use it for clarity. If it's annoying (e.g. in the output of mv -v), you can leave off the extra slash.
When I executed command make, I got an error message
Makefile:4: *** missing separator. Stop.
The command in Makefile is:
$(shell ./makejce common/jce jce)
What's wrong with it?
-------makejce---------
#!/bin/bash
FLAGS=""
local_protoc=""
dir0=`pwd`
dir=`pwd`
......
if [ $# -gt 1 ]
then
mkdir -p $2
cd $2
dir=`pwd`
cd $dir0
fi
cd $1
jce_dir=`pwd`
#sub dir
for d in `ls -d */`
do
if [ -d $d ]
then
cd $d
for f in `find . -name '*.jce'`
do
${local_protoc} ${FLAGS} --dir=${dir} $f
done
cd $jce_dir
fi
done
#current dir
for f in `ls *.jce`
do
${local_protoc} ${FLAGS} --dir=${dir} $f
done
cd $dir0
-----makefile------
......
$(shell ./makejce common/jce jce)
......
With so little info it looks extremely bizarre (why are you running all the build steps in a shell script then invoking that script with a shell makefile function? The entire point of a makefile is to manage the build steps...) but without more information I'll just answer your specific question:
The make shell function works like backticks or $(...) in shell scripts: that is it runs the command in a shell and expands to the stdout of the command.
In your makefile if you have:
$(shell echo hi)
then it runs the shell command echo hi and expands to the stdout (i.e., hi). Then make will attempt to interpret that as some makefile text, because that's where you have put the function invocation (on a line all by itself). That's a syntax error because make doesn't know what to do with the string hi.
If you want to run a shell function then either (a) redirect its output so it doesn't output anything:
$(shell ...command... >/dev/null 2>&1)
or (b) capture the output somewhere that it won't bother make, such as in a variable like this:
_dummy := $(shell ...command...)
(by using := here we ensure the shell function is evaluated when the makefile is parsed).
This is printing the current dir, not the parent:
run:
#cd ..; \
echo $(shell pwd)
I need the parent dir in a command like:
run:
#cd ..; \
docker run -it --rm -p 8080:8080 -v $(shell pwd):/go/src/hello golang bash
Remember that make works by invoking a shell and sending the recipe to the shell for execution. Make doesn't have a shell "built in", so it's not running recipes directly.
The problem is that $(shell ..) is a make function. All make variables and functions are expanded before the shell is invoked (consider: the shell doesn't know how to handle make functions).
That means that a make function like $(shell ...) is first expanded and pwd is run, which gives you the current directory that the make process is running in, then the resulting string is passed to the shell for execution. So the shell sees this:
cd ..; echo /path/to/make/dir
You never need to use the $(shell ...) function inside a recipe; the recipe is already running in a shell! Instead you want to use shell syntax inside a recipe. The one caveat to this is that you have to escape dollar signs (replacing shell $ with $$) so that make doesn't interpret them as make variables. So if you write:
run:
#cd ..; echo $$(pwd)
then make expands that string and sends this command to the shell:
cd ..; echo $(pwd)
which works as you want.
Why not use the POSIXly mandated variable PWD?
run:
#cd ..; echo $$PWD
Save a process today!
I want to translate this bash-script intro a zsh-script. Hence I have no experience with this I hope I may get help here:
bash script:
SCRIPT_PATH="${BASH_SOURCE[0]}";
if([ -h "${SCRIPT_PATH}" ]) then
while([ -h "${SCRIPT_PATH}" ]) do SCRIPT_PATH=`readlink "${SCRIPT_PATH}"`; done
fi
pushd . > /dev/null
cd `dirname ${SCRIPT_PATH}` > /dev/null
SCRIPT_PATH=`pwd`;
popd > /dev/null
What I already know is that I can use
SCRIPT_PATH="$0"; to get the path were the script is located at. But then I get errors with the "readlink" statement.
Thanks for your help
Except for BASH_SOURCE I see no changes that you need to make. But what is the purpose of the script? If you want to get directory your script is located at there is ${0:A:h} (:A will resolve all symlinks, :h will truncate last path component leaving you with a directory name):
SCRIPT_PATH="${0:A:h}"
and that’s all. Note that original script has something strange going on:
if(…) and while(…) launch … in a subshell. You do not need subshell here, it is faster to do these checks using just if … and while ….
pushd . is not needed at all. While using pushd you normally replace the cd call with it:
pushd "$(dirname $SCRIPT_PATH)" >/dev/null
SCRIPT_PATH="$(pwd)"
popd >/dev/null
cd `…` will fail if … outputs something with spaces. It is possible for a directory to contain a space. In the above example I use "$(…)", "`…`" will also work.
You do not need trailing ; in variable declarations.
There is readlink -f that will resolve all symlinks thus you may consider reducing original script to SCRIPT_PATH="$(dirname $(readlink -f "${BASH_SOURCE[0]}"))" (the behavior may change as your script seems to resolve symlinks only in last component): this is bash equivalent to ${0:A:h}.
if [ -h "$SCRIPT_PATH" ] is redundant since while body with the same condition will not be executed unless script path is a symlink.
readlink $SCRIPT_PATH will return symlink relative to the directory containing $SCRIPT_PATH. Thus original script cannot possibly used to resolve symlinks in last component.
There is no ; between if(…) and then. I am surprised bash accepts this.
All of the above statements apply both to bash and zsh.
If resolving only symlinks only in last component is essential you should write it like this:
SCRIPT_PATH="$0:a"
function ResolveLastComponent()
{
pushd "$1:h" >/dev/null
local R="$(readlink "$1")"
R="$R:a"
popd >/dev/null
echo $R
}
while test -h "$SCRIPT_PATH" ; do
SCRIPT_PATH="$(ResolveLastComponent "$SCRIPT_PATH")"
done
.
To illustrate 7th statement there is the following example:
Create directory $R/bash ($R is any directory, e.g. /tmp).
Put your script there without modifications, e.g. under name $R/bash/script_path.bash. Add line echo "$SCRIPT_PATH" at the end of it and line #!/bin/bash at the start for testing.
Make it executable: chmod +x $R/bash/script_path.bash.
Create a symlink to it: cd $R/bash && ln -s script_path.bash link.
cd $R
Launch $R/bash/1. Now you will see that your script outputs $R while it should output $R/bash like it does when you launch $R/bash/script_path.bash.