I want to run this cmd line script
$ script.sh lib/* ../test_git_thing
I want it to process all the files in the /lib folder.
FILES=$1
for f in $FILES
do
echo "Processing $f file..."
done
Currently it only prints the first file. If I use $#, it gives me all the files, but also the last param which I don't want. Any thoughts?
The argument list is being expanded at the command line when you invoke "script.sh lib/*" your script is being called with all the files in lib/ as args. Since you only reference $1 in your script, it's only printing the first file. You need to escape the wildcard on the command line so it's passed to your script to perform the globbing.
As correctly noted, lib/* on the command line is being expanded into all files in lib. To prevent expansion, you have 2 options. (1) quote your input:
$ script.sh 'lib/*' ../test_git_thing
Or (2), turn file globbing off. However, the option set -f will disable pathname expansion within the shell, but it will disable all pathname expansion (setting it within the script doesn't help as expansion is done by the shell before passing arguments to your script). In your case, it is probably better to quote the input or pass the first arguments as a directory name, and add the expansion in the script:
DIR=$1
for f in "$DIR"/*
In bash and ksh you can iterate through all arguments except the last like this:
for f in "${#:1:$#-1}"; do
echo "$f"
done
In zsh, you can do something similar:
for f in $#[1,${#}-1]; do
echo "$f"
done
$# is the number of arguments and ${#:start:length} is substring/subsequence notation in bash and ksh, while $#[start,end] is subsequence in zsh. In all cases, the subscript expressions are evaluated as arithmetic expressions, which is why $#-1 works. (In zsh, you need ${#}-1 because $#- is interpreted as "the length of $-".)
In all three shells, you can use the ${x:start:length} syntax with a scalar variable, to extract a substring; in bash and ksh, you can use ${a[#]:start:length} with an array to extract a subsequence of values.
This answers the question as given, without using non-POSIX features, and without workarounds such as disabling globbing.
You can find the last argument using a loop, and then exclude that when processing the list of files. In this example, $d is the directory name, while $f has the same meaning as in the original answer:
#!/bin/sh
if [ $# != 0 ]
then
for d in "$#"; do :; done
if [ -d "$d" ]
then
for f in "$#"
do
if [ "x$f" != "x$d" ]
then
echo "Processing $f file..."
fi
done
fi
fi
Additionally, it would be a good idea to also test if "$f" is a file, since it is common for shells to pass the wildcard character through the argument list if no match is found.
Related
So, lets say I have 6 files that are all the same type. In my specific case all of them are zip files and I want to select all of them and "pass them through" a shell script that "unzips" all of them.
I can already do it selecting one by one as the script simply does:
#!/bin/bash
DIR=$(dirname "$#")
exec unzip "$#" -d "${DIR}"
So it unzips the "zip file" exactly where I have it.
Now, when I select multiple files (aka more than one file). I don't know what happens as I don't fully understand what is "parsed" into the script.
I found this What does $# mean in a shell script?
So I would like to know how to do it right.
Thanks a lot.
Fixing Your Script: Iterating Over Arguments
If you're calling commands (like unzip) that only take one argument (of the type you want to pass) at a time, then you need to iterate over them. That is:
#!/bin/bash
for arg in "$#"; do # or just "for arg do"
dir=$(dirname "$arg")
unzip "$arg" -d "$dir"
done
Literal Answer (What The Original Syntax Did)
"$#" expands to the complete list of positional arguments. What does that mean in practice?
Let's say your code were called with:
./yourscript "Directory One/file1.zip" "Directory Two/file2.zip"
In this case, you would have:
# this is what your code would try to do (it's an error!)
DIR=$(dirname "Directory One/file1.zip" "Directory Two/file2.zip")
...followed by:
# also doesn't work, since "unzip" only takes a single zipfile argument
# ...and because the above dirname fails, DIR is empty here
unzip "Directory One/file1.zip" "Directory Two/file2.zip" -d "$DIR"
Quoting from the official manual:
# - Expands to the positional parameters, starting from one. When the expansion occurs within double quotes, each parameter expands to a separate word. That is, "$#" is equivalent to "$1" "$2" .... If the double-quoted expansion occurs within a word, the expansion of the first parameter is joined with the beginning part of the original word, and the expansion of the last parameter is joined with the last part of the original word. When there are no positional parameters, "$#" and $# expand to nothing (i.e., they are removed).
Is it possible to use basename and readlink in a one line? Something like:
ln -s /usr/local/src symlink
echo `basename <(readlink -f "./symlink")`
except that script above prints 63 instead of src.
Use the command substitution instead of the process substitution:
echo "$(basename "$(readlink -f "./symlink")")"
or, if that's your complete line, echo is redundant:
basename "$(readlink -f "./symlink")"
Multiple $(..) command substitutions can be nested without any escaping or quoting needed (unlike with the old-style backquote version). Also note that if the substitution appears within double quotes, word splitting and filename expansion are not performed on the results.
To clarify the difference: when you say <(cmd), the cmd is executed and the result is made available in a file, handle to which is returned, something like /dev/fd/63. Since the basename acts on a filename given, not its contents, it returns 63.
Unlike process substitution, the $(cmd) will execute the cmd and return the result of command (its standard output). You can then store it in a variable, like res=$(cmd), or reuse it in-place, like cmd "$(cmd)".
I want to run this cmd line script
$ script.sh lib/* ../test_git_thing
I want it to process all the files in the /lib folder.
FILES=$1
for f in $FILES
do
echo "Processing $f file..."
done
Currently it only prints the first file. If I use $#, it gives me all the files, but also the last param which I don't want. Any thoughts?
The argument list is being expanded at the command line when you invoke "script.sh lib/*" your script is being called with all the files in lib/ as args. Since you only reference $1 in your script, it's only printing the first file. You need to escape the wildcard on the command line so it's passed to your script to perform the globbing.
As correctly noted, lib/* on the command line is being expanded into all files in lib. To prevent expansion, you have 2 options. (1) quote your input:
$ script.sh 'lib/*' ../test_git_thing
Or (2), turn file globbing off. However, the option set -f will disable pathname expansion within the shell, but it will disable all pathname expansion (setting it within the script doesn't help as expansion is done by the shell before passing arguments to your script). In your case, it is probably better to quote the input or pass the first arguments as a directory name, and add the expansion in the script:
DIR=$1
for f in "$DIR"/*
In bash and ksh you can iterate through all arguments except the last like this:
for f in "${#:1:$#-1}"; do
echo "$f"
done
In zsh, you can do something similar:
for f in $#[1,${#}-1]; do
echo "$f"
done
$# is the number of arguments and ${#:start:length} is substring/subsequence notation in bash and ksh, while $#[start,end] is subsequence in zsh. In all cases, the subscript expressions are evaluated as arithmetic expressions, which is why $#-1 works. (In zsh, you need ${#}-1 because $#- is interpreted as "the length of $-".)
In all three shells, you can use the ${x:start:length} syntax with a scalar variable, to extract a substring; in bash and ksh, you can use ${a[#]:start:length} with an array to extract a subsequence of values.
This answers the question as given, without using non-POSIX features, and without workarounds such as disabling globbing.
You can find the last argument using a loop, and then exclude that when processing the list of files. In this example, $d is the directory name, while $f has the same meaning as in the original answer:
#!/bin/sh
if [ $# != 0 ]
then
for d in "$#"; do :; done
if [ -d "$d" ]
then
for f in "$#"
do
if [ "x$f" != "x$d" ]
then
echo "Processing $f file..."
fi
done
fi
fi
Additionally, it would be a good idea to also test if "$f" is a file, since it is common for shells to pass the wildcard character through the argument list if no match is found.
I would like to run recursively myscript.sh, to execute all files in the directory:
It has been discussed here that I could do like this:
#!/bin/bash
for file in * ; do
echo $file
done
But I would like myscript.sh to execute with this syntax, so that I could select only certain filetypes to be executed:
./myscript.sh *.dat
Thus I modify the script above:
#!/bin/bash
for file in $1 ; do
echo $file
done
In which when executing, it only executes first occurrence, not all files with *.dat extensions.
What is wrong here?
The wildcard *.dat is expanded by the shell before your script ever sees it. So the filenames show up in your script as $1, $2, $3, etc.
You can work with them all at once by using the special $# variable:
for file in "$#"; do
echo $file
done
Note that the double quotes around "$#" is special. From man bash:
# Expands to the positional parameters, starting from one. When the expansion
occurs within double quotes, each parameter expands to a separate word. That
is, "$#" is equivalent to "$1" "$2" ... If the double-quoted expansion occurs
within a word, the expansion of the first parameter is joined with the begin-
ning part of the original word, and the expansion of the last parameter is
joined with the last part of the original word. When there are no positional
parameters, "$#" and $# expand to nothing (i.e., they are removed).
You could do the same thing with just: ls *.dat | xargs echo
You need to get the actual contents of *, like so:
for file in $* ; do
echo $file
done
I am trying achieve the same effect as typing
mv ./images/*.{pdf,eps,jpg,svg} ./images/junk/
at the command line, from inside a bash script. I have:
MYDIR="./images"
OTHERDIR="./images/junk"
SUFFIXES='{pdf,eps,jpg,svg}'
mv "$MYDIR/"*.$SUFFIXES "$OTHERDIR/"
which, when run, gives the not unexpected error:
mv: rename ./images/*.{pdf,eps,jpg,svg} to ./images/junk/*.{pdf,eps,jpg,svg}:
No such file or directory
What is the correct way to quote all this so that mv will actually do the desired expansion? (Yes, there are plenty of files that match the pattern in ./images/.)
A deleted answer was on the right track. A slight modification to your attempt:
shopt -s extglob
MYDIR="./images"
OTHERDIR="./images/junk"
SUFFIXES='#(pdf|eps|jpg|svg)'
mv "$MYDIR/"*.$SUFFIXES "$OTHERDIR/"
Brace expansion is done before variable expansion, but variable expansion is done before pathname expansion. So the braces are still braces when the variable is expanded in your original, but when the variable instead contains pathname elements, they have already been expanded when the pathname expansion gets done.
You'll need to eval that line in order for it to work, like so:
MYDIR="./images"
OTHERDIR="./images/junk"
SUFFIXES='{pdf,eps,jpg,svg}'
eval "mv \"$MYDIR\"/*.$SUFFIXES \"$OTHERDIR/\""
Now, this has problems, in particular, if you don't trust $SUFFIXES, it might contain an injection attack, but for this simple case it should be alright.
If you are open to other solutions, you might want to experiment with find and xargs.
You can write a function:
function expand { for arg in "$#"; do [[ -f $arg ]] && echo $arg; done }
then call it with what you want to expand:
expand "$MYDIR/"*.$SUFFIXES
You can also make it a script expand.sh if you like.