I am trying to create a "Lambda" style WHERE script.
I want lambdaWHERE to take piped input and pass it through if condition after given as arguments is met. Like xargs I use {} to represent what comes down the pipe.
I call command like:
ls -d EqAAL* | lambdaWHERE.sh -f {}/INFO_ACTIVETICK
I want the folder names passed through if they contain a file called INFO_ACTIVETICK
Here is the script:
#!/bin/sh
#set -x
ARGS=$*
while read i
do
CMD=`echo $ARGS | sed 's/{}/'$i'/g'`
if [[ $CMD ]]
then
echo $i
fi
done
But when I run it a mysterious "-n" appears...
$ ls -d EqAAL* | /q/lambdaWHERE.sh -f {}/INFO_ACTIVETICK
+ ARGS='-f {}/INFO_ACTIVETICK'
+ read i
++ echo -f '{}/INFO_ACTIVETICK'
++ sed 's/{}/EqAAL-1m/g'
+ CMD='-f EqAAL-1m/INFO_ACTIVETICK'
+ [[ -n -f EqAAL-1m/INFO_ACTIVETICK ]]
+ echo EqAAL-1m
EqAAL-1m
+ read i
How can I get the bit in the [[ ]] correct?
You were quite close. you only need to switch to the standard POSIX [ $CMD ] and it will work.
The main difference between using [[ $CMD ]] and [ $CMD ] is that the first has fewer surprises and you need not quote variables. That also means that a variable is though of as one token and cannot have a whole expression in it like you are trying. [ $CMD ] however works the same way as the original shell where [ was just a command an thus need explicit quotations in order to interpret something with spaces as one argument.
There is a relevant question about the differences between [[ ...]] and [ ..]
Related
I have a script below that sources a directory of bash scripts and then parses the flags of the command to run a specific function from the sourced files.
Given this function within the scripts dir:
function reggiEcho () {
echo $1
}
Here are some examples of current output
$ reggi --echo hello
hello
$ reggi --echo hello world
hello
$ reggi --echo "hello world"
hello
$ reggi --echo "hello" --echo "world"
hello
world
As you can see quoted parameters are not honored as they should be `"hello world" should echo properly.
This is the script, the issue is within the while loop.
How do I parse these flags, and maintain passing in quoted parameters into the function?
DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" && pwd )"
STR="$(find $DIR/scripts -type f -name '*.sh' -print)"
ARR=( $STR )
TUSAGE="\n"
for f in "${ARR[#]}"; do
if [ -f $f ]
then
. $f --source-only
if [ -z "$USAGE" ]
then
:
else
TUSAGE="$TUSAGE \t$USAGE\n"
fi
USAGE=""
else
echo "$f not found"
fi
done
TUSAGE="$TUSAGE \t--help (shows this help output)\n"
function usage() {
echo "Usage: --function <args> [--function <args>]"
echo $TUSAGE
exit 1
}
HELP=false
cmd=()
while [ $# -gt 0 ]; do # loop until no args left
if [[ $1 = '--help' ]] || [[ $1 = '-h' ]] || [[ $1 = '--h' ]] || [[ $1 = '-help' ]]; then
HELP=true
fi
if [[ $1 = --* ]] || [[ $1 = -* ]]; then # arg starts with --
if [[ ${#cmd[#]} -gt 0 ]]; then
"${cmd[#]}"
fi
top=`echo $1 | tr -d -` # remove all flags
top=`echo ${top:0:1} | tr '[a-z]' '[A-Z]'`${top:1} # make sure first letter is uppercase
top=reggi$top # prepend reggi
cmd=( "$top" ) # start new array
else
echo $1
cmd+=( "$1" )
fi
shift
done
if [[ "$HELP" = true ]]; then
usage
elif [[ ${#cmd[#]} -gt 0 ]]; then
${cmd[#]}
else
usage
fi
There are many places in this script where you have variable references without double-quotes around them. This means the variables' values will be subject to word spitting and wildcard expansion, which can have various weird effects.
The specific problem you're seeing is due to an unquoted variable reference on the fourth-from-last line, ${cmd[#]}. With cmd=( echo "hello world" ), word splitting makes this equivalent to echo hello world rather than echo "hello world".
Fixing that one line will fix your current problem, but there are a number of other unquoted variable references that may cause other problems later. I recommend fixing all of them. Cyrus' recommendation of shellcheck.net is good at pointing them out, and will also note some other issues I won't cover here. One thing it won't mention is that you should avoid all-caps variable names (DIR, TUSAGE, etc) -- there are a bunch of all-caps variables with special meanings, and it's easy to accidentally reuse one of them and wind up with weird effects. Lowercase and mixed-case variables are safer.
I also recommend against using \t and \n in strings, and counting on echo to translate them into tabs and newlines, respectively. Some versions of echo do this automatically, some require the -e option to tell them to do it, some will print "-e" as part of their output... it's a mess. In bash, you can use $'...' to translate those escape sequences directly, e.g:
tusage="$tusage"$' \t--help (shows this help output)\n' # Note mixed quoting modes
echo "$tusage" # Note that double-quoting is *required* for this to work right
You should also fix the file listing so it doesn't depend on being unquoted (see chepner's comment). If you don't need to scan subdirectories of $DIR/scripts, you can do this with a simple wildcard (note lowercase vars and that the var is double-quoted, but the wildcard isn't):
arr=( "$dir/scripts"/*.sh )
If you need to look in subdirectories, it's more complicated. If you have bash v4 you can use a globstar wildcard, like this:
shopt -s globstar
arr=( "$dir/scripts"/**/*.sh )
If your script might have to run under bash v3, see BashFAQ #20: "How can I find and safely handle file names containing newlines, spaces or both?", or just use this:
while IFS= read -r -d '' f <&3; do
if [ -f $f ]
# ... etc
done 3< <(find "$dir/scripts" -type f -name '*.sh' -print0)
(That's my favorite it-just-works idiom for iterating over find's matches. Although it does require bash, not some generic POSIX shell.)
I'd like to add an argument to a command in bash only if a variable evaluates to a certain value. For example this works:
test=1
if [ "${test}" == 1 ]; then
ls -la -R
else
ls -R
fi
The problem with this approach is that I have to duplicate ls -R both when test is 1 or if it's something else. I'd prefer if I could write this in one line instead such as this (pseudo code that doesn't work):
ls (if ${test} == 1 then -la) -R
I've tried the following but it doesn't work:
test=1
ls `if [ $test -eq 1 ]; then -la; fi` -R
This gives me the following error:
./test.sh: line 3: -la: command not found
A more idiomatic version of svlasov's answer:
ls $( (( test == 1 )) && printf %s '-la' ) -R
Since echo understands a few options itself, it's safer to use printf %s to make sure that the text to print is not mistaken for an option.
Note that the command substitution must not be quoted here - which is fine in the case at hand, but calls for a more robust approach in general - see below.
However, in general, the more robust approach is to build up arguments in an array and pass it as a whole:
# Build up array of arguments...
args=()
(( test == 1 )) && args+=( '-la' )
args+=( '-R' )
# ... and pass it to `ls`.
ls "${args[#]}"
Update: The OP asks how to conditionally add an additional, variable-based argument to yield ls -R -la "$PWD".
In that case, the array approach is a must: each argument must become its own array element, which is crucial for supporting arguments that may have embedded whitespace:
(( test == 1 )) && args+= ( '-la' "$PWD" ) # Add each argument as its own array element.
As for why your command,
ls `if [ $test -eq 1 ]; then -la; fi` -R
didn't work:
A command between backticks (or its modern, nestable equivalent, $(...)) - a so-called command substitution - is executed just like any other shell command (albeit in a sub-shell) and the whole construct is replaced with the command's stdout output.
Thus, your command tries to execute the string -la, which fails. To send it to stdout, as is needed here, you must use a command such as echo or printf.
Print the argument with echo:
test=1
ls `if [ $test -eq 1 ]; then echo "-la"; fi` -R
I can't say how acceptable this is, but:
test=1
ls ${test:+'-la'} -R
See https://stackoverflow.com/revisions/16753536/1 for a conditional truth table.
Another answer without using eval and using BASH arrays:
myls() { local arr=(ls); [[ $1 -eq 1 ]] && arr+=(-la); arr+=(-R); "${arr[#]}"; }
Use it as:
myls
myls "$test"
This script builds whole command in an array arr and preserves the original order of command options.
I have a directory with lots of config files in it, they look like this:
us-sfo-building1.foo us-sfo-building1.bar
mx-mex-building15.foo mx-mex-building15.bar
Now, I want to execute a script which takes us-sfo-building1.foo and us-sfo-building1.bar as input parameters.
I want basically this: ./script $.foo $.bar but I have to make sure that I always have the matching pair of foo and bar, otherwise the script complains. I tried this, but it did not work as expected:
#/bin/bash
for x in "*.foo*; do
x=${x%.foo}
if [ -e "$x.bar" ]; then
./script "$x.foo" "$x.bar"
fi
done
Any idea on how to solve this or where my mistake is?
for x in "*.foo*; do doesn't need a ".
Fix:
#/bin/bash
for x in *.foo; do
x=${x%.foo}
if [ -e "$x.bar" ]; then
./script "$x.foo" "$x.bar"
fi
done
And a suggestion (includes preferred use of [[ ]]):
#/bin/bash
for x in *.foo; do
y=${x%.foo}.bar
[[ -e $y ]] && ./script "$x" "$y"
done
I am trying to run the below logic
if [-f $file] && [$variable -lt 1] ; then
some logic
else
print "This is wrong"
fi
It fails with the following error
MyScipt.ksh[10]: [-f: not found
Where 10th line is the if condition , I have put in .
I have also tried
if [-f $file && $variable -lt 1] ; then
which gives the same error.
I know this is a syntax mistake somehwere , but I am not sure , what is the correct syntax when I am using multiple conditions with && in a if block
[ is not an operator, it's the name of a program (or a builtin, sometimes). Use type [ to check. Regardless, you need to put a space after it so that the command line parser knows what to do:
if [ -f $file ]
The && operator might not do what you want in this case, either. You should probably read the bash(1) documentation. In this specific case, it seems like what you want is:
if [ -f $file -a $variable -lt 1 ]
Or in more modern bash syntax:
if [[ -f $file && $variable -lt 1 ]]
The [ syntax is secretly a program!
$ type [
[ is a shell builtin
$ ls -l $(which [)
-rwxr-xr-x 1 root root 35264 Nov 19 16:25 /usr/bin/[
Because of the way the shell parses (technically "lexes") your command line, it sees this:
if - keyword
[-f - the program [-f
$file] - A string argument to the [-f program, made by the value of $file and ]. If $file was "asdf", then this would be asdf]
And so forth, down your command. What you need to do is include spaces, which the shell uses to separate the different parts (tokens) of your command:
if [ -f "$file" ]; then
Now [ stands on its own, and can be recognized as a command/program. Also, ] stands on its own as an argument to [, otherwise [ will complain. A couple more notes about this:
You don't need to put a space before or after ;, because that is a special separator that the shell recognizes.
You should always "double quote" $variables because they get expanded before the shell does the lexing. This means that if an unquoted variable contains a space, the shell will see the value as separate tokens, instead of one string.
Using && in an if-test like that isn't the usual way to do it. [ (also known as test) understands -a to mean "and," so this does what you intended:
if [ -f "$file" -a "$variable" -lt 1 ]; then
Use -a in an if block to represent AND.
Note the space preceding the -f option.
if [ -f $file -a $variable -lt 1] ; then
some logic
else
print "This is wrong"
fi
This question has 3 parts, and each alone is easy, but combined together is not trivial (at least for me) :)
Need write a script what should take as its arguments:
one name of another command
several arguments for the command
list of files
Examples:
./my_script head -100 a.txt b.txt ./xxx/*.txt
./my_script sed -n 's/xxx/aaa/' *.txt
and so on.
Inside my script for some reason I need distinguish
what is the command
what are the arguments for the command
what are the files
so probably the most standard way write the above examples is:
./my_script head -100 -- a.txt b.txt ./xxx/*.txt
./my_script sed -n 's/xxx/aaa/' -- *.txt
Question1: Is here any better solution?
Processing in ./my_script (first attempt):
command="$1";shift
args=`echo $* | sed 's/--.*//'`
filenames=`echo $* | sed 's/.*--//'`
#... some additional processing ...
"$command" "$args" $filenames #execute the command with args and files
This solution will fail when the filenames will contain spaces and/or '--', e.g.
/some--path/to/more/idiotic file name.txt
Question2: How properly get $command its $args and $filenames for the later execution?
Question3: - how to achieve the following style of execution?
echo $filenames | $command $args #but want one filename = one line (like ls -1)
Is here nice shell solution, or need to use for example perl?
First of all, it sounds like you're trying to write a script that takes a command and a list of filenames and runs the command on each filename in turn. This can be done in one line in bash:
$ for file in a.txt b.txt ./xxx/*.txt;do head -100 "$file";done
$ for file in *.txt; do sed -n 's/xxx/aaa/' "$file";done
However, maybe I've misinterpreted your intent so let me answer your questions individually.
Instead of using "--" (which already has a different meaning), the following syntax feels more natural to me:
./my_script -c "head -100" a.txt b.txt ./xxx/*.txt
./my_script -c "sed -n 's/xxx/aaa/'" *.txt
To extract the arguments in bash, use getopts:
SCRIPT=$0
while getopts "c:" opt; do
case $opt in
c)
command=$OPTARG
;;
esac
done
shift $((OPTIND-1))
if [ -z "$command" ] || [ -z "$*" ]; then
echo "Usage: $SCRIPT -c <command> file [file..]"
exit
fi
If you want to run a command for each of the remaining arguments, it would look like this:
for target in "$#";do
eval $command \"$target\"
done
If you want to read the filenames from STDIN, it would look more like this:
while read target; do
eval $command \"$target\"
done
The $# variable, when quoted will be able to group parameters as they should be:
for parameter in "$#"
do
echo "The parameter is '$parameter'"
done
If given:
head -100 test this "File name" out
Will print
the parameter is 'head'
the parameter is '-100'
the parameter is 'test'
the parameter is 'this'
the parameter is 'File name'
the parameter is 'out'
Now, all you have to do is parse the loop out. You can use some very simple rules:
The first parameter is always the file name
The parameters that follow that start with a dash are parameters
After the "--" or once one doesn't start with a "-", the rest are all file names.
You can check to see if the first character in the parameter is a dash by using this:
if [[ "x${parameter}" == "x${parameter#-}" ]]
If you haven't seen this syntax before, it's a left filter. The # divides the two parts of the variable name. The first part is the name of the variable, and the second is the glob filter (not regular expression) to cut off. In this case, it's a single dash. As long as this statement isn't true, you know you have a parameter. BTW, the x may or may not be needed in this case. When you run a test, and you have a string with a dash in it, the test might mistake it for a parameter of the test and not the value.
Put it together would be something like this:
parameterFlag=""
for parameter in "$#" #Quotes are important!
do
if [[ "x${parameter}" == "x${parameter#-}" ]]
then
parameterFlag="Tripped!"
fi
if [[ "x${parameter}" == "x--" ]]
then
print "Parameter \"$parameter\" ends the parameter list"
parameterFlag="TRIPPED!"
fi
if [ -n $parameterFlag ]
then
print "\"$parameter\" is a file"
else
echo "The parameter \"$parameter\" is a parameter"
fi
done
Question 1
I don't think so, at least not if you need to do this for arbitrary commands.
Question 3
command=$1
shift
while [ $1 != '--' ]; do
args="$args $1"
shift
done
shift
while [ -n "$1" ]; do
echo "$1"
shift
done | $command $args
Question 2
How does that differ from question 3?