While writing a bash script to help creating polaroid thumbnail using Imagick's convert commmand. I encounter a problem. Although, I manage to work around with this (actually, because convert is flexible enough), I still want to know how to solve this without such specific workaround.
So basically, the bash script will get a caption value which may contain space. I want to use that caption as parameter of convert. If the caption is empty (''), I will not use the option '-caption' for convert command. Like this:
CAPTION="Is this Cute?" # The actual value will be tacked from the parameter of this bash.
IN_FILE="resources/puppy.png"
OUTFILE="resources/puppy_polaroid.png"
# If CAPTION is not empty, reformat CAPTION
if [ "$CAPTION" != "" ]; then CAPTION="-caption \"$CAPTION\""; fi
# otherwise, do not use '-caption' add all
COMMAND="convert $CAPTION \"$IN_FILE\" \"$OUTFILE\""
echo "Command: $COMMAND" #This echo a value command
`$COMMAND`
The echo echoes the value command that can be copied can pasted in a terminal and run. BUT the bash does not run. How I can do this?
NOTE: In case of convert, -caption "" do the job. I know this and currently use this as work around.
Thanks in advance for helps.
EDIT: From the answer, here is the code that work for me now.
... # Get CAPTION and GRAVITY from parameters
if [ "$CAPTION" != "" ]; then ARGS_CAPTION=(-caption "$CAPTION"); fi
if [ "$GRAVITY" != "" ]; then ARGS_GRAVITY=(-gravity "$GRAVITY"); fi
if [ ! -f "$IN_FILE" ]; then echo "The input file does not exist: '$IN_FILE'"; exit; fi
if [ "$OUTFILE" == "" ]; then OUTFILE=${IN_FILE%.*}-${IN_FILE#*.}-polaroid.png; fi
ARGS=("${ARGS_CAPTION[#]}" -thumbnail 480x480 -border 5x5 -pointsize 60 "${ARGS_GRAVITY[#]}" +polaroid -thumbnail 120x120)
echo convert "${ARGS[#]}" "$IN_FILE" "$OUTFILE";
convert "${ARGS[#]}" "$IN_FILE" "$OUTFILE"
I hope that this will be useful for those seeking similar solution.
You'll want to read entry 050 in the BASH FAQ:
I'm trying to put a command in a variable, but the complex cases always fail!
Variables hold data. Functions hold code. Don't put code inside variables! There are many situations in which people try to shove commands, or command arguments, into variables and then run them. Each case needs to be handled separately.
...
I'm constructing a command based on information that is only known at run time
The root of the issue described above is that you need a way to maintain each argument as a separate word, even if that argument contains spaces. Quotes won't do it, but an array will. (We saw a bit of this in the previous section, where we constructed the addrs array on the fly.)
If you need to create a command dynamically, put each argument in a separate element of an array. A shell with arrays (like Bash) makes this much easier. POSIX sh has no arrays, so the closest you can come is to build up a list of elements in the positional parameters. Here's a POSIX sh version of the sendto function from the previous section:
Use an array, as so:
#!/bin/bash
# ^^^ - note the shebang line explicitly using bash, not /bin/sh
CAPTION="Is this Cute?" # The actual value will be tacked from the parameter of this bash.
IN_FILE="resources/puppy.png"
OUTFILE="resources/puppy_polaroid.png"
extra_args=( )
if [[ $CAPTION ]] ; then
extra_args+=( -caption "$1" )
fi
convert "${extra_args[#]}" "$INFILE" "$OUTFILE"
This construct presumes that you're potentially going to be appending numerous extra arguments. Note that += is unsupported in some older versions of bash which are still present on systems deployed in the field (most notably RHEL4). For such older releases it can be necessary to write extra_args=( "${extra_args[#]}" -caption "$1" ) to append to an array.
Putting backticks around $COMMAND on the last line causes the script to try to execute the output of the command rather than the command itself.
$ c='echo hi'
$ `$c`
hi: command not found
This will work:
if [[ "$CAPTION" != "" ]]
then
convert -caption "$CAPTION" "$IN_FILE" "$OUTFILE"
else
convert "$IN_FILE" "$OUTFILE"
fi
CAPTION="$1"
IN_FILE="resources/puppy.png"
OUTFILE="resources/puppy_polaroid.png"
case "$CAPTION" in
"" ) CAPTION="-caption ''";;
* ) CAPTION='-caption "$CAPTION"';;
esac
convert $CAPTION "$IN_FILE" "$OUTFILE"
Related
I need to verify that all images mentioned in a csv are present inside a folder. I wrote a small shell script for that
#!/bin/zsh
red='\033[0;31m'
color_Off='\033[0m'
csvfile=$1
imgpath=$2
cat $csvfile | while IFS=, read -r filename rurl
do
if [ -f "${imgpath}/${filename}" ]
then
echo -n
else
echo -e "$filename ${red}MISSING${color_Off}"
fi
done
My CSV looks something like
Image1.jpg,detail-1
Image2.jpg,detail-1
Image3.jpg,detail-1
The csv was created by excel.
Now all 3 images are present in imgpath but for some reason my output says
Image1.jpg MISSING
Upon using zsh -x to run the script i found that my CSV file has a BOM at the very beginning making the image name as \ufeffImage1.jpg which is causing the whole issue.
How can I ignore a BOM(byte-order marker) in a while read operation?
zsh provides a parameter expansion (also available in POSIX shells) to remove a prefix: ${var#prefix} will expand to $var with prefix removed from the front of the string.
zsh also, like ksh93 and bash, supports ANSI C-like string syntax: $'\ufeff' refers to the Unicode sequence for a BOM.
Combining these, one can refer to ${filename#$'\ufeff'} to refer to the content of $filename but with the Unicode sequence for a BOM removed if it's present at the front.
The below also makes some changes for better performance, more reliable behavior with odd filenames, and compatibility with non-zsh shells.
#!/bin/zsh
red='\033[0;31m'
color_Off='\033[0m'
csvfile=$1
imgpath=$2
while IFS=, read -r filename rurl; do
filename=${filename#$'\ufeff'}
if ! [ -f "${imgpath}/${filename}" ]; then
printf '%s %bMISSING%b\n' "$filename" "$red" "$color_Off"
fi
done <"$csvfile"
Notes on changes unrelated to the specific fix:
Replacing echo -e with printf lets us pick which specific variables get escape sequences expanded: %s for filenames means backslashes and other escapes in them are unmodified, whereas %b for $red and $color_Off ensures that we do process highlighting for them.
Replacing cat $csvfile | with < "$csvfile" avoids the overhead of starting up a separate cat process, and ensures that your while read loop is run in the same shell as the rest of your script rather than a subshell (which may or may not be an issue for zsh, but is a problem with bash when run without the non-default lastpipe flag).
echo -n isn't reliable as a noop: some shells print -n as output, and the POSIX echo standard, by marking behavior when -n is present as undefined, permits this. If you need a noop, : or true is a better choice; but in this case we can just invert the test and move the else path into the truth path.
I have 2 bash scripts. One is calling another.
Caller.sh
arg1="+hcpu_extra=111 bbb"
str="-y +hcpu_extra=111 bbb"
local cmd_re="(-y)(.*)"
if [[ $str =~ $cmd_re ]]
then
opt=${BASH_REMATCH[1]}
arg=${BASH_REMATCH[2]}
echo "matched $opt"
echo "matched $arg"
fi
./callee.sh -y $arg
## ./callee.sh -y $arg1
I found if I print $arg1 and $arg, they show the same value "+hcpu_extra=111 bbb" on the screen. But when I pass them respectively to the callee.sh as the argument. I got different results . So my question is , what is the difference between $arg and $arg1 from bash interpreter's point of view? .
First, the code won't work right as posted, because local can't be used except in a function.
If you remove the local or put this in a function, the only difference between arg and arg1 is that arg starts with a space (the one that was between "-y" and "+hcpu". but since you're expanding those variables without double-quotes around them, that'll be removed... unless you changed IFS to something that doesn't contain a space.
(BTW, variable references without double-quotes and changes to IFS are both things that can have weird effects, and are best avoided when possible.)
Anyway, my summary is: the posted code doesn't show the effect you've described; you appear to have left out something important. See How to create a Minimal, Complete, and Verifiable example.
I am trying to write a bash script that will do the following:
Take a directory or file as input (will always begin with /mnt/user/)
Search other mount points for same file or directory (will always begin with /mnt/diskx)
Return value
So, for example, the input will be "/mnt/user/my_files/file.txt". It will search if ""/mnt/disk1/my_files/file.txt" exists and will incrementally look for each disk (disk2, disk3, etc) until it finds it or disk20.
This is what I have so far:
#/user/bin/bash
var=$1
i=0
while [ -e $check_var = echo $var | sed 's:/mnt/user:/mnt/disk$i+1:']
do
final=$check_var
done
It's incomplete yes, but I am not that proficient in bash so I'm doing a little at a time. I'm sure my command won't work properly yet either but right now I am getting an "unexpected end of file" and I can't figure out why.
There are many issues here:
If this is the actual code you're getting "unexpected end of file" on, you should save the file in Unix format, not DOS format.
The shebang should be #!/usr/bin/bash or #!/bin/bash depending on your system
You have to assign check_var before running [ .. ] on it.
You have to use $(..) to expand a command
Variables like $i are not expanded in single quotes
sed can't add numbers
i is never incremented
the loop logic is inverted, it should loop until it matches and not while it matches.
You'd want to assign final after -- not in -- the loop.
Consider doing it in even smaller pieces, it's easier to debug e.g. the single statement sed 's:/mnt/user:/mnt/disk$i+1:' than your entire while loop.
Here's a more canonical way of doing it:
#!/bin/bash
var="${1#/mnt/user/}"
for file in /mnt/disk{1..20}/"$var"
do
[[ -e "$file" ]] && final="$file" && break
done
if [[ $final ]]
then
echo "It exists at $final"
else
echo "It doesn't exist anywhere"
fi
Using bash scripting, I am looking to try to search for a file based upon a path, however I would like to search from the bottom of the path up. something like /path/to/directory/here and then search "here" for a file ".important" , then go up to "directory" and search for ".important" and so forth up the tree. I don't want to recurse downward an any point in the path.
Thanks
Easy enough once you understand string manipulation in bash.
dest=/path/to/directory/here
curr=
# quote right-hand side to prevent interpretation as glob-style pattern
while [[ $curr != "$dest" ]]; do
if [[ -e $curr/.important ]]; then
printf 'Found ' >&2
printf '%s\n' "$curr/.important"
else
printf '%s\n' "Not found at $curr" >&2
fi
rest=${dest#$curr/} # strip $curr/ from $dest to get $rest
next=${rest%%/*} # strip anything after the first / from next
[[ $next ]] || break # break if next is empty
curr=$curr/$next # otherwise, add next to curr and recur
done
See http://wiki.bash-hackers.org/syntax/pe for more on the string expansion syntax used here.
Alternately:
( set -f; cd /; IFS=/; for dir in $dest; do
cd "$dir" || break
if [ -e .important ]; then
pwd
break
fi
done )
Key points:
set -f disables globbing; otherwise, this will behave very badly for a directory named *.
IFS=/ sets string-splitting on expansion to operate on /.
for dir in $dest is only safe after the two above operations have been done.
breaking if cd fails is essential to ensure that your script is actually in the directory that it thinks it's in.
Note that this is done in a subshell (per the parenthesis) to prevent its changes to shell settings (the set -f and IFS=) from impacting the larger script. This means you can use it in $() and read its output via stdout into a shell variable, but that you can't set a variable inside it and expect that variable to still be set in the parent script.
I have written a small bash script called "isinFile.sh" for checking if the first term given to the script can be found in the file "file.txt":
#!/bin/bash
FILE="file.txt"
if [ `grep -w "$1" $FILE` ]; then
echo "true"
else
echo "false"
fi
However, running the script like
> ./isinFile.sh -x
breaks the script, since -x is interpreted by grep as an option.
So I improved my script
#!/bin/bash
FILE="file.txt"
if [ `grep -w -- "$1" $FILE` ]; then
echo "true"
else
echo "false"
fi
using -- as an argument to grep. Now running
> ./isinFile.sh -x
false
works. But is using -- the correct and only way to prevent code/option injection in bash scripts? I have not seen it in the wild, only found it mentioned in ABASH: Finding Bugs in Bash Scripts.
grep -w -- ...
prevents that interpretation in what follows --
EDIT
(I did not read the last part sorry). Yes, it is the only way. The other way is to avoid it as first part of the search; e.g. ".{0}-x" works too but it is odd., so e.g.
grep -w ".{0}$1" ...
should work too.
There's actually another code injection (or whatever you want to call it) bug in this script: it simply hands the output of grep to the [ (aka test) command, and assumes that'll return true if it's not empty. But if the output is more than one "word" long, [ will treat it as an expression and try to evaluate it. For example, suppose the file contains the line 0 -eq 2 and you search for "0" -- [ will decide that 0 is not equal to 2, and the script will print false despite the fact that it found a match.
The best way to fix this is to use Ignacio Vazquez-Abrams' suggestion (as clarified by Dennis Williamson) -- this completely avoids the parsing problem, and is also faster (since -q makes grep stop searching at the first match). If that option weren't available, another method would be to protect the output with double-quotes: if [ "$(grep -w -- "$1" "$FILE")" ]; then (note that I also used $() instead of backquotes 'cause I find them much easier to read, and quotes around $FILE just in case it contains anything funny, like whitespace).
Though not applicable in this particular case, another technique can be used to prevent filenames that start with hyphens from being interpreted as options:
rm ./-x
or
rm /path/to/-x