I want to create a simple dialog with bash-dialog. I work with (X)DSL and bash-3.2. The latest (X)DSL is based on Linux 2.4.31 and comes with bash-2.05, however, bash-3.2 is downloadable from MyDSL/Testing. So, my script runs under '#!/bin/bash-3.2/bin/bash'.
The menu items the users can choose from come from a database.
Example database file 'armatures':
Indoor Lighting|Lighting for Indoor use
Outdoor Lighting|Lighting for Outdoor use
I retrieve the data into an array 'options' from the 'armatures' file with:
options=($(awk -F"|" '{ print $1,$2 }' armatures)
and in terminal 'echo' the array:
echo ${options[#]}
which shows:
"Indoor Armatures" "Lighting for Indoor use" "Outdoor Armatures" "Lighting for Outdoor use"
This looks OK to use as a selection menu with 'whiptail' but it isn't. The command line:
whiptail --clear --title "Armatures" --menu "Choose an armature" 50 80 10 ${options[#]}
shows:
column1-column2
Indoor-Armatures
Lighting-for
Indoor-use
Outdoor-Armatures
Lighting-for
Outdoor-use
in stead of:
column1-column2
Indoor armatures-Lighting for Indoor use
Outdoor armatures-Ligthing for Outdoor use
It seems that array elements with double quotes are ignored or not seen by 'whiptail'. I also tried "${options[#]}" but that always results on the first word 'Indoor'.
Aside from 'whiptail' I tried 'dialog' but they are the same: version information shows 'cdialog (ComeOn Dialog!) version 1.1-20080316' in both cases.
I have very limited resources and don't want to venture (yet) into 'xdialog', 'zenity', 'dzen' and the like, even if that would solve this. I am also limited to Linux 2.4.31 due to XDSL (for XBOX).
I've been browsing the Internet a lot but to no avail. What could be the solution with 'whiptail/dialog'?
The basic problem you are having comes from the order in which the shell parses command lines: it parses (and removes) quotes and escapes before it replaces variables with their values, and it never goes back and re-parses for any quotes or escapes within the replaced values. Consider this example:
var='"foo bar"' # Note that the single-quotes will be removed, and the
# double-quotes will be treated as part of the variable value.
somecmd $var # This runs cmd with 2 arguments: '"foo' and 'bar"'
In your case, I'm not sure where the double-quotes are coming from; they're not in the file listing you provided and the awk command won't add them. But in any case, you don't want them stored as part of the value, you want them around the variable reference:
var='foo bar' # Again, the single-quotes are removed.
somecmd "$var" # This runs cmd with a single argument: 'foo bar'
Your case is a little more complicated since you're using an array, but the same principle applies. Note that echoing a variable is highly misleading; if it shows what looks like proper quoting, that actually means there's something horribly wrong because it should show the arguments after quote removal.
So, how do you solve it? Try this:
options=()
while IFS="|" read col1 col2 || [ -n "$col1" ]; do
options+=("$col1" "$col2") # Note the double-quotes around the variable references
done <armatures
echo "options:"
printf " '$s'\n" "${options[#]}" # This prints each array element on a separate line
whiptail --clear --title "Armatures" --menu "Choose an armature" 50 80 10 "${options[#]}" # Again, double-quotes around the reference
UPDATE: I added a test ([ -n "$col1" ]) to execute the loop for an unterminated last line in the database file.
If the double-quotes are actually in the database file, you'll have to remove them; the easiest way to handle this probably to strip quotes while adding the strings to the array, using bash's ability to so string replacement (replacing '"' with blank) while building the array:
options=()
while IFS="|" read col1 col2 || [ -n "$col1" ]; do
options+=("${col1//'"'/}" "${col2//'"'/}")
done <armatures
The main problem is with the way bash (or any other shell) splits the command lines into "words", as mentioned in the first answer. The line splitting is done on the basis of the IFS - the Internal Field Separator - which is set to <space><tab><newline> by default. The clever chaps (Stephen Bourne etc) who developed the early shells obviously thought it was a good idea for users to be able to change it. It's also nice that it can be set to a multi-character value.
So all you really need to do is set the field separator to a newline, run the awk script, then set it back. You should also quote the array when you use it.
This should work for bash:
IFS=$'\n'
options=($(awk -F"|" '{ print $1,$2 }' armatures)
IFS=$' \t\n'
The array members are now properly defined. Then:
whiptail --clear --title "Armatures" --menu "Choose an armature" 50 80 10 "${options[#]}"
For more primitive shells like the Bourne Shell, you will probably need to set IFS by entering "IFS=" (without the quotes), opening single quotes, pressing <Enter>, then closing with a single quote. To reset it again, enter "IFS=" (without the quotes), open single quotes, enter a <space>, quote (e.g. Ctrl-V) a <tab>, hit <Enter>, then close with a single quote.
Related
The goal is to get rid of (by changing) filenames that give headaches for scripting by translating them to something else. The reason is that in this nearly 30 year Unix / Linux environment, with a lot of existing scripts that may not be "written correctly", a new, large and important cache of files arrived that have to be managed, and so, a colleague has asked me to write a script to help with "problematic filenames" and translate them. They've got a list of chars to turn into dots, such as the comma, and another list to turn into underscores, such as whitespace, as but two examples and ran into problems which I asked about over here.
I was using tr to do it, but commenters to it said I should perhaps ask just about this instead of how to get tr to work. So, I have!
Parameter expansion can do this for you.
Note that unlike when using tr (as requested on your other question), when using parameter expansion you don't need to use backslashes inside your character class definitions: put the expansion in double quotes and bash will treat the results of that expansion as literal.
#!/usr/bin/env bash
toDots='\,;:|+##$%^&*~'
toUnderscores='}{]['"'"'="()`!'
# requires bash 5+: if debug=1, then print what we would do instead of doing it
runOrDebug() {
if (( debug )); then
printf '%s\n' "${*#Q}"
else
"$#"
fi
}
renameFiles() {
local name subDots subBoth
for name; do
subDots=${name//["$toDots"]/.}
subBoth=${subDots//["$toUnderscores"]/_}
if [[ $subBoth != "$name" ]]; then
runOrDebug mv -- "$name" "$subBoth"
fi
done
}
debug=1 renameFiles '[/a],/;[p:r|o\b+lem#a#t$i%c]/#(%$^!/(e^n&t*ry)~='
Note that toUnderscores is (except for the single quote in the middle) in single quotes, so all the backslashes in it are part of the variable's data rather than being syntax; because globs use character class syntax from REs, they're parsed as POSIX regular expression character class syntax.
See a demonstration of the technique running at https://ideone.com/kKE7IJ
I'm writing a shell script intended to edit audio files using the sox command. I've been running into a strange problem I never encountered in bash scripting before: When defining space separated effects in sox, the command will work when that effect is written directly, but not when it's stored in a variable. This means the following works fine and without any issues:
sox ./test.in.wav ./test.out.wav delay 5
Yet for some reason the following will not work:
IFS=' ' # set IFS to only have a tab character because file is tab-separated
while read -r file effects text; do
sox $file.in.wav $file.out.wav $effects
done <in.txt
...when its in.txt is created with:
printf '%s\t%s\t%s\n' "test" "delay 5" "other text here" >in.txt
The error indicates this is causing it to see the output file as another input.
sox FAIL formats: can't open input file `./output.wav': No such file or directory
I tried everything I could think of: Using quotation marks (sox "$file.in.wav" "$file.out.wav" "$effects"), echoing the variable in-line (sox $file.in.wav $file.out.wav $(echo $effects)), even escaping the space inside the variable (effects="delay\ 5"). Nothing seems to work, everything produces the error. Why does one command work but not the other, what am I missing and how do I solve it?
IFS does not only change the behavior of read; it also changes the behavior of unquoted expansions.
In particular, unquoted expansions' content are split on characters found in IFS, before each element resulting from that split is expanded as a glob.
Thus, if you want the space between delay and 5 to be used for word splitting, you need to have a regular space, not just a tab, in IFS. If you move your IFS assignment to be part of the same simple command as the read, as in IFS=$'\t' read -r file effects text; do, that will stop it from changing behavior in the rest of the script.
However, it's not good practice to use unquoted expansions for word-splitting at all. Use an array instead. You can split your effects string into an array with:
IFS=' ' read -r -a effects_arr <<<"$effects"
...and then run sox "$file.in.wav" "$file.out.wav" "${effects_arr[#]}" to expand each item in the array as a separate word.
By contrast, if you need quotes/escapes/etc to be allowed in effects, see Reading quoted/escaped arguments correctly from a string
I've written a bunch of environment variables in Docker format, but now I want to use them outside of that context. How can I source them with one line of bash?
Details
Docker run and compose have a convenient facility for importing a set of environment variables from a file. That file has a very literal format.
The value is used as is and not modified at all. For example if the value is surrounded by quotes (as is often the case of shell variables), the quotes are included in the value passed
Lines beginning with # are treated as comments and are ignored
Blank lines are also ignored.
"If no = is provided and that variable is…exported in your local environment," docker "passes it to the container"
Thankfully, whitespace before the = will cause the run to fail
so, for example, this env-file:
# This is a comment, with an = sign, just to mess with us
VAR1=value1
VAR2=value2
USER
VAR3=is going to = trouble
VAR4=this $sign will mess with things
VAR5=var # with what looks like a comment
#VAR7 =would fail
VAR8= but what about this?
VAR9="and this?"
results in these env variables in the container:
user=ubuntu
VAR1=value1
VAR2=value2
VAR3=is going to = trouble
VAR4=this $sign will mess with things
VAR5=var # with what looks like a comment
VAR8= but what about this?
VAR9="and this?"
The bright side is that once I know what I'm working with, it's pretty easy to predict the effect. What I see is what I get. But I don't think bash would be able to interpret this in the same way without a lot of changes. How can I put this square Docker peg into a round Bash hole?
tl;dr:
source <(sed -E -e "s/^([^#])/export \1/" -e "s/=/='/" -e "s/(=.*)$/\1'/" env.list)
You're probably going to want to source a file, whose contents
are executed as if they were printed at the command line.
But what file? The raw docker env-file is inappropriate, because it won't export the assigned variables such that they can be used by child processes, and any of the input lines with spaces, quotes, and other special characters will have undesirable results.
Since you don't want to hand edit the file, you can use a stream editor to transform the lines to something more bash-friendly. I started out trying to solve this with one or two complex Perl 5 regular expressions, or some combination of tools, but I eventually settled on one sed command with one simple and two extended regular expressions:
sed -E -e "s/^([^#])/export \1/" -e "s/=/='/" -e "s/(=.*)$/\1'/" env.list
This does a lot.
The first expression prepends export to any line whose first character is anything but #.
As discussed, this makes the variables available to anything else you run in this session, your whole point of being here.
The second expression simply inserts a single-quote after the first = in a line, if applicable.
This will always enclose the whole value, whereas a greedy match could lop off some of (e.g.) VAR3, for example
The third expression appends a second quote to any line that has at least one =.
it's important here to match on the = again so we don't create an unmatched quotation mark
Results:
# This is a comment, with an =' sign, just to mess with us'
export VAR1='value1'
export VAR2='value2'
export USER
export VAR3='is going to = trouble'
export VAR4='this $sign will mess with things'
export VAR5='var # with what looks like a comment'
#VAR7 ='would fail'
export VAR8=' but what about this?'
export VAR9='"and this?"'
Some more details:
By wrapping the values in single-quotes, you've
prevented bash from assuming that the words after the space are a command
appropriately brought the # and all succeeding characters into the VAR5
prevented the evaluation of $sign, which, if wrapped in double-quotes, bash would have interpreted as a variable
Finally, we'll take advantage of process substitution to pass this stream as a file to source, bring all of this down to one line of bash.
source <(sed -E -e "s/^([^#])/export \1/" -e "s/=/='/" -e "s/(=.*)$/\1'/" env.list)
Et voilà!
This question already has answers here:
When to wrap quotes around a shell variable?
(5 answers)
Closed 7 years ago.
I want to create some scripts for filling some templates and inserting them into my project folder. I want to use a shell script for this, and the templates are very small so I want to embed them in the shell script. The problem is that echo seems to ignore the line breaks in my string. Either that, or the string doesn't contain line breaks to begin with. Here is an example:
MY_STRING="
Hello, world! This
Is
A
Multi lined
String."
echo -e $MY_STRING
This outputs:
Hello, world! This Is A Multi lined String.
I'm assuming echo is the culprit here. How can I get it to acknowledge the line breaks?
You need double quotes around the variable interpolation.
echo -e "$MY_STRING"
This is an all-too common error. You should get into the habit of always quoting strings, unless you specifically need to split into whitespace-separated tokens or have wildcards expanded.
So to be explicit, the shell will normalize whitespace when it parses your command line. You can see this if you write a simple C program which prints out its argv array.
argv[0]='Hello,'
argv[1]='world!'
argv[2]='This'
argv[3]='Is'
argv[4]='A'
argv[5]='Multi'
argv[6]='lined'
argv[7]='String.'
By contrast, with quoting, the whole string is in argv[0], newlines and all.
For what it's worth, also consider here documents (with cat, not echo):
cat <<"HERE"
foo
Bar
HERE
You can also interpolate a variable in a here document.
cat <<HERE
$MY_STRING
HERE
... although in this particular case, it's hardly what you want.
echo is so nineties. The new (POSIX) kid on the block is printf.
printf '%s\n' "$MY_STRING"
No -e or SYSV vs BSD echo madness and full control over what gets printed where and how wide, escape sequences like in C. Everybody please start using printf now and never look back.
Try this :
echo "$MY_STRING"
I seem to have some difficulty getting what I want to work. Basically, I have a series of variables that are assigned strings with some quotes and \ characters. I want to remove the quotes to embed them inside a json doc, since json hates quotes using python dump methods.
I figured it would be easy. Just determine how to remove the characters easy and then write a simple for loop for the variable substitution, well it didn't work that way.
Here is what I want to do.
There is a variable called "MESSAGE23", it contains the following "com.centrify.tokend.cac", I want to strip out the quotes, which to me is easy, a simple echo $opt | sed "s/\"//g". When I do this from the command line:
$> MESSAGE23="com."apple".cacng.tokend is present"
$> MESSAGE23=`echo $MESSAGE23 | sed "s/\"//g"`
$> com.apple.cacng.tokend is present
This works. I get the properly formatted string.
When I then try to throw this into a loop, all hell breaks loose.
for i to {1..25}; do
MESSAGE$i=`echo $MESSAGE$i | sed "s/\"//g"`
done
This doesn't work (either it throws a bunch of indexes out or nothing), and I'm pretty sure I just don't know enough about arg or eval or other bash substitution variables.
But basically I want to do this for another set of variables with the same problems, where I strip out the quotes and incidentally the "\" too.
Any help would be greatly appreciated.
You can't do that. You could make it work using eval, but that introduces another level of quoting you have to worry about. Is there some reason you can't use an array?
MESSAGE=("this is MESSAGE[0]" "this is MESSAGE[1]")
MESSAGE[2]="I can add more, too!"
for (( i=0; i<${#MESSAGE[#]}; ++i )); do
echo "${MESSAGE[i]}"
done
Otherwise you need something like this:
eval 'echo "$MESSAGE'"$i"'"'
and it just gets worse from there.
First, a couple of preliminary problems: MESSAGE23="com."apple".cacng.tokend is present" will not embed double-quotes in the variable value, use MESSAGE23="com.\"apple\".cacng.tokend is present" or MESSAGE23='com."apple".cacng.tokend is present' instead. Second, you should almost always put double-quotes around variable expansions (e.g. echo "$MESSAGE23") to prevent parsing oddities.
Now, the real problems: the shell doesn't allow variable substitution on the left side of an assignment (i.e. MESSAGE$i=something won't work). Fortunately, it does allow this in a declare statement, so you can use that instead. Also, when the sees $MESSAGE$i it replaces it will the value of $MESSAGE followed by the value of $i; for this you need to use indirect expansion (`${!metavariable}').
for i in {1..25}; do
varname="MESSAGE$i"
declare $varname="$(echo "${!varname}" | tr -d '"')"
done
(Note that I also used tr instead of sed, but that's just my personal preference.)
(Also, note that #Mark Reed's suggestion of an array is really the better way to do this sort of thing.)