Am trying to print a bunch of strings in a script (in zsh) and it doesn't seem to work. The code would work if I place the array in a variable and use it instead. Any ideas why this doesn't work otherwise?
for string in (some random strings to print) ; echo $string
The default form of the for command in zsh does not use parentheses (if there are any they are not interpreted as part of the for statement):
for string in some random strings to show
do
echo _$string
done
This results in the following output:
_some
_random
_strings
_to
_show
So, echo _$string was run for each word after in. The list ends with the newline.
It is possible to write the whole statement in a single line:
for string in some random strings to show; do echo _$string; done
As usual when putting multiple shell commands in the same line, newlines just need to be replaced by ;. The exception here is the newline after do; while zsh allows a ; to be placed after do, it is usually not done, and in bash it would be a syntax error.
There are also several short forms available for for, all of which are equivalent to the default form above and produce the same output:
for single commands (to be exact: single pipelines or multiple pipelines linked with && or ||, where a pipeline can also be just a single command), there are two options:
the default form, just without do or done:
for string in some random strings to show ; echo _$string
without in but with parentheses, also without do or done
for string (some random strings to show) ; echo _$string
for a list of commands (like in the default form), foreach instead of for, no in, with parentheses and terminated by end:
foreach string (some random strings to show) echo _$string ; end
In your case, you mixed the two short forms for single commands. Due to the presence of in, zsh did not take the parentheses as a syntactic element of the for command. Instead they are interpreted as a glob qualifier. Aside from the fact that you did not intend any filename expansions, this fails for two reasons:
there is no pattern (with or without actual globs) before the glob qualifier. So any matching filename would have to exactly match an empty string, which is just not possible
but mainly "some random strings to print" is not a valid glob qualifier. You probably get an error like "zsh: unknown file attribute: i" (at least with zsh 5.0.5, it may depend on the zsh version).
Check the zsh forloop documentation:
for x (1 2 3); do echo $x; done
for x in 1 2 3; do echo $x; done
You are probably trying to do this:
for string in some random strings to print ;do
echo $string
done
Related
I'm trying to figure out what I thought would be a trivial issue in BASH, but I'm having difficulty finding the correct syntax. I want to loop over an array of values, one of them being an asterisk (*), I do not wish to have any wildcard expansion happening during the process.
WHITELIST_DOMAINS="* *.foo.com *.bar.com"
for domain in $WHITELIST_DOMAINS
do
echo "$domain"
done
I have the above, and I'm trying to get the following output:
*
*.foo.com
*.bar.com
Instead of the above, I get a directory listing on the current directory, followed by *.foo.com and *.bar.com
I know I need some escaping or quoting somewhere.. the early morning haze is still thick on my brain.
I've reviewed these questions:
How to escape wildcard expansion in a variable in bash?
Stop shell wildcard character expansion?
Your problem is that you want an array, but you wrote a single string that contains the elements with spaces between them. Use an array instead.
WHITELIST_DOMAINS=('*' '*.foo.com' '*.bar.com')
Always use double quotes around variable substitutions (i.e. "$foo"), otherwise the shell splits the the value of the variable into separate words and treats each word as a filename wildcard pattern. The same goes for command substitution: "$(somecommand)". For an array variable, use "${array[#]}" to expand to the list of the elements of the array.
for domain in "${WHITELIST_DOMAINS[#]}"
do
echo "$domain"
done
For more information, see the bash FAQ about arrays.
You can use array to store them:
array=('*' '*.foo.com' '*.bar.com')
for i in "${array[#]}"
do
echo "$i"
done
For the following variable:
var="/path/to/my/document-001_extra.txt"
i need only the parts between the / [slash] and the _ [underscore].
Also, the - [dash] needs to be stripped.
In other words: document 001
This is what I have so far:
var="${var##*/}"
var="${var%_*}"
var="${var/-/ }"
which works fine, but I'm looking for a more compact substitution pattern that would spare me the triple var=...
Use of sed, awk, cut, etc. would perhaps make more sense for this, but I'm looking for a pure bash solution.
Needs to work under GNU bash, version 3.2.51(1)-release
After editing your question to talk about patterns instead of regular expressions, I'll now show you how to actually use regular expressions in bash :)
[[ $var =~ ^.*/(.*)-(.*)_ ]] && var="${BASH_REMATCH[#]:1:2}"
Parameter expansions like you were using previously unfortunately cannot be nested in bash (unless you use ill-advised eval hacks, and even then it will be less clear than the line above).
The =~ operator performs a match between the string on the left and the regular expression on the right. Parentheses in the regular expression define match groups. If a match is successful, the exit status of [[ ... ]] is zero, and so the code following the && is executed. (Reminder: don't confuse the "0=success, non-zero=failure" convention of process exit statuses with the common Boolean convention of "0=false, 1=true".)
BASH_REMATCH is an array parameter that bash sets following a successful regular-expression match. The first element of the array contains the full text matched by the regular expression; each of the following elements contains the contents of the corresponding capture group.
The ${foo[#]:x:y} parameter expansion produces y elements of the array, starting with index x. In this case, it's just a short way of writing ${BASH_REMATCH[1]} ${BASH_REMATCH[2]}. (Also, while var=${BASH_REMATCH[*]:1:2} would have worked as well, I tend to use # anyway to reinforce the fact that you almost always want to use # instead of * in other contexts.)
Both of the following should work correctly. Though the second is sensitive to misplaced characters (if you have a / or - after the last _ it will fail).
var=$(IFS=_ read s _ <<<"$var"; IFS=-; echo ${s##*/})
var=$(IFS=/-_; a=($var); echo "${a[#]:${#a[#]} - 3:2}")
I know the difference in purpose between parentheses () and curly braces {} when grouping commands in bash.
But why does the curly brace construct require a semicolon after the last command, whereas for the parentheses construct, the semicolon is optional?
$ while false; do ( echo "Hello"; echo "Goodbye"; ); done
$ while false; do ( echo "Hello"; echo "Goodbye" ); done
$ while false; do { echo "Hello"; echo "Goodbye"; }; done
$ while false; do { echo "Hello"; echo "Goodbye" }; done
bash: syntax error near unexpected token `done'
$
I'm looking for some insight as to why this is the case. I'm not looking for answers such as "because the documentation says so" or "because it was designed that way". I'd like to know why it was designed this is way. Or maybe if it is just a historical artifact?
This may be observed in at least the following versions of bash:
GNU bash, version 3.00.15(1)-release (x86_64-redhat-linux-gnu)
GNU bash, version 3.2.48(1)-release (x86_64-apple-darwin12)
GNU bash, version 4.2.25(1)-release (x86_64-pc-linux-gnu)
Because { and } are only recognized as special syntax if they are the first word in a command.
There are two important points here, both of which are found in the definitions section of the bash manual. First, is the list of metacharacters:
metacharacter
A character that, when unquoted, separates words. A metacharacter is a blank or one of the following characters: ‘|’, ‘&’, ‘;’, ‘(’, ‘)’, ‘<’, or ‘>’.
That list includes parentheses but not braces (neither curly nor square). Note that it is not a complete list of characters with special meaning to the shell, but it is a complete list of characters which separate tokens. So { and } do not separate tokens, and will only be considered tokens themselves if they are adjacent to a metacharacter, such as a space or a semi-colon.
Although braces are not metacharacters, they are treated specially by the shell in parameter expansion (eg. ${foo}) and brace expansion (eg. foo.{c,h}). Other than that, they are just normal characters. There is no problem with naming a file {ab}, for example, or }{, since those words do not conform to the syntax of either parameter expansion (which requires a $ before the {) or brace expansion (which requires at least one comma between { and }). For that matter, you could use { or } as a filename without ever having to quote the symbols. Similarly, you can call a file if, done or time without having to think about quoting the name.
These latter tokens are "reserved words":
reserved word
A word that has a special meaning to the shell. Most reserved words introduce shell flow control constructs, such as for and while.
The bash manual doesn't contain a complete list of reserved words, which is unfortunate, but they certainly include the Posix-designated:
! { }
case do done elif else
esac fi for if in
then until while
as well as the extensions implemented by bash (and some other shells):
[[ ]]
function select time
These words are not the same as built-ins (such as [), because they are actually part of the shell syntax. The built-ins could be implemented as functions or shell scripts, but reserved words cannot because they change the way that the shell parses the command line.
There is one very important feature of reserved words, which is not actually highlighted in the bash manual but is made very explicit in Posix (from which the above lists of reserved words were taken, except for time):
This recognition [as a reserved word] shall only occur when none of the characters is quoted and when the word is used as:
The first word of a command …
(The full list of places where reserved words is recognized is slightly longer, but the above is a pretty good summary.) In other words, reserved words are only reserved when they are the first word of a command. And, since { and } are reserved words, they are only special syntax if they are the first word in a command.
Example:
ls } # } is not a reserved word. It is an argument to `ls`
ls;} # } is a reserved word; `ls` has no arguments
There is lots more I could write about shell parsing, and bash parsing in particular, but it would rapidly get tedious. (For example, the rule about when # starts a comment and when it is just an ordinary character.) The approximate summary is: "don't try this at home"; really, the only thing which can parse shell commands is a shell. And don't try to make sense of it: it's just a random collection of arbitrary choices and historical anomalies, many but not all based on the need to not break ancient shell scripts with new features.
The following script is showing me "unexpected end of file" error. I have no clue why am I facing this error. My all the quotes are closed properly.
#!/usr/bin/sh
insertsql(){
#sqlite3 /mnt/rd/stats_flow_db.sqlite <<EOF
echo "insert into flow values($1,$2,$3,$4,$5,$6,$7,$8,$9,$10,$11,$12,$13,$14,$15,$16,$17,$18)"
#.quit
}
for i in {1..100}
do
src_ip = "10.1.2."+$i
echo $src_ip
src_ip_octets = ${src_ip//,/}
src_ip_int = $src_ip_octets[0]*1<<24+$src_ip_octets[1]*1<<16+$src_ip_octets[2]*1<<8+$src_ip_octets[3]
dst_ip = "10.1.1."+$i
dst_ip_octets = ${dst_ip//,/}
dst_ip_int = $dst_ip_octets[0]*1<<24+$dst_ip_octets[1]*1<<16+$dst_ip_octets[2]*1<<8+$dst_ip_octets[3]
insertsql(1, 10000, $dst_ip, 20000, $src_ip, "2012-08-02,12:30:25.0","2012-08-02,12:45:25.0",0,0,0,"flow_a010105_a010104_47173_5005_1_50183d19.rrd",0,12,$src_ip_int,$dst_ip_int,3,50000000,80000000)
done
That error is caused by <<. When encountering that, the script tries to read until it finds a line which has exactly (starting in the first column) what is found after the <<. As that is never found, the script searches to the end and then complains that the file ended unexpectedly.
That will not be your only problem, however. I see at least the following other problems:
You can only use $1 to $9 for positional parameters. If you want to go beyond that, the use of the shift command is required or, if your version of the shell supports it, use braces around the variable name; e.g. ${10}, ${11}...
Variable assignments must not have whitespace arount the equal sign
To call your insertsql you must not use ( and ); you'd define a new function that way.
The cass to your insertsql function must pass the parameters whitespace separated, not comma separated.
A couple of problems:
There should be no space between equal sign and two sides of an assignment: e.g.,: dst_ip="10.1.1.$i"
String concatenation is not done using plus sign e.g., dst_ip="10.1.1.$i"
There is no shift operator in bash, no <<: $dst_ip_octets[0]*1<<24 can be done with expr $dst_ip_octets[0] * 16777216 `
Functions are called just like shell scripts, arguments are separated by space and no parenthesis: insertsql 1 10000 ...
That is because you don't follow shell syntax.
To ser variable you are not allowed to use space around = and to concatenate two parts of string you shouldn't use +. So the string
src_ip = "10.1.2."+$i
become
src_ip="10.1.2.$i"
Why you're using the string
src_ip_octets = ${src_ip//,/}
I don't know. There is absolutely no commas in you variable. So even to delete all commas it should look like (the last / is not required in case you're just deleting symbols):
src_ip_octets=${src_ip//,}
The next string has a lot of symbols that shell intepreter at its own way and that's why you get the error about unexpected end of file (especially due to heredoc <<)
src_ip_int = $src_ip_octets[0]*1<<24+$src_ip_octets[1]*1<<16+$src_ip_octets[2]*1<<8+$src_ip_octets[3]
So I don't know what exactly did you mean, though it seems to me it should be something like
src_ip_int=$(( ${src_ip_octets%%*.}+$(echo $src_ip_octets|sed 's/[0-9]\+\.\(\[0-9]\+\)\..*/\1/')+$(echo $src_ip_octets|sed 's/\([0-9]\+\.\)\{2\}\(\[0-9]\+\)\..*/\1/')+${src_ip_octets##*.} ))
The same stuff is with the next strings.
You can't do this:
dst_ip_int = $dst_ip_octets[0]*1<<24+$dst_ip_octets[1]*1<<16+$dst_ip_octets[2]*1<<8+$dst_ip_octets[3]
The shell doesn't do math. This isn't C. If you want to do this sort of calculation, you'll need to use something like bc, dc or some other tool that can do the sort of math you're attempting here.
Most of those operators are actually shell metacharacters that mean something entirely different. For example, << is input redirection, and [ and ] are used for filename globbing.
I am trying to read part of a file and stop and a particular line, using bash. I am not very familiar with bash, but I've been reading the manual and various references, and I don't understand why something like the following does not work (but instead produces a syntax error):
while { read -u 4 line } && (test "$line" != "$header_line")
do
echo in loop, line=$line
done
I think I could write a loop that tests a "done" variable, and then do my real tests inside the loop and set "done" appropriately, but I am curious as to 1) why the above does not work, and 2) is there some small correction that would make it work? I still fairly confused about when to use [, (, {, or ((, so perhaps some other combination would work, though I have tried several.
(Note: The "read -u 4 line" works fine when I call it above the loop. I have opened a file on file descriptor 4.)
I think what you want is more like this:
while read -u 4 line && test "$line" != "$header_line"
do
...
done
Braces (the {} characters) are used to separate variables from other parts of a string when whitespace cannot be used. For example, echo "${var}x" will print the value of the variable var followed by an x, but echo "$varx" will print the value of the variable varx.
Brackets (the [] characters) are used as a shortcut for the test program. [ is another name for test, but when test detects that it was called with [ it required a ] as its last argument. The point is clarity.
Parenthesis (the () characters) are used in a number of different situations. They generally start subshells, although not always (I'm not really certain in case #3 here):
Retrieving a single exit code from a series of processes, or a single output stream from a sequence of commands. For example, (echo "Hi" ; echo "Bye") | sed -e "s/Hi/Hello/" will print two lines, "Hello" and "Bye". It is the easiest way to get multiple echo statements to produce a single stream.
Evaluating commands as if they were variables: $(expr 1 + 1) will act like a variable, but will produce the value 2.
Performing math: $((5 * 4 / 3 + 2 % 1)) will evaluate like a variable, but will compute the result of that mathematical expression.
The && operator is a list operator - he seperates two commands and only executes when the first is true, but in this case the first is the while and he is expecting his do stuff. And then he reaches do and the while stuff is already history.
Your intention is to put it into the expression. So you put it together with (). E.g. this a solution with just a small change
while ( read -u 4 line && test "$line" != "$header_line" )