I'm trying to write a small piece of code that passes a small formula to another program, however i've found that something strange happens when the formula starts with 11*(:
$ echo 11*15
Neatly prints '11*15'
$ echo 21*(15)
Neatly prints '21*(15)', while
echo 11*(15)
Only gives '11'. As far as I've found this only happens with '11*('. I know that this can be solved by using proper quotation marks, but I'm still curious as to why this happens.
Does anyone know?
How is your program coded? If its coded to take in parameters, then pass your formula like
./myprogram "11*15"
or
echo '11*15' | myprogram
If you do echo just like that on the command line, you may inadvertently display files that has 11 in its file name
11*(15) uses a Bash-specific extended glob syntax. You've stumbled across it accidentally, emphasizing why quotation marks are a good idea. (I also learned a lot tracking down why it was working differently for me; thanks for that.)
The behavior of
echo 11*(15)
in bash is going to vary depending on whether extglob is enabled. If it's enabled *(PATTERN-LIST) matches zero or more occurrences of the patterns. If it's disabled, it doesn't, and the resulting ( is likely to cause a syntax error.
For example:
$ ls
11 115 1155 11555 115555
$ shopt -u extglob
$ echo 11*(55)
bash: syntax error near unexpected token `('
$ shopt -s extglob
$ echo 11*(55)
11 1155 115555
$
(This explains the odd behavior I discussed in comments.)
Quoting from the bash 4.2.8 documentation (info bash):
If the `extglob' shell option is enabled using the `shopt' builtin,
several extended pattern matching operators are recognized. In the
following description, a PATTERN-LIST is a list of one or more
patterns separated by a `|'. Composite patterns may be formed using
one or more of the following sub-patterns:
`?(PATTERN-LIST)'
Matches zero or one occurrence of the given patterns.
`*(PATTERN-LIST)'
Matches zero or more occurrences of the given patterns.
`+(PATTERN-LIST)'
Matches one or more occurrences of the given patterns.
`#(PATTERN-LIST)'
Matches one of the given patterns.
`!(PATTERN-LIST)'
Matches anything except one of the given patterns.
Related
Here is the code at the bash shell. How is the file mask supposed to be specified, if not this way? I expected both commands to find the search expression, but it's not happening. In this example, I know in advance that I prefer to restrict the search to python source code files only, because unqualified searches are silly time wasters.
So, this works as expected:
grep -rni '/home/ga/projects' -e 'def Pr(x,u,v)'
/home/ga/projects/anom/anom.py:27:def Pr(x,u,v): blah, blah, ...
but this won't work:
grep --include=\*.{py} -rni '/home/ga/projects' -e 'def Pr(x,u,v)'
I'm using GNU grep version 2.16.
--include=\*.{py} looks like a broken attempt to use brace expansion (an unquoted {...} expression).
However, for brace expansion
to occur in bash (and ksh and zsh), you must either have:
a list of at least 2 items, separated with ,; e.g. {py,txt}, which expands to 2 arguments, py and txt.
or, a range of items formed from two end points, separated with ..; e.g., {1..3}, which expands to 3 arguments, 1, 2, and 3.
Thus, with a single item, simply do not use brace expansion:
--include=\*.py
If you did have multiple extensions to consider, e.g., *.py as well as *.pyc files, here's a robust form that illustrates the underlying shell features:
'--include=*.'{py,pyc}
Here:
Brace expansion is applied, because {...} contains a 2-item list.
Since the {...} directly follows the literal (single-quoted) string --include=*., the results of the brace expansion include the literal part.
Therefore, 2 arguments are ultimately passed to grep, with the following literal content:
--include=*.py
--include=*.pyc
Your command fails because of the braces '{}'. It will search for it in the file name. You can create a file such as 'myscript.{py}' to convince yourself. You'll see it will appear in the results.
The correct option parameter would be '*.py' or the equivalent \*.py. Either way will protect it from being (mis)interpreted by the shell.
On the other side, I can only advise to use the command find for such jobs :
find /home/ga/projects -regex '.*\.py$' -exec grep -e "def Pr(x,u,v)" {} +
That will protect you from hard to understand shell behaviour.
Try like this (using quotes to be safe; also better readability than backslash escaping IMHO):
grep --include='*.py' ...
your \*.{py} brace expansion usage isn't supported at all by grep. Please see the comments below for the full investigation regarding this. For the record, blame this answer for the resulting brace wars ;)
By the way, the brace expansion works generally fine in Bash. See mklement0 answer for more details.
Ack. As an alternative, you might consider switching to ack instead from now on. It's a tool just like grep, but fully optimized for programmers.
It's a great fit for what you are doing. A nice quote about it:
Every once in a while something comes along that improves an idea so much, you can't ignore it. Such a thing is ack, the grep replacement.
I am reading a page and trying to extract some data from it. I am interested in using bash and after going through few links, i came to know that 'Shell Parameter Expansion' might help however, i am finding difficulty using it in my script. I know that using sed might be easier but just for my knowledge i want to know how can i achieve this in bash.
shopt -s extglob
str='My work</u><br /><span style="color: rgb(34,34,34);"></span><span>abc-X7-27ABC | </span><span style="color: rgb(34,34,34);">build'
echo "${str//<.*>/|}"
I want my output to be like this: My work|abc-X7-27ABC |build
I thought of checking whether it accepts only word instead of pattern and it seems to be working with words.
For instance,
echo "${str//span style/|}" works but
echo "${str//span.*style/|}" doesn't
On the other hand, i saw in one of the link that it does accept pattern. I am confused why it's not working with the patern i am using above.
How to make sed do non-greedy match?
(User konsolebox's solution)
One mistake you're making is by mixing shell globbing and regex. In shell glob dot is taken literally as dot character not as 0 or more of any character.
If you try this code instead:
echo "${str//<*>/|}"
then it will print:
My work|build
This is not an answer, so much as a demonstration of why pattern-matching is not recommended for this kind of HTML editing. I attempted the following.
shopt -s extglob
set +H # Turn off history expansion, if necessary, to allow the !(...) pattern
echo ${str//+(<+(!(>))>)/|}
First: it didn't work, even for a simpler string like str='My work</u><br />bob<foo>build'. Second, for the string in the original question, it appeared to lock up the shell; I suspect such a complex pattern triggers exponential backtracking.
Here's how it's intended to work:
!(>) is any thing other than a single >
+(!(>)) is one or more non-> characters.
<+(!(>))> is one or more non-> characters enclosed in < and >
+(<+(!(>))>) is one or more groups of <...>-enclosed non->s.
My theory is that since !(>) can match a multi-character string as well as a single character, there is a ton of backtracking required.
I just found out about the bash extglob shell option here:-
How can I use inverse or negative wildcards when pattern matching in a unix/linux shell?
All the answers that used shopt -s extglob also mentioned shopt -u extglob to turn it off.
Why would I want to turn something so useful off? Indeed why isn't it on by default?
Presumably it has the potential for giving some nasty surprises.
What are they?
No nasty surprises -- default-off behavior is only there for compatibility with traditional, standards-compliant pattern syntax.
Which is to say: It's possible (albeit unlikely) that someone writing fo+(o).* actually intended the + and the parenthesis to be treated as literal parts of the pattern matched by their code. For bash to interpret this expression in a different manner than what the POSIX sh specification calls for would be to break compatibility, which is right now done by default in very few cases (echo -e with xpg_echo unset being the only one that comes immediately to mind).
This is different from the usual case where bash extensions are extending behavior undefined by the POSIX standard -- cases where a baseline POSIX shell would typically throw an error, but bash instead offers some new and different explicitly documented behavior -- because the need to treat these characters as matching themselves is defined by POSIX.
To quote the relevant part of the specification, with emphasis added:
An ordinary character is a pattern that shall match itself. It can be any character in the supported character set except for NUL, those special shell characters in Quoting that require quoting, and the following three special pattern characters. Matching shall be based on the bit pattern used for encoding the character, not on the graphic representation of the character. If any character (ordinary, shell special, or pattern special) is quoted, that pattern shall match the character itself. The shell special characters always require quoting.
When unquoted and outside a bracket expression, the following three characters shall have special meaning in the specification of patterns:
? - A question-mark is a pattern that shall match any character.
* - An asterisk is a pattern that shall match multiple characters, as described in Patterns Matching Multiple Characters.
[ - The open bracket shall introduce a pattern bracket expression.
Thus, the standard explicitly requires any non-NUL character other than ?, * or [ or those listed elsewhere as requiring quoting to match themselves. Bash's behavior of having extglob off by default allows it to conform with this standard in its default configuration.
However, for your own scripts and your own interactive shell, unless you're making a habit of running code written for POSIX sh with unusual patterns included, enabling extglob is typically worth doing.
Being a Kornshell person, I have extglob on in my .bashrc by default because that's the way it is in Kornshell, and I use it a lot.
For example:
$ find !(target) -name "*.xml"
In Kornshell, this is no problem. In BASH, I need to set extglob. I also set lithist and set -o vi. This allows me to use VI commands in using my shell history, and when I hit v, it shows my code as a bunch of lines.
Without lithist set:
for i in *;do;echo "I see $i";done
With listhist set:
for i in *
do
echo "I see $i"
done
Now, only if BASH had the print statement, I'd be all set.
I'm having trouble understanding what the following code does in a bash completion script:
case "$last" in
+\(--import|-i\))
_filedir '+(txt|html)';;
When is that case ever met? I thought the second line above would be something like
--import|-i)
which does make sense to me. I grepped my bash_completion.d directory for '+\\(' but that one was the only one that came up so I guess it's not that common.
This code is indeed puzzling without context. As it is, it matches two literal strings -
$ case "+(--import" in +\(--import|-i\)) echo match ;; esac
match
$ case "-i)" in +\(--import|-i\)) echo match ;; esac
match
It looks similar to the extended glob pattern +(--import|-i), but in this form it's neither a match for the literal pattern (would need to escape the pipe) nor the actual pattern (would need to unescape the parentheses). I'd guess "bug", but bash completion is a minefield of crazy metaprogramming, so it's impossible to say without seeing the entire script.
From bash(1)
If the extglob shell option is enabled using the shopt builtin,
several extended pattern matching operators are recognized. In the
following description, a pattern-list is a list of one or more
patterns separated by a |. Composite patterns may be formed using one
or more of the following sub-patterns:
[...]
+(pattern-list)
Matches one or more occurrences of the given patterns
I run the following code
zgrep -c compinit /usr/share/man/man{1..9}/zsh*
I get
zsh: no matches found: /usr/share/man/man2/zsh*
This is strange, since the following works
echo Masi{1..9}/masi
This suggests me that the problem may be a bug in Zsh.
Is the above a bug in Zsh for {1..9}?
It's not a bug, and it is working inside words fine. The trouble you're having here is that {1..9} is not a wildcard expression like * is; as your echo example shows, it's an iterative expansion. So your zgrep example is exactly the same as if you had typed each alternate version into the command line, and then since there are no man pages starting with zsh in man2, it errors out. (It's erroring out on a failure to find a match, not anything intrinsically related to your brace sequence expansion.)
If you did this, on the other hand:
zgrep -c compinit /usr/share/man/man[1-9]/zsh*
you'd get the results you expect, because [1-9] is a normal wildcard expression.
In zsh, if you want to use ranges in filenames, zle offers <1-n> on any real names it can expand on. That is to say:
$ touch a0b a1b a5b a7b
$ print a<0-100>b
And then hit <Tab> right after the final b would leave you with print a0b a1b a5b a7b expanded on the line.
For all other intents and purposes - perhaps full range requirements, non-file and scripting use - I'd express this using the rather succinct idiomatic zsh loop as:
for n ({1..50}); do print $n; done
Will allow you process the whole sequence range of numbers 1 to 50 :) after which you can do all sorts of useful things with, such as a file collection that doesn't exist yet:
arr=($(for n ({1..50}); do print /my/path/file$n.txt; done)) && print $arr[33]