I'm trying to exclude multiple directories when using grep as in the following command
grep -r --exclude-dir={folder1, folder2} 'foo'
However, an error is raised grep: foo: No such file or directory. Maybe I'm doing something wrong with --exclude-dir option since the command below works as expected
grep -r 'foo'
How can I use --exclude-dir option correctly? Thanks in advance.
The --exclude-dir flag of GNU grep takes a glob expression as an argument. The glob expression with more than items then becomes a brace expansion sequence which is expanded by the shell.
The expansion involves words separated by a comma character and doesn't like spaces between the words. So ideally it should have been
--exclude-dir={folder1,folder2}
You can see this as a simple brace expansion in your shell by running
echo {a,b} # produces 'a b'
echo {a, b} # this doesn't undergo expansion by shell
echo --exclude-dir={folder1, folder2}
--exclude-dir={folder1, folder2}
so, your original command becomes
grep -r '--exclude-dir={folder1,' 'folder2}' foo
i.e. the exclude-dir takes a unexpanded glob expansion string as {folder1,' and 'folder2}' becomes the content that you are trying to search for, leaving foo as an unwanted extra argument, which the argparser of grep doesn't like throwing a command line parse error.
Remember brace expansion is a feature of the shell (e.g. bash), and not grep. In shells that don't support the feature, putting directories between {..} will be treated literally and might not work desirably.
I wonder whether there is a way to specify an one-or-more modifier for a character class in a parameter substitution pattern in bash, similar to a regex: i.e. [a-z]+. It seems like that for instance to remove all trailing whitespaces from a variable I would need to use ${fn##*[[:space:]]} (would only work for fn=" test" but not for fn=" test a") however I was hoping for something like ${fn##[[:space:]]+}. Is there a special syntax to specify regex instead of the patterns? Where is the pattern format specified?
Using extglob, you can do this:
shopt -s extglob
fn=" test a"
echo "${fn##+([[:space:]])}"
test a
Here glob expression +([[:space:]]) matches 1 or more whitespace characters.
You cannot use regular expressions in parameter expansions like that. However, the extended pattern syntax enabled by extglob is equivalent in power.
$ fn=" test a"
$ echo "$fn"
test a
$ shopt -s extglob
$ echo "${fn##*([[:space:]])}"
test a
I am trying to exclude a directory from a glob.
This works at the command line:
$ export exclude=BigDir
$ for d in ^$exclude/ ; do echo "$d" ; done
SmallDir/
SmallerDir/
$
But in a file it doesn't work at all
#!/bin/zsh
exclude=BigDir
for d in ^$exclude/ ; do echo "$d" ; done
Running ./test or however I saved it prints the literal string
^BigDir/
How do I get it to correctly expand in the script file?
You are incorrectly using the glob characters ? used by the shell and the regular expression constructs ^, $. The for loop in your example can not undergo a regex match to exclude the directory provided, since it undergoes only pathname expansion (aka. glob expansion)
Unless you let know the shell to treat ^ and $ as special by enabling extended glob options extglob in bash and extendedglob in zsh, you cannot achieve what you wanted to do.
So you probably just need
setopt extendedglob
print -rl ^BigDir*
meaning print anything except the the filenames matching with BigDir.
I am trying to write a bash script. I am not sure why in my script:
ls {*.xml,*.txt}
works okay, but
name="{*.xml,*.txt}"
ls $name
doesn't work. I get
ls: cannot access {*.xml,*.txt}: No such file or directory
The expression
ls {*.xml,*.txt}
results in Brace expansion and shell passes the expansion (if any) to ls as arguments. Setting shopt -s nullglob makes this expression evaluate to nothing when there are no matching files.
Double quoting the string suppresses the expansion and shell stores the literal contents in your variable name (not sure if that is what you wanted). When you invoke ls with $name as the argument, shell does the variable expansion but no brace expansion is done.
As #Cyrus has mentioned, eval ls $name will force brace expansion and you get the same result as that of ls {\*.xml,\*.txt}.
The reason your expansion doesn't work is that brace expansion is performed before variable expansion, see Shell expansions in the manual.
I'm not sure what it is you're trying to do, but if you want to store a list of file names, use an array:
files=( {*.txt,*.xml} ) # these two are the same
files=(*.txt *.xml)
ls -l "${files[#]}" # give them to a command
for file in "${files[#]}" ; do # or loop over them
dosomething "$file"
done
"${array[#]}" expands to all elements of the array, as separate words. (remember the quotes!)
I am trying the following command:
ls myfile.h1.{`seq -s ',' 3501 3511`}*
But ls raises the error:
ls: cannot access myfile.h1.{3501,3502,3503,3504,3505,3506,3507,3508,3509,3510,3511}*: No such file or directory
Seems like ls is thinking the entire line is a filename and not a wildcard pattern. But if I just copy that command ls myfile.h1.{3501,3502,3503,3504,3505,3506,3507,3508,3509,3510,3511}* in the terminal I get the listing as expected.
Why does typing out the command in full work, but not the usage with seq?
seq is not needed for your case, try
$ ls myfile.h1.{3500..3511}
if you want to use seq I would suggest using format option
$ ls $(seq -f 'myfile.h1.%g' 3501 3511)
but I don't think there is any reason to do so.
UPDATE:
Note that I didn't notice the globbing in the original post. With that, the brace extension still preferred way
$ ls myfile.h1.{3500..3511}*
perhaps even factoring the common digit out, if your bash support zero padding
$ ls myfile.h1.35{00..11}*
if not you can extract at least 3 out
$ ls myfile.h1.3{500..511}*
Note that the seq alternative won't work with globbing.
Other answer has more details...
karakfa's answer, which uses a literal sequence brace expansion expression, is the right solution.
As for why your approach didn't work:
Bash's brace expansion {...} only works with literal expressions - neither variable references nor, as in your case, command substitutions (`...`, or, preferably, $(...)) work[1] - for a concise overview, see this answer of mine.
With careful use of eval, however, you can work around this limitation; to wit:
from=3501 to=3511
# CAVEAT: Only do this if you TRUST that $from and $to contain
# decimal numbers only.
eval ls "myfile.h1.{$from..$to}*"
#ghoti suggests the following improvement in a comment to make the use of eval safe here:
# Use parameter expansion to remove all non-digit characters from the values
# of $from and $to, thus ensuring that they either contain only a decimal
# number or the empty string; this expansion happens *before* eval is invoked.
eval ls "myfile.h1.{${from//[^0-9]/}..${to//[^0-9]/}}*"
As for how your command was actually evaluated:
Note: Bash applies 7-8 kinds of expansions to a command line; only the ones that actually come into play here are discussed below.
first, the command in command substitution `seq -s ',' 3501 3511` is executed, and replaced by its output (also note the trailing ,):
3501,3502,3503,3504,3505,3506,3507,3508,3509,3510,3511,
the result then forms a single word with its prefix, myfile.h1.{ and its suffix, }*, yielding:
myfile.h1.{3501,3502,3503,3504,3505,3506,3507,3508,3509,3510,3511,}*
pathname expansion (globbing) is then applied to the result - in your case, since no files match, it is left as-is (by default; shell options shopt -s nullglob or shopt -s failglob could change that).
finally, literal myfile.h1.{3501,3502,3503,3504,3505,3506,3507,3508,3509,3510,3511,}* is passed to ls, which - because it doesn't refer to an existing filesystem item - results in the error message you saw.
[1] Note that the limitation only applies to sequence brace expansions (e.g., {1..3}); list brace expansions (e.g, {1,2,3}) are not affected, because no up-front interpretation (interpolation) is needed; e.g. {$HOME,$USER} works, because brace expansion results expanding the list to separate words $HOME, and $USER, which are only later expanded.
Historically, sequence brace expansions were introduced later, at a time when the order of shell expansions was already fixed.