I have a folder called .dir. When I type ls -al | grep .dir it finds it and shows me the result, but when I type ls -al | grep *dir or (*i*, *d*) it shows nothing. I can not understand why it does not find any matches.
When you do grep *dir two things can happen:
If there are any non-hidden file ending in dir say for example (foodir and bardir), the command will be expanded to grep foodir bardir and it will not do what you expect. Moreover the behavior will differ depending on there was just one file or more than one...
There are no non-hidden files ending in dir, then *dir is not expanded but used as the regular expression in the grep call. But the asterisk in grep is used to repeat the last match any number of times, and since there is no previous character it stands for itself. Conclusion: grep is looking for "*dir", literally.
For the first problem just use quotes. For the second use .*dir (. stands for any character):
ls -la | grep ".*dir"
Or if you want to see just your directory:
ls -la | grep "\.dir"
Note that if you do not escape the . with a \ it show also files such as adir, _dir...
Unquoted asterisks are expanded before being passed to grep. That means if your directory is empty except for the .dir directory, the string passed to grep depends on the status of the dotglob and nullglob settings (in Bash; I don't know their Korn shell equivalents):
$ cd -- "$(mktemp -d)"
$ mkdir .dir
$ shopt -u dotglob nullglob
$ echo *dir
*dir
$ shopt -s nullglob
$ echo *dir
$ shopt -s dotglob
$ echo *dir
.dir
grep works with regular expressions, not globs. * is a valid glob, but not a valid regular expression without anything before it.
Related
I've found a few similar questions to this but surprisingly none of them work for me.
I have this written in a script:
for d in $(ls -d "$1"); do
echo $d
done
$1 is the parent directory for which I wish to print out the list of subdirectories for, however, running this prints out, for example, a directory named "dir with spaces" as 3 words on separate lines.
You can use shell globbing instead of process substitution, which doesn't suffer from word expansion problem:
# to include dotfiles and not iterate empty directory
shopt -s dotglob nullglob
for d in "$1"/*; do
echo "$d"
done
Or you can resort to pretty common find ... -print0 | xargs -0 ... pattern.
In my dir, I have 'one.yaml', 'two.yaml', 'cron.yaml' and maybe lots of other yaml files. I'd like with ls (or another cmd if not possible) to select all the .yaml but cron.yaml
I tried ls *.yaml -I cron.yaml but it also returns cron.yaml...
Do it with the extended glob option. Use the syntax !() to match any pattern not inside the parenthesis.
shopt -s extglob
ls -- !(cron).yaml
if this is just for listing and not meant to be used in a program, do this in a sub-shell to not make the glob option made permanent
(
shopt -s extglob
ls -- !(cron).yaml) ;
)
written in one line as ( shopt -s extglob; ls -- !(cron).yaml; ).
POSIX shell does not support these extended glob options on which you could do
set -- *.yaml
for arg; do
if [ "$arg" != "script.sh" ]; then
printf '%s\n' "$arg"
fi
done
Unfortunately, ls -Ipattern doesn't apply to files explicitly listed on the command line, as cron.yaml will be when you use *.yaml.
A quick way to do what you want would be to just use grep -v:
ls *.yaml | grep -v cron.yaml
I'm having problems creating an if statement to check the files in my directory for a certain string in their names.
For example, I have the following files in a certain directory:
file_1_ok.txt
file_2_ok.txt
file_3_ok.txt
file_4_ok.txt
other_file_1_ok.py
other_file_2_ok.py
other_file_3_ok.py
other_file_4_ok.py
another_file_1_not_ok.sh
another_file_2_not_ok.sh
another_file_3_not_ok.sh
another_file_4_not_ok.sh
I want to copy all files that contain 1_ok to another directory:
#!/bin/bash
directory1=/FILES/user/directory1/
directory2=/FILES/user/directory2/
string="1_ok"
cd $directory
for every file in $directory1
do
if [$string = $file]; then
cp $file $directory2
fi
done
UPDATE:
The simpler answer was made by Faibbus, but refer to Inian if you want to remove or simply move files that don't have the specific string you want.
The other answers are valid as well.
cp directory1/*1_ok* directory2/
Use find for that:
find directory1 -maxdepth 1 -name '*1_ok*' -exec cp -v {} directory2 \;
The advantage of using find over the glob solution posted by Faibbus is that it can deal with an unlimited number of files which contain 1_ok were the glob solution will lead to an argument list too long error when calling cp with too many arguments.
Conclusion: For interactive use with a limited number of input files the glob will be fine, for a shell script, which has to be stable, I would use find.
With your script I suggest:
#!/bin/bash
source="/FILES/user/directory1"
target="/FILES/user/directory2"
regex="1_ok"
for file in "$source"/*; do
if [[ $file =~ $regex ]]; then
cp -v "$file" "$target"
fi
done
From help [[:
When the =~ operator is used, the string to the right of the operator
is matched as a regular expression.
Please take a look: http://www.shellcheck.net/
Using extglob matching in bash with the below pattern,
+(pattern-list)
Matches one or more occurrences of the given patterns.
First enable extglob by
shopt -s extglob
cp -v directory1/+(*not_ok*) directory2/
An example,
$ ls *.sh
another_file_1_not_ok.sh another_file_3_not_ok.sh
another_file_2_not_ok.sh another_file_4_nnoot_ok.sh
$ shopt -s extglob
$ cp -v +(*not_ok*) somedir/
another_file_1_not_ok.sh -> somelib/another_file_1_not_ok.sh
another_file_2_not_ok.sh -> somelib/another_file_2_not_ok.sh
another_file_3_not_ok.sh -> somelib/another_file_3_not_ok.sh
To remove the files except the one containing this pattern, do
$ rm -v !(*not_ok*) 2>/dev/null
So far I have this:
ls /usr/bin | grep "^[\.]"
The cmd still gets files with a "." in there.
I have looked at [[:punct:]] but still returns the same thing.
There's grep -v to exclude things. So try
ls /usr/bin | grep -v \\.
man grep says
-v, --invert-match
Selected lines are those not matching any of the specified patterns.
It's generally considered a bad idea to parse ls.
If I understand you correctly, you want all files in /usr/bin that don't have a dot in the name. You can use find to do that:
find /usr/bin -not -name "*.*"
It is more portable (thanks #Adrian) to use a ! instead of -not:
find /usr/bin ! -name "*.*"
Not really clear, what you want:
your command:
ls /usr/bin | grep "^[\.]"
mean, filter the output from ls to show only files, what are start with a dot.
grep "^[\.]"
^ ^^ - escaped dot
+- at the begining of the line
If you want, exclude all files what contains dot, use
ls /usr/bin | grep -v '\.' #or see HenrikN's answer and comments (grep -vF .)
it you want exclude only entries what are starting with dot, use
grep '^[^\.]'
whats mean anything, but dot at the start
Ps: anyway, parsing output form ls is usually an very bad idea. (http://mywiki.wooledge.org/ParsingLs)
You can change your regex to exclude files starting with ".":
ls -a /usr/bin | grep "^[^.]"
This regex selects only files which do not have "." at the start. By the way only ls -a shows files that starts with ".". How did you manage to get them without "-a" ?
This can be achieved with pure bash, if the extglob shell option is enabled.
shopt -s extglob
echo /usr/bin/!(*.*)
# or alternatively:
echo /usr/bin/+([!.])
You may replace echo with ls -d if you want to pipe the list to another command line-wise.
I think you are referring to the current working directory and parent dirctory and not a command with "a dot" in it.
Try this as you probably have ls aliased:
/bin/ls /usr/bin
Say I want to copy the contents of a directory excluding files and folders whose names contain the word 'Music'.
cp [exclude-matches] *Music* /target_directory
What should go in place of [exclude-matches] to accomplish this?
In Bash you can do it by enabling the extglob option, like this (replace ls with cp and add the target directory, of course)
~/foobar> shopt extglob
extglob off
~/foobar> ls
abar afoo bbar bfoo
~/foobar> ls !(b*)
-bash: !: event not found
~/foobar> shopt -s extglob # Enables extglob
~/foobar> ls !(b*)
abar afoo
~/foobar> ls !(a*)
bbar bfoo
~/foobar> ls !(*foo)
abar bbar
You can later disable extglob with
shopt -u extglob
The extglob shell option gives you more powerful pattern matching in the command line.
You turn it on with shopt -s extglob, and turn it off with shopt -u extglob.
In your example, you would initially do:
$ shopt -s extglob
$ cp !(*Music*) /target_directory
The full available extended globbing operators are (excerpt from man bash):
If the extglob shell option is enabled using the shopt builtin, several extended
pattern matching operators are recognized.A pattern-list is a list of one or more patterns separated by a |. Composite patterns may be formed using one or more of the following sub-patterns:
?(pattern-list)
Matches zero or one occurrence of the given patterns
*(pattern-list)
Matches zero or more occurrences of the given patterns
+(pattern-list)
Matches one or more occurrences of the given patterns
#(pattern-list)
Matches one of the given patterns
!(pattern-list)
Matches anything except one of the given patterns
So, for example, if you wanted to list all the files in the current directory that are not .c or .h files, you would do:
$ ls -d !(*#(.c|.h))
Of course, normal shell globing works, so the last example could also be written as:
$ ls -d !(*.[ch])
Not in bash (that I know of), but:
cp `ls | grep -v Music` /target_directory
I know this is not exactly what you were looking for, but it will solve your example.
If you want to avoid the mem cost of using the exec command, I believe you can do better with xargs. I think the following is a more efficient alternative to
find foo -type f ! -name '*Music*' -exec cp {} bar \; # new proc for each exec
find . -maxdepth 1 -name '*Music*' -prune -o -print0 | xargs -0 -i cp {} dest/
A trick I haven't seen on here yet that doesn't use extglob, find, or grep is to treat two file lists as sets and "diff" them using comm:
comm -23 <(ls) <(ls *Music*)
comm is preferable over diff because it doesn't have extra cruft.
This returns all elements of set 1, ls, that are not also in set 2, ls *Music*. This requires both sets to be in sorted order to work properly. No problem for ls and glob expansion, but if you're using something like find, be sure to invoke sort.
comm -23 <(find . | sort) <(find . | grep -i '.jpg' | sort)
Potentially useful.
You can also use a pretty simple for loop:
for f in `find . -not -name "*Music*"`
do
cp $f /target/dir
done
In bash, an alternative to shopt -s extglob is the GLOBIGNORE variable. It's not really better, but I find it easier to remember.
An example that may be what the original poster wanted:
GLOBIGNORE="*techno*"; cp *Music* /only_good_music/
When done, unset GLOBIGNORE to be able to rm *techno* in the source directory.
My personal preference is to use grep and the while command. This allows one to write powerful yet readable scripts ensuring that you end up doing exactly what you want. Plus by using an echo command you can perform a dry run before carrying out the actual operation. For example:
ls | grep -v "Music" | while read filename
do
echo $filename
done
will print out the files that you will end up copying. If the list is correct the next step is to simply replace the echo command with the copy command as follows:
ls | grep -v "Music" | while read filename
do
cp "$filename" /target_directory
done
One solution for this can be found with find.
$ mkdir foo bar
$ touch foo/a.txt foo/Music.txt
$ find foo -type f ! -name '*Music*' -exec cp {} bar \;
$ ls bar
a.txt
Find has quite a few options, you can get pretty specific on what you include and exclude.
Edit: Adam in the comments noted that this is recursive. find options mindepth and maxdepth can be useful in controlling this.
The following works lists all *.txt files in the current dir, except those that begin with a number.
This works in bash, dash, zsh and all other POSIX compatible shells.
for FILE in /some/dir/*.txt; do # for each *.txt file
case "${FILE##*/}" in # if file basename...
[0-9]*) continue ;; # starts with digit: skip
esac
## otherwise, do stuff with $FILE here
done
In line one the pattern /some/dir/*.txt will cause the for loop to iterate over all files in /some/dir whose name end with .txt.
In line two a case statement is used to weed out undesired files. – The ${FILE##*/} expression strips off any leading dir name component from the filename (here /some/dir/) so that patters can match against only the basename of the file. (If you're only weeding out filenames based on suffixes, you can shorten this to $FILE instead.)
In line three, all files matching the case pattern [0-9]*) line will be skipped (the continue statement jumps to the next iteration of the for loop). – If you want to you can do something more interesting here, e.g. like skipping all files which do not start with a letter (a–z) using [!a-z]*, or you could use multiple patterns to skip several kinds of filenames e.g. [0-9]*|*.bak to skip files both .bak files, and files which does not start with a number.
this would do it excluding exactly 'Music'
cp -a ^'Music' /target
this and that for excluding things like Music?* or *?Music
cp -a ^\*?'complete' /target
cp -a ^'complete'?\* /target