I'm trying to use the ? wildcard to display only 1 character files, and ?.* to display 1 character files with extensions.
what works:
cd /mydir
ls ? ?.*
I'm trying to use this in a shell script so therefor i cant use "cd"
What i'm trying to get to work
ls ? ?.* /mydir
and it gives me the output:
ls: cannot access ?.*: No such file or directory
I've also tried:
ls /mydir ? ?.*
which gives me the exact same output as before.
From a comment you wrote:
im in college for linux administrator and 1 of my current classes in shell scripting. My teacher is just going over basic stuff. And, my current assingment is to get the number of files in the tmp directory of our class server, the number of files that end in .log and the number of files that only have 1 character names and store the data in a file and then display the stored data to the user. I know it's stupid, but it's my assignment.
I only hope that they don't teach you to parse the output of ls in college... it's one of the most terrible things to do. Please refer to these links:
Why you shouldn't parse the output of ls(1)
Don't ever do these
The solution you chose
ls /mydir/? /mydir/?.* | wc -l
is broken in two cases:
If there are no matching files, you'll get an error. You can fix that in two ways: use shopt -s nullglob or just redirect stderr to devnull.
If there's a newline in a file name. Try it: touch $'a.lol\nlol\n\lol\nlol\nlol'. LOL.
The proper bash way is the following:
shopt -s nullglob
shopt -u failglob
files=( /mydir/? /mydir/?.* )
echo "There are ${#files[#]} files found."
When you write ls ? ?.* /mydir, you're trying to display the files matching three distincts patterns: ?, ?.*, and /mydir. You want to match only /mydir/? and /mydir/?.*, hence this command: ls /mydir/? /mydir/?.*.
Edit: while this is a correct answer to the initial question (listing /mydir/? and /mydir/?.*), OP wanted to do this to parse the output and get the file count. See #gniourf_gniourf's answer, which is a much better way to do this.
cd works perfectly within a shell script, use it. For minimal impact on the script, I would use a subshell:
( cd /mydir && ls ? ?.* )
That way, you don't change the current working directory of the script (and neither $OLDPWD, which would be clobbered with cd /mydir; ...; cd -;).
While ls seems like an obvious choice, find is probably more suitable:
find /mydir \! -name "." -a \( -name "?" -o -name "?.*" \)
Related
I'm trying to use the ? wildcard to display only 1 character files, and ?.* to display 1 character files with extensions.
what works:
cd /mydir
ls ? ?.*
I'm trying to use this in a shell script so therefor i cant use "cd"
What i'm trying to get to work
ls ? ?.* /mydir
and it gives me the output:
ls: cannot access ?.*: No such file or directory
I've also tried:
ls /mydir ? ?.*
which gives me the exact same output as before.
From a comment you wrote:
im in college for linux administrator and 1 of my current classes in shell scripting. My teacher is just going over basic stuff. And, my current assingment is to get the number of files in the tmp directory of our class server, the number of files that end in .log and the number of files that only have 1 character names and store the data in a file and then display the stored data to the user. I know it's stupid, but it's my assignment.
I only hope that they don't teach you to parse the output of ls in college... it's one of the most terrible things to do. Please refer to these links:
Why you shouldn't parse the output of ls(1)
Don't ever do these
The solution you chose
ls /mydir/? /mydir/?.* | wc -l
is broken in two cases:
If there are no matching files, you'll get an error. You can fix that in two ways: use shopt -s nullglob or just redirect stderr to devnull.
If there's a newline in a file name. Try it: touch $'a.lol\nlol\n\lol\nlol\nlol'. LOL.
The proper bash way is the following:
shopt -s nullglob
shopt -u failglob
files=( /mydir/? /mydir/?.* )
echo "There are ${#files[#]} files found."
When you write ls ? ?.* /mydir, you're trying to display the files matching three distincts patterns: ?, ?.*, and /mydir. You want to match only /mydir/? and /mydir/?.*, hence this command: ls /mydir/? /mydir/?.*.
Edit: while this is a correct answer to the initial question (listing /mydir/? and /mydir/?.*), OP wanted to do this to parse the output and get the file count. See #gniourf_gniourf's answer, which is a much better way to do this.
cd works perfectly within a shell script, use it. For minimal impact on the script, I would use a subshell:
( cd /mydir && ls ? ?.* )
That way, you don't change the current working directory of the script (and neither $OLDPWD, which would be clobbered with cd /mydir; ...; cd -;).
While ls seems like an obvious choice, find is probably more suitable:
find /mydir \! -name "." -a \( -name "?" -o -name "?.*" \)
Is there a way to have the find command return a value when it does not find a match? Basically, I have an old backup, and I want to search for each of the files in it on my current computer. Here is a cheesy way I was going to do it:
first run the following from the home directory:
$ ls -lR * > AllFiles.txt;
This will build my listing of all of my files on the current computer.
Next run the following script for each file in the back up:
#! /bin/bash
if ! grep $1 ~/AllFiles.txt; then
echo file $1 not found;
exit 1;
fi
Of course this is clunky, and it does not account for filename changes, but it's close enough. Alternatively, I'd like to do a find command for each of the back up files.
You can use standard return value test if using a standard gnu find, such as:
find . ! -readable -prune -o -name 'filenameToSearch.ext'
then check for return value using:
echo $?
if any value other than 0 means it did not find a match.
If I understood you correctly;
grep -r valueORpatternToSearchinTEXT $(find . -type f) |wc -l
This will find for every file in the working/existing directory you are and its subdirs, search for what you need, then count for lines, if it is not found, you will get 0 at the end. Remove pipe and afterwards if you want to see what is found and where.
Its a interview question. Interviewer asked this "basic" shell script question when he understand i don't have experience in shell scripting. Here is question.
Copy files from one directory which has size greater than 500 K to another directory.
I can do it immediately in c lang but seems difficult in shell script as never tried it.I am familiar with unix basic commands so i tried it, but i can just able to extract those file names using below command.
du -sk * | awk '{ if ($1>500) print $2 }'
Also,Let me know good shell script examples book.
It can be done in several ways. I'd try and use find:
find $FIRSTDIRECTORY -size +500k -exec cp "{\} $SECONDDIRECTORY \;
To limit to the current directory, use -maxdepth option.
du recurses into subdirectories, which is probably not desired (you could have asked for clarification if that point was ambiguous). More likely you were expected to use ls -l or ls -s to get the sizes.
But what you did works to select some files and print their names, so let's build on it. You have a command that outputs a list of names. You need to put the output of that command into the command line of a cp. If your du|awk outputs this:
Makefile
foo.c
bar.h
you want to run this:
cp Makefile foo.c bar.h otherdirectory
So how you do that is with COMMAND SUBSTITUTION which is written as $(...) like this:
cd firstdirectory
cp $(du -sk * | awk '{ if ($1>500) print $2 }') otherdirectory
And that's a functioning script. The du|awk command runs first, and its output is used to build the cp command. There are a lot of subtle drawbacks that would make it unsuitable for general use, but that's how beginner-level shell scripts usually are.
find . -mindepth 1 -maxdepth 1 -type f -size +BYTESc -exec cp -t DESTDIR {}\+
The c suffix on the size is essential; the size is in bytes. Otherwise, you get probably-unexpected rounding behaviour in determining the result of the -size check. If the copying is meant to be recursive, you will need to take care of creating any destination directory also.
Here's how one might list all files matching a pattern in bash:
ls *.jar
How to list the complement of a pattern? i.e. all files not matching *.jar?
Use egrep-style extended pattern matching.
ls !(*.jar)
This is available starting with bash-2.02-alpha1.
Must first be enabled with
shopt -s extglob
As of bash-4.1-alpha there is a config option to enable this by default.
ls | grep -v '\.jar$'
for instance.
Little known bash expansion rule:
ls !(*.jar)
With an appropriate version of find, you could do something like this, but it's a little overkill:
find . -maxdepth 1 ! -name '*.jar'
find finds files. The . argument specifies you want to start searching from ., i.e. the current directory. -maxdepth 1 tells it you only want to search one level deep, i.e. the current directory. ! -name '*.jar' looks for all files that don't match the regex *.jar.
Like I said, it's a little overkill for this application, but if you remove the -maxdepth 1, you can then recursively search for all non-jar files or what have you easily.
POSIX defines non-matching bracket expressions, so we can let the shell expand the file names for us.
ls *[!j][!a][!r]
This has some quirks though, but at least it is compatible with about any unix shell.
If your ls supports it (man ls) use the --hide=<PATTERN> option. In your case:
$> ls --hide=*.jar
No need to parse the output of ls (because it's very bad) and it scales to not showing multiple types of files. At some point I needed to see what non-source, non-object, non-libtool generated files were in a (cluttered) directory:
$> ls src --hide=*.{lo,c,h,o}
Worked like a charm.
Another approach can be using ls -I flag (Ignore-pattern).
ls -I '*.jar'
And if you want to exclude more than one file extension, separate them with a pipe |, like ls test/!(*.jar|*.bar). Let's try it:
$ mkdir test
$ touch test/1.jar test/1.bar test/1.foo
$ ls test/!(*.jar|*.bar)
test/1.foo
Looking at the other answers you might need to shopt -s extglob first.
One solution would be ls -1|grep -v '\.jar$'
Some mentioned variants of this form:
ls -d *.[!j][!a][!r]
But this seems to be only working on bash, while this seems to work on both bash and zsh:
ls -d *.[^j][^a][^r]
ls -I "*.jar"
-I, --ignore=PATTERN
do not list implied entries matching shell PATTERN
It works without having to execute anything before
It works also inside watch quotes: watch -d 'ls -I "*.gz"', unlike watch 'ls !(*.jar)' which produces: sh: 1: Syntax error: "(" unexpected
Note: For some reason in Centos requires quoting the pattern after -I while Ubuntu does not
In the book "Beginning Portable Shell Scripting" by Peter Seebach there is an example how to list the contents of all subdirectories of the current directory:
#!/bin/sh
/bin/ls | while read file
do
if test -d "$file"; then
( cd "$file" && ls )
fi
done
I learned that parsing ls is bad and globbing should be prefered. Do you think the author chooses parsing because there is a portability issue?
I would do:
#!/bin/sh
for file in *
do
if test -d "$file"; then
( cd "$file" && ls )
fi
done
Thanks,
Somebody
Both solutions are not robust against weird filenames, nor do they handle directories which begin with ".". I would write this using find, e.g.:
find . -maxdepth 1 -type d -exec ls '{}' ';'
but first I'd question what output is actually required, either for a person to eyeball or a further script to digest.
You'll probably be able to do in a single "find" what is going to cost a lot of process forks with the for/while ... do ... done loop.
Globbing is much preferred over parsing ls since it will handle filenames that include spaces and other characters.
For the specific case in your question, you may be able to use nothing more than:
ls */