shell script to count readable files - bash

How can I write a shell script file_readable which:
accepts some number of names as arguments,
checks each name to see if it is a regular file and readable, and
outputs a count of the number of such files.
For example:
$ sh file_readable /etc/fstab /etc/ssh/ssh_host_rsa_key /etc/does-not-exist
1
Of these, only /etc/fstab is likely to both exist and be readable.
So far I have put this together but it does not work correctly - can anybody help me please ?:
#!/bin/sh
for filename in "$#"
do
if test -f "$filename"
then echo | wc -l
else echo $?
fi
done

then echo | wc -l
If file exists and is a regular you print number of lines in empty string plus "\n", which is equal 1 always. Sound not quite usable, isn't it?
All you need is incrementing some counter and printing it in the end.
#!/bin/sh
readable_files=0
for filename in "$#"; do
if test -f "$filename"; then
readable_files=$(( readable_files + 1 ))
fi
done
echo "${readable_files}"

Related

Reading in File line by line w/ Bash

I'm creating a bash script to read a file in line by line, that is formatted later to be organised by name and then date. I cannot see why this code isn't working at this time though no errors show up even though I have tried with the input and output filename variables on their own, with a directory finder and export command.
export inputfilename="employees.txt"
export outputfilename="branch.txt"
directoryinput=$(find -name $inputfilename)
directoryoutput=$(find -name $outputfilename)
n=1
if [[ -f "$directoryinput" ]]; then
while read line; do
echo "$line"
n=$((n+1))
done < "$directoryoutput"
else
echo "Input file does not exist. Please create a employees.txt file"
fi
All help is very much appreciated, thank you!
NOTE: As people noticed, I forgot to add in the $ sign on the data transfer to file, but it was just in copying my code, I do have the $ sign in my actual application and still no result
Reading in File line by line w/ Bash
The best and idiomatic way to read file line by line is to do:
while IFS= read -r line; do
// parse line
printf "%s" "$line"
done < "file"
More on this topic can be found on bashfaq
However don't read files in bash line by line. You can (ok, almost) always not read a stream line by line in bash. Reading a file line by line in bash is extremely slow and shouldn't be done. For simple cases all the unix tools with the help of xargs or parallel can be used, for more complicated awk and datamesh are used.
done < "directoryoutput"
The code is not working, because you are passing to your while read loop as input to standard input the content of a file named directoryoutput. As such a file does not exists, your script fails.
directoryoutput=$(find -name $outputfilename)
One can simply append the variable value with newline appended to a read while loop using a HERE-string construction:
done <<< "$directoryoutput"
directoryinput=$(find -name $inputfilename)
if [[ -f "$directoryinput" ]]
This is ok as long as you have only one file named $inputfilename in your directory. Also it makes no sense to find a file and then check for it's existance. In case of more files, find return a newline separated list of names. However a small check if [ "$(printf "$directoryinput" | wc -l)" -eq 1 ] or using find -name $inputfilename | head -n1 I think would be better.
while read line;
do
echo "$line"
n=$((n+1))
done < "directoryoutput"
The intention is pretty clear here. This is just:
n=$(<directoryoutput wc -l)
cat "directoryoutput"
Except that while read line removed trailing and leading newlines and is IFS dependent.
Also always remember to quote your variables unless you have a reason not to.
Have a look at shellcheck which can find most common mistakes in scripts.
I would do it more like this:
inputfilename="employees.txt"
outputfilename="branch.txt"
directoryinput=$(find . -name "$inputfilename")
directoryinput_cnt=$(printf "%s\n" "$directoryinput" | wc -l)
if [ "$directoryinput_cnt" -eq 0 ]; then
echo "Input file does not exist. Please create a '$inputfilename' file" >&2
exit 1
elif [ "$directoryinput_cnt" -gt 1 ]; then
echo "Multiple file named '$inputfilename' exists in the current path" >&2
exit 1
fi
directoryoutput=$(find . -name "$outputfilename")
directoryoutput_cnt=$(printf "%s\n" "$directoryoutput" | wc -l)
if [ "$directoryoutput_cnt" -eq 0 ]; then
echo "Input file does not exist. Please create a '$outputfilename' file" >&2
exit 1
elif [ "$directoryoutput_cnt" -gt 1 ]; then
echo "Multiple file named '$outputfilename' exists in the current path" >&2
exit 1
fi
cat "$directoryoutput"
n=$(<"$directoryoutput" wc -l)

Test existence of files in shell

I have a list of files and I would like to pipe to a command to test if each files exists
cat files.txt | command
Does it exist any command to do that ? Do I have to write a script ? Can you help me to write it ?
For example:
> cat files.txt
libs/cakephp/app_exp/views/elements/export-menu.ctp
libs/cron/prad_import.php
main/css/admin/remarketing-graph.css
main/images/dropd-arrow.png
main/includes/forms/export/export_menu.php
main/jquery/jquery/jquery.cookie.js
main/mvc/controllers/remarketing/prad_controller.php
main/mvc/controllers/remarketing/remarketing_controller.php
but some of this files doesn't exits, so I want to do
> cat files.txt | command
libs/cakephp/app_exp/views/elements/export-menu.ctp
main/css/admin/remarketing-graph.css
main/images/dropd-arrow.png
main/includes/forms/export/export_menu.php
main/jquery/jquery/jquery.cookie.js
that returns only the existing files
Are you really using #!/bin/sh? In what context? An old Unix env, or in modern, stripped-down env with minimal cmds?
Can you use #!/bin/bash or #!/bin/ksh? That will make it easier.
But in any shell this should work
while read line ; do if [ -f "$line" ] ; then echo "$line" ; fi ; done < files.txt
This should allow for file/paths with spaces in them, but if other odd chars are embedded in the filenames then more work maybe required.
IHTH
test -f "files.txt" && cat "files.txt" | command
Also
[[ -f "files.txt" ]] && cat "files.txt" | command # in BASH
take your pick. test is more portable.
Understanding that you want command to test for file existence, then you do not need a pipe at all. You can do something like this with a while loop:
while read file; do
if test -f "$file"; then
echo "the file exists" # or whatever you want to do
else
echo "the file does NOT exist" # or whatever you want to do
fi
done < "files.txt"
This will read the file line-by-line and test each file for existence. You can also read all filenames from files.txt into an array and then loop through the array if you prefer. And yes, the above can be a script.

list in script shell bash

I did this script
#!/bin/bash
liste=`ls -l`
for i in $liste
do
echo $i
done
The problem is I want the script displays each result line by line, but it displays word by word :
I have :
my_name
etud
4096
Oct
8
10:13
and I want to have :
my_name etud 4096 Oct 8 10:13
The final aim of the script is to analyze each line ; it is the reason I want to be able to recover the entire line. Maybe the list is not the best solution but I don't know how to recover the lines.
To start, we'll assume that none of your filenames ever contain newlines:
ls -l | IFS= while read -r line; do
echo "$line"
# Do whatever else you want with $line
done
If your filenames could contain newlines, things get tricky. In this case, it's better (although slower) to use stat to retrieve the desired metadata from each file individually. Consult man stat for details about how your local variety of stat works, as it is unfortunately not very standardized.
for f in *; do
line=$(stat -c "%U %n %s %y" "$f") # One possibility
# Work with $line as if it came from ls -l
done
You can replace
echo $i
with
echo -n "$i "
echo -n outputs to console without newline.
Another to do it with a while loop and without a pipe:
#!/bin/bash
while read line
do
echo "line: $line"
done < <(ls -l)
First, I hope that you aren't genuinely using ls in your real code, but only using it as an example. If you want a list of files, ls is the wrong tool; see http://mywiki.wooledge.org/ParsingLs for details.
Second, modern versions of bash have a builtin called readarray.
Try this:
readarray -t my_array < <(ls -l)
for entry in "${my_array[#]}"; do
read -a pieces <<<"$entry"
printf '<%s> ' "${pieces[#]}"; echo
done
First, it creates an array (called my_array) with all the output from the command being run.
Then, for each line in that output, it creates an array called pieces, and emits each piece with arrow brackets around them.
If you want to read a line at a time, rather than reading the entire file at once, see http://mywiki.wooledge.org/BashFAQ/001 ("How can I read a file (data stream, variable) line-by-line (and/or field-by-field)?")
Joinning the previous answers with the need to store the list of files in a variable. You can do this
echo -n "$list"|while read -r lin
do
echo $lin
done

Script to call either from file or user input

I'm trying to write a small script that either takes input from a file or from user, then it gets rid of any blank lines from it.
I'm trying to make it so that if there is no file name specified it will prompt the user for input. Also is the best way to output the manual input to a file then run the code or to store it in a variable?
So far I have this but when I run it with a file it give 1 line of error before returning the output I want. The error says ./deblank: line 1: [blank_lines.txt: command not found
if [$# -eq "$NO_ARGS"]; then
cat > temporary.txt; sed '/^$/d' <temporary.txt
else
sed '/^$/d' <$#
fi
Where am I going wrong?
You need spaces around [ and ]. In bash, [ is a command and you need spaces around it for bash to interpret it so.
You can also check for the presence of arguments by using (( ... )). So your script could be rewritten as:
if ((!$#)); then
cat > temporary.txt; sed '/^$/d' <temporary.txt
else
sed '/^$/d' "$#"
fi
If you want to use only the first argument, then you need to say $1 (and not $#).
Try using this
if [ $# -eq 0 ]; then
cat > temporary.txt; sed '/^$/d' <temporary.txt
else
cat $# | sed '/^$/d'
fi
A space is needed between [ and $# and your usage of $# is not good. $# represents all arguments and -eq is used to compare numeric values.
There are multiple problems here:
You need to leave a space between the square brackets [ ] and the variables.
When using a string type, you cannot use -eq, use == instead.
When using a string comparison you need to use double square brackets.
So the code should look like:
if [[ "$#" == "$NO_ARGS" ]]; then
cat > temporary.txt; sed '/^$/d' <temporary.txt
else
sed '/^$/d' <$#
fi
Or else use $# instead.
Instead of forcing user input to a file, I'd force the given file to stdin:
#!/bin/bash
if [[ $1 && -r $1 ]]; then
# it's a file
exec 0<"$1"
elif ! tty -s; then
: # input is piped from stdin
else
# get input from user
echo "No file specified, please enter your input, ctrl-D to end"
fi
# now, let sed read from stdin
sed '/^$/d'

shell script: if statement

I'm following the tutorial here: http://bash.cyberciti.biz/guide/If..else..fi#Number_Testing_Script
My script looks like:
lines=`wc -l $var/customize/script.php`
if test $lines -le 10
then
echo "script has less than 10 lines"
else
echo "script has more than 10 lines"
fi
but my output looks like:
./boot.sh: line 33: test: too many arguments
script has more than 10 lines
Why does it say I have too many arguments? I fail to see how my script is different from the one in the tutorial.
wc -l file command will print two words. Try this:
lines=`wc -l file | awk '{print $1}'`
To debug a bash script (boot.sh), you can:
$ bash -x ./boot.sh
It'll print every line executed.
wc -l file
outputs
1234 file
use
lines=`wc -l < file`
to get just the number of lines. Also, some people prefer this notation instead of backticks:
lines=$(wc -l < file)
Also, since we don't know if $var contains spaces, and if the file exists:
fn="$var/customize/script.php"
if test ! -f "$fn"
then
echo file not found: $fn
elif test $(wc -l < "$fn") -le 10
then
echo less than 11 lines
else
echo more than 10 lines
fi
Also, you should use
if [[ $lines -gt 10 ]]; then
something
else
something
fi
test condition is really outdated, and so is it's immediate successor, [ condition ], mainly because you have to be really careful with those forms. For example, you must quote any $var you pass to test or [ ], and there are other details that get hairy. (tests are treated in every way as any other command). Check out this article for some details.

Resources