Bash check is file with variable name inside loop exists - bash

I would like to check if a file exists. Of course this is explained in many places. Now I am inside a loop like:
for ((l=0;l<5;l+=1));
do
if -a FILENAMEl #FILENAME contains l!!!!!!!!!
then "FILENAMEl exists"
else
do
.............
fi
done
Any ideas?
Thanks so much!!!

The main problem is that you are mixing the syntax of a variable name l with that of a file name. If you wish to use them together, to form part of a filename with a variable, you need a syntax break (caused by "$"), or use braces ({}).
If the file name has a variable in the middle, then braces work best. For example: "my_file_${l}_head.txt" would create files like my_file_1_head.txt, my_file_2_head.txt, etc.
Here is your original example corrected:
for ((l=0;l<5;l+=1))
do
if test -a FILENAME$l
then echo "FILENAME$l exists"
else echo "FILENAME$l doesn't exist"
fi
done
However, I wouldn't write code this way.
I only took your example and changed it as little as possible to show you the essential difference.
Here's another way to write it, using a more DRY (Don't Repeat Yourself) approach:
for l in {1..5}; do
file="filename$l"
if [[ -a "$file" ]]; then
echo "$file exists"
else
echo "$file does not exist"
fi
done
If you want more minimalism, here's yet another approach:
for l in {1..5}; do f="filename$l"
[[ -a "$f" ]] && echo "$f exists" || echo "$f does not exist"
done
Now, if you need to do something other than just print out the status, using function calls to make the extra work modular works well:
for l in {1..5} ; do f="$filename$l"
[[ -a "$f" ] && process_file $f || non_existant_file $f
done
Then, elsewhere, you should define both process_file and non_existant_file:
process_file() {
local file="$1"
# do whatever is needed for an existing file
}
non_existant_file() {
local file="$1"
# do whatever is needed for a non-existant file
}

Assume you're trying to find which files exist in the filename format file1.csv, file2.csv, etc...
for i in {1..5};
do f="file$i.csv";
if test -e $f;
then echo "$f exists";
else echo "$f does not exist";
fi
done
Perhaps what you need is simply a find
find . -name "file?.csv" -size +10k
You can restrict the file name to suffix 1..5 and do an action on the find result (check for find's -exec or more generally xargs as below).
find . -name "file[1-5].csv" -size +10c | xargs head -1

Related

Shell Script: Get newest file (of an extension) from a directory

I want to get newest file in a directory. I am using
BashFAQ/099 as reference.
My Code:
#!/bin/bash
function newest_file() {
local file_ext="$2"
local files=("$1"/*."${file_ext}")
local newest="${files[#]:0:1}"
for f in "${files[#]}"; do
if [[ "$f" -nt "$newest" ]]; then
newest="$f"
fi
done
echo "$newest"
}
file_path
newest_file "${1:-$HOME/Downloads}" "${2:-*}"
Output:
➜ bash func_get_new_file.sh $HOME/reports txt
/home/ashwin/reports/test4.txt
➜ bash func_get_new_file.sh $HOME/reports
newest_file:2: no matches found: /home/ashwin/reports/*.*
I get error when file extension is *. I want to understand why . does not work.
I also want to understand the (..) syntax used for creating array of files in directory, i.e. files=("$1"/*.${file_ext}) in my code above.
As suggested by #Cyrus, updating code as per shellcheck.net.
On futher debug and reading other posts, I figured that passing asterisk as parameter is troublesome. I am updating my code to check any special characters being passed in $2 as below:
#!/bin/bash
newest_file() {
local file_ext="$2"
if [[ ! "${file_ext}" =~ ^[.a-zA-Z0-9]*$ ]]; then
echo "ERROR: file extension can have "." or alphanumberic characters. Don't provide any ext if all files should be checked"
return 1
fi
[[ -z "${file_ext}" ]] && local files=("$1"/*) || local files=("$1"/*.${file_ext})
local newest="${files[#]:0:1}"
for f in "${files[#]}"; do
if [[ $f -nt $newest ]]; then
newest="$f"
fi
done
echo "$newest"
}
newest_file "${1:-$HOME/Downloads}" "${2}"
Unless anyone can provide a simple solution to passing asterisk, I will go with the above solution.

Shell script with absolute path and control errors

I was doing this little script in which the first argument must be a path to an existing directory and the second any other thing.
Each object in the path indicated in the first argument must be renamed so that the new
name is the original that was added as a prefix to the character string passed as the second argument. Example, for the string "hello", the object OBJECT1 is renamed hello.OBJECT1 and so on
Additionally, if an object with the new name is already present, a message is shown by a standard error output and the operation is not carried out continuing with the next object.
I have the following done:
#! /bin/bash
if [ "$#" != 2 ]; then
exit 1
else
echo "$2"
if [ -d "$1" ]; then
echo "directory"
for i in $(ls "$1")
do
for j in $(ls "$1")
do
echo "$i"
if [ "$j" = "$2"."$i" ]; then
exit 1
else
mv -v "$i" "$2"."$i"
echo "$2"."$i"
fi
done
done
else
echo "no"
fi
fi
I am having problems if I run the script from another file other than the one I want to do it, for example if I am in /home/pp and I want the changes to be made in /home/pp/rr, since that is the only way It does in the current.
I tried to change the ls to catch the whole route with
ls -R | sed "s;^;pwd;" but the route catches me badly.
Using find you can't because it puts me in front of the path and doesn't leave the file
Then another question, to verify that that object that is going to create new is not inside, when doing it with two for I get bash errors for all files and not just for coincidences
I'm starting with this scripting, so it has to be a very simple solution thing
An obvious answer to your question would be to put a cd "$2 in the script to make it work. However, there are some opportunities in this script for improvement.
#! /bin/bash
if [ "$#" != 2 ]; then
You might put an error message here, for example, echo "Usage: $0 dir prefix" or even a more elaborate help text.
exit 1
else
echo $2
Please quote, as in echo "$2".
if [ -d $1 ]; then
Here, the quotes are important. Suppose that your directory name has a space in it; then this if would fail with bash: [: a: binary operator expected. So, put quotes around the $1: if [ -d "$1" ]; then
echo "directory"
This is where you could insert the cd "$1".
for i in $(ls $1)
do
It is almost always a bad idea to parse the output of ls. Once again, this for-loop will fail if a file name has a space in it. A possible improvement would be for i in "$1"/* ; do.
for j in $(ls $1)
do
echo $i
if [ $j = $2.$i ]; then
exit 1
else
The logic of this section seems to be: if a file with the prefix exists, then exit instead of overwriting. It is always a good idea to tell why the script fails; an echo before the exit 1 will be helpful.
The question is why you use the second loop? a simple if [ -f "$2.$i ] ; then would do the same, but without the second loop. And it will therefore be faster.
mv -v $i $2.$i
echo $2.$i
Once again: use quotes!
fi
done
done
else
echo "no"
fi
fi
So, with all the remarks, you should be able to improve your script. As tripleee said in his comment, running shellcheck would have provided you with most of the comment above. But he also mentioned basename, which would be useful here.
With all that, this is how I would do it. Some changes you will probably only appreciate in a few months time when you need some changes to the script and try to remember what the logic was that you had in the past.
#!/bin/bash
if [ "$#" != 2 ]; then
echo "Usage: $0 directory prefix" >&2
echo "Put a prefix to all the files in a directory." >&2
exit 1
else
directory="$1"
prefix="$2"
if [ -d "$directory" ]; then
for f in "$directory"/* ; do
base=$(basename "$f")
if [ -f "Sdirectory/$prefix.$base" ] ; then
echo "This would overwrite $prefix.$base; exiting" >&2
exit 1
else
mv -v "$directory/$base" "$directory/$prefix.$base"
fi
done
else
echo "$directory is not a directory" >&2
fi
fi

Why is "ls -1 $fl | wc -l" not returning value 0 in my for loop?

I am trying to add a condition in a for loop to check the existence of a file as well as check for file size > 0 KB.
Period file contains monthly data:
20180101
20180201
20180301
20180401
20180501
There are individual files created for each month. Suppose a file is not created for one month, (20180201), then the loop below should terminate.
For example:
xxx_20180101.txt
xxx_20180301.txt
xxx_20180401.txt
xxx_20180501.txt
if [[ $STATUS -eq 0 ]]; then
for per in `cat ${PATH}/${PERIOD}.txt | cut -f 1 -d";"`
do
for fl in `ls -1 ${PATH}/${FILE} | grep ${per}`
do
if [[ `ls -1 $fl | wc -l` -eq 0 ]]; then
echo "File not found"
STATUS=1
else
if [[ -s "$fl" ]]; then
echo "$fl contain data.">>/dev/null
else
echo "$fl File size is 0KB"
STATUS=1
fi
fi
done
done
fi
but ls -1 $fl | wc -l is not returning 0 value when the if condition is executed.
The following is a demonstration of what a best-practices rewrite might look like.
Note:
We do not (indeed, must not) use a variable named PATH to store a directory under which we look for data files; doing this overwrites the PATH environment variable used to find programs to execute.
ls is not used anywhere; it is a tool intended to generate output for human consumption, not machines.
Reading through input is accomplished with a while read loop; see BashFAQ #1 for more details. Note that the input source for the loop is established at the very end; see the redirection after the done.
Finding file sizes is done with stat -c here; for more options, portable to platforms where stat -c is not supported, see BashFAQ #87.
Because your filename format is well-formed (with an underscore before the substring from your input file, and a .txt after that substring), we're refining the glob to look only for names matching that restriction. This prevents a search for 001 to find xxx_0015.txt, xxx_5001.txt, etc. as well.
#!/usr/bin/env bash
# ^^^^ -- NOT /bin/sh; this lets us use bash-only syntax
path=/provided/by/your/code # replacing buggy use of PATH in original code
period=likewise # replacing use of PERIOD in original code
shopt -s nullglob # generate a zero-length list for unmatched globs
while IFS=';' read -r per _; do
# populate an array with a list of files with names containing $per
files=( "$path/$period/"*"_${per}.txt" )
# if there aren't any, log a message and proceed
if (( ${#files[#]} == 0 )); then
echo "No files with $per found in $path/$period" >&2
continue
fi
# if they *do* exist, loop over them.
for file in "${files[#]}"; do
if [[ -s "$file" ]]; then
echo "$file contains data" >&2
if (( $(stat -c +%s -- "$file") >= 1024 )); then
echo "$file contains 1kb of data or more" >&2
else
echo "$file is not empty, but is smaller than 1kb" >&2
fi
else
echo "$file is empty" >&2
fi
done
done < "$path/$period.txt"
Here's a refactoring of Mikhail's answer with the standard http://shellcheck.net/ warnings ironed out. I have not been able to understand the actual question well enough to guess whether this actually solves the OP's problem.
while IFS='' read -r per; do
if [ -e "xxx_$per.txt" ]; then
echo "xxx_$per.txt found" >&2
else
echo "xxx_$per.txt not found" >&2
fi
done <periods.txt
You are over engineering here. Just iterate over content of file with periods and search each period in a list of files. Like this:
for per in `cat periods.txt`
do
if ls | grep -q "$per"; then
echo "$per found";
else
echo "$per not found"
fi
done

While loop does not execute

I currently have this code:
listing=$(find "$PWD")
fullnames=""
while read listing;
do
if [ -f "$listing" ]
then
path=`echo "$listing" | awk -F/ '{print $(NF)}'`
fullnames="$fullnames $path"
echo $fullnames
fi
done
For some reason, this script isn't working, and I think it has something to do with the way that I'm writing the while loop / declaring listing. Basically, the code is supposed to pull out the actual names of the files, i.e. blah.txt, from the find $PWD.
read listing does not read a value from the string listing; it sets the value of listing with a line read from standard input. Try this:
# Ignoring the possibility of file names that contain newlines
while read; do
[[ -f $REPLY ]] || continue
path=${REPLY##*/}
fullnames+=( $path )
echo "${fullnames[#]}"
done < <( find "$PWD" )
With bash 4 or later, you can simplify this with
shopt -s globstar
for f in **/*; do
[[ -f $f ]] || continue
path+=( "$f" )
done
fullnames=${paths[#]##*/}

How to test filename expansion result in bash?

I want to check whether a directory has files or not in bash.
My code is here.
for d in {,/usr/local}/etc/bash_completion.d ~/.bash/completion.d
do
[ -d "$d" ] && [ -n "${d}/*" ] &&
for f in $d/*; do
[ -f "$f" ] && echo "$f" && . "$f"
done
done
The problem is that "~/.bash/completion.d" has no file.
So, $d/* is regarded as simple string "~/.bash/completion.d/*", not empty string which is result of filename expansion.
As a result of that code, bash tries to run
. "~/.bash/completion.d/*"
and of course, it generates error message.
Can anybody help me?
If you set the nullglob bash option, through
shopt -s nullglob
then globbing will drop patterns that don't match any file.
# NOTE: using only bash builtins
# Assuming $d contains directory path
shopt -s nullglob
# Assign matching files to array
files=( "$d"/* )
if [ ${#files[#]} -eq 0 ]; then
echo 'No files found.'
else
# Whatever
fi
Assignment to an array has other benefits, including desirable (correct!) handling of filenames/paths containing white-space, and simple iteration without using a sub-shell, as the following code does:
find "$d" -type f |
while read; do
# Process $REPLY
done
Instead, you can use:
for file in "${files[#]}"; do
# Process $file
done
with the benefit that the loop is run by the main shell, meaning that side-effects (such as variable assignment, say) made within the loop are visible for the remainder of script. Of course, it's also way faster, if performance is an issue.
Finally, an array can also be inserted in command line arguments (without splitting arguments containing white-space):
$ md5sum fileA "${files[#]}" fileZ
You should always attempt to correctly handle files/paths containing white-space, because one day, they will happen!
You could use find directly in the following way:
for f in $(find {,/usr/local}/etc/bash_completion.d ~/.bash/completion.d -maxdepth 1 -type f);
do echo $f; . $f;
done
But find will print a warning if some of the directory isn't found, you can either put a 2> /dev/null or put the find call after testing if the directories exist (like in your code).
find() {
for files in "$1"/*;do
if [ -d "$files" ];then
numfile=$(ls $files|wc -l)
if [ "$numfile" -eq 0 ];then
echo "dir: $files has no files"
continue
fi
recurse "$files"
elif [ -f "$files" ];then
echo "file: $files";
:
fi
done
}
find /path
Another approach
# prelim stuff to set up d
files=`/bin/ls $d`
if [ ${#files} -eq 0 ]
then
echo "No files were found"
else
# do processing
fi

Resources