if I write this string in my script:
list=$(ls /path/to/some/files/*/*/*.dat)
it works fine. But what I need is
files="files/*/*/*.dat"
list=$(ls /path/to/some/${files})
and it says
ls: /path/to/some/files/*/*/*.dat: No such file or directory
How should I do it?
If you only get that message where there truly are no matching .dat files, add this to your script:
shopt -s nullglob
It will cause the glob to expand to an empty list if there are no matching files, rather than being treated literally.
Try this:
list=$(find /path/to/some/files/ -mindepth 3 -maxdepth 3 -name '*.dat')
Related
As a part of my bash routine I am trying to add IF condition which should remove all csv filles contained pattern "filt" in their names:
# this is a folder contained all subfolders
results=./results
# looping all directories located in the $results
for d in "${results}"/*/; do
if [ -f "${d}"/*filt*.csv ]; check if csv file is present within dir $d
rm "${d}"/*filt*.csv # remove this csv file
fi
done
Although a version without condition find and removes the csv properly:
rm "${d}"/*filt*.csv
While executing my example with IF gives the following error:
line 27: [: too many arguments
where the line 27 corresponds to the IF condition. How it could be fixed?
can I use something like without any IF statement?
# find all CSV filles matching pattern within the "${d}"
find "${d}" -type f -iname *filt*.csv* -delete
You could use shopt -s nullglob and then skip the test and use rm -f "$d"/*filt*.csv directly. The nullglob option makes sure that the glob not matching anything expands to the empty string, and -f would silence rm.
You could also skip the outer loop and simplify everything to
shopt -s nullglob
rm -f results/*filt*.csv
This could fail if the glob matches so many files that the maximum line length is exceeded. In that case, you're better off with find:
find results -name '*filt*.csv' -exec rm {} +
or, with GNU find:
find results -name '*filt*.csv' -delete
If there are subdirectories you want to skip, use -maxdepth 1. If there are directories matching the pattern, use -type f.
Im trying to write a command that deletes all files/folders starting with "test", expcet zip files that also start with test.
So far i have:
rm -rf test*[!zip]
This almost works, but not quite.
If i have following files:
test
test1
test.zip
Then after the command remaining are:
test
test.zip
So the problem is that test should also be deleted. I understand that my command only matches things that have at least one extra character besides "test", but not sure how to fix it.
Your glob pattern test*[!zip] will require at least one character after test due to presence of [!zip] part.
You may use this extended glob pattern in bash:
shopt -s extglob nullglob
printf '%s\n' test!(*zip)
Or else, you may use this find:
find . -maxdepth 1 -mindepth 1 -type f -name 'test*' -not -name 'test*zip'
I want to iterate all the files in a directory in a bash script.
List all files with extentions .LOG .txt .MAP .TL9*
List all files which have no extention.
I am trying this :
for file in *.{LOG,txt,MAP,TL9*}; do
I want to list the files, that only ends with above extension.
So, I do not want to list a file: temp.txt.EXT because it does not end with above given extentions. Similarly I don't want this to be reported temp.TL94.JPG or temp.TL9.JPG.
But in this above for loop, how do i insert the check which gives me the file with no extention?
Please help.
Using extglob, you can do this:
shopt -s nullglob
shopt -s extglob
for file in #(!(*.*)|*.#(csv|LOG|TL9!(*.*))); do
echo "$file"
done
With extglob:
*.#(LOG|txt|MAP|TL9) !(*.*)
*.#(LOG|txt|MAP|TL9) matches all .LOG, txt, .MAP, and .TL9 files
!(*.*) matches all files except ones having . in name
Enable extglob first if not enabled:
shopt -s extglob
You could also use the find command to list files with extension MAP, LOG, TL9or without any extension at all.
#!/bin/bash
files=`find . -type f -regex ".*[\.LOG|\.MAP|\.TL9]" -o ! -name "*.*"`
for file in $files
do
echo $file
done
I'm trying to pass parameter to rm in bash script to clean my system automatically. For example, I want to remove everything except the *.doc files. So I wrote the following codes.
#!/bin/bash
remove_Target="!*.txt"
rm $remove_Target
However, the output always say
rm: cannot remove ‘!*.txt’: No such file or directory
It is obviously that bash script add single quotes for me when passing the variable to rm. How can I remove the single quotes?
Using Bash
Suppose that we have a directory with three files
$ ls
a.py b.py c.doc
To delete all except *.doc:
$ shopt -s extglob
$ rm !(*.doc)
$ ls
c.doc
!(*.doc) is an extended shell glob, or extglob, that matches all files except those ending in .doc.
The extglob feature requires a modern bash.
Using find
Alternatively:
find . -maxdepth 1 -type f ! -name '*.doc' -delete
#!/bin/bash
outbound=/home/user/outbound/
putfile=DATA_FILE_PUT_*.CSV
cd $outbound
filecnt=0
for file in $putfile; do let filecnt=filecnt+1; done
echo "Filecount: " $filecnt
So this code works well when there are files located in the outbound directory. I can place files into the outbound path and as long as they match the putfile mask then the files are incremented as expected.
Where the problem comes in is if I run this while there are no files located in $outbound.
If there are zero files there $filecnt still returns a 1 but I'm looking to have it return a 0 if there are no files there.
Am I missing something simple?
Put set -x just below the #! line to watch what your script is doing.
If there is no matching file, then the wildcard is left unexpanded, and the loop runs once, with file having the value DATA_FILE_PUT_*.CSV.
To change that, set the nullglob option. Note that this only works in bash, not in sh.
shopt -s nullglob
putfile=DATA_FILE_PUT_*.CSV
for file in $putfile; do let filecnt=filecnt+1; done
Note that the putfile variable contains the wildcard pattern, not the list of file names. It might make more sense to put the list of matches in a variable instead. This needs to be an array variable, and you need to change the current directory first. The number of matching files is then the length of the array.
#!/bin/bash
shopt -s nullglob
outbound=/home/user/outbound/
cd "$outbound"
putfiles=(DATA_FILE_PUT_*.CSV)
echo "Filecount: " ${#putfiles}
If you need to iterate over the files, take care to protect the expansion of the array with double quotes, otherwise if a file name contains whitespace then it will be split over several words (and if a filename contains wildcard characters, they will be expanded).
#!/bin/bash
shopt -s nullglob
outbound=/home/user/outbound/
cd "$outbound"
putfiles=(DATA_FILE_PUT_*.CSV)
for file in "${putfiles[#]}"; do
echo "Processing $file"
done
You could test if file exists first
for file in $putfile; do
if [ -f "$file" ] ; then
let filecnt=filecnt+1
fi
done
Or look for your files with find
for file in $(find . -type f -name="$putfile"); do
let filecnt=filecnt+1
done
or simply (fixed)
filecnt=$(find . -type f -name "$putfile" | wc -l); echo $filecnt
This is because when no matches are found, bash by default expands the wildcard DATA_FILE_PUT_*.CSV to the word DATA_FILE_PUT_*.CSV and therefore you end up with a count of 1.
To disable this behavior, use shopt -s nullglob
Not sure why you need a piece of code here. Following one liner should do your job.
ls ${outbound}/${putfile} | wc -l
Or
find ${outbound} -maxdepth 1 -type f -name "${putfile}" | wc -l