I need to copy a list of filenames in a textfile. Trying by this:
#!/bin/sh
mjdstart=55133
mjdend=56674
datadir=/nfs/m/ir1/ssc/evt
hz="h"
for mjd in $(seq $mjdstart $mjdend); do
find $datadir/ssc"${hz}"_allcl_mjd"${mjd}".evt -maxdepth 1 -type f -printf $datadir'/%f\n' > ssc"${hz}".list
done
I tried also:
find $datadir/ssc"${hz}"_allcl_mjd"${mjd}".evt -maxdepth 1 -type f -printf $datadir'/%f\n' | split -l999 -d - ssc"${hz}".list
Or other combinations, but clearly I am missing something: the textfile is empty. Where is my mistake?
Use >> (append) instead of > (overwrite) otherwise you will have output of last command only:
> ssc"${hz}".list
for mjd in $(seq $mjdstart $mjdend); do
find $datadir/ssc"${hz}"_allcl_mjd"${mjd}".evt -maxdepth 1 -type f -printf $datadir'/%f\n' >> ssc"${hz}".list
done
You don't need to use find here, as you simply have a range of specific file names whose existence you are checking for:
#!/bin/sh
mjdstart=55133
mjdend=56674
datadir=/nfs/m/ir1/ssc/evt
hz="h"
for mjd in $(seq $mjdstart $mjdend); do
fnname="$datadir/ssc${hz}_allcl_mjd${mjd}.evt"
[[ -f $fname ]] && printf "$fname\n"
done > "ssc$hz.list"
You are using find wrong. The first argument is the directory, in which it should search. Also, using > overwrites your list file in every turn. Use >> to concatenate:
find $datadir -maxdepth 1 -type f -name "src${hz}_allcl_mjd${mjd}.evt" >> ssc"${hz}".list
Related
I am using the following bash line to find text files in a subfolder with a given a pattern inside it and move them to a subfolder:
find originalFolder/ -maxdepth 1 -type f -exec grep -q 'mySpecificPattern' {} \; -exec mv -i {} destinationFolder/ \;
Now instead of grepping a pattern, I would like to move the files to a subfolder if they consist only of a single line (of text): how can I do that?
You can do it this way:
while IFS= read -r -d '' file; do
[[ $(wc -l < "$file") -eq 1 ]] && echo mv -i "$file" destinationFolder/
done < <(find originalFolder/ -maxdepth 1 -type f -print0)
Note use of echo in front of mv so that you can verify output before actually executing mv. Once you're satisfied with output, remove echo before mv.
Using wc as shown above is the most straightforward way, although it reads the entire file to determine the length. It's also possible to do length checks with awk, and the exit function lets you fit that into a find command.
find . -type f -exec awk 'END { exit (NR==1 ? 0 : 1) } NR==2 { exit 1 }' {} \; -print
The command returns status 0 if there has been only 1 input record at end-of-file, and it also exits immediately with status 1 when line 2 is encountered; this should easily outrun wc if large files are a performance concern.
If you want a breakdown of how many files are in each dir under your current dir:
for i in $(find . -maxdepth 1 -type d) ; do
echo -n $i": " ;
(find $i -type f | wc -l) ;
done
It does not work when the directory name has a blank in the name. Can anyone here tell me how I must edite this shell script so that such directory names also accepted for counting its file contents?
Thanks
Your code suffers from a common issue described in http://mywiki.wooledge.org/BashPitfalls#for_i_in_.24.28ls_.2A.mp3.29.
In your case you could do this instead:
for i in */; do
echo -n "${i%/}: "
find "$i" -type f | wc -l
done
This will work with all types of file names:
find . -maxdepth 1 -type d -exec sh -c 'printf "%s: %i\n" "$1" "$(find "$1" -type f | wc -l)"' Counter {} \;
How it works
find . -maxdepth 1 -type d
This finds the directories just as you were doing
-exec sh -c 'printf "%s: %i\n" "$1" "$(find "$1" -type f | wc -l)"' Counter {} \;
This feeds each directory name to a shell script which counts the files, similarly to what you were doing.
There are some tricks here: Counter {} are passed as arguments to the shell script. Counter becomes $0 (which is only used if the shell script generates an error. find replaces {} with the name of a directory it found and this will be available to the shell script as $1. This is done is a way that is safe for all types of file names.
Note that, wherever $1 is used in the script, it is inside double-quotes. This protects it for word splitting or other unwanted shell expansions.
I found the solution what I have to consider:
Consider_this
#!/bin/bash
SAVEIFS=$IFS
IFS=$(echo -en "\n\b")
for i in $(find . -maxdepth 1 -type d); do
echo -n " $i: ";
(find $i -type f | wc -l) ;
done
IFS=$SAVEIFS
Is there any sort option available in find command to get directory with least access date/time
find . -type d -printf "%A# %p\n" | sort -n | tail -n 1 | cut -d " " -f 2-
If you prefer the filename without leading path, replace %p by %f.
the below linux command displays the access and modified time along with size
stat -f
find -type d -printf '%T+ %p\n' | sort | head -1
source
find -type d -printf '%T+ %p\n' | sort
This sound like more of a job for ls:
ls -ultd *|grep ^d
The problem with using find, at least on my system (cygwin/bash), is that find accesses the dirs, so all access-times result in current time, defeating your apparent purpose.
A simple shell script will also do:
unset -v oldest
for i in "$dir"/*; do
[ "$i" -ot "$oldest" -o "$oldest" = "" ] && oldest="$i"
done
note: to find the oldest directory use "$dir"/*/ above (thanks Cyrus) and -type d below with the find command.
In bash if you need a recursive solution, then you can rewrite it as a while loop with process substitution using find
unset -v oldest
while IFS= read -r i; do
[ "$i" -ot "$oldest" -o "$oldest" = "" ] && oldest="$i"
done < <(find "$dir" -type f)
I have a bash script which searches through all the sub-directories (at all levels) given a target directory:
#! /bin/bash
DIRECTORIES="/home/me/target_dir_1"
for curr in $DIRECTORIES
do
...
Now I want the script to search multiple target directories such as target_dir_1, target_dir_2, target_dir_3. How should I modify the script to do this?
use find instead.
find /home/me/target_dir_1 -type d
You can put that in a for loop:
for d in target_dir_1 target_dir_2
do
find /home/me/"$d" -type d
done
If it is always /home/me, and you want to search all the directories under that, do the follwing:
find /home/me -type d
#!/bin/bash
GIVEN_DIR=$1 ## Or you could just set the value here instead of using $1.
while read -r DIR; do
echo "$DIR" ## do something with subdirectory.
done < <(exec find "$GIVEN_DIR" -type d -mindepth 1)
Run with:
bash script.sh dir
Note that word splitting is a bad idea so don't do this:
IFS=$'\n'
for DIR in $(find "$GIVEN_DIR" -type d -mindepth 1); do
echo "$DIR" ## do something with subdirectory.
done
Neither with other forms like when you could use -print0 for find, although it's fine if you still use while:
while read -r DIR -d $'\0'; do
echo "$DIR" ## do something with subdirectory.
done < <(exec find "$GIVEN_DIR" -type d -mindepth 1 -print0)
Lastly you could record those on an array:
readarray -t SUBDIRS < <(exec find "$GIVEN_DIR" -type d -mindepth 1)
for DIR in "${SUBDIRS[#]}"; do
echo "$DIR" ## do something with subdirectory.
done
Say:
for i in /home/me/target_dir_{1..5}; do
echo $i;
done
This would result in:
/home/me/target_dir_1
/home/me/target_dir_2
/home/me/target_dir_3
/home/me/target_dir_4
/home/me/target_dir_5
Alternatively, you can specify the variable as an array and loop over it:
DIRECTORIES=( /home/me/target_dir_1 /home/me/target_dir_2 /home/me/target_dir_3 )
for i in ${DIRECTORIES[#]}; do echo $i ; done
which would result in
/home/me/target_dir_1
/home/me/target_dir_2
/home/me/target_dir_3
I have a list of 3,900 ID numbers and I need to find on our FTP server the matching files.
Finding one file is quite simple e.g.
find . -name "*IDNumber*" -exec ls '{}' ';' -print
but how do I do this for 3,900 IDs numbers? I created a file with the IDs like so
028892663163
028923481973
...
but how do I pass the list of ID numbers as argument? Can you provide some pointers?
Thanks!
I would try to reduce the number of times you have to invoke find:
find . -type f -print | grep -f id.file | xargs cp -t target_dir
You may try to optimize it by running find with more than one id at a time.
With bash (100 at a time, you may try with more):
c= p=
while IFS= read -r; do
p+=" -name '*$REPLY*' -o "
(( ++c ))
(( c % 100 )) || {
eval find . ${p% -o }
p=
}
done < id_list_all
[[ $p ]] &&
eval find . ${p% -o }
Figured it out.
put all my 3,900 ID numbers in a file outfile
typed the command line:
cat outfile | while read line
do
find . -name "$line" -exec cp '{}' /target_directory ';' -print
done
Worked awesome!
I read your question wrong the first time... arguments from find to other things. What you want is arguments from a file passed to find. So, here's the correct answer with xargs:
xargs --max-args=1 -I X -d '\n' find . -name X -exec [...] < your_list