The directories are variables set to the full-path
for e in "$DIR_0" "$DIR_1" "$DIR_2"
do
for i in $e/*
do
echo $i
done
The output for each line is the full path. I want only the name of each file
You are looking for basename.
This is the Bash equivalent of basename:
echo "${i##*/}"
It strips off everything before and including the last slash.
If you truly do not wish to recurse you can achieve that more succinctly with this find command:
find "$DIR_0" "$DIR_1" "$DIR_2" -type f -maxdepth 1 -exec basename{} \;
If you wish to recurse over subdirs simply leave out maxdepth:
find "$DIR_0" "$DIR_1" "$DIR_2" -type f -exec basename{} \;
to traveling a directory recursively with bash
try this you can find it here
#! /bin/bash
indent_print()
{
for((i=0; i < $1; i++)); do
echo -ne "\t"
done
echo "$2"
}
walk_tree()
{
local oldifs bn lev pr pmat
if [[ $# -lt 3 ]]; then
if [[ $# -lt 2 ]]; then
pmat=".*"
else
pmat="$2"
fi
walk_tree "$1" "$pmat" 0
return
fi
lev=$3
[ -d "$1" ] || return
oldifs=$IFS
IFS=""
for el in $1/ *; do
bn=$(basename "$el")
if [[ -d "$el" ]]; then
indent_print $lev "$bn/"
pr=$( walk_tree "$el" "$2" $(( lev + 1)) )
echo "$pr"
else
if [[ "$bn" =~ $2 ]]; then
indent_print $lev "$bn"
fi
fi
done
IFS=$oldifs
}
walk_tree "$1" "\.sh$"
See also the POSIX compliant Bash functions to replace basename & dirname here:
http://cfaj.freeshell.org/src/scripts/
Related
I am trying to find all files by passing a directory name in all sub directories meaning the process is recursive here is my code
myrecursive() {
if [ -f $1 ]; then
echo $1
elif [ -d $1 ]; then
for i in $(ls $1); do
if [ -f $1 ]; then
echo $i
else
myrecursive $i
fi
done
else
echo " sorry"
fi
}
myrecursive $1
However when I pass directory with another directory I get 2 times sorry,where is my mistake?
The goal that you are trying to achieve could be simply done by using find command:
# will search for all files recursively in current directory
find . * -exec echo {} \;
# will search for all *.txt file recursively in current directory
find . -name "*.txt" -exec echo {} \;
# will search for all *.txt file recursively in current directory
# but depth is limited to 3
find . -name "*.txt" -max-depth 3 -exec echo {} \;
See man find for manual. How to run find -exec?
The problem with your code is quite simple.
The ls command will return a list of filenames, but they aren't valid for
recursion. Use globbing instead. The loop below simply replaces $(ls) with $1/*
myrecursive() {
if [ -f $1 ]; then
echo $1
elif [ -d $1 ]; then
for i in $1/*; do
if [ -f $1 ]; then
echo $i
else
myrecursive $i
fi
done
else
echo " sorry"
fi
}
myrecursive $1
Hope that helps
#!/bin/bash
myrecursive() {
if [ -f "$1" ]; then
echo "$1"
elif [ -d "$1" ]; then
for i in "$1"/*; do
if [ -f "$i" ]; then #here now our file is $i
echo "$i"
else
myrecursive "$i"
fi
done
else
echo " sorry"
fi
}
myrecursive "$1"
I want to find and print files in directory tree, that have the sname name as theirs dirs.
This is my code so far:
#!bin/bash
if [ $# -eq 0 ]
then
echo "No args"
fi
if [[ -d $1 ]] #if its dir
then
find $1 -type f | (while read var1 #for every regular file in dir tree
do
if [[ -f $var1 ]]
then
echo $var1 #full path
# I dont know how to get the dir name
echo $(basename $var1) #file name
echo
#then compare it and print full path
fi
done)
fi
I want to do this using FIND function in bash linux. Thanks
You can use this script with find:
while IFS= read -rd '' f; do
d="${f%/*}"
[[ ${d##*/} == ${f##*/} ]] && echo "$f"
done < <(find . -type f -print0)
I've written an OCR wrapper batch & service script for tesseract and abbyyocr11 found here: https://github.com/deajan/pmOCR
The main function is a find command that passes it's arguments to xargs with -print0 in order to deal with special filenmames.
The find command became more and more complex and ended up as a VERY long one liner that becomes difficult to maintain:
find "$DIRECTORY_TO_PROCESS" -type f -iregex ".*\.$FILES_TO_PROCES" ! -name "$find_excludes" -print0 | xargs -0 -I {} bash -c 'export file="{}"; function proceed { eval "\"'"$OCR_ENGINE_EXEC"'\" '"$OCR_ENGINE_INPUT_ARG"' \"$file\" '"$OCR_ENGINE_ARGS"' '"$OCR_ENGINE_OUTPUT_ARG"' \"${file%.*}'"$FILENAME_ADDITION""$FILENAME_SUFFIX$FILE_EXTENSION"'\" && if [ '"$_BATCH_RUN"' -eq 1 ] && [ '"$_SILENT"' -ne 1 ];then echo \"Processed $file\"; fi && echo -e \"$(date) - Processed $file\" >> '"$LOG_FILE"' && if [ '"$DELETE_ORIGINAL"' == \"yes\" ]; then rm -f \"$file\"; fi"; }; if [ "'$CHECK_PDF'" == "yes" ]; then if ! pdffonts "$file" 2>&1 | grep "yes" > /dev/null; then proceed; else echo "$(date) - Skipping file $file already containing text." >> '"$LOG_FILE"'; fi; else proceed; fi'
Is there a nicer way to pass the find results to a human readable function (without impacting too much speed) ?
Thanks.
Don't use bash -c. You are already committed to starting a new bash process for each file from the find command, so just save the code to a file and run that with
find "$DIRECTORY_TO_PROCESS" -type f -iregex ".*\.$FILES_TO_PROCES" \
! -name "$find_excludes" -print0 |
xargs -0 -I {} bash script.bash {}
You can replace find altogether. It's easier in bash 4 (which I'll show here), but doable in bash 3.
proceed () {
...
}
shopt -s globstar
extensions=(pdf tif tiff jpg jpeg bmp pcx dcx)
for ext in "${extensions[#]}"; do
for file in /some/path/**/*."$ext"; do
[[ ! -f $file || $file = *_ocr.pdf ]] && continue
# Rest of script here
done
done
Prior to bash 4, you can write your own recursive function to descend through a directory hierarchy.
descend () {
for fd in "$1"/*; do
if [[ -d $fd ]]; then
descend "$fd"
elif [[ ! -f $fd || $fd != *."$ext" || $fd = *_ocr.pdf ]]; then
continue
else
# Rest of script here
fi
done
}
for ext in "${extensions[#]}"; do
descend /some/path "$ext"
done
OK, create the script, then run find.
#!/bin/bash
trap cleanup EXIT
cleanup() { rm "$script"; }
script=$(mktemp)
cat <<'END' > "$script"
########################################################################
file="$1"
function proceed {
"$OCR_ENGINE_EXEC" "$OCR_ENGINE_INPUT_ARG" "$file" "$OCR_ENGINE_ARGS" "$OCR_ENGINE_OUTPUT_ARG" "${file%.*}$FILENAME_ADDITION$FILENAME_SUFFIX$FILE_EXTENSION"
if [ "$_BATCH_RUN" -eq 1 ] && [ "$_SILENT" -ne 1 ]; then
echo "Processed $file"
fi
echo -e "$(date) - Processed $file" >> "$LOG_FILE"
if [ "$DELETE_ORIGINAL" == "yes" ]; then
rm -f "$file"
fi
}
if [ "$CHECK_PDF" == "yes" ]; then
if ! pdffonts "$file" 2>&1 | grep "yes" > /dev/null; then
proceed
else
echo "$(date) - Skipping file $file already containing text." >> '"$LOG_FILE"';
fi
else
proceed
fi
########################################################################
END
find "$DIRECTORY_TO_PROCESS" -type f \
-iregex ".*\.$FILES_TO_PROCES" \
! -name "$find_excludes" \
-exec bash "$script" '{}' \;
The 'END' of the heredoc is quoted, so the variables are not expanded until the script is actually executed.
I finished using a while loop with a substituted find command, ie:
while IFS= read -r -d $'\0' file; do
if ! lsof -f -- "$file" > /dev/null 2>&1; then
if [ "$_BATCH_RUN" == true ]; then
Logger "Preparing to process [$file]." "NOTICE"
fi
OCR "$file" "$fileExtension" "$ocrEngineArgs" "$csvHack"
else
if [ "$_BATCH_RUN" == true ]; then
Logger "Cannot process file [$file] currently in use." "ALWAYS"
else
Logger "Deferring file [$file] currently being written to." "ALWAYS"
kill -USR1 $SCRIPT_PID
fi
fi
done < <(find "$directoryToProcess" -type f -iregex ".*\.$FILES_TO_PROCES" ! -name "$findExcludes" -and ! -wholename "$moveSuccessExclude" -and ! -wholename "$moveFailureExclude" -and ! -name "$failedFindExcludes" -print0)
The while loop reads every file from the find command in file variable.
Using -d $'\0' in while and -print0 in find command helps dealing with special filenames.
I'm trying to write a shell script that will recursively count all the files and sub-directories in a directory and also all the hidden ones. My script can count them however it can't detect hidden files and directories that are in a sub-directory. How can i change it so that it is able to do this? Also i cannot use find, du or ls -R
#!/bin/bash
cd $1
dir=0
hiddendir=0
hiddenfiles=0
x=0
items=( $(ls -A) )
amount=( $(ls -1A | wc -l) )
counter() {
if [ -d "$i" ]; then
let dir+=1
if [[ "$i" == .* ]]; then
let hiddendir+=1
let dir-=1
fi
search "$i"
elif [ -f "$i" ]; then
let files+=1
if [[ "$i" == .* ]]; then
let files-=1
let hiddenfiles+=1
fi
fi
}
search() {
for i in $1/*; do
counter "$i"
done
}
while [ $x -lt $amount ]; do
i=${items[$x]}
counter "$i"
let x+=1
done
#!/bin/bash -e
shopt -s globstar dotglob # now ** lists all entries recursively
cd "$1"
dir=0 files=0 hiddendir=0 hiddenfiles=0
counter() {
if [ -f "$1" ]; then local typ=files
elif [ -d "$1" ]; then local typ=dir
else continue
fi
[[ "$(basename "$1")" == .* ]] && local hid=hidden || local hid=""
((++$hid$typ))
}
for i in **; do
counter "$i"
done
echo $dir $files $hiddendir $hiddenfiles
Consider using this:
find . | wc -l
I have to search all subdirs recursively and print *(number of * = depth of file/dir) type and name. The problem comes when i enter dir and then want to get out but nothing happens.
my test file
DIR test
*FILE ace
*FILE base
*DIR father
**FILE cookies
*DIR mother
**DIR how
***FILE youdoing
*FILE zebra
my code
maxDepth is how far in to the dir it can go(default 3) and currDepth is 1 at the beginning
function tree(){
maxDepth=$2
currDepth=$3
#print the starting file
if [ "$currDepth" -eq 0 ];then
printf "%s %s\n" DIR "$1"
currDepth=1
fi
for path in "$1"/*;do
for i in $( seq 1 $currDepth );do
echo -n *
done
if [ -d "$path" ];then
printf "%s %s\n" DIR "${path##*/}"
if [[ "$currDepth" -lt "$maxDepth" ]];then
tree "$path" "$maxDepth" "$(( currDepth + 1 ))"
fi
continue
fi
if [ -f "$path" ];then
printf "%s %s\n" FILE "${path##*/}"
continue
fi
if [ -L "$path" ];then
printf "%s %s\n" LINK "${path##*/}"
continue
fi
done
}
my output
DIR test
*FILE ace
*FILE base
*DIR father
**FILE cookies
**DIR mother
***DIR how
***FILE zebra
what am i doing wrong
Debug your script by doing set -x before you run it.
Make sure integers are always integers by declaring them with the -i integer attribute.
Use expression syntax consistently. It's a good idea to always use [[ ]] tests for string comparisons and (( )) for arithmetic and numeric comparisons, if your target shell is bash.
Use (( )) for loops instead of seq, which is nonstandard.
Explicitly declare your variables in the function (using local or declare) to ensure they are scoped to the function.
Actually call the inner tree.
#!/bin/bash
tree() {
local -i maxDepth=$2 # Make sure these values are always integers
local -i currDepth=$3
# print the starting file
if (( currDepth == 0 )); then # use bash arithmetic
printf "%s %s\n" DIR "$1"
currDepth=1
fi
for path in "$1"/*;do
for ((i=0; i<currDepth; i++)); do
printf '*'
done
if [[ -d "$path" ]];then
printf "%s %s\n" DIR "${path##*/}"
if [[ "$currDepth" -lt "$maxDepth" ]];then
tree "$path" "$maxDepth" "$(( currDepth + 1 ))"
fi
continue
fi
if [[ -f "$path" ]];then
printf "%s %s\n" FILE "${path##*/}"
continue
fi
if [[ -L "$path" ]];then
printf "%s %s\n" LINK "${path##*/}"
continue
fi
done
}
Here a solution using find, stat and sed:
find <DIR> -exec stat --printf="%n,%F\n" "{}" \; | \
sed -r -e "s/[^\/]+\//\*/g" -e "s/regular file/FILE/" -e "s/directory/DIR/" | \
sed -r -e "s/([\*]+)([^,]+),(.+)/\1 \3 \2/"
IMPORTANT: Use DIR not DIR/ otherwise DIR name will not appear in results.
Explanation:
find returns recursively all files and directory within DIR.
-exec option in find allows to pass each result to another command.
Here I'm passing each result to the command stat
stat has an option to format the output -printf (see manpage) :
%n is the filename (with relavtive path)
%F is the file type ( regular file, directory,symbolic link,block special file...)
So,
find <DIR> -exec stat --printf="%n,%F\n" "{}" \;
returns the following, one result by line (assuming that there are only regular files and directories in DIR) :
DIR/path/to/file,regular file
DIR/path/to/dir,directory
Then, I'm using sed to transform each line the way you required using regular expression:
Replace string/ by * -> ***basename,file type
Replace "regular file" by FILE
Replace "directory" by DIR
Shuffle around basename and filetype using back referencing in sed.
NOTE: I will not explain in details how regular expressions work as it would be too long.
I should have used local in front of currDepth=$3