Is there a way to quickly know whether an executable in my $PATH contains a particular string? For instance, I want to quickly list the executables that contain SRA.
The reason I'm asking is that I have several scripts with the characters SRA in them. The problem is that I always forget the starting character of the file (if I do remember, I use tab completion to find it).
You can store all the paths in an array and then use find with its various options:
IFS=":" read -ra paths <<< "$PATH"
find "${paths[#]}" -type f -executable -name '*SRA*'
IFS=":" read -ra paths <<< "$PATH" reads all the paths into an array, setting the field separator temporarily to :, as seen in Setting IFS for a single statement.
-type f looks for files.
-executable looks for files that are executable. You may use -perm +111 instead in OSX (source).
Since the -executable option is not available in FreeBSD or OSX, ghoti nicely recommends to use the -perm option:
find -perm -o=x,-g=x,-u=x
For example:
find ${PATH//:/ } -maxdepth 1 -executable -name '*SRA*'
And if you happen to have spaces (or other hazardous characters) in the $PATH (the <<< trick borrowed from the answer of #fedorqui):
tr ":\n" "\\000" <<< "$PATH" | \
xargs -0r -I{} -n1 find {} -maxdepth 1 -executable -name '*SRA*'
It also handles the empty $PATH correctly.
A bit clumsy:
find $(echo $PATH | tr : ' ') -name \*SRA\*
I wrote a bash script that wraps this up for OSX, based off this great answer on this page.
I think will work for other operating systems as well. Note that it also ignores errors, sorts the results, and only shows unique values too!
executables_in_path_matching_substring.sh
#!/usr/bin/env bash
function show_help()
{
ME=$(basename "$0")
IT=$(cat <<EOF
returns a list of files in the path that match a substring
usage: $ME SUBSTRING
e.g.
# Find all files in the path that have "git" in their name
$ME git
EOF
)
echo "$IT"
echo
exit
}
if [ -z "$1" ]
then
show_help
fi
if [ "$1" == "help" ] || [ "$1" == '?' ] || [ "$1" == "--help" ] || [ "$1" == "h" ]; then
show_help
fi
SUBSTRING="$1"
IFS=":" read -ra paths <<< "$PATH"
find "${paths[#]}" -type f -perm +111 -name "*$SUBSTRING*" 2>/dev/null | sort | uniq
Related
Prompt the user for a file name, without the path (Ex: xyz.out)
- Using the find command, provide the full path to the file
- At the end, “print number of locations of that file found”
- If it’s not found, then display “not found
and this is my script
#! /bin /bash
echo "please enter your file name"
read filename
if [ -f $filename ];
then
echo "file $filename found"
find $PWD -type f | grep $filename
#find "$(cd ..; pwd)" -name $filename
else
echo "file $filename was not found"
fi
but the thing is At the end, i need to “print number of locations of that file found”
help me out with this
Something like this to get the count:
find $PWD -type f -name $filename 2>/dev/null | wc -l
This should work:
find "$PWD" -type f -name "$fname" |grep "" -c
In trying to keep it as short as possible, one approach with Posix shell would be to fill a temporary file with the file names returned by find, cat the file to provide your output, and use wc to provide the line count (note: you use your own pattern instead of "*cmpf*" shown below):
$ find . -name "*cmpf*" -printf "%P\n" >tmp; cat tmp; c=$(wc -l <tmp); \
rm tmp; printf "[%s] files found\n" $c
cmpf1f2.c
cmpf1f2_2.c
bin/cmpf1f2_2
bin/cmpf1f2
snip/cmpf1f2_notes.txt
cmpf1f2_for.c
[6] files found
If bash is available, another approach is to read the matching files into an array and then use the number of elements as your count. Example:
$ a=( $(find . -name "*cmpf*" -printf "%P\n") ); printf "%s\n" ${a[#]}; \
printf -- "--\n'%s' files found.\n" ${#a[#]}
cmpf1f2.c
cmpf1f2_2.c
bin/cmpf1f2_2
bin/cmpf1f2
snip/cmpf1f2_notes.txt
cmpf1f2_for.c
--
'6' files found.
Both approaches give you both the files and directories in which they reside as well as the count of the files returned.
Note: if you would like ./ before each file and directory names, use the %p format instead of %P above.
How to verify that a file has absolutely no content. [ -s $file ] gives if file is zero bytes but how to know if file is absolutely empty with no data that including empty lines ?
$cat sample.text
$ ls -lrt sample.text
-rw-r--r-- 1 testuser userstest 1 Jul 31 16:38 sample.text
When i "vi" the file the bottom has this - "sample.text" 1L, 1C
Your file might have new line character only.
Try this check:
[[ $(tr -d "\r\n" < file|wc -c) -eq 0 ]] && echo "File has no content"
A file of 0 size by definition has nothing in it, so you are good to go. However, you probably want to use:
if [ \! -s f ]; then echo "0 Sized and completely empty"; fi
Have fun!
Blank lines add data to the file and will therefore increase the file size, which means that just checking whether the file is 0 bytes is sufficient.
For a single file, the methods using the bash built-in -s (for test, [ or [[). ([[ makes dealing with ! less horrible, but is bash-specific)
fn="file"
if [[ -f "$fn" && ! -s "$fn" ]]; then # -f is needed since -s will return false on missing files as well
echo "File '$fn' is empty"
fi
A (more) POSIX shell compatible way: (The escaping of exclamation marks can be shell dependant)
fn="file"
if test -f "$fn" && test \! -s "$fn"; then
echo "File '$fn' is empty"
fi
For multiple files, find is a better method.
For a single file you can do: (It will print the filename if empty)
find "$PWD" -maxdepth 1 -type f -name 'file' -size 0 -print
for multiple files, matching the glob glob*:(It will print the filenames if empty)
find "$PWD" -maxdepth 1 -type f -name 'glob*' -size 0 -print
To allow subdirectories:
find "$PWD" -type f -name 'glob*' -size 0 -print
Some find implementations does not require a directory as the first parameter (some do, like the Solaris one). On most implementations, the -print parameter can be omitted, if it is not specified, find defaults to printing matching files.
I have been looking for a way to list file that do not exist from a list of files that are required to exist. The files can exist in more than one location. What I have now:
#!/bin/bash
fileslist="$1"
while read fn
do
if [ ! -f `find . -type f -name $fn ` ];
then
echo $fn
fi
done < $fileslist
If a file does not exist the find command will not print anything and the test does not work. Removing the not and creating an if then else condition does not resolve the problem.
How can i print the filenames that are not found from a list of file names?
New script:
#!/bin/bash
fileslist="$1"
foundfiles="~/tmp/tmp`date +%Y%m%d%H%M%S`.txt"
touch $foundfiles
while read fn
do
`find . -type f -name $fn | sed 's:./.*/::' >> $foundfiles`
done < $fileslist
cat $fileslist $foundfiles | sort | uniq -u
rm $foundfiles
#!/bin/bash
fileslist="$1"
while read fn
do
FPATH=`find . -type f -name $fn`
if [ "$FPATH." = "." ]
then
echo $fn
fi
done < $fileslist
You were close!
Here is test.bash:
#!/bin/bash
fn=test.bash
exists=`find . -type f -name $fn`
if [ -n "$exists" ]
then
echo Found it
fi
It sets $exists = to the result of the find. the if -n checks if the result is not null.
Try replacing body with [[ -z "$(find . -type f -name $fn)" ]] && echo $fn. (note that this code is bound to have problems with filenames containing spaces).
More efficient bashism:
diff <(sort $fileslist|uniq) <(find . -type f -printf %f\\n|sort|uniq)
I think you can handle diff output.
Give this a try:
find -type f -print0 | grep -Fzxvf - requiredfiles.txt
The -print0 and -z protect against filenames which contain newlines. If your utilities don't have these options and your filenames don't contain newlines, you should be OK.
The repeated find to filter one file at a time is very expensive. If your file list is directly compatible with the output from find, run a single find and remove any matches from your list:
find . -type f |
fgrep -vxf - "$1"
If not, maybe you can massage the output from find in the pipeline before the fgrep so that it matches the format in your file; or, conversely, massage the data in your file into find-compatible.
I use this script and it works for me
#!/bin/bash
fileslist="$1"
found="Found:"
notfound="Not found:"
len=`cat $1 | wc -l`
n=0;
while read fn
do
# don't worry about this, i use it to display the file list progress
n=$((n + 1))
echo -en "\rLooking $(echo "scale=0; $n * 100 / $len" | bc)% "
if [ $(find / -name $fn | wc -l) -gt 0 ]
then
found=$(printf "$found\n\t$fn")
else
notfound=$(printf "$notfound\n\t$fn")
fi
done < $fileslist
printf "\n$found\n$notfound\n"
The line counts the number of lines and if its greater than 0 the find was a success. This searches everything on the hdd. You could replace / with . for just the current directory.
$(find / -name $fn | wc -l) -gt 0
Then i simply run it with the files in the files list being separated by newline
./search.sh files.list
I need to loop through a directory recursively and remove all files with extension .pdf and .doc. I'm managing to loop through a directory recursively but not managing to filter the files with the above mentioned file extensions.
My code so far
#/bin/sh
SEARCH_FOLDER="/tmp/*"
for f in $SEARCH_FOLDER
do
if [ -d "$f" ]
then
for ff in $f/*
do
echo "Processing $ff"
done
else
echo "Processing file $f"
fi
done
I need help to complete the code, since I'm not getting anywhere.
As a followup to mouviciel's answer, you could also do this as a for loop, instead of using xargs. I often find xargs cumbersome, especially if I need to do something more complicated in each iteration.
for f in $(find /tmp -name '*.pdf' -or -name '*.doc'); do rm $f; done
As a number of people have commented, this will fail if there are spaces in filenames. You can work around this by temporarily setting the IFS (internal field seperator) to the newline character. This also fails if there are wildcard characters \[?* in the file names. You can work around that by temporarily disabling wildcard expansion (globbing).
IFS=$'\n'; set -f
for f in $(find /tmp -name '*.pdf' -or -name '*.doc'); do rm "$f"; done
unset IFS; set +f
If you have newlines in your filenames, then that won't work either. You're better off with an xargs based solution:
find /tmp \( -name '*.pdf' -or -name '*.doc' \) -print0 | xargs -0 rm
(The escaped brackets are required here to have the -print0 apply to both or clauses.)
GNU and *BSD find also has a -delete action, which would look like this:
find /tmp \( -name '*.pdf' -or -name '*.doc' \) -delete
find is just made for that.
find /tmp -name '*.pdf' -or -name '*.doc' | xargs rm
Without find:
for f in /tmp/* tmp/**/* ; do
...
done;
/tmp/* are files in dir and /tmp/**/* are files in subfolders. It is possible that you have to enable globstar option (shopt -s globstar).
So for the question the code should look like this:
shopt -s globstar
for f in /tmp/*.pdf /tmp/*.doc tmp/**/*.pdf tmp/**/*.doc ; do
rm "$f"
done
Note that this requires bash ≥4.0 (or zsh without shopt -s globstar, or ksh with set -o globstar instead of shopt -s globstar). Furthermore, in bash <4.3, this traverses symbolic links to directories as well as directories, which is usually not desirable.
If you want to do something recursively, I suggest you use recursion (yes, you can do it using stacks and so on, but hey).
recursiverm() {
for d in *; do
if [ -d "$d" ]; then
(cd -- "$d" && recursiverm)
fi
rm -f *.pdf
rm -f *.doc
done
}
(cd /tmp; recursiverm)
That said, find is probably a better choice as has already been suggested.
Here is an example using shell (bash):
#!/bin/bash
# loop & print a folder recusively,
print_folder_recurse() {
for i in "$1"/*;do
if [ -d "$i" ];then
echo "dir: $i"
print_folder_recurse "$i"
elif [ -f "$i" ]; then
echo "file: $i"
fi
done
}
# try get path from param
path=""
if [ -d "$1" ]; then
path=$1;
else
path="/tmp"
fi
echo "base path: $path"
print_folder_recurse $path
This doesn't answer your question directly, but you can solve your problem with a one-liner:
find /tmp \( -name "*.pdf" -o -name "*.doc" \) -type f -exec rm {} +
Some versions of find (GNU, BSD) have a -delete action which you can use instead of calling rm:
find /tmp \( -name "*.pdf" -o -name "*.doc" \) -type f -delete
For bash (since version 4.0):
shopt -s globstar nullglob dotglob
echo **/*".ext"
That's all.
The trailing extension ".ext" there to select files (or dirs) with that extension.
Option globstar activates the ** (search recursivelly).
Option nullglob removes an * when it matches no file/dir.
Option dotglob includes files that start wit a dot (hidden files).
Beware that before bash 4.3, **/ also traverses symbolic links to directories which is not desirable.
This method handles spaces well.
files="$(find -L "$dir" -type f)"
echo "Count: $(echo -n "$files" | wc -l)"
echo "$files" | while read file; do
echo "$file"
done
Edit, fixes off-by-one
function count() {
files="$(find -L "$1" -type f)";
if [[ "$files" == "" ]]; then
echo "No files";
return 0;
fi
file_count=$(echo "$files" | wc -l)
echo "Count: $file_count"
echo "$files" | while read file; do
echo "$file"
done
}
This is the simplest way I know to do this:
rm **/#(*.doc|*.pdf)
** makes this work recursively
#(*.doc|*.pdf) looks for a file ending in pdf OR doc
Easy to safely test by replacing rm with ls
The following function would recursively iterate through all the directories in the \home\ubuntu directory( whole directory structure under ubuntu ) and apply the necessary checks in else block.
function check {
for file in $1/*
do
if [ -d "$file" ]
then
check $file
else
##check for the file
if [ $(head -c 4 "$file") = "%PDF" ]; then
rm -r $file
fi
fi
done
}
domain=/home/ubuntu
check $domain
There is no reason to pipe the output of find into another utility. find has a -delete flag built into it.
find /tmp -name '*.pdf' -or -name '*.doc' -delete
The other answers provided will not include files or directories that start with a . the following worked for me:
#/bin/sh
getAll()
{
local fl1="$1"/*;
local fl2="$1"/.[!.]*;
local fl3="$1"/..?*;
for inpath in "$1"/* "$1"/.[!.]* "$1"/..?*; do
if [ "$inpath" != "$fl1" -a "$inpath" != "$fl2" -a "$inpath" != "$fl3" ]; then
stat --printf="%F\0%n\0\n" -- "$inpath";
if [ -d "$inpath" ]; then
getAll "$inpath"
#elif [ -f $inpath ]; then
fi;
fi;
done;
}
I think the most straightforward solution is to use recursion, in the following example, I have printed all the file names in the directory and its subdirectories.
You can modify it according to your needs.
#!/bin/bash
printAll() {
for i in "$1"/*;do # for all in the root
if [ -f "$i" ]; then # if a file exists
echo "$i" # print the file name
elif [ -d "$i" ];then # if a directroy exists
printAll "$i" # call printAll inside it (recursion)
fi
done
}
printAll $1 # e.g.: ./printAll.sh .
OUTPUT:
> ./printAll.sh .
./demoDir/4
./demoDir/mo st/1
./demoDir/m2/1557/5
./demoDir/Me/nna/7
./TEST
It works fine with spaces as well!
Note:
You can use echo $(basename "$i") # print the file name to print the file name without its path.
OR: Use echo ${i%/##*/}; # print the file name which runs extremely faster, without having to call the external basename.
Just do
find . -name '*.pdf'|xargs rm
If you can change the shell used to run the command, you can use ZSH to do the job.
#!/usr/bin/zsh
for file in /tmp/**/*
do
echo $file
done
This will recursively loop through all files/folders.
The following will loop through the given directory recursively and list all the contents :
for d in /home/ubuntu/*;
do
echo "listing contents of dir: $d";
ls -l $d/;
done
I'm trying to write a script that searches a directory for files and greps for a pattern. Something similar to the below except the find expression is much more complicated (excludes particular directories and files).
#!/bin/bash
if [ -d "${!#}" ]
then
path=${!#}
else
path="."
fi
find $path -print0 | xargs -0 grep "$#"
Obviously, the above doesn't work because "$#" still contains the path. I've tried variants of building up an argument list by iterating over all the arguments to exclude path such as
args=${#%$path}
find $path -print0 | xargs -0 grep "$path"
or
whitespace="[[:space:]]"
args=""
for i in "${#%$path}"
do
# handle the NULL case
if [ ! "$i" ]
then
continue
# quote any arguments containing white-space
elif [[ $i =~ $whitespace ]]
then
args="$args \"$i\""
else
args="$args $i"
fi
done
find $path -print0 | xargs -0 grep --color "$args"
but these fail with quoted input. For example,
# ./find.sh -i "some quoted string"
grep: quoted: No such file or directory
grep: string: No such file or directory
Note that if $# doesn't contain the path, the first script does do what I want.
EDIT : Thanks for the great solutions! I went with a combination of the answers:
#!/bin/bash
path="."
end=$#
if [ -d "${!#}" ]
then
path="${!#}"
end=$((end - 1))
fi
find "$path" -print0 | xargs -0 grep "${#:1:$end}"
EDIT:
Original was just slightly off. No removal is to be done if the last argument is not a directory.
#!/bin/bash
if [ -d "${!#}" ]
then
path="${!#}"
remove=1
else
path="."
remove=0
fi
find "$path" -print0 | xargs -0 grep "${#:1:$(($#-remove))}"