I'm trying to list all files, except hidden ones, in only the subdirectories of a folder in bash by doing:
$ find ./public -mindepth 3 -type f -not -path '*/\.*'
That returns:
./public/mobile/images/image1.jpg
./public/mobile/images/image2.png
./public/mobile/images/image3.jpg
./public/mobile/javascripts/java1.js
./public/mobile/javascripts/java2.js
./public/mobile/javascripts/java3.js
./public/mobile/stylesheets/main.css
./public/mobile/views/doc1.html
./public/mobile/views/doc2.html
./public/mobile/views/doc3.html
How can I ignore the file path and show only the file name with the extension?
Thank you :)
Use -printf additionally to the find command, instead of -print.
find ./public -mindepth 3 -type f -not -path '*/\.*' -printf %f\\n
Note the usage of \\n - you need \n to add a new line after file name, but add another \ as escape or add some quotes to prevent interpreting \n by shell
If you are using bash 4 or later, you can skip using find and using a file pattern instead.
shopt -s globstar # For **
printf "%s\n" public/*/*/**/*.*
If you expect some files to have no extension, you'll need to use a loop and filter out
non-file matches manually.
for f in */*/*/**/*; do
[[ -f $f ]] || continue
printf "%s\n" "${f##*/}"
done
Related
How do I rename files in directory and subdirectory?
I found this program, but I need to go change files in subdirectory.
for file in *#me01
do
mv "$file" "${file/#me01/_me01}"
done
n#me01
to
n_me01
The following one-liner will likely work for you:
find . -type f -name '*#me01' -execdir rename '#me01' '_me01' {} \;
The following form is likely more correct as it will change only the last # to _ if there are multiple occurrences of #me01 in the file:
for f0 in $(find . -type f -name '*#me01')
do
f1=$(printf '%s' "$f0" | sed 's/#me01$/_me01/')
mv "$f0" "$f1"
done
This latter form is also more flexible and can be built upon more easily as the regex language in sed is much more powerful than rename expressions.
If rename of directories is also required the following can easily be added...
Either:
find . -type d -name '*#me01' -execdir rename '#me01' '_me01' {} \;
Or:
for d0 in $(find . -type d -name '*#me01')
do
d1=$(printf '%s' "$d0" | sed 's/#me01$/_me01/')
mv "$d0" "$d1"
done
Using bash:
shopt -s globstar
for name in **/*#me01; do
mv "$name" "${name%#me01}_me01"
done
This enables the globstar shell option in bash which makes ** match across path separators in pathnames.
It also uses a standard parameter substitution to delete the #me01 portion at the very end of the found pathname and replace it with _me01.
I am trying to delete old files, I first need to store the files in a variable and delete them one by one, my code works for normal files but if filename has white spaces in it, it cant delete that file and throws an error
code-
OLD_FILES=`find . -name "*.txt" -type f -newermt $2000-01-01 ! -newermt $2017-12-12`
for i in $OLD_FILES
do
rm $i
done
I can't use
OLD_FILES=`find . -name "*.$FILE_TYPE" -type f -newermt $START_DATE ! -newermt $DATE -delete `
because find and delete needs to be separate functions and to avoid code repetition
Filenames on UNIX may contain more or less any character, especially characters which are used by the shell to split input into words, like whitespace and newlines. If you use a for in loop, word splitting happens and that's what you are seeing.
I recommend to use the -print0 option of find which would separate files by null bytes and then use while read with the null byte as the delimiter to read them in the shell one by one:
find ... -print0 | while read -d $'\0' file ; do
do_something "${file}"
rm "${file}"
done
Have a look at the following link
https://unix.stackexchange.com/questions/208140/deleting-files-with-spaces-in-their-names
There are many solutions you might be able to apply to your problem:
like deleting the files via their inum
using a regex with space to fetch the file: find . -regex '.* .*' -delete
using xargs and find . -type -f -print 0
make a function to escape all spaces of your filenames \, before running the rm command
I am facing a problem with the following shell script:
#!/bin/bash
searchPattern=".*\/.*\.abc|.*\/.*\.xyz|.*\/.*\.[0-9]{3}"
subFolders=$(find -E * -type d -regex ".*201[0-4][0-1][0-9].*|.*20150[1-6].*" -maxdepth 0 | sed 's/.*/"&"/')
echo "subFolders: $subFolders"
# iterate through subfolders
for thisFolder in $subFolders
do
echo "The current subfolder is: $thisFolder"
find -E $thisFolder -type f -iregex $searchPattern -maxdepth 1 -print0 | xargs -0 7z a -mx=9 -uz1 -x!.DS_Store ${thisFolder}/${thisFolder}_data.7z
done
The idea behind it is to archive filetypes with the ending .abc, .xyz and .000-.999 in one 7z archive per subfolder. However, I can't manage to deal with folders including spaces. When I run the script as shown above I always get the following error:
find: "20130117_test": No such file or directory
If I run the script with the line
subFolders=$(find -E * -type d -regex ".*201[0-4][0-1][0-9].*|.*20150[1-6].*" -maxdepth 0 | sed 's/.*/"&"/')
changed to
subFolders=$(find -E * -type d -regex ".*201[0-4][0-1][0-9].*|.*20150[1-6].*" -maxdepth 0)
the script works like charm, but of course not for folders containing space.
Strangely enough, when I execute the following line directly in shell, it works as expected:
find -E "20130117_test" -type f -iregex ".*\/.*\.abc|.*\/.*\.xyz|.*\/.*\.[0-9]{3}" -maxdepth 1 -print0 | xargs -0 7z a -mx=9 -uz1 -x!.DS_Store "20130117_test"/"20130117_test"_data.7z
I know the issue is somehow related to the storing of a list of folders (in quotes) in the subFolders variable, but I simply cannot find a way to make it work properly.
I hope someone more advanced in shell can help me out here.
In general, you should not use find in an attempt to generate a list of file names. You especially cannot build a quoted list the way you are attempting; there is a difference between quotes in a parameter value and quotes around a parameter expansion. Here, especially, you can just use simple patterns:
shopt -s nullglob
subFolders=(
*201[0-4][0-1][0-9]*
*20150[1-6]*
)
for thisFolder in "${subFolders[#]}"; do
echo "The current subfolder is: $thisFolder"
to_archive=(
*/*.abc
*/*.xyz
*/*.[0-9][0-9][0-9]
)
7z a -mx9 -uz1 -x!.DS_Store "$thisFolder/$thisFolder_data.7z" "${to_archive[#]}"
done
Combining the input from gniourf_gniourf and chepner I was able to produce the following code, which does exactly what I want.
#!/bin/bash
shopt -s nullglob
find -E "$PWD" -type d -maxdepth 1 -regex ".*201[0-5][0-1][0-9].*" -print0 | while IFS="" read -r -d "" thisFolder ; do
echo "The current folder is: $thisFolder"
to_archive=( "$thisFolder"/*.[Aa][Bb][Cc] "$thisFolder"/*.[Xx][Yy][Zz] "$thisFolder"/*.[0-9][0-9][0-9] )
if [ ${#to_archive[#]} != 0 ]
then
7z a -mx=9 -uz1 -x!.DS_Store "$thisFolder"/"${thisFolder##*/}"_data.7z "${to_archive[#]}" && rm "${to_archive[#]}"
fi
done
shopt -s nullglob leads to ignorance towards non-matching characters
find... searches for directories matching the regex pattern and streams each matching folder to the while loop using the null separator.
inside the while loop I can safely quote the $thisFolder variable expansion and therefore deal with possible spaces.
using absolute paths instead of relative paths instructs 7z to create no folders inside the archive
I try to find folders without dot symbol.
I Search it in users directory via this script:
!#/bin/bash
users=$(ls /home)
for user in $users;
do
find /home/$user/web/ -maxdepth 1 -type d -iname '*' ! -iname "*.*"
done
But I see in result users with folders with dot, for example - test.uk or test.cf
What I do wrong?
Thanks in advance!
You can use find with -regex option for that:
find /home/$user/web/ -maxdepth 1 -type d -regex '\./[^.]*$'
'\./[^.]*$' will match names without any DOT.
The problem is that your command finds directories in /home/username/web/ where the directory doesn't contain a dot.
It does not check to see if username itself contains a dot.
To see if there's a dot anywhere, you can use ipath instead of iname:
!#/bin/bash
users=$(ls /home)
for user in $users;
do
find /home/$user/web/ -maxdepth 1 -type d -iname '*' ! -ipath "*.*"
done
Or more correctly and succinctly:
#!/bin/bash
find /home/*/web/ -maxdepth 1 -type d ! -ipath "*.*"
No need for find; just use an extended glob to match any files not containing a .
shopt -s extglob
for dir in /home/*/;
do
printf '%s\n' "$dir"/!(*.*)
done
You could even do away with the loop entirely:
shopt -s extglob
printf '%s\n' /home/*/!(*.*)
To exclude directories in /home that contain a ., you can change /home/*/ to /home/!(*.*)/ in either example.
i think that i don't understand very well how the find command in Unix works; i have this code for counting the number of files in each folder but i want to count the number of lines of each file found and save the total in variable.
find "$d_path" -type d -maxdepth 1 -name R -print0 | while IFS= read -r -d '' file; do
nb_fichier_R="$(find "$file" -type f -maxdepth 1 -iname '*.R' | wc -l)"
nb_ligne_fichier_R= "$(find "$file" -type f -maxdepth 1 -iname '*.R' -exec wc -l {} +)"
echo "$nb_ligne_fichier_R"
done
output:
43 .//system d exploi/r-repos/gbm/R/basehaz.gbm.R
90 .//system d exploi/r-repos/gbm/R/calibrate.plot.R
45 .//system d exploi/r-repos/gbm/R/checks.R
178 total: File name too long
can i just save to total number of lines in my variable? here in my example just save 178 and that for each files in my folder "$d_path"
Many Thanks
Maybe I'm missing something, but wouldn't this do what you want?
wc -l R/*.[Rr]
Solution:
find "$d_path" -type d -maxdepth 1 -name R | while IFS= read -r file; do
nb_fichier_R="$(find "$file" -type f -maxdepth 1 -iname '*.R' | wc -l)"
echo "$nb_fichier_R" #here is fine
find "$file" -type f -maxdepth 1 -iname '*.R' | while IFS= read -r fille; do
wc -l $fille #here is the problem nothing shown
done
done
Explanation:
adding -print0 the first find produced no newline so you had to tell read -d '' to tell it not to look for a newline. Your subsequent finds output newlines so you can use read without a delimiter. I removed -print0 and -d '' from all calls so it is consistent and idiomatic. Newlines are good in the unix world.
For the command:
find "$d_path" -type d -maxdepth 1 -name R -print0
there can be at most one directory that matches ("$d_path/R"). For that one directory, you want to print:
The number of files matching *.R
For each such file, the number of lines in it.
Allowing for spaces in $d_path and in the file names is most easily handled, I find, with an auxilliary shell script. The auxilliary script processes the directories named on its command line. You then invoke that script from the main find command.
counter.sh
shopt -s nullglob;
for dir in "$#"
do
count=0
for file in "$dir"/*.R; do ((count++)); done
echo "$count"
wc -l "$dir"/*.R </dev/null
done
The shopt -s nullglob option means that if there are no .R files (with names that don't start with a .), then the glob expands to nothing rather than expanding to a string containing *.R at the end. It is convenient in this script. The I/O redirection on wc ensures that if there are no files, it reads from /dev/null, reporting 0 lines (rather than sitting around waiting for you to type something).
On the other hand, the find command will find names that start with a . as well as those that do not, whereas the globbing notation will not. The easiest way around that is to use two globs:
for file in "$dir"/*.R "$dir"/.*.R; do ((count++)); done
or use find (rather carefully):
find . -type f -name '*.R' -exec sh -c 'echo $#' arg0 {} +
Using counter.sh
find "$d_path" -type d -maxdepth 1 -name R -exec sh ./counter.sh {} +
This script allows for the possibility of more than one sub-directory (if you remove -maxdepth 1) and invokes counter.sh with all the directories to be examined as arguments. The script itself carefully handles file names so that whether there are spaces, tabs or newlines (or any other character) in the names, it will work correctly. The sh ./counter.sh part of the find command assumes that the counter.sh script is in the current directory. If it can be found on $PATH, then you can drop the sh and the ./.
Discussion
The technique of having find execute a command with the list of file name arguments is powerful. It avoids issues with -print0 and using xargs -0, but gives you the same reliable handling of arbitrary file names, including names with spaces, tabs and newlines. If there isn't already a command that does what you need (but you could write one as a shell script), then do so and use it. If you might need to do the job more than once, you can keep the script. If you're sure you won't, you can delete it after you're done with it. It is generally much easier to handle files with awkward names like this than it is to fiddle with $IFS.
Consider this solution:
# If `"$dir"/*.R` doesn't match anything, yield nothing instead of giving the pattern.
shopt -s nullglob
# Allows matching both `*.r` and `*.R` in one expression. Using them separately would
# give double results.
shopt -s nocaseglob
while IFS= read -ru 4 -d '' dir; do
files=("$dir"/*.R)
echo "${#files[#]}"
for file in "${files[#]}"; do
wc -l "$file"
done
# Use process substitution to prevent going to a subshell. This may not be
# necessary for now but it could be useful to future modifications.
# Let's also use a custom fd to keep troubles isolated.
# It works with `-u 4`.
done 4< <(exec find "$d_path" -type d -maxdepth 1 -name R -print0)
Another form is to use readarray which allocates all found directories at once. Only caveat is that it can only read normal newline-terminated paths.
shopt -s nullglob
shopt -s nocaseglob
readarray -t dirs < <(exec find "$d_path" -type d -maxdepth 1 -name R)
for dir in "${dirs[#]}"; do
files=("$dir"/*.R)
echo "${#files[#]}"
for file in "${files[#]}"; do
wc -l "$file"
done
done