I have a rather interesting problem that I am trying to find the optimal solution for. I am creating an file autocompletion backend for Emacs. This means that I am using the linux find command to get files and directories.
The backend is given a file with a partially completed path (e.g. /usr/folder/foo) and I want to grab all files and directories that could match the partial path for two directories down (e.g. for example it could provide foo_bar/, foo_bar/bar, foo_bar/baz, foo_bar/bat/ foo_baz). So far I have only been to break this down into 3 steps
find all files in the current directory that may match the prefix
find foo* -type f -maxdepth 1
collect all possible directories we may want to look through
find foo* -type d -maxdepth 1
use each of those directories to make 2 more calls to find (I need to be able to differentiate between files and directories)
find foo_bar/ -type d -maxdepth 1
find foo_bar/ -type f -maxdepth 1
This solution involves a lot of calls to find (especially because the last step has to be called for every matching directory). This makes getting candidates slow, especially in large file systems. Ideally I would like to only make one call to get all the candidates. But I have not found a good way to do that. Does anyone know an optimal solution?
looking though the find manpage, I ended up using -printf.
find -L foo* -maxdepth 1 -printf '%p\t%y\n'
gives me everything I needed. only one command, differentiate between files and directories, search depth, etc.
Related
We have an Amazon EC2 instance where we upload output from our security cameras. Every now and then, the cameras have an issue, stop uploaded, and need to be rebooted. The easy way for us to determine this is by seeing if the files are not being created. The problem is it creates lots and lots of files. If I use find with -ctime, it takes a very long time for this script to run. Is there a faster way to check to see if files have been created since yesterday? I just need to capture the result, (yes there are some files, or not there are not,) and email a message, but it would be nice to have something that didn't take half an hour to run.
#!/bin/bash
find /vol/security_ftp/West -ctime -1
find /vol/security_ftp/BackEntrance -ctime -1
find /vol/security_ftp/BoardroomDoor -ctime -1
find /vol/security_ftp/MainEntrance -ctime -1
find /vol/security_ftp/North -ctime -1
find /vol/security_ftp/South -ctime -1
Using find is a natural solution, but if you really must avoid it, you can see the newest file in a directory using ls and sorting the output according to ctime, eg.
ls /vol/security_ftp/West -clt | head --lines=1
This would be enough if you want to see the date.
If you need better formatted output (or only ctime to process it further) you can feed the filename to stat:
stat --format="%z" $( ls /vol/security_ftp/West -ct | head --lines=1 )
This does not answer automatically if any file was created recently, though.
The simple (and recommended man find) solution is:
find /vol/security_ftp/ -mtime 0
To find files in /vol/security_ftp modified within the last 24 hours. Give it a try and see if it will meet your time requirements. We can look for another solution if the default can't do it quick enough. If the delay is due to numerous subdirectories under /vol/security_ftp, then limit the depth and type with:
find /vol/security_ftp/ -maxdepth 1 -type f -mtime 0
I've been reading up on find's -prune action. One common task I do is to process only the files of a directory, ignoring all directories.
Prune, from what I've learned, is great for ignoring directories if you know their names (or wildcards matching their names). But what if you don't know their names (or a pattern that matches files as well as directories)?
I found that -maxdepth achieves what I'm trying to do. I'm just wondering what the equivalent -prune approach might be.
For example, say I want to process all the files of my home directory, but not recurse into any subdirectory. Let's say my directory structure and files look like this (directories ending in '/'):
~/tmpData.dat
~/.bashrc
~/.vimrc
~/.Xdefaults
~/tmp/
~/tmp/.bashrc
~/bkups/.bashrc
~/bkups/.vimrc
~/bkups/.Xdefaults
~/bkups/tmpData.dat
.. what would be the correct find/prune command?
OK, I found my own solution. I simply specify pattern(s) that match everything in my
home directory ('~/*' for example). But in order to include all my dot files (.bashrc,
etc.), I have to use two patterns; one for non-dotted filenames and one for the files
starting with dots:
find ~/* ~/.* -type d -prune -o -type f -print
I am trying to figure out a way to search a directory for a file older than 365 days. If it finds a match, I'd like it to both delete the file and locate any other files in the directory that have the same basename, and delete those as well.
File name examples: 12345.pdf (Search for) then delete, 12345_a.pdf, 12345_xyz.pdf (delete if exist).
Thanks! I am very new to BASH scripting, so patience is appreciated ;-))
I doubt this can be done cleanly in a single pass.
Your best bet is to use -mtime or a variant to collect names and then use another find command to delete files matching those names.
UPDATE
With respect to your comment, I mean something like:
# find basenames of old files
find .... -printf '%f\n' | sort -u > oldfiles
for file in ($<oldfiles); do find . -name $file -exec rm; done
Tricky question for a bash noob like me, but i'm sure this this easier that it seems to me.
I'm currently using the find command as follows :
run "find #{current_release}/migration/ -name '*.sql'| sort -n | xargs cat >#{current_release}/#{stamp}.sql"
in my capistrano recipe.
Problem is #{current_release}/migration/ contains subfolders, and I'd like the find command to include only one of these, depending on it's name (that I know, it's based on the target environment.
As a recap, folder structure is
Folder
|- sub1
|- sub2
and i'm trying to make a find specifying to recurse ONLY on sub1 for example. I'm sure this is possible, just couldn't find how.
Thanks.
Simply specify the directory you want as argument to find, e.g. find #{current_release}/migration/sub1 ....
EDIT: As per your clarification, you should use the -maxdepth argument for find, to limit the recursion depth. So, for example, you can use find firstdir firstdir/sub1 -maxdepth 1.
You just need to append that to your find invocation:
find #{current_release}/migration/sub_you_want -name ...
Depending on how you make the determination of the sub-directory you want, you should be able to script that as well.
I understand that the wildcard * (by itself) will expand in such a way that it means "all non-hidden files in the current folder" with hidden files being those prefixed by a period.
There are two use cases that I would think are useful, but I don't know how to properly do:
How can you glob for... "All files in the current folder, including hidden files, but not including . or .."?
How can you glob for... "All hidden files (and only hidden files) in the current folder, but not including . or .."?
To expand on paviums answer and answer the second part of your question, all files except . and .. could be specified like this:
{.[!.]*,*}
Depending on your exact use case it might be better to set the dotglob shell option, so that bash includes dotfiles in expansions of * by default:
$ shopt -s dotglob
$ echo *
.tst
The Bash Cookbook suggests a solution to your 2nd requirement.
.[!.]*
as a way of specifying 'dot files' but avoiding . and ..
Of course, ls has the -A option, but that's not globbing.
Combining sth and pavium answers
# dot files but avoiding . and ..
.[!.]*
# all files but avoiding . and ..
{.[!.]*,*}
To meet your first case:
echo {.,}[^.]*
or
echo {.,}[!.]*
Edit:
This one seems to get everything, but is shorter than ephemient's
echo {.*,}[^.]*
By "all files" and "all hidden files" do you mean files-only, or do you mean both files and directories? Globbing operates on names irrespective of it belonging to a file or a directory. The other folks give good answers for using globbing to find hidden vs non-hidden names, but you may want to turn to the find command as an easier alternative that can distinguish between the types.
To find "All files in the current folder, including hidden files, but not including . or ..":
find . -type f
To find "All files and directories in the current folder, including hidden files, but not including . or ..":
find . ! -name .
To find "All hidden files (and only hidden files) in the current folder, but not including . or ..":
find . -name '.*' -type f
To find "All hidden files and directories (and only hidden files and directories) in the current folder, but not including . or ..":
find . -name '.*' ! -name .
Note that by default find will recurse through subdirectories, too, so if you want to limit it to only the current directory you can use:
find . -maxdepth 1 -type f
So, even though this is old - without using shopt, this doesn't seem to have been answered fully. But, expanding on what has been given as answers so far, these work for me:
1:
{*,.[!.]*,..?*}
2:
{.[!.]*,..?*}