Hi I'm looking to see what file is changing in a directory i'd like to get the md5sum of every file and write it to a text file. Then after i know a file has changed i'd like to run it again so i can diff the output files to see what exactly changed. Here is what i've tried however it doesn't work as i need.
Also not only do i need to get the md5sum of every file in a folder including subdirectories i need it to not follow symlinks
#!/bin/bash
#
cd /sys/class
for i in $(find . -type f)
do
ls -lt "$i" >> /home/george/Desktop/before.txt
done
echo "Finished!"
Thank you for any help
===Edit===
I put my actual paths in as i don't really see a need to hide them. Anyway running this returned only a few files (outputted file below) which are the files in the folders meaning it's not going into subdirectories and finding those files too. Btw sorry my bash is way rusty
--w------- 1 root root 4096 Jun 20 03:03 ./gpio/export
--w------- 1 root root 4096 Jun 20 03:03 ./gpio/unexport
-rw-r--r-- 1 root root 4096 Jun 20 03:03 ./firmware/timeout
-r--r--r-- 1 root root 4096 Jun 20 03:04 ./drm/version
===Edit2===
Not exactly sure why some of these files aren't being found for instance
/sys/class/backlight/intel_backlight/brightness
And many others like that there are so many files that aren't being found for some reason
The cd is unnecessary, and with type -f you are already in fact bypassing symlinks. So the loop is unnecessary, too:
find /path/to/directory -type f -exec md5sum {} + >before.txt
If your find is too old to support -exec {} + try with -exec {} \; instead.
For the md5sum comparison, you could try simply removing identical lines;
fgrep -vxf before.txt after.txt | less
This is assuming the list in before.txt will fit into fgrep; but if you are dealing with a few dozen thousand files tops, it can probably cope. This will not identify deleted files from before.txt, though.
If your file list size is small enough that you can do it all in memory, you might consider sorting before.txt by the hash. If you do the same for after.txt you'd be able to go line by line on each of the files and identify matches even if the filename has changed. You'd also be able to skip over deleted or added files with less problems than if you had to interpret a diff before.txt after.txt
If using file modification date is an option, what you can do is use ls -lt | head to filter out the newest file and keep that. Then when you want to check for changes, ls -lt again and go through anything that's newer than the date you stored. This should work nicely regardless of file list size, but will be vulnerable to someone modifying last modification date (which would require root privileges)
Related
I am trying to get a directory to replace an existing folder but can't get it done with mv - I believe there's a way and I just don't know it (yet). Even after consulting the man page and searching the web.
If /path/to/ only contains directory, the following command will move /path/to/directory (vanishes) to /path/to/folder
mv /path/to/directory /path/to/folder
It is basically a rename, which is what I try to achieve.
But if /path/to/folder already exists, the same command moves the /path/to/directory to /path/to/folder/directory.
I do not want to use cp command to avoid IO.
Instead of using cp to actually copy the data in each file, use ln to make "copies" of the pointers to the file.
ln /path/to/directory/* /path/to/folder && rm -rf /path/to/directory
Note this is slightly more atomic than using cp; each individual file appears in /path/to/folder in a single step (i.e., there is no chance that /path/to/folder/foo.txt is ever partially copied), but there is still a small window where some, but not all, files from /path/to/directory have been linked to folder. Also, the rm -rf is not atomic, but assuming no one is interested in directory, that's not an issue. (Although, as files from /path/to/directory are unlinked, you can see changes to the link counts of files under /path/to/foldoer changing from 2 to 1. It's unlikely that anyone will care about that.)
What you think of as a file is really just a file system entry to an otherwise anonymous file managed by the file system. For example, consider a simple example.
$ mkdir d
$ cd d
$ echo hello > file.txt
$ cp file.txt file_copy.txt
$ ln file.txt file_link.txt
$ ls -li
total 24
12890456377 -rw-r--r-- 2 chepner staff 6 Mar 3 12:46 file.txt
12890456378 -rw-r--r-- 1 chepner staff 6 Mar 3 12:47 file_copy.txt
12890456377 -rw-r--r-- 2 chepner staff 6 Mar 3 12:46 file_link.txt
The -i option adds each entries inode number (the first column) to the output; an inode can be thought of as a unique identifier for a file. In this output, you can see that file_copy.txt is an entirely new file, with a different inode than file.txt. file_link.txt has the exact same inode, meaning that file.txt and file_link.txt are simply two different names for the same thing. The number just before the owner is the link count; file.txt and file_link.txt both refer to a file with a link count of 2.
When you use rm, you are just removing a link to a file, not the file itself. A file is not removed until the link count is reduced to 0. To demonstrate, we'll remove file.txt and file_copy.txt.
$ rm file.txt file_copy.txt
$ ls -li
total 8
12890456377 -rw-r--r-- 1 chepner staff 6 Mar 3 12:46 file_link.txt
As you can see, the only link to file_copy is gone, so inode 12890456378 no longer appears in the output. (Whether or not the data is really gone is a matter of file-system implementation.) file_link.txt, though, still refers to the same file as before, but now with a link count of 1, because file.txt was removed.
Links to a file do not have to appear in the same directory; they can appear in any directory on the same file system, which is the only caveat using this trick. ln will, IIRC, give you an error if you try to create a link to a file on another file system.
Could I please know how I can print contents of the file with same extension (for example, .coords) in multiple directories to a text file using a shell script, confining to specific date and time of the directory created.
Would be thankful to your replies.
EDIT:
ls -ltr
total 16
drwxrwxr-x 2 prakkisr prakkisr 4096 Jul 28 13:23 A
drwxrwxr-x 2 prakkisr prakkisr 4096 Jul 29 09:56 B
drwxrwxr-x 2 prakkisr prakkisr 4096 Jul 31 12:15 C
drwxrwxr-x 2 prakkisr prakkisr 4096 Jul 31 14:34 D
All the folders A,B,C,D have a file which ends with .coords (a.coords in A folder, b.coords in B folder etc..)
Firstly, I want only those folders generated on Jul 31 (i.e C and D folder) to be accessed and want to print the contents of c.coords and d.coords files in those folder into a text file. ( this is according to date)
Secondly, Is it possible to print it according to time. Like suppose, I want only those ".coords" file from the folder (in this case 'D' folder), which are generated after time 14:00 today and get printed into another file. (this is according to date as well as time)
The following command will print the contents of all *.coords files that are in directories with a modification date within the last day:
find . -type d -mtime 0 -exec sh -c 'cat {}/*.coords 2>/dev/null' \;
If you wanted to see the names of the *.coords files rather than their content, then use:
find . -type d -mtime 0 -exec sh -c 'ls {}/*.coords 2>/dev/null' \;
The age of the directory can be specified in many other ways. For example:
To specify the directories age in minutes, use -mmin in place of -mtime.
To specify the directories creation date, rather than its last modification date, use -cmin or -ctime.
If your file system supports it, it is also possible to select directories based on their last access time. Use -amin or -atime.
It is also possible to select directories based some range in times by prepending the age with a + or - sign. To select directories with a creation date more recent than two days, use -ctime -2. By including two such specifiers, you can select from a range of dates.
See man find for full details.
Variation
Suppose that we want to search based on the date of the file, rather than the date of the directory in which the file resides. In this case, a simpler command may be used to print the contents of the matching files:
find . -name '*.coords' -mtime 0 -exec cat {} \;
Suppose that we want to both print the file's name and its contents. Then, we include to actions to the find command:
find . -name '*.coords' -mtime 0 -print -exec cat {} \;
Note the use of quotation marks around *.coords. This assures that the command will work in case that the current directory happens to have .coords file in it.
I have a directory that contains sub-directories and other files and would like to update the date/timestamps recursively with the date/timestamp of another file/directory.
I'm aware that:
touch -r file directory
changes the date/timestamp for the file or directory with the others, but nothing within it. There's also the find version which is:
find . -exec touch -mt 201309300223.25 {} +\;
which would work fine if i could specify the actual file/directory and use anothers date/timestamp. Is there a simple way to do this? even better, is there a way to avoid changing/updating timestamps when doing a 'cp'?
even better, is there a way to avoid changing/updating timestamps when doing a 'cp'?
Yes, use cp with the -p option:
-p
same as --preserve=mode,ownership,timestamps
--preserve
preserve the specified attributes (default:
mode,ownership,timestamps), if possible additional attributes:
context, links, xattr, all
Example
$ ls -ltr
-rwxrwxr-x 1 me me 368 Apr 24 10:50 old_file
$ cp old_file not_maintains <----- does not preserve time
$ cp -p old_file do_maintains <----- does preserve time
$ ls -ltr
total 28
-rwxrwxr-x 1 me me 368 Apr 24 10:50 old_file
-rwxrwxr-x 1 me me 368 Apr 24 10:50 do_maintains <----- does preserve time
-rwxrwxr-x 1 me me 368 Sep 30 11:33 not_maintains <----- does not preserve time
To recursively touch files on a directory based on the symmetric file on another path, you can try something like the following:
find /your/path/ -exec touch -r $(echo {} | sed "s#/your/path#/your/original/path#g") {} \;
It is not working for me, but I guess it is a matter of try/test a little bit more.
In addition to 'cp -p', you can (re)create an old timestamp using 'touch -t'. See the man page of 'touch' for more details.
touch -t 200510071138 old_file.dat
I want to copy all the files in a directory that were modified this month. I can list those files like this:
ls -l * | grep Jul
And then to copy them I was trying to pipe the result into cp via xargs but had no success (I think) because I couldn't figure out how to parse the ls -l output to just grab the filename for cp.
I'm sure there are many ways of doing this; I'll give the correct answer out to the person who can show me how to parse ls -l in this manner (or talk me down from that position) though I'd be interested in seeing other methods as well.
Thanks!
Of course, just doing grep Jul is bad because you might have files with Jul in their name.
Actually, find is probably the right tool for your job. Something like this:
find $DIR -maxdepth 1 -type f -mtime -30 -exec cp {} $DEST/ \;
where $DIR is the directory where your files are (e.g. '.') and $DEST is the target directory.
The -maxdepth 1 flag means it doesn't look inside sub-directories for files (isn't recursive)
The -type f flag means it looks only at regular files (e.g. not directories)
The -mtime -30 means it looks at files with modification time newer than 30 days (+30 would be older than 30 days)
the -exec flag means it executes the following command on each file, where {} is replaced with the file name and \; is the end of the command
interested in seeing how this might be done with zsh
ls -lt *.*(mM0)
last month
ls -lt *.*(mM1)
or for precise date date ranges
autoload -U age
ls -tl *.*(e#age 2014/06/07 now#)
ls -tl *.*(e#age 2014/06/01 2014/06/20#)
my log directory containing following files
access.log
defaultAuditrecorder20110901.log (this is 31st jun generated log file)
defaultAuditrecorder20110901.log (this is 1 st aug generated log file)
defaultAuditrecorder20110902.log (this is 2 nd aug generated log file)
defaultAuditrecorder.log (this is currentdey running log file)
mng1.log001
mng1.log002
mng1.log003 .............. so on......
my requirement is using shell script i need to only delete defaultauditrecord log files except current and previous day.
Consider using logrotate. It lets you delete (or compress, rotate, etc.) log files, and is quite configurable. It is likely more robust than rolling your own script.
Edit: Here's a tutorial.
the simplest mechanism is to use the find command.
find /var/log -mtime +2d -a -type f -print
This will find all files that have been modified more than 2 days ago. To chain it into a removal command you would use:
find /var/log -mtime +2d -a -type f -print0 | xargs -0 rm
In this example, I used /var/log, you would substitute the directory that contains the logs. The reason for using the -print0 and the xargs -0, is that if the file contains whitespace it would not get processed by the rm command properly.