I have a large number of photos on my machine where I'd like to parse the standard naming convention I have created for each file, and then pipe it to the touch command.
For example, I have these files:
2016-08-06-00h28m34.jpg
2016-08-06-00h28m35.jpg
2016-08-06-00h28m36.jpg
I would like to generate (and then run) the following commands:
touch -t 201608060028.34 2016-08-06-00h28m34.jpg
touch -t 201608060028.35 2016-08-06-00h28m35.jpg
touch -t 201608060028.36 2016-08-06-00h28m36.jpg
I can do this manually in a text editor, but it's extremely time-consuming, due to the number of files in each directory. I could also do it in C# and run it over my LAN, but that seems like overkill. Heck, I can even do this in SQL Server, but ... it's OS X and I'm sure there's a simple command-line thing I'm missing.
I've looked at Windows version of Unix touch command to modify a file's modification date from filename part, and Split Filename Up to Define Variables, but I can't seem to figure out how to add in the period for the seconds portion of the script, plus I don't want to add the batch script to each of the hundreds of folders I have.
Any assistance will be greatly appreciated.
Simple Option
I presume you are trying to set the filesystem time to match the EXIF capture time of thousands of photos. There is a tool for that, which runs on OSX, Linux (and Windows if you must). It is called jhead and I installed it on OSX using homebrew with:
brew install jhead
There may be other ways to install it - jhead website.
Please make a back up before trying this, or try it out on a small subset of your files, as I may have misunderstood your needs!
Basically the command to set the filesystem timestamp to match the EXIF timestamp on a single file is:
jhead -ft SomeFile.jpg
So, if you wanted to set the timestamps for all files in $HOME/photos/tmp and all subdirectories, you would do:
find $HOME/photos/tmp -iname \*.jpg -exec jhead -ft {} \;
Option not requiring any extra software
Failing that, you could do it with Perl which is installed on OSX by default anyway:
find . -name \*.jpg | perl -lne 'my $a=$_; s/.*(\d{4})-(\d+)-(\d+)-(\d+)h(\d+)m(\d+).*/$1$2$3$4$5.$6/ && print "touch -t $_\ \"$a\"" '
which gives this sort of output on my machine:
touch -t 201608060028.34 "./2016-08-06-00h28m34.jpg"
touch -t 201608060028.35 "./2016-08-06-00h28m35.jpg"
touch -t 201501060028.35 "./tmp/2015-01-06-00h28m35.jpg"
and if that looks good on your machine, you could send those commands into bash to be executed like this:
find . -name \*.jpg | perl -lne 'my $a=$_;s/.*(\d{4})-(\d+)-(\d+)-(\d+)h(\d+)m(\d+).*/$1$2$3$4$5.$6/ && print "touch -t $_\ \"$a\"" ' | bash -x
And, for the Perl purists out there, yes, I know Perl could do the touch itself and save invoking a whole touch process per file, but that would require modules and explanation and a heap of other extraneous stuff that is not really necessary for a one-off, or occasional operation.
Related
I'm trying convert some files to read only in backup environment. Data Domain has retention-lock feature that can lock files with external trigger which touch -a -t "dateuntillocked" /backup/foo.
In this situation there is also metadata files in folder that should not be locked otherwise next backup job cannot update metadata file and fails.
I extracted metadata file names but file count can be changed. For exp.
foo1.meta foo2.meta . . fooN.meta
Is it possible to create a variable for each entry and add to command dynamically?
Like:
var1=/backup/foo234.meta
var2=/backup/foo322.meta
.
.
varN=/backup/fooNNN.meta
<find command> | grep -v $var1 $var2....varN | while read line; do touch -a -t "$dateuntillocked" "$line"; done
another elaboration of the case is
for example you executed a ls in a folder but amount of file can differs in time. script will create a variable for every file and use in a touch command with while loop. if 3 files in folder, script will create 3 variable and use 3 variable with touch in while loop. if "ls" result find 4 files, script dynamically create 4 variable fof files and use all in while loop etc. I am not a programmer so my logic can differ. May be another way to do this with easier way.
Just guessing what your intentions might be.
You can combine find | grep | command into a single command:
find /backup -name 'foo*.meta' -exec touch -a -t "$dateuntillocked" {} +
I want to make a bash script in unix where the user gives a number between 1 and 24.then it has to scan every file in the directory that the user is and find all the files that got created the same time as the number.
I know that unix wont store teh birth time for most of the files.So I found that each file has crtime, which you can find with this line of code: debugfs -R 'stat /path/to/file' /dev/sda2
The problem is that i have to know every crtime so I can search them by the hour.
thanks in advance and sorry for the complicated explenation and bad english
Use a loop to execute stat for each file, then use grep -q to check whether the crtime matches the time given by the user. Since paths in debugfs -R 'stat path' do not necessarily correspond to paths on your system and there might be quoting issues, we use inode numbers instead.
#! /usr/bin/env bash
hour="$1"
for f in ./*; do
debugfs -R "stat <$(stat -c %i "$f")>" /dev/sda2 2> /dev/null |
grep -Eq "^crtime: .* 0?$hour:" &&
echo "$f"
done
Above script assumes that your working directory is on an ext4 file systems on /dev/sda2. Because of debugfs you have to run the script as the superuser (for instance by using sudo). Depending on your system there might be an alternative to debugfs which can be run as a regular user.
Example usage:
Print all files and directories which were created between 8:00:00 am and 8:59:59 am.
$ ./scriptFromAbove 8
./some file
./another file
./directories are matched too
If you want to exclude directories you can add [ -f "$f" ] && in front of debugfs.
The OP has already identified the issue that Unix system calls do not retrieve the file creation time - just the modification times (either mtime or ctime). Assuming this is acceptable substitution, an EFFICIENT way to find all the files in the current directory created on a specific hour is to leverage 'ls' and an awk filter.
#! /bin/sh
printf -v hh '%02d' $1
ls -l --time-style=long-iso | awk -v hh=$hh '+$7 == hh'
While this solution does not access the actual creation time, it does not require super user access (which is usually required for debugfs)
Unfortunately my shell skills are very bad and I would need some help in running a simple script on my QNAP to fix some date problem on some videos.
The script I put in place is very easy:
in a given folder
check if there are .mp4 files starting with VID_
if so, for each of them run a given exiftool command
Here is the script so far, but I guess I am not using the right way to call the variable:
#!/bin/sh
# set target directories
dir="/share/Multimedia/Pictures/"
# move to target directory
cd "$dir"
# check if there is some .mp4 file starting with "VID_" in the folder
VID=$(ls -A $dir | grep 'VID_' | grep './mp4')
if
["$VID"];
then
# for each file in the list
for f in $VID
do
# change all date metadata according to its filename
exiftool "-*date<filename" -wm w $f
done
else
fi
Thanks for your help!
ps: the exiftool instruction is correct (except probably for the variable)
There isn't a need to script this, doing so just slows you down as you have to call ExifTool for every file. ExifTool can do all the files in one pass:
ExifTool -ext mp4 '-*date<filename' -wm w /path/to/dir/VID_*
The -ext mp4 options limits the command to only mp4 files. Since you seem to be on a linux/mac system, the double quotes had to be changed to single quotes. Double quotes are needed on Windows systems, single quotes on linux/mac systems.
Your code is probably failing due to use of:
grep './mp4'
Since there is no / before mp4.
Better to have your script as:
#!/bin/sh
# set target directories
dir="/share/Multimedia/Pictures/"
# move to target directory
cd "$dir"
for f in VID_*.mp4; do
exiftool "-*date<filename" -wm w "$f"
done
No need to parse output of ls here and there is no need to use grep since glob VID_*.mp4 will do the job of finding correct files.
I am trying to grep for a string as below but running into error shown below,can anyone suggest how to fix it?
find . | xargs grep 'bin data doesn't exist for HY11' -sl
Error:-
args: unmatched single quote; by default quotes are special to xargs unless you use the -0 option
Your grep pattern contains a quotation mark!
Use double quotes round the pattern: "bin doesn't exist for HY11" rather than 'bin ... HY11'.
You also want to add -print0 to the find command, and -0 to xargs.
The better way is to do this all directly:
find . -type f -exec grep -H "bin doesn't exist for HY11" "{}" "+"
That doesn't even need xargs.
If you have GNU Parallel you can run:
find . | parallel -X -q grep "bin data doesn't exist for HY11" -sl
All new computers have multiple cores, but most programs are serial in nature and will therefore not use the multiple cores. However, many tasks are extremely parallelizeable:
Run the same program on many files
Run the same program for every line in a file
Run the same program for every block in a file
GNU Parallel is a general parallelizer and makes is easy to run jobs in parallel on the same machine or on multiple machines you have ssh access to.
If you have 32 different jobs you want to run on 4 CPUs, a straight forward way to parallelize is to run 8 jobs on each CPU:
GNU Parallel instead spawns a new process when one finishes - keeping the CPUs active and thus saving time:
Installation
A personal installation does not require root access. It can be done in 10 seconds by doing this:
(wget -O - pi.dk/3 || curl pi.dk/3/ || fetch -o - http://pi.dk/3) | bash
For other installation options see http://git.savannah.gnu.org/cgit/parallel.git/tree/README
Learn more
See more examples: http://www.gnu.org/software/parallel/man.html
Watch the intro videos: https://www.youtube.com/playlist?list=PL284C9FF2488BC6D1
Walk through the tutorial: http://www.gnu.org/software/parallel/parallel_tutorial.html
Sign up for the email list to get support: https://lists.gnu.org/mailman/listinfo/parallel
I tried to make an alias for committing several different git projects. I tried something like
cat projectPaths | \
xargs -I project git --git-dir=project/.git --work-tree=project commit -a
where projectPaths is a file containing the paths to all the projects I want to commit. This seems to work for the most part, firing up vi in sequence for each project so that I can write a commit msg for it. I do, however, get a msg:
"Vim: Warning: Input is not from a terminal"
and afterward my terminal is weird: it doesn't show the text I type and doesn't seem to output any newlines. When I enter "reset" things pretty much back to normal, but clearly I'm doing something wrong.
Is there some way to get the same behavior without messing up my shell?
Thanks!
Using the simpler example of
ls *.h | xargs vim
here are a few ways to fix the problem:
xargs -a <( ls *.h ) vim
or
vim $( ls *.h | xargs )
or
ls *.h | xargs -o vim
The first example uses the xargs -a (--arg-file) flag which tells xargs to take its input from a file rather than standard input. The file we give it in this case is a bash process substitution rather than a regular file.
Process substitution takes the output of the command contained in <( ) places it in a filedescriptor and then substitutes the filedescriptor, in this case the substituted command would be something like xargs -a /dev/fd/63 vim.
The second command uses command substitution, the commands are executed in a subshell, and their stdout data is substituted.
The third command uses the xargs --open-tty (-o) flag, which the man page describes thusly:
Reopen stdin as /dev/tty in the child process before executing the
command. This is useful if you want xargs to run an interactive
application.
If you do use it the old way and want to get your terminal to behave again you can use the reset command.
The problem is that since you're running xargs (and hence git and hence vim) in a pipeline, its stdin is taken from the output of cat projectPaths rather than the terminal; this is confusing vim. Fortunately, the solution is simple: add the -o flag to xargs, and it'll start git (and hence vim) with input from /dev/tty, instead of its own stdin.
The man page for GNU xargs shows a similar command for emacs:
xargs sh -c 'emacs "$#" < /dev/tty' emacs
(in this command, the second "emacs" is the "dummy string" that wisbucky refers to in a comment to this answer)
and says this:
Launches the minimum number of copies of Emacs needed, one after the
other, to edit the files listed on xargs' standard input. This example
achieves the same effect as BSD's -o option, but in a more flexible and
portable way.
Another thing to try is using -a instead of cat:
xargs -a projectPaths -I project git --git-dir=project/.git --work-tree=project commit -a
or some combination of the two.
If you have GNU Parallel http://www.gnu.org/software/parallel/ installed you should be able to do this:
cat projectPaths |
parallel -uj1 git --git-dir={}/.git --work-tree={} commit -a
In general this works too:
cat filelist | parallel -Xuj1 $EDITOR
in case you want to edit more than one file at a time (and you have set $EDITOR to your favorite editor).
-o for xargs (as mentioned elsewhere) only works for some versions of xargs (notably it does not work for GNU xargs).
Watch the intro video to learn more about GNU Parallel http://www.youtube.com/watch?v=OpaiGYxkSuQ
Interesting! I see the exact same behaviour on Mac as well, doing something as simple as:
ls *.h | xargs vim
Apparently, it is a problem with vim:
http://talideon.com/weblog/2007/03/xargs-vim.cfm