I am trying to write a script that takes 2 arguments : the first file contain some files extensions , and the second a directory .
My script move the files which their extension existed in the extension's file to the directory.
This is my script:
BEGIN{
}
{
file_ext = $1
folder = $2
isexist = "[ -e " $1 " ]"
if( ( system(isexist) ) != 0 )
{
getline < file_ext
system("find *." $0" -exec mv {} " folder " \;")
next
}
}
END{
}
But when I call the script in the shell , I am getting this error :
mv: missing destination file operand after `koko.cpp'
When I put folder name directly it works fine , but when I pass it by argument it doesn't work, why?
Command line arguments are accessed via the ARGV array:
BEGIN {
# "pop" the directory name off the arguments list
folder = ARGV[2]
ARGV[2] = ""
}
{
# then process the "file_ext" file
system("find . -name \\*." $0 " -exec mv {} " folder " \\;")
}
Related
I have a FTP folder receiving files from a remote camera. The camera stores the video file name always as ./rec_YYYY-MM-DD_HH-MM.mkv. The video files are stored all in the same folder, the root folder from the FTP server.
I need to move these files to another folder, with this new scheme:
Remove rec_ from the file name.
Change date format to DD-MM-YY.
Remove date from the file name and make it a folder instead, where that same file and all the others in the same date will be stored in.
Final file path would be: ./DD-MM-YYYY/HH-MM.mkv.
The process would continue to all the files, putting them in the folder corresponding to the day it was created.
Summing up: ./rec_YYYY-MM-DD_HH-MM.mkv >> ./DD-MM-YYYY/HH-MM.mkv. The same should apply to all files that are in the same folder.
As I can't make it happen directly from the camera, this needs to be done with Bash on the server that is receiving the files.
So far, what I got is script, which would get the file's creation date and use it to make a folder, and then get creation time to move the file with the new name, based on it's creation time.:
for f in *.mp4
do
mkdir "$f" "$(date -r "$f" +"%d-%m-%Y")"
mv -n "$f" "$(date -r "$f" +"%d-%m-%Y/%H-%M-%S").mp4"
done
I'm getting this output (with testfile 1.mp4):
It creates the folder based on the file's creation date;
it renames the file to it's creation time;
Then, it returns mkdir: cannot create directory ‘1.mp4’: File exists
If two or more files, only one gets renamed and moved as described. The others stay the same and terminal returns:
mkdir: cannot create directory ‘1.mp4’: File exists
mkdir: cannot create directory ‘2.mp4’: File exists
mkdir: cannot create directory ‘12-12-2018’: File exists
Could someone help me out? Better suggestions? Thanks!
Honestly I would just use Perl or Python for this. Here's how to embed either in a shell script.
Here's a perl script that doesn't use any libraries, even ones that ship with Perl (so it'll work without extra packages on distributions like CentOS that don't ship with the entire Perl library). The perl script launches one new process per file in order to perform the copy.
perl -e '
while (<"*.m{p4,kv}">) {
my $path = $_;
my ($prefix, $year, $month, $day, $hour, $minute, $ext) =
split /[.-_]/, $path;
my $sec = q[00];
die "unexpected prefix ($prefix) in $path"
unless $prefix eq q[rec];
die "unexpected extension ($ext) in $path"
unless $ext eq q[mp4] or $ext eq q[mkv];
my $dir = "$day-$month-$year";
my $name = "$hour-$min-$sec" . q[.] . $ext;
my $destpath = $dir . q[/] . $name;
die "$dir . $name is unexpectedly a directory" if (-d $dir);
system("cp", "--", $path, $destpath);
}
'
Here's a Python example, it's compatible with either Python 2 or Python 3 but does use the standard library. The Python script does not spawn any additional processes.
python3 -c '
import os.path as path
import re
from glob import iglob
from itertools import chain
from os import mkdir
from shutil import copyfile
for p in chain(iglob("*.mp4"), iglob("*.mkv")):
fields = re.split("[-]|[._]", p)
prefix = fields[0]
year = fields[1]
month = fields[2]
day = fields[3]
hour = fields[4]
minute = fields[5]
ext = fields[6]
sec = "00"
assert prefix == "rec"
assert ext in ["mp4", "mkv"]
directory = "".join([day, "-", month, "-", year])
name = "".join([hour, "-", minute, "-", sec, ".", ext])
destpath = "".join([directory, "/", name])
assert not path.isdir(destpath)
try:
mkdir(directory)
except FileExistsError:
pass
copyfile(src=p, dst=destpath)
'
Finally, here's a bash solution. It splits paths using -, ., and _ and then extracts various subfields by indexing into $# inside a function. The indexing trick is portable, although regex substitution on variables is a bash extension.
#!/bin/bash
# $1 $2 $3 $4 $5 $6 $7 $8
# path rec YY MM DD HH MM ext
process_file() {
mkdir "$5-$4-$3" &> /dev/null
cp -- "$1" "$5-$4-$3"/"$6-$7-00.$8"
}
for path in *.m{p4,kv}; do
[ -e "$path" ] || continue
# NOTE: two slashes are needed in the substitution to replace everything
# read -a ARRAYVAR <<< ... reads the words of a string into an array
IFS=' ' read -a f <<< "${path//[-_.]/ }"
process_file "$path" "${f[#]}"
done
If you cd /to/some/directory/containing_your_files then you could use the following script
#!/usr/bin/env bash
for f in rec_????-??-??_??-??.m{p4,kv} ; do
dir=${f:4:10} # skip 4 chars ('rec_') take 10 chars ('YYYY_MM_DD')
fnm=${f:15} # skip 15 chars, take the remainder
test -d "$dir" || mkdir "$dir"
mv "$f" "$dir"/"$fnm"
done
note ① that I have not exchanged the years and the days, if you absolutely need to do the swap you can extract the year like this, year=${dir::4} etc and ② that this method of parameter substitution is a Bash-ism, e.g., it doesn't work in dash.
your problem is: mkdir creates folder but you are giving filename for folder creation.
if you want to use fileName for folder creation then use it without extension.
the thing is you are trying to create folder with the already existing fileName
I have media server with two directories: Movies and TV Shows. Within each of those directories, each entry exists in a sub-directory which contains the video file and subtitle files.
I've scoured the web and have found an excellent perl script from Michelle Sullivan, posted here:
#!/usr/bin/perl
use strict;
use warnings;
open DIR, "ls -1 |";
while (<DIR>)
{
chomp;
next if ( -d "$_"); # skip directories
next unless ( -r "$_"); # if it's not readable skip it!
my $file = $_;
open PROBE, "ffprobe -show_streams -of csv '$file' 2>/dev/null|" or die ("Unable to launch ffmpeg for $file! ($!)");
my ($v, $a, $s, #c) = (0,0,0);
while (<PROBE>)
{
my #streaminfo = split(/,/, $_);
push(#c, $streaminfo[2]) if ($streaminfo[5] eq "video");
$a++ if ($streaminfo[5] eq "audio");
$s++ if ($streaminfo[5] eq "subtitle");
}
close PROBE;
$v = scalar #c;
if (scalar #c eq 1 and $c[0] eq "ansi")
{
warn("Text file detected, skipping...\n");
next;
}
warn("$file: Video Streams: $v, Audio Streams: $a, Subtitle Streams: $s, Video Codec(s): " . join (", ", #c) . "\n");
if (scalar #c > 1)
{
warn("$file has more than one video stream, bailing!\n");
next;
}
if ($c[0] eq "hevc")
{
warn("HEVC detected for $file ...converting to AVC...\n");
system("mkdir -p h265");
my #params = ("-hide_banner", "-threads 2");
push(#params, "-map 0") if ($a > 1 or $s > 1 or $v > 1);
push(#params, "-c:a copy") if ($a);
push(#params, "-c:s copy") if ($s);
push(#params, "-c:v libx264 -pix_fmt yuv420p") if ($v);
if (system("mv '$file' 'h265/$file'"))
{
warn("Error moving $file -> h265/$file\n");
next;
}
if (system("ffmpeg -xerror -i 'h265/$file' " . join(" ", #params) . " '$file' 2>/dev/null"))
{
warn("FFMPEG ERROR. Cannot convert $file restoring original...\n");
system("mv 'h265/$file' '$file'");
next;
}
} else {
warn("$file doesn't appear to need converting... Skipping...\n");
}
}
close DIR;
The script performs perfectly - as long as it is run from within the directory containing the media.
My question: Can this script be modified to run recursively from the root directory? How?
Thanks in advance.
(Michelle's script can be seen here: http://www.michellesullivan.org/blog/1636)
Why do you want to run recursively? Do you mean that you want to run it on all the files under a particular directory?
In this problems, I'd rather separate the part that generates the list of files to process from the processing. With a long list of files, I might take the lines from standard input instead:
while( <> ) {
...
}
Pipe the list into the script:
$ find ... | script
Or take it from a file:
$ script list_of_files.txt
With a short list, I might use a favorite xargs trick:
$ find ... -print0 | xargs -0 script
In that case I go through the command-line arguments:
foreach ( #ARGV ) {
...
}
If you want to do all of this in the program, you can use File::Find.
Beyond that, it sounds like you are asking someone to do the work for you.
I am trying to achieve this is Mac OS, tried to achieve similar by using fdupes but didn't work. Here is what I am trying to achieve:
There are 100 files in directory 'alpha'
Pick one file A and compare it with each remaining file in the directory 'alpha'
If content of file A matches any file (duplicate), delete the duplicate file
Move to file B, and compare with the remaining file, and do the same (check for duplicate)
Repeat the same until all files are checked for duplicates. Remaining files should be unique
Update
I modified a bit something similar I found here, but I have to run it multiple times to take out the duplicates. It is not detecting duplicates in a single run (have to run it multiple times to detect duplicate). Not sure if it is working correctly
use Digest::MD5;
%check = ();
while (<*>) {
-d and next;
$fname = "$_";
print "checking .. $fname\n";
$md5 = getmd5($fname) . "\n";
if ( !defined( $check{$md5} ) ) {
$check{$md5} = "$fname";
}
else {
print "Found duplicate files: $fname and $check{$md5}\n";
print "Deleting duplicate $check{$md5}\n";
unlink $check{$md5};
}
}
sub getmd5 {
my $file = "$_";
open( FH, "<", $file ) or die "Cannot open file: $!\n";
binmode(FH);
my $md5 = Digest::MD5->new;
$md5->addfile(FH);
close(FH);
return $md5->hexdigest;
}
You should limit the number of times that you have to read each file's contents:
Inventory the files using Path::Class or some similar method.
a. Build a hash relating file sizes and MD5::Digest to a list of file names.
Compare likely duplicates only. Matching file size and digest.
The following is untested:
use strict;
use warnings;
use Path::Class;
use Digest::MD5;
my $dir = dir('.');
my %files_per_digest;
# Inventory Directory
while ( my $file = $dir->next ) {
my $size = $file->stat->size;
my $digest = do {
my $md5 = Digest::MD5->new;
$md5->addfile( $file->openr );
$md5->hexdigest;
};
push #{ $files_per_digest{"$size - $digest"} }, $file;
}
# Compare likely duplicates only
for my $files ( grep { #$_ > 1 } values %files_per_digest ) {
# Sort by alpha
#$files = sort #$files;
print "Comparing: #files\n";
for my $i ( reverse 0 .. $#files ) {
for my $j ( 0 .. $i - 1 ) {
my $fh1 = $files->[$i]->openr;
my $fh2 = $files->[$j]->openr;
my $diff = 0;
while ( !eof($fh1) && !eof($fh2) ) {
$diff = 1, last if scalar(<$fh1>) ne scalar(<$fh2>);
}
if ( $diff or !eof($fh1) or !eof($fh2) ) {
print " $files->[$i] ($i) is duplicate of $files->[$j] ($j)\n";
$files->[$i]->remove();
splice #$files, $i, 1;
}
}
}
}
I've used rdfind in the past with very good success. It's very accurate, fast, and seems to run leaner than fdupes. According to RDFind's web site (http://rdfind.pauldreik.se/), it can be installed using MacPorts.
I have a series of subfolders, each with images on them.
Structure looks like
/stuff/
IMG_012.JPG
IMG_013.JPG
IMG_014.JPG
/morestuff/
IMG_022.JPG
IMG_023.JPG
IMG_024.JPG
I would like to write a script on my mac terminal to loop through each folder and rename the images sequentially including the folder name. So, the above structure would look like:
/stuff/
stuff_1.JPG
stuff_2.JPG
stuff_3.JPG
/morestuff/
morestuff_1.JPG
morestuff_1.JPG
morestuff_1.JPG
I orignally tried creating a Automator workflow and using variables, but had difficulty assigning the folder name as the variable and getting the loop to work.
I'm hoping there is an elegant solution with the terminal.
Any ideas?
This should work nicely for you. Save it in your HOME directory as "RenameImages", then make it executable like this:
cd
chmod +x RenameImages
Then you can run it (-exec) it on every directory (-type d) like this:
find . -type d -exec ./RenameImages {} \;
Here is the script:
#!/bin/bash
# Ignore case, i.e. process *.JPG and *.jpg
shopt -s nocaseglob
shopt -s nullglob
# Go to where the user says
echo Processing directory $1
cd "$1" || exit 1
# Get last part of directory name
here=$(pwd)
dir=${here/*\//}
i=1
for image in *.JPG
do
echo mv "$image" "${dir}${i}.jpg"
((i++))
done
At the moment it does nothing except show you what it would do. If you like what it is doing, simply remove the word echo from the 3rd to last line and run it again.
I would just like to throw out another way to do this since you mentioned you tried using Automator. This file search and process software: http://www.softpedia.com/get/File-managers/JFileProcessor.shtml https://github.com/stant/jfileprocessor
Will let you search for files with glob or regex, in subfolders to X or all depth, by name, size, date. You can save to a List window or file. Then you can run a groovy (think java) script to do whatever you want to the list of files; zip or tar them, modify the list strings like sed, delete, move, copy files, grep or ls -l them, whatever. In your case you can modify an existing groovy example to do something like:
int numItems = defaultComboBoxModel.getSize();
System.out.println( "defaultComboBoxModel.getSize() num of items =" + numItems + "=" );
String str = "";
for( int i = 0; i < numItems; i++ )
{
str = defaultComboBoxModel.getElementAt( i ).toString();
System.out.println( "file of index =" + i + " str =" + str + "=" );
String cmd = "mv " + str + " " + i + ".jpg"; // or whatever
def list = cmd.execute().text // this stuff just captures output and write to a log file
list.eachLine{
outFile << it;
}
outFile << System.getProperty("line.separator") + "-------------------------------------" + System.getProperty("line.separator");
}
It will also let you massage your list like add to, delete from, subtract one list from another.
I'm trying to emulate RapidCRC's ability to check crc32 values within filenames on Windows Vista Ultimate 64-bit. However, I seem to be running into some kind of argument limitation.
I wrote a quick Perl script, created a batch file to call it, then placed a shortcut to the batch file in %APPDATA%\Microsoft\Windows\SendTo
This works great when I select about 20 files or less, right-click and "send to" my batch file script. However, nothing happens at all when I select more than that. I suspect there's a character or number of arguments limit somewhere.
Hopefully I'm missing something simple and that the solution or a workaround isn't too painful.
References:
batch file (crc32_inline.bat):
crc32_inline.pl %*
Perl notes:
I'm using (strawberry) perl v5.10.0
I have C:\strawberry\perl\bin in my path, which is where crc32.bat exists.
perl script (crc32_inline.pl):
#!/usr/bin/env perl
use strict;
use warnings;
use Cwd;
use English qw( -no_match_vars );
use File::Basename;
$OUTPUT_AUTOFLUSH = 1;
my $crc32_cmd = 'crc32.bat';
my $failure_report_basename = 'crc32_failures.txt';
my %failures = ();
print "\n";
foreach my $arg (#ARGV) {
# if the file has a crc, check to see if it matches the calculated
# crc.
if (-f $arg and $arg =~ /\[([0-9a-f]{8})\]/i) {
my $crc = uc $1;
my $basename = basename($arg);
print "checking ${basename}... ";
my $calculated_crc = uc `${crc32_cmd} "${arg}"`;
chomp($calculated_crc);
if ($crc eq $calculated_crc) {
print "passed.\n";
}
else {
print "FAILED (calculated ${calculated_crc})\n";
my $dirname = dirname($arg);
$failures{$dirname}{$basename} = $calculated_crc;
}
}
}
print "\nReport Summary:\n";
if (scalar keys %failures == 0) {
print " All files OK\n";
}
else {
print sprintf(" %d / %d files failed crc32 validation.\n" .
" See %s for details.\n",
scalar keys %failures,
scalar #ARGV,
$failure_report_basename);
my $failure_report_fullname = $failure_report_basename;
if (defined -f $ARGV[0]) {
$failure_report_fullname
= dirname($ARGV[0]) . '/' . $failure_report_basename;
}
$OUTPUT_AUTOFLUSH = 0;
open my $fh, '>' . $failure_report_fullname or die $!;
foreach my $dirname (sort keys %failures) {
print {$fh} $dirname . "\n";
foreach my $basename (sort keys %{$failures{$dirname}}) {
print {$fh} sprintf(" crc32(%s) basename(%s)\n",
$failures{$dirname}{$basename},
$basename);
}
}
close $fh;
$OUTPUT_AUTOFLUSH = 1;
}
print sprintf("\n%s done! (%d seconds elapsed)\n" .
"Press enter to exit.\n",
basename($0),
time() - $BASETIME);
<STDIN>;
I will recommend just putting a shortcut to your script in the "Send To" directory instead of doing it via a batch file (which is subject to cmd.exes limits on command line length).