I am not a shell expert and I would like to know how to write a small algorithm (I think it is possible) to solve my problem.
I have an output directory which itself contains several folders for example data_1 , data_2, etc. These folders also contain different versions, for example version_1, version_2. And finally these versions all contain an image folder photos which contains several thousand images in the form 000001.jpg, 000002.jpg, ...
I'm looking to convert all these photos folders into a video that takes the photos frame by frame. For example for the 2nd version of the dataset data_1 I will do the following commands:
ffmpeg -i data_1/version_1/photos/%6d.jpg data_1/version_1/data_1_ version_1.mov
rm -r data_1/version_1/photos
But this implies that I have to rewrite by hand the command once the program is finished for a folder by changing the folder names and also the .mov file which must imperatively be of the form data_version.mov and saved at data/version/data_version.mov.
I would like a script that automates this procedure by going through all the data folders to create the videos BUT also by checking that if the video exists the script is not run on the current version.
Thank you in advance for any help
Assuming the directory name photos is fixed, would you please try:
while IFS= read -r dir; do # assign "dir" to the path to "photos"
dir2=${dir%/photos} # remove trailing "photos" dirname
name=${dir2#*/} # remove leading slash
name=${name//\//_} # convert slashes to underscores
echo ffmpeg -i "$dir/%6d.jpg" "$dir2/${name}.mov" # execute ffmpeg
echo rm -rf -- "$dir" # rm "photos" directories
done < <(find . -type d -name "photos") # find dir named "photos" recursively
If the output looks good, drop echo and execute.
Related
I have a script that, when I right click on a folder, combines all pngs/jpgs/tifs inside the folder into a PDF and renames the PDF to the name of the folder it resides in.
cd %~dpnx1
for %%a in (.) do set currentfolder=%%~na
start cmd /k magick "*.{png,jpg,tif}" "%currentfolder%.pdf"
However, I have quite a lot of folders and currently have to do this one by one.
How can I create a function where I can right click on a folder, which searches subfolders and combines the jpgs to PDF?
So in the example below, Im wanting to create 3 PDFS (Folder A, Folder B and Folder C) by right clicking and running batch on the parent folder.
Example:
Parent folder (one that I would right click and run script from)
|- Folder A
||- test1.jpg
||- test2.jpg
||- test3.jpg
|- Folder B
||- example1.jpg
|| - example2.jpg
|- Folder C
|| Folder D
|||- temp.jpg
|||- temp2.jpg
I have also recently moved to Mac so I'm looking to use zsh. I've had some help to attempt to use the following myself but no luck:
#!/bin/bash
# Set the output directory
output_dir='./pdfs/'
# Make the output directory if it doesn't exist
mkdir -p "$output_dir"
# Check if an input directory was provided as a command-line argument
if [ $# -eq 0 ]
then
# Use the current directory as the input directory if none was provided
input_dir='./'
else
# Use the first command-line argument as the input directory
input_dir="$1"
fi
# Find all the directories in the input directory
find "$input_dir" -type d | while read dir; do
# Extract the base directory name
dirname=$(basename "$dir")
# Create a PDF file with the same name as the base directory name
output_file="$output_dir/$dirname.pdf"
# Find all the JPEG files in the current directory
find "$dir" -type f -name '*.jpg' | while read file; do
# Convert the JPEG file to PDF and append it to the output file
convert "$file" "$file.pdf"
done
# Concatenate all the PDF files in the current directory into a single PDF
gs -q -dNOPAUSE -dBATCH -sDEVICE=pdfwrite -sOutputFile="$output_file" "$dir"/*.pdf
# Remove the temporary PDF files
rm "$dir"/*.pdf
done
Hope you can help. Thank you.
There are several aspects to this question, and judging by your attempted solution, they will all be non-trivial for you. It is more than a normal question so I'll just give you an outline so you can tackle it in chunks. You'll need to:
install homebrew
install ImageMagick
use Automator to make a workflow for right-click
learn some bash scripting to recurse through directories
learn some ImageMagick to make PDFs
Install homebrew
Go to here and follow instructions to install homebrew. I am not repeating the instructions here as they may change.
You'll likely need to install Xcode command-line tools with:
xcode-select --install
You'll need to set your PATH properly afterwards. Don't omit this step.
Install ImageMagick
You'll need to do:
brew install imagemagick
Setup workflow with Automator for right-click
Next you need to make a script that will be executed when you right-click on a directory. It will look like this when we have done it. I right-clicked on the Junk directory on my desktop and went down to Quick Actions and across to makePDFs.
So, in order to do that you need to start the Automator by clicking ⌘ SPACE and typing Automator and hitting ENTER when it guesses.
Then select New Document and Quick Action. Now navigate the orange areas in the diagram till you find Run Shell Script then drag Run Shell Script over to the right side and drop it in the blue zone. Go on the Edit menu and click ⌘ and then Save As and enter makePDFs in the box. This is the name that will appear in future on your right-click menu.
Now set the options in the green box like I have done.
Now replace all the code in the blue box with the code copied from below:
#!/bin/bash
################################################################################
# Recurse into all subdirectories specified in parameter and make PDF in each
# directory of all images found in there.
################################################################################
# Add ImageMagick from homebrew to PATH
PATH=$PATH:/opt/homebrew/bin
# Check we got a directory as parameter
if [ $# -ne 1 ] ; then
>&2 echo "Usage: $0 DIRECTORY"
exit 1
fi
# Find and process all subdirectories
shopt -s nullglob
while read -rd $'\0' dir; do
# Start a subshell so we don't have to cd back somewhere
(
cd "$dir" || exit 1
# Make list of all images in directory
declare -a images
for image in *.jpg *.png *.tif ; do
images+=("$image")
done
numImages=${#images[#]}
if [ $numImages -gt 0 ] ; then
pdfname=${PWD##*/}
magick "${images[#]}" "${pdfname}.pdf"
fi
)
done < <(find "$1" -type d -print0)
Finally, set the options like I did in the cyan coloured box.
Now save the whole workflow again and everything should work nicely.
For bash, try this (tested on Linux):
for d in */; do convert "$d"/*.{png,jpg,tif} "$d/${d%/}.pdf" ; done
In slo-mo:
for d in */: loop on all directories in current directory (the / restrict matches to directories)). Variable $d contains the directory name, and name has a final /.
"$d"/*.{png,jpg,tif}: all files with png, jpg or tif extension in the directory "$d"
"$d/${d%/}.pdf": the directory name, a slash, the directory name with the ending slash removed, and .pdf`
If you look carefully, the explicit /s in the code aren't necessary since there is already one at the end of $d, but leaving them in makes the code a bit more readable and the multiple '//' are coalesced into a single one.
This code may however complain that there are no png/jpg/tif. A slightly different form makes it behave more nicely:
shopt -s extglob # this is possibly already set by default
for d in */; do convert "$d"/*.#(png|jpg|tif) "$d/${d%/}".pdf ; done
for zsh this could be (untested!):
# shopt -s extglob # no shopt necessary
for d in */; do convert "$d"/*.png(N) "$d"/*.jpg(N) "$d"/*.tif(N) "$d/${d%/}".pdf ; done
with the caveat that if no pattern matches, the command will just be convert whatever_output.pdf and you will get the built-in help.
The difference is that *.{png,jpg,tif} is expanded to *.png *.jpg *.tif before any pattern matching is done, so this represents three file patterns and the shell tries to match each pattern in turn (and leaves the literal pattern in case there is no match), while *.#(png|jpg|tif) is a single file pattern that matches any of the three extensions. This can also make a difference for you because the files do not appear in the same order, *.{png,jpg,tif} lists all the PNG, then the JPG,then the TIF, while *.#(png|jpg|tif) has them all sorted in alphabetical order without regard for the extension.
First, please forgive me and be easy on me if this question seems easy; the first time I tried posting a question about another subject, I didn't provide enough information a few months ago. My apologies.
I'm trying to scan my incoming media folder for new audio files and convert them to my preferred format into another folder, without removing the originals.
I've written the script below and while it seems to work for one-offs, I can't seem to get it to create the destination directory name based off the source directory name; and I can't seem to figure out how to keep it looping, "scanning", for new media to arrive without processing what it's already processed.
I hope this makes sense...
#! /bin/bash
srcExt=$1
destExt=$2
srcDir=$3
destDir=$4
opts=$5
# Creating the directory name - not currently working
# dirName="$(basename "$srcDir")"
# mkdir "$destDir"/"$dirName"
for filename in "$srcDir"/*.flac; do
basePath=${filename%.*}
baseName=${basePath##*/}
ffmpeg -i "$filename" $opts "$destDir"/"$baseName"."$destExt"
done
for filename in "$srcDir"/*.mp3; do
basePath=${filename%.*}
baseName=${basePath##*/}
ffmpeg -i "$filename" $opts "$destDir"/"$baseName"."$destExt"
done
there are different ways of doing this, the easiest way might just be to look at the "modification date" of the file and seeing if it has changed, something like:
#! /bin/bash
srcExt=$1
destExt=$2
srcDir=$3
destDir=$4
opts=$5
# Creating the directory name - not currently working
# dirName="$(basename "$srcDir")"
# mkdir "$destDir"/"$dirName"
for filename in ` find "$srcDir" \( -name '*.mp3' -o -name '*.flac' \) -mmin -10`; do
basePath=${filename%.*}
baseName=${basePath##*/}
ffmpeg -i "$filename" $opts "$destDir"/"$baseName"."$destExt"
done
Consider using mkdir -p which will a) create all necessary intermediate directories, and b) not complain if they already exist.
If you want the new items to be processesd immediately they arrive, look at inotify or fswatch on macOS. In general, if less urgent, schedule your job to run every 10 minutes under cron, maybe prefixing with nice so as not to be a CPU "hog".
Decide which files to generate by changing directory to the source directory and iterating over all files. For each file, work out what the corresponding output file should be according to your rules, test if it already exists, if not create it.
Don't repeat all your for loop code like that, just do:
cd "$srcDir"
for filename in *.flac *.mp3 ; do
GENERATE OUTPUT FILENAME
if [ ! -f "$outputfilename" ] ; then
mkdir -p SOMETHING
ffmpeg -i "$filename" ... "$outputfilename"
fi
done
I have a few files with the format ReportsBackup-20140309-04-00 and I would like to send the files with same pattern to the files as the example to the 201403 file.
I can already create the files based on the filename; I would just like to move the files based on the name to their correct folder.
I use this to create the directories
old="directory where are the files" &&
year_month=`ls ${old} | cut -c 15-20`&&
for i in ${year_month}; do
if [ ! -d ${old}/$i ]
then
mkdir ${old}/$i
fi
done
you can use find
find /path/to/files -name "*201403*" -exec mv {} /path/to/destination/ \;
Here’s how I’d do it. It’s a little verbose, but hopefully it’s clear what the program is doing:
#!/bin/bash
SRCDIR=~/tmp
DSTDIR=~/backups
for bkfile in $SRCDIR/ReportsBackup*; do
# Get just the filename, and read the year/month variable
filename=$(basename $bkfile)
yearmonth=${filename:14:6}
# Create the folder for storing this year/month combination. The '-p' flag
# means that:
# 1) We create $DSTDIR if it doesn't already exist (this flag actually
# creates all intermediate directories).
# 2) If the folder already exists, continue silently.
mkdir -p $DSTDIR/$yearmonth
# Then we move the report backup to the directory. The '.' at the end of the
# mv command means that we keep the original filename
mv $bkfile $DSTDIR/$yearmonth/.
done
A few changes I’ve made to your original script:
I’m not trying to parse the output of ls. This is generally not a good idea. Parsing ls will make it difficult to get the individual files, which you need for copying them to their new directory.
I’ve simplified your if ... mkdir line: the -p flag is useful for “create this folder if it doesn’t exist, or carry on”.
I’ve slightly changed the slicing command which gets the year/month string from the filename.
I have a large repository of media files that follow torrent naming conventions- something unpleasant to read. At one point, I had properly named the folders that contain said files, but not want to dump all the .avi, .mkv, etc files into my main media directory using a bash script.
Overview:
Current directory tree:
Proper Movie Title/
->Proper.Movie.Title.2013.avi
->Proper.Movie.Title.2013.srt
Title 2/
->Title2[proper].mkv
Movie- Epilogue/
->MOVIE EPILOGUE .AVI
Media Movie/
->MEDIAMOVIE.CD1.mkv
->MEDIAMOVIE.CD2.mkv
.
.
.
Desired directory tree:
Proper Movie Title/
->Proper Movie Title.avi
->Proper Movie Title.srt
Title 2.mkv
Movie- Epilogue.avi
Media Movie/
->Media Movie.cd1.mkv
->Media Movie.cd2.mkv
Though this would be an ideal, my main wish is for the directories with only a single movie file within to have that file be renamed and moved into the parent directory.
My current approach is to use a double for loop in a .sh file, but I'm currently having a hard time keeping new bash knowledge in my head.
Help would be appreciated.
My current code (Just to get access to the internal movie files):
#!/bin/bash
FILES=./*
for f in $FILES
do
if [[ -d $f ]]; then
INFILES=$f/*
for file in $INFILES
do
echo "Processing >$file< folder..."
done
#cat $f
fi
done
Here's something simple:
find * -type f -maxdepth 1 | while read file
do
dirname="$(dirname "$file")"
new_name="${dirname##*/}"
file_ext=${file##*.}
if [ -n "$file_ext" -a -n "$dirname" -a -n "$new_name" ]
then
echo "mv '$file' '$dirname/$new_name.$file_ext'"
fi
done
The find * says to run find on all items in the current directory. The -type f says you only are interested in files, and -maxdepth 1 limits the depth of the search to the immediate directory.
The ${file##*.} is using a pattern match. The ## says the largest left hand match to *. which is basically pulling everything off to the file extension.
The file_dir="$(dirname "$file")" gets the directory name.
Note quotes everywhere! You have to be careful about white spaces.
By the way, I echo instead of doing the actual move. I can pipe the output to a file, examine that file and make sure everything looks okay, then run that file as a shell script.
So I have a music folder full of different file formats all mixed together. It IS well-structured: Music/[Artist Name]/[Album Name], with compilation albums in a folder called "Various". Some folders contain just .mp3, .m4a, or .flac files, and some have multiple versions of the album in different file formats. There are also of course various .jpegs of cover art and many of the artist folders contain a .jpeg of the artist's portrait, and there are also many miscellaneous .cue and .log files and other junk.
What I want to end up with is a separate music folder for .flac files, retaining the existing folder structure, removeing them from the existing library. I don't want to end up with empty folders in my current library either, where the album was only in flac. In cases of album folders with multiple formats, I want to move the flacs to the new library along with the cover art, but of course keep the existing cover art in place as well.
I haven't been able to find an application capable of handling this, but I figured a shell script could probably handle it. However I am pathetic with bash and really don't want to break my library.
The files are on a remote disk that I can access with mac, windows, or linux so any approach is good.
Just to fully clarify, here's the logic I'm hoping to code:
Find each subdirectory of /Music that contains .flac files
Copy each of these directories in their entirety (and intermediate parent directories) to a new location in /FLAC, but exclude other audio filetypes (.mp3, .m4a, etc.) from the copy.
If the directory has no other audio filetypes than .flac, delete the entire directory. If it DOES have other audio files, just delete the .flac files.
Do one final sweep through all of /Music to delete any directory that contains no audio files in itself or any subdirectory.
Here's another approach:
find . -type f -name \*.flac -printf "%h\n" |
sort -u |
while read -r dirname; do
new="../flac/$dirname"
echo mkdir -p "$new"
echo mv "$dirname"/*.flac "$new"
jpgs=("$dirname"/*.jpg)
[[ ${#jpgs[#]} -gt 0 ]] && echo cp "$dirname"/*.jpg "$new"
done
find with -printf will print out the directory name of each flac file (relative to the current directory
sort -u to remove the duplicates -- now you have a list of directories containing flac files
the while loop iterates over the directories, clones the directory hierarchy under another directory (amend to suit your needs), moves the flac files, and copies jpg files if there are any.
Remove the echo commands if you're satisfies it works for you.
First make a new parent folder, e.g. "MusicNew" which should be equivalent to the existing 'Music' folder. Then assuming your description of sub-folders and file type that you want to copy, the following lines of code should work.
IFS='\n'
for i in `find Music -mindepth 2 -maxdepth 3 -type f -name "*flac"`;
do
echo ${i};
newfold=`echo ${i} | sed -e 's/Music/MusicNew/g'`;
filename=`basename ${i}`;
fulldirpath="${newfold:0:${#newfold} - ${#filename}}";
echo $fulldirpath/${filename};
mkdir -p ${fulldirpath};
mv ${i} ${fulldirpath}/;
done