Bash Use Origin File as Destination File - bash

I'm working with the ImageMagick bash tool, which uses commands of the form:
<command> <input filename> <stuff> <output filename>
I'm trying to do the following command:
<command> x.png <stuff> x.png
but for every file in a directory. I tried:
<command> *.png <stuff> *.png
But that didn't work. What's the correct way to perform such a command on every file in a directory?

Many Unix command-line tools, such as awk, sed, grep are stream-based or line-oriented, which means they process files one line at a time. When using these, it is necessary to write output to an intermediate file and then rename that (with mv) back over the original input file. The reason for that is that you may be writing over the input file before you have read it all, so you will clobber your inputs. In that case, #sjsam's answer is absolutely correct - especially since he is careful to use the && in between so that the mv is not done if the command is not successful.
In the specific case of ImageMagick however, and its convert, compare, compose commands, this is NOT the case. These programs are file-oriented, not line-oriented, so they read the entire input file into memory before writing any outputs. The reason is that they often need to know if there are any transparent pixels in an image before they can start processing, or how many colours there are in an image and they cannot know this till they have read the entire file. As such, it is perfectly safe, and in fact idiomatic to use the same input filename as output filename, like this:
convert image.jpg ... -crop 100x100 -colors 16 ... image.jpg
In answer to your question, when you have multiple files to process, the preferred method is to use the mogrify tool (also part of the ImageMagick suite) which is specifically intended for processing multiple files. So, you would do this:
mogrify -crop 100x100 -colors 16 *.jpg
which would overwrite your input files with the results - which is what you asked for. If you did not want it to overwrite your input files, you would add a -path telling ImageMagick the path to an output directory like this:
mogrify -path /path/to/thumbnails -thumbnail -colors 16 *.jpg
If you had too many files for the Windows COMMAND.EXE/CMD.EXE to expand and pass to mogrify, you would let ImageMagick expand the glob (the asterisk) internally like this and be able to deal with unlimited numbers of files:
mogrify -crop 100x100 '*.jpg'
The point is that not only is the mogrify syntax concise, and portable across Windows/OSX/Linux, it also means you only need to create a single process to do all your images, rather than having to write a FOR loop and create and execute a convert process and a mv process for each and every one of potentially thousands of files - so it is much more efficient and less error-prone.

For a single file do like this :
<command> x.png <stuff> temp.png && mv temp.png x.png
For a set of files do like this :
#!/bin/bash
find `pwd` -name \*.png | while read line
do
<command> "$line" <stuff> temp.png && mv temp.png "$line"
done
Save the script in the folder containing the png files as processpng. Make it an executable and run it.
./processpng
A general version of the above script which takes the path to folder containing the above files as argument is below :
#!/bin/bash
find $1 -name \*.png | while read line
do
<command> "$line" <stuff> temp.png && mv temp.png "$line"
done
Save the script as processpng anywhere in the computer. Make it an executable and run it like :
./processpng /path/to/your/png/folder
Edit:
Incorporating #anishsane's solution, a compact way of achieving the same
results would be
#!/bin/bash
find $1 -name \*.png -exec bash -c "<command> {} <stuff> temp.png && mv temp.png {}"
In the context of find .... -exec:
{} indicates (contains) the result(s) from the find expression. Note
that empty curly braces {} have no special meaning to shell so we can
get away without escaping {}
Note: Emphasis mine.
Reference: This AskUbuntu Question.

Related

Loop through the directories to get the image files

I have a directory which includes multiple sub-directories. I would like to go through the directories and subdirectories and find the jpg files and convert the size using mogrify command. I would like to do it as dynamic as possible that's why I wrote a script. The $1 is the first argument that I pass through when executing the bash script. After running the script, it gives me an error about 'mogrify can not read [#]%'. I guess something is very wrong with my code and I am not mature in bash. Can anyone tell me how to do this script dynamically so that would be fast.
p.s: the name of jpg files are not in especial format...just bunch of numbers.
for folder in $1/*
do
for file in "$folder"/*
do
if [ -e "${file[#]%.jpg}" ]; then
mogrify -resize 112x112! "${file[#]%.jpg}"
fi
done
done
If you're open to using find, then this becomes pretty easy:
#!/usr/bin/env bash
find "$1" \( -iname \*.jpg -o -iname \*.jpeg \) -print0 | while read -r -d $'\0' file; do
# base="${file##*/}" $base is the file name with all the directory stuff stripped off
# dir="${file%/*} $dir is the directory with the file name stripped off
mogify -resize '112x112!' "$file"
done
Put that in a file named mymog.bash then
$ chmod 755 mymog.bash
$ mymog.bash /some/dir
Notes:
! is special to bash, so putting that in the single quotes make it "unspecial", passing it along to the mogrify command unmolested.
The double quotes around $1 and $file are needed in case a directory or file name has spaces in it. If you had a directory named /Users/alice/my pictures and didn't use the quotes, mogrify would get one argument named /Users/alice/my and another one named pictures.
Make sure you use the \( and \) for find. That makes the whole condition ("match *.jpg" OR "match *.jpeg") apply to the action -print0.
I used find 's -print0 action which prints each matching file name with a null-terminated (zero-terminated) string. You can have filenames that have newline characters in the middle. This protects against that.
bash 's built-in read command reads until a newline by default. I used the -d $'\0' to make it read each "line" or "record" (filename) up to the null (zero) character at its end. (Each ends with null because of the -print0.)
This solution (one of many) has two parts:
It uses the find utility to find (under the directory given) all files that end in .jpg or .jpeg, ignoring the case of the filenames. [So it will match .JPG or even \.JpEg.]
It spits out one record for each file.
If you give it an absolute path like /some/dir, it will find /some/dir/a.jpg and /some/dir/sub1/sub2/sub3/b.jpg.
If you give it a relative path like ../../nearby/dir, it will find ../../nearby/dir/c.jpg and ../../nearby/dir/sub1/sub2/sub3/d.jpeg.
The find part ends with the first | on that line. After that, it is a bash while…do loop.
The variable file takes on the value of each record spit out by find.
The loop (everything between do and done) runs once for each value that file takes on.
The two rows that start with # are comments. They contain commands that are ignored (skipped). You can remove the # to have bash run those commands too. I included them as examples in case you needed the directory part or just the filename part of the record.
find "$1" -type f -name "*.jpg" -exec mogrify -resize 112x112! {} \;
Try find and a while read loop:
find "$1" -type f -name '*.jpg' -print | while read fname
do
....
done
If your filenames may contain special chars like line feed, the use:
find "$1" -type f -name '*.jpg' -print0 | while IFS= read -r -d '' fname
do
....
done
There are many more options to tune the search.

I need some slightly basic BASH help - find + execdir + sox

Ok, hello. I have some scripts that I use on the command line to rip mp3s from .wav and .aif files. Lately I've been playing with this sampler that wants only mono wav files. WAIT - I promise this is a programming question.
I want to run a command to split all wav files inside a directory to mono files with a different file name.
So, for example Drummerrolleasy.wav, would then have DrummerrolleasyL.wav in the same directory as the left mono split of the original stereo wav.
I will ensure that when the command runs, there are only stereo wav files in the directory.
I have this command:
find . -name "*.wav" -execdir echo $1 {} \;
that works as expected, and provides me the names of the files, one by one, that I would like to then put inside a call to sox.
this:
find . -type f -name “*.wav” -exec sox {} "$1" L"$1" \;
does not work (runs but does nothing), and I'm not finding anything particularly illuminating for how to do this. I might need to spawn a new shell, but this doesn't work either because the shell just sits there:
find . -name "*.wav" -execdir sh -c 'sox "{}" "$1" “L””$1"’ \;
If I have to figure this out on my own, that can work too, and when I get there I will update but if someone would like to point me in the right direction I would be much obliged.
Goes like this:
for file in *.wav; do sox $file L$file remix 1; done
thanks Mix .L and .R files into a stereo file using SOX in bulk
saves me a lot of time. BASH rules.

Batch convert PNGs to individual PDFs while maintaining deep folder hierarchy in bash

I've found a solution that claims to do one folder, but I have a deep folder hierarchy of sheet music that I'd like to batch convert from png to pdf. What do my solutions look like?
I will run into a further problem down the line, which may complicate things. Maybe I should write a script? (I'm a total n00b fyi)
The "further problem" is that some of my sheet music spans more than one page, so if the script can parse filenames that include "1of2" and "2of2" to be turned into a single pdf, that'd be neat.
What are my options here?
Thank you so much.
Updated Answer
As an alternative, the following should be faster (as it does the conversions in parallel) and also able to handle larger numbers of files:
find . -name \*.png -print0 | parallel -0 convert {} {.}.pdf
It uses GNU Parallel which is readily available on Linux/Unix and which can be simply installed on OSX with homebrew using:
brew install parallel
Original Answer (as accepted)
If you have bash version 4 or better, you can use extended globbing to recurse directories and do your job very simply:
First enable extended globbing with:
shopt -s globstar
Then recursively convert PNGs to PDFs:
mogrify -format pdf **/*.png
You can loop over png files in a folder hierarchy, and process each one as follows:
find /path/to/your/files -name '*.png' |
while read -r f; do
g=$(basename "$f" .png).pdf
your_conversion_program <"$f" >"$g"
done
To merge pdf-s, you could use pdftk. You need to find all pdf files that have a 1of2 and 2of2 in their name, and run pdftk on those:
find /path/to/your/files -name '*1of2*.pdf' |
while read -r f1; do
f2=${f1/1of2/2of2} # name of second file
([ -f "$f1" ] && [ -f "$f2" ]) || continue # check both exist
g=${f1/1of2//} # name of output file
(! [ -f "$g" ]) || continue # if output exists, skip
pdftk "$f1" "$f2" output "$g"
done
See:
bash string substitution
Regarding a deep folder hierarchy you may use find with -exec option.
First you find all the PNGs in every subfolder and convert them to PDF:
find ./ -name \*\.png -exec convert {} {}.pdf \;
You'll get new PDF files with extension ".png.pdf" (image.png would be converted to image.png.pdf for example)
To correct extensions you may run find command again but this time with "rename" after -exec option.
find ./ -name \*\.png\.pdf -exec rename s/\.png\.pdf/\.pdf/ {} \;
If you want to delete source PNG files, you may use this command, which deletes all files with ".png" extension recursively in every subfolder:
find ./ -name \*\.png -exec rm {} \;
if i understand :
you want to concatenate all your png files from a deep folders structure into only one single pdf.
so...
insure you png are ordered as you want in your folders
be aware you can redirect output of a command (say a search one ;) ) to the input of convert, and tell convert to output in one pdf.
General syntax of convert :
convert 1.png 2.png ... global_png.pdf
The following command :
convert `find . -name '*'.png -print` global_png.pdf
searches for png files in folders from cur_dir
redirects the output of the command find to the input of convert, this is done by back quoting find command
converts works and output to pdf file
(this very simple command line works fine only with unspaced filenames, don't miss quoting the wild char, and back quoting the find command ;) )
[edit]Care....
be sure of what you are doing.
if you delete your png files, you will just loose your original sources...
it might be a very bad practice...
using convert without any tricky -quality output option could create an enormous pdf file... and you might have to re-convert with -quality "60" for instance...
so keep your original sources until you do not need them any more

bash: moving files to original directory based on filename?

I've got a bunch of subdirectories with a couple thousand PNG files that will be sent through Photoshop, creating PSD files. Photoshop can only output those to a single folder, and I want to move each one back to their original directory - so the new file foo_bar_0005.psd should go to where foo_bar_0005.png already is. Every filename only exists once.
Can somebody help me with this? I'm on OSX.
You might start from this minimal script:
#!/bin/bash
search_dir="search/png/from/this/directory/"
psd_dir="path/to/psd/directory/"
for psd_file in "$psd_dir"*.psd; do
file_name="$(echo $psd_file | sed 's/.*\/\(.*\).psd$/\1/g')"
png_dir="$(find $search_dir -name $file_name.png | grep -e '.*/' -o)"
mv $psd_file $png_dir
done
But note that this script doesn't include any error handlers e.g. file collision issue, file not found issue, etc.
Each file found with this find is piped to a Bash command that successively make the psd conversion and move the .psd to the .png original directory.
psd_dir=/psd_dir/
export psd_dir
find . -type f -name '*.png' | xargs -L 1 bash -c 'n=${1##*/}; echo photoshop "$1" && echo mv ${psd_dir}${n%.png}.psd ${1%/*}/; echo' \;
The echo are here to give you an overview of the result.
You should remove them to launch the real photoshop command.

Converting all .pdf files in folder tree to .png

I'm on Ubuntu, and I have a tree of folders containing .pdf files. I need to convert each one to a .png format. The bash script I am currently using is:
for f in $(find ./polkadots -name 'image.pdf'); do
convert -transparent white -fuzz 10% $f image.png;
done
I have tested the for loop by itself, and it works (it produces a list of all the .pdf files under the ./polkadots folder that I need to convert):
for f in $(find ./polkadots -name 'image.pdf'); do
echo "$f";
done
I have tested the imagemagic convert command by itself, and it works (it converts a single file in my current directory from .pdf to .png):
convert -transparent white -fuzz 10% image.pdf image.png
However, when I combine them... the console sits and thinks for a while, and then concludes.. but no files have been created or changed, and no error messages are produced. What am I doing wrong?
EDIT: The new .png files are being created, but they are being created in my current directory, instead of in the sub-directory where the .pdf was found. How do I fix this?
Try using find alone. No need to use a loop.
I haven't tested this command but it should work.
find ./polkadots -name 'image.pdf' -exec convert -transparent white -fuzz 10% {} image.png \; -print
The -print at the end is optional. I prefer to see which files have been modified.
Maybe you can find output option in convert command directly, which can export the png file to your expect folder. Anyway follow your idea, here is the updated code:
find ./polkadots -type f -name "image.pdf" |while read line
do
dir=${line%/*}
convert -transparent white -fuzz 10% $line image.png
mv image.png ${dir}/image.png
done
If you need convert all pdf files under polkadots folder, try this:
find ./polkadots -type f -name "*.pdf" |while read line
do
dir=${line%/*}
file=${line##*/}
file=${file%.*}
convert -transparent white -fuzz 10% $line ${file}.png
echo mv ${file}.png ${dir}/${file}.png
done
If you are using bash 4+, you should use globstar instead of find
#!/bin/bash
shopt -s globstar
for f in ./polkadots/**/image.pdf; do
convert -transparent white -fuzz 10% "$f" "${f%/*}/image.png"
done
If you're using an older bash, your original answer is close to fine, but has a few bugs/potential bugs,
If you directories have spaces in them, your original script will case errors, so use the | while read -r -d '' f syntax.
Don't put a ; at the end of command lines
Quote all variables to prevent expansion problems
As you pointed out, main issue in your case was not specifying destination dir, so you can use ${f%/*} parameter expansion to get the dir (this will delete everything including and after the last / in $f, then put the / back and append the filename like below.
Edited
#!/bin/bash
find ./polkadots -name "image.pdf" -print0 | while read -r -d '' f; do
convert -transparent white -fuzz 10% "$f" "${f%/*}/image.png"
done

Resources