Combine multiple images using vips (ruby-vips8) - ruby

Question

How do I apply a function to corresponding pixels of two images of the same resolution? Like Photoshop does when covering one layer with another one. What about more than two images?
If it was Wolfram Mathematica I would take a List of those images and transpose them to get a single "image" where each "pixel" would be an array of N pixels -- there I would apply a Mean[] function to them.
But how do I do that with vips? There are so many Vips::Image methods and only here I could find some minimal description on what do they all mean. So for example:
images = Dir["shots/*"].map{ |i| Vips::Image.new_from_file(i) }
ims = images.map(&:bandmean)
(ims.inject(:+) / ims.size).write_to_file "temp.png"
I wanted it to mean "calculating an average image" but I'm not sure what I've done here.

Answer 1

ruby-vips8 comes with a complete set of operator overloads, so you can just do arithmetic on images. It also does automatic common-subexpression elimination, so you don't need to be too careful about ordering or grouping, you can just write an equation and it should work well.
In your example:
require 'vips8'
images = Dir["shots/*"].map{ |i| Vips::Image.new_from_file(i) }
sum = images.reduce (:+)
avg = sum / images.length
avg.write_to_file "out.tif"
+-*/ with a constant always makes a float image, so you might want to cast the result down to uchar before saving (or maybe ushort?) or you'll have a HUGE output tiff. You could write:
avg = sum / images.length
avg.cast("uchar").write_to_file "out.tif"
By default, new_from_file opens images for random access. If your sources images are JPG or PNG, this will involve decompressing them entirely to memory (or to a disk temp if they are very large) before processing can start.
In this case, you only need to scan the input images from top to bottom as you write the result, so you can stream the images through your system. Change the new_from_file to be:
images = Dir["shots/*"].map { |i| Vips::Image.new_from_file(i, :access => "sequential") }
to hint that you will only be using the image pixels sequentially, and you should see a nice drop in memory and CPU use.
PNG is a horribly slow format, I would use tiff if possible.
You could experiment with bandrank This does something like a median filter over a set of images: you give it an array of images and at each pixel position it sorts the images by pixel value and selects the Nth one. It's a very effective way to remove transitory artifacts.
You can use condition.ifthenelse(then, else) to compute more complex functions. For example, to set all pixels greater than their local average equal to the local average, you could write:
(image > image.gaussblur(1)).ifthenelse(image.gaussblur(1), image)
You might be curious how vips will execute the program above. The code:
(images.reduce(:+) / images.length).cast("uchar")
will construct a pipeline of image processing operations: a series of vips_add() to sum the array, then a vips_linear() to do the divide, and finally a vips_cast() to knock it back to uchar.
When you call write_to_file, each core on your machine will be given a copy of the pipeline and they will queue up to process tiles from the source images as they arrive from the decompressor. Each time a line of output tiles is completed, a background thread will use the selected image write library (libtiff in my example) to send those scanlines back to disk.
You should see low memory use and good CPU utilization.