how to convert bayerrg8 format image to rgb image - scikit-image

Question

I've got a camera that provides images in Bayer RG8 format.
I'm using skimage for processing images, but I could not find away to convert the Bayer RG8 format to standard RGB (to display on screen).
Is there any way to do this with skimage?
I did find a reference to opencv conversion, but I'm trying to avoid including opencv in my app (unless it is absolutely necessary).

Answer 1

As you have not provided any input data, I took the greyscale image from here and made it into a raw Bayer8 file with GBRG ordering using ImageMagick as follows:
magick mandi.png -trim -depth 8 gray:bayer.bin
which gives me an 1013x672 pixel file of 680,736 bytes.
Then I read it like this and made it into an image that skimage can understand like this:
#!/usr/bin/env python3
import numpy as np
from skimage.io import imsave
# Width and height of Bayer image, not original which is w/2 x h/2
w, h = 1013, 672
ow, oh = w//2, h//2
# Load in Bayer8 image, assumed raw 8-bit GBRG
bayer = np.fromfile('bayer.bin', dtype=np.uint8).reshape((h,w))
# Pick up raw uint8 samples
R = bayer[1::2, 0::2] # rows 1,3,5,7 columns 0,2,4,6
B = bayer[0::2, 1::2] # rows 0,2,4,6 columns 1,3,5,7
G0 = bayer[0::2, 0::2] # rows 0,2,4,6 columns 0,2,4,6
G1 = bayer[1::2, 1::2] # rows 1,3,5,7 columns 1,3,5,7
# Chop any left-over edges and average the 2 Green values
R = R[:oh,:ow]
B = B[:oh,:ow]
G = G0[:oh,:ow]//2 + G1[:oh,:ow]//2
# Formulate image by stacking R, G and B and save
out = np.dstack((R,G,B))
imsave('result.png',out)
And get this:
Copyright Mathworks, Inc.
Of course, there are more sophisticated methods of interpolating, but this is the most basic and you are welcome to take it and improve it!
Ok, I had some time and I tried to do a 2d-interpolation of the missing values in the Bayer array. I am not 100% confident of my answer, but I think it should be pretty close.
Basically, I copy the original Bayer array at full resolution, and overwrite all green and blue samples with np.Nan and call that Red. Then I do a 2d-interpolation to replace the Nans.
Same again for green and blue, that gives this:
#!/usr/bin/env python3
import numpy as np
from skimage.io import imsave
from scipy.interpolate import griddata
def interp2d(im):
"""Interpolate in 2d array, replacing NaNs with interpolated values"""
x, y = np.indices(im.shape)
im[np.isnan(im)] = griddata(
(x[~np.isnan(im)], y[~np.isnan(im)]),
im[~np.isnan(im)],
(x[np.isnan(im)], y[np.isnan(im)]))
im = np.nan_to_num(im)
return np.clip((im),0,255)
# Width and height of Bayer image
w, h = 1013, 672
# Calculate output width and height as multiples of 4
ow = (w//4) * 4
oh = (h//4) * 4
# Load in Bayer8 image, assumed raw 8-bit GBRG, reshape and make sides multiple of 4
bayer = np.fromfile('bayer.bin', dtype=np.uint8).reshape((h,w)).astype(np.float)[:oh, :ow]
# In following code you'll see "cell" which is the basic repeating 2x2 cell of a Bayer matrix
#
# cell = G B
# R G
#
# Set everything not Red in bayer array to Nan, then replace Nans with interpolation
cell = np.array([[np.NaN, np.NaN],
[1.0 , np.NaN]])
R = bayer*np.tile(cell,(oh//2,ow//2))
R = interp2d(R).astype(np.uint8)
# Set everything not Green in bayer array to Nan, then replace Nans with interpolation
cell = np.array([[1.0 , np.NaN],
[np.NaN, 1.0 ]])
G = bayer*np.tile(cell,(oh//2,ow//2))
G = interp2d(G).astype(np.uint8)
# Set everything not Blue in bayer array to Nan, then replace Nans with interpolation
cell = np.array([[np.NaN, 1.0 ],
[np.NaN, np.NaN]])
B = bayer*np.tile(cell,(oh//2,ow//2))
B = interp2d(B).astype(np.uint8)
# Form image by stacking R, G and B and save
imsave('result.png',np.dstack((R,G,B)))
Keywords: Python, bayer, bayer8, debayer, de-bayer, de-mosaic, de-mosaicking, image, raw, CFA, skimage, scikit-image, image processing.