I can understand the explanation in tutorial 6, which is:
// Func gradient("gradient");
// Var x("x"), y("y");
// gradient(x, y) = x + y;
// gradient.realize(8, 8);
//
// This does three things internally:
// 1) Generates code than can evaluate gradient over an arbitrary
// rectangle.
// 2) Allocates a new 8 x 8 image.
// 3) Runs the generated code to evaluate gradient for all x, y
// from (0, 0) to (7, 7) and puts the result into the image.
// 4) Returns the new image as the result of the realize call.
However, follow the description, I can't figure it out how such a example works:
Func histogram("hist_serial");
histogram(i) = 0;
RDom r(0, input.width(), 0, input.height());
histogram(input(r.x, r.y) / 32) += 1;
histogram.vectorize(i, 8);
histogram.realize(8);
What I am confusing is: in the "gradient" example, evaluating gradient for all x, y from (0,0) to (7,7) can give us a result, like gradient(1,1)=1+1=2. But in the second example, evaluating histogram for i from 0 to 7 looks strange to me, as I think that we are trying to calculate the result from back to front. A more natural way is to evaluate the input first, then calculate histogram.
So, how the "realize" in the second example works?
Halide automatically infers all of the values which need to be computed to produce a requested region of output. realize just asks the pipeline to compute the requested region of the output Func(s). Halide then automatically infers what regions of which earlier Funcs are required, and recursively evaluates all of the those, up to the inputs, before producing the requested region of output.
Related
This question already has an answer here:
Efficient inpaint with neighbouring pixels
(1 answer)
Closed 6 years ago.
I am trying to replace all pixels with certain value in an image with the average values of the neighbors. Can interp2 be useful here? I tried this -
I = imread('test_image.JPG');
[r c] = size(I);
class_of_image = class(I);
[xi,yi] = meshgrid(1:0.5:c,1:0.5:r);
I1 = cast(interp2(double(image),xi,yi,'linear'),class_of_image);
[x_z,y_z] = find(I1==0);
I1(x_z,y_z) = I1(x_z-1,y_z)+I1(x_z+1,y_z)+I1(x_z,y_z-1)+I1(x_z,y_z+1);
This fails spectacularly with an error message - Index exceeds matrix dimensions.
I realize that the error is in trying to access I1 indices beyond r and c. Is there a generic way to incorporate this in the code?
Please help!
If you are trying to replace pixels in an image that are at a certain value to be the average of its 4 neighbours, then you don't have to use interp2. It looks like you are doubling the size of the image and then sampling from that image when you're done.
If you want to do what you're asking, you need to use column-major indices to facilitate the vectorized access of pixels. Specifically, you need to use sub2ind to help determine the locations you need to access in your matrix.
However, you will need to account for pixels that go out of bounds. There are many ways to accommodate this, but what I will implement is known as zero-padding where the border pixels are simply set to 0. I would create a zero-padded image where the top and bottom rows as well as the left and right values are all some sentinel value (like -1), use find on this image to find the coordinates then do the inpainting. Make sure you set the border pixels back to 0 before doing this so that you don't use -1 as part of the inpainting. You would then crop the border pixels of this new image when you're done to obtain the final output image.
Therefore, if you want to perform your "inpainting" try this instead:
% Read in image
I = imread('test_image.JPG');
% Create padded image with border pixels set to -1
Ipad = -ones(size(I) + 2);
% Place image in the middle
Ipad(2:end-1,2:end-1) = I;
% Find zero pixels
[r,c] = find(I == 0);
% Now set border pixels to 0
Ipad(Ipad == -1) = 0;
% Find column major indices for those elements that are 0
% as well as their 4 neighbours
ind = sub2ind(size(I), r, c);
ind_up = sub2ind(size(I), r-1, c);
ind_down = sub2ind(size(I), r+1, c);
ind_left = sub2ind(size(I), r, c-1);
ind_right = sub2ind(size(I), r, c+1);
% Perform the inpainting by averaging
Ipad(ind) = (Ipad(ind_up) + Ipad(ind_down) + Ipad(ind_left) + Ipad(ind_right))/4;
% Store the output in I1 after removing border pixels
I1 = Ipad(2:end-1,2:end-1);
However, a possibly shorter way to do this even though you would operate on the entire image would be to perform 2D convolution using a 3 x 3 kernel whose elements are 1 in the cardinal directions and ensuring that you divide by 4 to find the average value per location. After, you would simply copy over those values in the output that are 0 in the original image. You can use conv2 to do that and make sure you specify the 'same' flag to ensure that the output size is the same as the input size. The behaviour of conv2 when you approach the border elements is to zero-pad, which is what I did already in the first implementation:
% Read in image
I = imread('test_image.JPG');
% Specify kernel
kernel = [0 1 0; 1 0 1; 0 1 0] / 4;
% Perform convolution - make sure you cast image to double
% as convolution in 2D only works for floating-point types
out = conv2(double(I), kernel, 'same');
% Copy over those values from the output that match the value
% to be inpainted for the input. Also cast back to original
% data type.
I1 = I;
I1(I == 0) = cast(out(I == 0), class(I));
I'm trying to compare the performance in Halide of a 2-pass, separable approach to the integral-image-based box filtering approach to gain a better understanding of Halide scheduling. I cannot find any examples of Integral Image creation in Halide where the Integral Image Function is used in the defintion of a subsequent function.
ImageParam input(type_of<uint8_t>(), 3, "image 1");
Var x("x"), y("y"), c("c"), xi("xi"), yi("yi");
Func ip("ip");
ip(x, y, c) = cast<float>(BoundaryConditions::repeat_edge(input)(x, y, c));
Param<int> radius("radius", 15, 1, 50);
RDom imageDomain(input);
RDom r(-radius, radius, -radius, radius);
// Make an integral image
Func integralImage = ip;
integralImage(x, imageDomain.y, c) += integralImage(x, imageDomain.y - 1, c);
integralImage(imageDomain.x, y, c) += integralImage(imageDomain.x - 1, y, c);
integralImage.compute_root(); // Come up with a better schedule for this
// Apply box filter to integral image
Func outputImage;
outputImage(x,y,c) = integralImage(x+radius,y+radius,c)
+ integralImage(x-radius,y-radius,c)
- integralImage(x-radius,y+radius,c)
- integralImage(x-radius,y+radius,c);
Expr normFactor = (2*radius+1) * (2*radius+1);
outputImage(x,y,c) = outputImage(x,y,c) / normFactor;
result(x,y,c) = cast<uint8_t>(outputImage(x,y,c));
result.parallel(y).vectorize(x,8)
I did find the following code in the tests:
https://github.com/halide/Halide/blob/master/test/correctness/multi_pass_reduction.cpp
But this example uses realize to compute the integral image as a buffer over a fixed domain and doesn't consume the definition of integral image as a function in the definition of a subsequent function.
When I run this code, I observe that:
The computation of the Integral Image is extremely slow (moves my pipeline to 0 fps)
I get an incorrect answer. I feel like I must be somehow misdefining my integral image
I also have a related question, how would one best schedule the computation of an integral image in this type of scenario in Halide?
My problem was in the definition of my integral image. If I change my implementation to the standard one pass definition of the integral image, I get expected behavior:
Func integralImage;
integralImage(x,y,c) = 0.0f; // Pure definition
integralImage(intImDom.x,intImDom.y,c) = ip(intImDom.x,intImDom.y,c)
+ integralImage(intImDom.x-1,intImDom.y,c)
+ integralImage(intImDom.x,intImDom.y-1,c)
- integralImage(intImDom.x-1,intImDom.y-1,c);
integralImage.compute_root();
I still have remaining questions about the most efficient algorithm/schedule in Halide for computing an integral image, but I'll re-post that as a more specific question, as my current post was kind of open-ended.
As an aside, there is a second problem in the code above in that padding of the input image is not handled correctly.
I want to Compute the mean absolute error between two image in Matlab and named it MAE
there is the code:
x=imread('duck.jpg');
imshow(x)
xmin=min(x);
xmax = max(x);
xmean=mean(x);
I = double(x) / 255;
v = var(I(:));
y = imnoise(x, 'gaussian', 0, v / 10);
y = double(y) / 255;
imshow(y)
There's no need to evaluate the min(), max(), mean() for the first image in order to evaluate the MAE.
Since the MAE is the sum of the (L1-norm) differences between corresponding pixels in your two images x and y (divided by the number of pixels), you can simply evaluate it as:
MAE=sum(abs(x(:)-y(:)))/numel(x);
where numel() is a function that returns the number of elements in its argument. In your case since x and y have the same number of elements you can either put numel(x) or numel(y).
Can anyone suggest any links, ideas or algorithms to generate flowers randomly like the one as my profile pic? The profile pic flower has only a 10 x 10 grid and the algorithm is not truly random. I would also prefer that the new algorithm use a grid of about 500 x 500 or even better, allow the user to pick the size of the grid.
[Plant[][] is declared as int plant[10][10];]
public void generateSimpleSky(){
for(int w2=0;w2<10;w2++)
for(int w3=0;w3<10;w3++)
plant[w2][w3]=5;
}
public void generateSimpleSoil(){
for(int q=0;q<10;q++)
plant[q][9]=1;
}
public void generateSimpleStem(){
int ry=rand.nextInt(4);
plant[3+ry][8]=4;
xr=3+ry;
for(int u=7;u>1;u--){
int yu=rand.nextInt(3);
plant[xr-1+yu][u]=4;
xr=xr-1+yu;
}
}
public void generateSimpleFlower(){
plant[xr][2]=3;
for(int q2=1;q2<4;q2++)
if((2-q2)!=0)
plant[xr][q2]=2;
for(int q3=xr-1;q3<=xr+1;q3++)
if((xr-q3)!=0)
plant[q3][2]=2;
}
It sounds like a reasonably simple problem where you just generate 1 parameter at a time, possibly based on the output of the previous variables.
My model of a flower will be: It has just a reasonably upright stem, a perfectly round center, some amount of leaves on the stem on alternating sides, petals perfectly distributed around the center.
random() is just a random number within some chosen bounds, the bounds may be unique for each variable. random(x1, x2, ..., xn) generates a random number within some bounds dependent on the variables x1, x2, ..., xn (as in stemWidth < stemHeight/2, a reasonable assumption).
The Stem
stemXPosition = width / 2
stemHeight = random()
stemWidth = random(stemHeight)
stemColour = randomColour()
stemWidthVariationMax = random(stemWidth, stemHeight)
stemWidthVariationPerPixel = random(stemWidth, stemHeight)
stemWidthVariationMax/-PerPixel are for generating a stem that isn't perfectly straight (if you want to do something that complicated, a low PerPixel is for smoothness). Generate the stem using these as follows:
pixelRelative[y-position][0] := left x-position at that y-position relative to the stem
pixelRelative[y-position][1] := right x-position at that y-position relative to the stem
pixelRelative[0][0] = randomInRange(-stemWidthVariationMax, stemWidthVariationMax)
for each y > 0:
pixelRelative[y-1][0] = max(min(randomInRange(pixel[y] - stemWidthVariationPerPixel,
pixel[y] + stemWidthVariationPerPixel),
-stemWidthVariationMax),
stemWidthVariationMax)
//pixelRelative[0][1] and pixelRelative[y-1][1] generated same as pixelRelative[y-1][i]
for each y:
pixelAbsolute[y][0] = width / 2 - stemWidth / 2 + pixelRelative[y][0]
pixelAbsolute[y][1] = width / 2 + stemWidth / 2 + pixelRelative[y][1]
You can also use arcs to simplify things and go more than 1 pixel at a time.
The Top
centerRadius = random(stemHeight)
petalCount = random() // probably >= 3
petalSize = random(centerRadius, petalCount)
It's not too easy to generate the petals, you need to step from 0 to 2*PI with step-size of 2*PI/petalCount and generate arcs around the circle. It requires either a good graphics API or some decent maths.
Here's some nicely generated tops of flowers, though seemingly not open-source. Note that they don't have a center at all. (or centerRadius = 0)
The Leaves
You could probably write an entire paper on this, (like this one) but a simple idea would just be to generate a 1/2 circle and extend lines outward from there to meet at 2*the radius of the circle and to draw parallel lines on the flower.
Once you have a leaf generation algorithm:
leafSize = random(stemHeight) // either all leaves are the same size or generate the size for each randomly
leafStemLength = random(leafSize) // either all leaves have the same stem length or generate for each randomly
leafStemWidth = random(leafStemLength)
leaf[0].YPosition = random(stemHeight)
leaf[0].XSide = randomly either left or right
leaf[0].rotation = random between say 0 and 80 degrees
for each leaf i:
leaf[i].YPosition = random(stemHeight, leaf[i-1]) // only generate new leaves above previous leaves
leaf[i].XSide = opposite of leaf[i].XSide
Last words
The way to determine the bounds of each random would be either to argue it out, or give it some fixed value, generate everything else randomly a few times, keep increasing / decreasing it until it starts to look weird.
10 x 10 versus 500 x 500 would probably require greatly different algorithms, I wouldn't recommend the above for below 100 x 100, maybe generate a bigger image and simply shrink it using averaging or something.
Code
I started writing some Java code, when I realised it may take a bit longer than I would like to spend on this, so I'll show you what I have so far.
// some other code, including these functions to generate random numbers:
float nextFloat(float rangeStart, float rangeEnd);
int nextInt(int rangeStart, int rangeEnd);
...
// generates a color somewhere between green and brown
Color stemColor = Color.getHSBColor(nextFloat(0.1, 0.2), nextFloat(0.5, 1), nextFloat(0.2, 0.8));
int stemHeight = nextInt(height/2, 3*height/4);
int stemWidth = nextInt(height/20, height/20 + height/5);
Color flowerColor = ??? // I just couldn't use the same method as above to generate bright colors, but I'm sure it's not too difficult
int flowerRadius = nextInt(Math.min(stemHeight, height - stemHeight)/4, 3*Math.min(stemHeight, height - stemHeight)/4);
I am having some problems in matlab i don't understand. The following piece of code analyses a collection of images, and should return a coherent image (and always did).
But since I've put an if-condition in the second for-loop (for optimisation purposes) it returns an interlaced image.
I don't understand why, and am getting ready to throw my computer out the window. I suspect it has something to do with ind2sub, but as far as i can see everything is working just fine! Does anyone know why it's doing this?
function imageMedoid(imageList, resizeFolder, outputFolder, x, y)
% local variables
medoidImage = zeros([1, y*x, 3]);
alphaImage = zeros([y x]);
medoidContainer = zeros([y*x, length(imageList), 3]);
% loop through all images in the resizeFolder
for i=1:length(imageList)
% get filename and load image and alpha channel
fname = imageList(i).name;
[container, ~, alpha] = imread([resizeFolder fname]);
% convert alpha channel to zeros and ones, add to alphaImage
alphaImage = alphaImage + (double(alpha) / 255);
% add (r,g,b) values to medoidContainer and reshape to single line
medoidContainer(:, i, :) = reshape(im2double(container), [y*x 3]);
end
% loop through every pixel
for i=1:(y * x)
% convert i to coordinates for alphaImage
[xCoord, yCoord] = ind2sub([x y],i);
if alphaImage(yCoord, xCoord) == 0
% write default value to medoidImage if alpha is zero
medoidImage(1, i, 1:3) = 0;
else
% calculate distances between all values for current pixel
distances = pdist(squeeze(medoidContainer(i,:,1:3)));
% convert found distances to matrix of distances
distanceMatrix = squareform(distances);
% find index of image with the medoid value
[~, j] = min(mean(distanceMatrix,2));
% write found medoid value to medoidImage
medoidImage(1, i, 1:3) = medoidContainer(i, j, 1:3);
end
end
% replace values larger than one (in alpha channel) by one
alphaImage(alphaImage > 1) = 1;
% reshape image to original proportions
medoidImage = reshape(medoidImage, y, x, 3);
% save medoid image
imwrite(medoidImage, [outputFolder 'medoid_modified.png'], 'Alpha', alphaImage);
end
I didn't include the whole code, just this function (for brevity's sake), if anyone needs more (for a better understanding of it), please let me know and i'll include it.
When you call ind2sub, you give the size [x y], but the actual size of alphaImage is [y x] so you are not indexing the correct location with xCoord and yCoord.