I'm trying to develop a small and simplistic webcam-controlled game, where the user moves a figure on the x-axis by tracking a lighting source with the webcam (flashlight eg.)
So far my code generates a target object every couple of seconds at a random location in the picture.
That object is stored as a Mat via
Mat target = imread("target.png");
In order to paint the object onto the background image, I'm using
bgClear.copyTo(temp);
for(int i = targetX; i < target.cols + targetX; i++){
for(int j = targetY; j < target.rows + targetY; j++){
temp.at<Vec3b>(j,i) = target.at<Vec3b>(j-targetY,i-targetX);
}
}
temp.copyTo(bg);
where bgClear represents the clean background, temp the background copy that is being edited and bg the final background thats being shown. including the object.
targetX and targetY are the starting coordinates of the object (whereas targetX is randomly generated beforehand so that the object spawns at a random location in the upper half of the image), relative to the background. (so I'm not iterating through the whole background, only the range of the object).
It works so far, but I have a problem:
The transparent area of the imported image is now white, and I dont seem to be able to fix it by checking the pixel values with something like
if(target.at<Vec3b>(Point(j-targetY,i-targetX))[0] != 255 &&
target.at<Vec3b>(Point(j-targetY,i-targetX))[1] != 255 &&
target.at<Vec3b>(Point(j-targetY,i-targetX))[2] != 255)
before I am actually replacing the pixel.
I've also tried loading the .png file by adding the -1 flag (alpha channel), but then the image just seems ghosty and can barely be seen.
In case I might you have problems imaging what I'm talking about, here's a partial screenshot of it: Screenshot
Any advice on how I might fix this ?
Regards,
Daniel
You need to handle transparency manually. General idea is, while copying to temp only copy pixels that are opaque i.e. alpha value is high.
use CV_LOAD_IMAGE_UNCHANGED (= -1) in imread.
split target to four single channel image using split.
merge first three channels to form a BGR image using merge.
in the paint loop, use newly formed BGR image as source and the unmerged fourth channel (alpha) as mask.
...as I was mentioning in my comment to asif's helpful answer:
Mat target = imread("target", CV_LOAD_IMAGE_UNCHANGED); // load image
Mat targetBGR(target.rows, target.cols, CV_8UC3); // create BGR mat
Mat targetAlpha(target.rows, target.cols, CV_8UC1); // create alpha mat
Mat out[] = {targetBGR, targetAlpha}; // create array of matrices
int from_to[] = { 0,0, 1,1, 2,2, 3,3 }; // create array of index pairs
mixChannels( &target, 1, out, 2, from_to, 4 ); // finally split target into 3
channel BGR plus 1 channel Alpha
...as described in this example. (minus the R-B-channel-swapping).
...later in the pixel-processing loop:
if(targetAlpha.at<uchar>(j-targetY,i-targetX) > 0)
temp.at<Vec3b>(j,i) = targetBGR.at<Vec3b>(j-targetY,i-targetX);
Working like a charm!
Related
Mapbox provides Global elevation data with height data encoded in PNG image. Height is decoded by height = -10000 + ((R * 256 * 256 + G * 256 + B) * 0.1). Details are in https://www.mapbox.com/blog/terrain-rgb/.
I want to import the height data to generate terrains in Unity3D.
Texture2D dem = (Texture2D)AssetDatabase.LoadAssetAtPath("Assets/dem/12/12_3417_1536.png", typeof(Texture2D));
for (int i = 0; i < width; i++)
for (int j = 0; j < height; j++)
{
Color c = dem.GetPixel(i, j);
float R = c.r*255;
float G = c.g*255;
float B = c.b*255;
array[i, j] = -10000 + ((R * 256 * 256 + G * 256 + B) * 0.1f);
}
Here I set a break point and the rgba value of the first pixel is RGBA(0.000, 0.592, 0.718, 1.000). c.r is 0. The height is incorrect as this point represent the height of somewhere on a mountain.
Then I open the image in Photoshop and get RGB of the first pixel: R=1,G=152,B=179.
I write a test program in C#.
System.Drawing.Bitmap bitmap = new System.Drawing.Bitmap("12_3417_1536.png");
Color a = bitmap.GetPixel(0, 0);
It shows Color a is (R,G,B,A)=(1,147,249,255)
Here is the image I test:
https://api.mapbox.com/v4/mapbox.terrain-rgb/12/3417/1536.pngraw?access_token=pk.eyJ1Ijoib2xlb3RpZ2VyIiwiYSI6ImZ2cllZQ3cifQ.2yDE9wUcfO_BLiinccfOKg
Why I got different RGBA value with different method? Which one is correct?
According to the comments below, different read order and compressed data in unity may result in different value of the rgba of pixel at (0,0).
Now I want to focus on----How to convert the rgba(0~1) to RGBA(0~255)?
r_ps=r_unity*255? But how can I explain r=0 in unity and r=1 in PS of pixel at (0,0)
?
Try disabling compression from the texture's import settings in Unity (No compression). Alternatively, if you fetch the data at runtime, you can use Texture.LoadBytes() to avoid compression artifacts.
I will assume you are using the same picture and that there aren't two 12_3417_1536.png files in separate folders.
Each of these functions has a different concept of which pixel is at (0,0). Not sure what you mean by "first" pixel when you tested with photoshop, but Texture coordinates in unity start at lower left corner.
When I tested the lower left corner pixel using paint, I got the same value as you did with photoshop. However, if you test the upper left corner, you get (1,147,249,255) which is the result bitmap.GetPixel returns.
The unity values that you're getting seem to be way off. Try calling dem.GetPixel(0,0) so that you're sure you're analyzing the simplest case.
I am trying to store the motion detected from optical flow for frames in a video sequence and then use these stored motion vectors in order to predict the already known frames using just the first frame as a reference. I am currently using two processing sketches - the first sketch draws a motion vector for every pixel grid (each of width and height 10 pixels). This is done for every frame in the video sequence. The vector is only drawn in a grid if there is sufficient motion detected. The second sketch aims to reconstruct the video frames crudely from just the initial frame of the video sequence combined with information about the motion vectors got from the first sketch.
My approach so far is as follows: I am able to determine the size, position and direction of each motion vector drawn in the first sketch from four variables. By creating four arrays (two for the motion vector's x and y coordinate and another two for its length in the x and y direction), every time a motion vector is drawn I can append each of the four variables to the arrays mentioned above. This is done for each pixel grid throughout an entire frame where the vector is drawn and for each frame in the sequence - via for loops. Once the arrays are full, I can then save them to a text file as a list of strings. I then load these strings from the text file into the second sketch, along with the first frame of the video sequence. I load the strings into variables within a while loop in the draw function and convert them back into floats. I increment a variable by one each time the draw function is called - this moves on to the next frame (I used a specific number as a separator in my text-files which appears at the end of every frame - the loop searches for this number and then increments the variable by one, thus breaking the while loop and the draw function is called again for the subsequent frame). For each frame, I can draw 10 by 10 pixel boxes and move then by the parameters got from the text files in the first sketch. My problem is simply this: How do I draw the motion of a particular frame without letting what I've have blitted to the screen in the previous frame affect what will be drawn for the next frame. My only way of getting my 10 by 10 pixel box is by using the get() function which gets pixels that are already drawn to the screen.
Apologies for the length and complexity of my question. Any tips would be very much appreciated! I will add the code for the second sketch. I can also add the first sketch if required, but it's rather long and a lot of it is not my own. Here is the second sketch:
import processing.video.*;
Movie video;
PImage [] naturalMovie = new PImage [0];
String xlengths [];
String ylengths [];
String xpositions [];
String ypositions [];
int a = 0;
int c = 0;
int d = 0;
int p;
int gs = 10;
void setup(){
size(640, 480, JAVA2D);
xlengths = loadStrings("xlengths.txt");
ylengths = loadStrings("ylengths.txt");
xpositions = loadStrings("xpositions.txt");
ypositions = loadStrings("ypositions.txt");
video = new Movie(this, "sample1.mov");
video.play();
rectMode(CENTER);
}
void movieEvent(Movie m) {
m.read();
PImage f = createImage(m.width, m.height, ARGB);
f.set(0, 0, m);
f.resize(width, height);
naturalMovie = (PImage []) append(naturalMovie, f);
println("naturalMovie length: " + naturalMovie.length);
p = naturalMovie.length - 1;
}
void draw() {
if(naturalMovie.length >= p && p > 0){
if (c == 0){
image(naturalMovie[0], 0, 0);
}
d = c;
while (c == d && c < xlengths.length){
float u, v, x0, y0;
u = float(xlengths[a]);
v = float(ylengths[a]);
x0 = float(xpositions[a]);
y0 = float(ypositions[a]);
if (u != 1.0E-19){
//stroke(255,255,255);
//line(x0,y0,x0+u,y0+v);
PImage box;
box = get(int(x0-gs/2), int(y0 - gs/2), gs, gs);
image(box, x0-gs/2 +u, y0 - gs/2 +v, gs, gs);
if (a < xlengths.length - 1){
a += 1;
}
}
else if (u == 1.0E-19){
if (a < xlengths.length - 1){
c += 1;
a += 1;
}
}
}
}
}
Word to the wise: most people aren't going to read that wall of text. Try to "dumb down" your posts so they get to the details right away, without any extra information. You'll also be better off if you post an MCVE instead of only giving us half your code. Note that this does not mean posting your entire project. Instead, start over with a blank sketch and only create the most basic code required to show the problem. Don't include any of your movie logic, and hardcode as much as possible. We should be able to copy and paste your code onto our own machines to run it and see the problem.
All of that being said, I think I understand what you're asking.
How do I draw the motion of a particular frame without letting what I've have blitted to the screen in the previous frame affect what will be drawn for the next frame. My only way of getting my 10 by 10 pixel box is by using the get() function which gets pixels that are already drawn to the screen.
Separate your program into a view and a model. Right now you're using the screen (the view) to store all of your information, which is going to cause you headaches. Instead, store the state of your program into a set of variables (the model). For you, this might just be a bunch of PVector instances.
Let's say I have an ArrayList<PVector> that holds the current position of all of my vectors:
ArrayList<PVector> currentPositions = new ArrayList<PVector>();
void setup() {
size(500, 500);
for (int i = 0; i < 100; i++) {
currentPositions.add(new PVector(random(width), random(height)));
}
}
void draw(){
background(0);
for(PVector vector : currentPositions){
ellipse(vector.x, vector.y, 10, 10);
}
}
Notice that I'm just hardcoding their positions to be random. This is what your MCVE should do as well. And then in the draw() function, I'm simply drawing each vector. This is like drawing a single frame for you.
Now that we have that, we can create a nextFrame() function that moves the vectors based on the ArrayList (our model) and not what's drawn on the screen!
void nextFrame(){
for(PVector vector : currentPositions){
vector.x += random(-2, 2);
vector.y += random(-2, 2);
}
}
Again, I'm just hardcoding a random movement, but you would be reading these from your file. Then we just call the nextFrame() function as the last line in the draw() function:
If you're still having trouble, I highly recommend posting an MCVE similar to mine and posting a new question. Good luck.
This is for the programing language Processing (2.0).
Say I wish to load a not square image (lets use a green circle for the example). If I load this on a black background you can visibly see the white square of the image(aka all parts of image that aren't the green circle). How would I go about efficiently removing them?
It can not think of an efficient way to do it, I will be doing it to hundreds of pictures about 25 times a second(since they will be moving).
Any help would be greatly appreciated, the more efficient the code the better.
As #user3342987 said, you can loop through the image's pixels to see if each pixel is white or not. However, it's worth noting that 255 is white (not 0, which is black). You also shouldn't hardcode the replacement color, as they suggested -- what if the image is moving over a striped background? The best approach is to change all the white pixels into transparent pixels using the image's alpha channel. Also, since you mentioned you would be doing it "about 25 times a second", you shouldn't be doing these checks more than once-- it will be the same every time and would be wasteful. Instead, do it when the images are first loaded, something like this (untested):
PImage[] images;
void setup(){
size(400,400);
images = new PImage[10];
for(int i = 0; i < images.length; i++){
// example filenames
PImage img = loadImage("img" + i + ".jpg");
img.beginDraw();
img.loadPixels();
for(int p = 0; p < img.pixels.length; p++){
//color(255,255,255) is white
if(img.pixels[p] == color(255,255,255)){
img.pixels[p] = color(0,0); // set it to transparent (first number is meaningless)
}
}
img.updatePixels();
img.endDraw();
images[i] = img;
}
}
void draw(){
//draw the images as normal, the white pixels are now transparent
}
So, this will lead to no lag during draw() because you edited out the white pixels in setup(). Whatever you're drawing the images on top of will show through.
It's also worth mentioning that some image filetypes have an alpha channel built in (e.g., the PNG format), so you could also change the white pixels to transparent in some image editor and use those edited files for your sketch. Then your sketch wouldn't have to edit them every time it starts up.
Pixels are stored in the Pixels[] array, you can use a for loop to check to see if the value is 0 (aka white). If it is white load it as the black background.
I'm drawing an image using rgb pixel data. I need to set transparent background color for that image. What value I can set for alpha to be a transparent image? Or is there any other solution for this?
If I understand what you need, you basically want to turn specific colors on an image transparent. To do that you need to use getImageData check out mdn for an explanation on pixel manipulation.
Heres some sample code
var imgd = ctx.getImageData(0, 0, imageWidth, imageHeight),
pix = imgd.data;
for (var i = 0, n = pix.length; i <n; i += 4) {
var r = pix[i],
g = pix[i+1],
b = pix[i+2];
if(g > 150){
// If the green component value is higher than 150
// make the pixel transparent because i+3 is the alpha component
// values 0-255 work, 255 is solid
pix[i + 3] = 0;
}
}
ctx.putImageData(imgd, 0, 0);
And a working demo
With the above code you could check for fuschia by using
if(r == 255 && g == 0 && b == 255)
I think you want the clearRect canvas method:
http://www.w3schools.com/html5/canvas_clearrect.asp
This will let you clear pixels to transparent (or any other RGBA color) without fuss or pixel manipulation.
an alpha of 0 indications that pixel is completely transparent an alpha value of 255 is completely opaque meaning that it will have no transparency.
if you portions of your image are completely transparent (an alpha of 0) it doesn't matter what you use for the RGB values as long as use an Alpha of 0. On a side note some older windows programs that I have used make an assumption like the upper left pixel or the lower right pixel is to be used as the transparency color. It would then loop through all of the pixels and set the alpha to 0 when it encountered this specific RGB value.
If you use an Alpha of 127 and the image appeared on top of another image it would look like the two images are equally visible or that the bottom image is bleeding 50% of it's colors through to the top image.
Set a variable for alpha if you want to test and see what it looks like when you apply it to the entire image.
I need to determine the amount/quality of color in an image in order to compare it with other images and recommend a user (owner of the image) maybe he needs to print it in black and white and not in color.
So far I'm analyzing the image and extracting some data of it:
The number of different colors I find in the image
The percentage of color in the whole page (color pixels / total pixels)
For further analysis I may need other characteristic of these images. Do you know what else is important (or I'm missing here) in image analysis?
After some time I found a missing characteristic (very important) which helped me a lot with the analysis of the images. I don't know if there is a name for that but I called it the average color of the image:
When I was looping over all the pixels of the image and counting each color I also retrieved the information of the RGB values and summarized all the Reds, Greens and Blues of all the pixels. Just to come up with this average color which, again, saved my life when I wanted to compare some kind of images.
The code is something like this:
File f = new File("image.jpg");
BufferedImage im = ImageIO.read(f);
int tot = 0;
int red = 0;
int blue= 0;
int green = 0;
int w = im.getWidth();
int h = im.getHeight();
// Going over all the pixels
for (int i=0;i<w;i++){
for (int j=0;j<h;j++){
int pix = im.getRGB(i, j); //
if (!sameARGB(pix)) { // Compares the RGB values
tot+=1;
red+=pix.getRed();
green+=pix.getGreen();
blue+=pix.getBlue();
}
}
}
And you should get the results like this:
// Percentage of color on the image
double per = (double)tot/(h*w);
// Average color <-------------
Color c = new Color((double)red/tot,(double)green/tot,(double)blue/tot);