Getting two consecutive frames from camera in Processing - image

Question

I've been trying a couple of ways to achieve this using Processing, but each time, it doesn't get some that are perfectly consecutive. Would anyone know the "right way" of doing this?
Thanks in advance!

Answer 1

In theory it should be a matter of listening to the captureEvent() to get a new frame and keep track if the first frame was previously recorded, if so recorded the second one afterwards.
Here's a basic commented sketch to illustrate the point (press any key to grab another pair of frames):
import processing.video.*;
Capture camera;
PImage firstFrame;
PImage secondFrame;
void setup(){
size(1920,480);
camera = new Capture(this,640,480);
camera.start();
}
void draw(){
image(camera,0,0);
if(firstFrame != null){
image(firstFrame,640,0);
}
if(secondFrame != null){
image(secondFrame,1280,0);
}
}
//this is the callback from the video library when a new camera frame is available
void captureEvent(Capture c){
//read a new frame
c.read();
//if the first frame wasn't recorded yet, record(copy) it's pixels
if(firstFrame == null){
firstFrame = c.get();
}
//same for the second frame, but check if the first frame has been recorded first
if(firstFrame != null && secondFrame == null){
secondFrame = c.get();
}
}
void keyPressed(){
//reset consecutive frames on keypress
firstFrame = secondFrame = null;
}
In theory (as you can see in the Processing Video Library's source code), captureEvent is fired only when a new camera sample is ready.
In practice, you will find that two consecutive frames may look identical (even though they could be a split second apart in time), even noise as you pointed out in your comments.
It feels like what you're after is a frame that is consecutive, but different enough from the previous. If that's the case you can have a play with the FrameDifferencing example (Processing > Examples > Libraries > Video > Capture > FrameDifferencing)
Here's a modified version of the above sketch, using Golan Levin's FrameDifferencing code to only grab a second frame if it's different by a little bit:
import processing.video.*;
Capture camera;
PImage firstFrame;
PImage secondFrame;
PImage diff;
void setup(){
size(1920,960);
camera = new Capture(this,640,480);
camera.start();
diff = createImage(640,480,RGB);
}
void draw(){
image(camera,0,0);
if(firstFrame != null){
image(firstFrame,640,0);
}
if(secondFrame != null){
image(secondFrame,1280,0);
}
image(diff,0,480);
}
//this is the callback from the video library when a new camera frame is available
void captureEvent(Capture c){
//read a new frame
c.read();
//if the first frame wasn't recorded yet, record(copy) it's pixels
if(firstFrame == null){
firstFrame = c.get();
println("recorded first frame at",new java.util.Date());
}
//same for the second frame, but check if the first frame has been recorded first
if(firstFrame != null && secondFrame == null){
//if the difference between the first frame cand the current frame is even ever so slightly off, record the second frame
if(difference(firstFrame,camera) > 100){
secondFrame = c.get();
}
}
}
int difference(PImage first,PImage second){
final int numPixels = 640*480;
camera.loadPixels();
int movementSum = 0; // Amount of movement in the frame
for (int i = 0; i < numPixels; i++) { // For each pixel in the video frame...
color currColor = first.pixels[i];
color prevColor = second.pixels[i];
// Extract the red, green, and blue components from current pixel
int currR = (currColor >> 16) & 0xFF; // Like red(), but faster
int currG = (currColor >> 8) & 0xFF;
int currB = currColor & 0xFF;
// Extract red, green, and blue components from previous pixel
int prevR = (prevColor >> 16) & 0xFF;
int prevG = (prevColor >> 8) & 0xFF;
int prevB = prevColor & 0xFF;
// Compute the difference of the red, green, and blue values
int diffR = abs(currR - prevR);
int diffG = abs(currG - prevG);
int diffB = abs(currB - prevB);
// Render the difference image to the screen
diff.pixels[i] = color(diffR, diffG, diffB);
// Add these differences to the running tally
movementSum += diffR + diffG + diffB;
}
diff.updatePixels();
return movementSum;
}
void keyPressed(){
//reset consecutive frames on keypress
firstFrame = secondFrame = null;
}
In the example above 100 is an arbitrary value.
The max would be 255*3*640*480 (0-255 per channel * number of channels * width * height)

Answer 2

I looked at the KetaiCamera sources and issue reports, especially this one. Unfortunately this code is not built to provide real time camera frames.
You can try to take the AndroidCapture project as a starter, and modify the native Android class to achieve your goal.