Video Delay/Buffer in Processing 2.0 - processing

I'm having a ton of trouble making a simple video delay in processing. I looked around on the internet and I keep finding the same bit of code and I can't get it to work at all. When I first tried it, it did nothing (at all). Here's my modified version (which at least seems to load frames into the buffer), I really have no idea why it doesn't work and I'm getting really tired of pulling out my hair. Please... please, for the love of god, please somebody point out the stupid mistake I'm making here.
And now, without further delay (hah, get it?), the code:
import processing.video.*;
VideoBuffer vb;
Movie myMovie;
Capture cam;
float seconds = 1;
void setup() {
size(320,240, P3D);
frameRate(30);
String[] cameras = Capture.list();
if (cameras.length == 0) {
println("There are no cameras available for capture.");
exit();
} else {
println("Available cameras:");
for (int i = 0; i < cameras.length; i++) {
println(cameras[i]);
}
cam = new Capture(this, cameras[3]);
cam.start();
}
vb = new VideoBuffer(90, width, height);
}
void draw() {
if (cam.available() == true) {
cam.read();
vb.addFrame(cam);
}
image(cam, 0, 0);
image( vb.getFrame(), 150, 0 );
}
class VideoBuffer
{
PImage[] buffer;
int inputFrame = 0;
int outputFrame = 0;
int frameWidth = 0;
int frameHeight = 0;
VideoBuffer( int frames, int vWidth, int vHeight )
{
buffer = new PImage[frames];
for(int i = 0; i < frames; i++)
{
this.buffer[i] = new PImage(vWidth, vHeight);
}
this.inputFrame = 0;
this.outputFrame = 1;
this.frameWidth = vWidth;
this.frameHeight = vHeight;
}
// return the current "playback" frame.
PImage getFrame()
{
return this.buffer[this.outputFrame];
}
// Add a new frame to the buffer.
void addFrame( PImage frame )
{
// copy the new frame into the buffer.
this.buffer[this.inputFrame] = frame;
// advance the input and output indexes
this.inputFrame++;
this.outputFrame++;
println(this.inputFrame + " " + this.outputFrame);
// wrap the values..
if(this.inputFrame >= this.buffer.length)
{
this.inputFrame = 0;
}
if(this.outputFrame >= this.buffer.length)
{
this.outputFrame = 0;
}
}
}

This works in Processing 2.0.1.
import processing.video.*;
Capture cam;
PImage[] buffer;
int w = 640;
int h = 360;
int nFrames = 60;
int iWrite = 0, iRead = 1;
void setup(){
size(w, h);
cam = new Capture(this, w, h);
cam.start();
buffer = new PImage[nFrames];
}
void draw() {
if(cam.available()) {
cam.read();
buffer[iWrite] = cam.get();
if(buffer[iRead] != null){
image(buffer[iRead], 0, 0);
}
iWrite++;
iRead++;
if(iRead >= nFrames-1){
iRead = 0;
}
if(iWrite >= nFrames-1){
iWrite = 0;
}
}
}
There is a problem inside your addFrame-Method. You just store a reference to the PImage object, whose pixels get overwritten all the time. You have to use buffer[inputFrame] = frame.get() instead of buffer[inputFrame] = frame. The get() method returns a copy of the image.

Related

How to debug "ArrayIndexOutOfBoundsException: 0" in Processing

I'm trying to make Kim Asendorf's pixel sorting code work with input from a webcam in processing.
I get the error: "ArrayIndexOutOfBoundsException: 0". Any ideas of how I can fix this?
//BLACK
int getFirstNotBlackY(int _x, int _y) {
int x = _x;
int y = _y;
color c;
if (y < height) {
while ( (c = cam.pixels[x + y * cam.width]) < blackValue) {
y++;
if (y >= height) return -1;
}
import processing.video.*;
Capture cam;
String[] cameras = Capture.list();
int mode = 0;
//MODE:
//0 -> black
//1 -> bright
//2 -> white
//b(16777216)
int loops = 1;
int blackValue = -10000000;
int brigthnessValue = 60;
int whiteValue = -6000000;
int row = 0;
int column =
boolean saved = false;
void setup() {
size(640, 480);
import processing.video.*;
String[] cameras = Capture.list();
import processing.video.*;
if (cameras.length == 0) {
println("There are no cameras available for capture.");
exit();
} else {
println("Available cameras:");
for (int i = 0; i < cameras.length; i++) {
println(cameras[i]);
}
// The camera can be initialized directly using an
// element from the array returned by list():
cam = new Capture(this, cameras[0]);
cam.start();
}
}
I've added the code I'm using to initialize the webcam.

Processing - deprecated OpenKinect library

I am trying to replicate a project for Kinect for this music video, but the code is seriously outdated.
After weeks searching, I have not found anything about this.
I would be greatly thankful to anyone who points out to me what is deprecated in the following code:
(I'm using Processing 3)
import org.openkinect.*;
import org.openkinect.processing.*;
import java.io.*;
// Kinect Library object
Kinect kinect;
float a = 0;
// Size of kinect image
int w = 640;
int h = 480;
// writing state indicator
boolean write = false;
// treshold filter initial value
int fltValue = 950;
// "recording" object. each vector element holds a coordinate map vector
Vector <Object> recording = new Vector<Object>();
// We'll use a lookup table so that we don't have to repeat the math over and over
float[] depthLookUp = new float[2048];
void setup() {
size(800,600,P3D);
kinect = new Kinect(this);
kinect.start();
kinect.enableDepth(true);
// We don't need the grayscale image in this example
// so this makes it more efficient
kinect.processDepthImage(false);
// Lookup table for all possible depth values (0 - 2047)
for (int i = 0; i < depthLookUp.length; i++) {
depthLookUp[i] = rawDepthToMeters(i);
}
}
void draw() {
background(0);
fill(255);
textMode(SCREEN);
text("Kinect FR: " + (int)kinect.getDepthFPS() + "\nProcessing FR: " + (int)frameRate,10,16);
// Get the raw depth as array of integers
int[] depth = kinect.getRawDepth();
// We're just going to calculate and draw every 4th pixel (equivalent of 160x120)
int skip = 4;
// Translate and rotate
translate(width/2,height/2,-50);
rotateY(a);
//noStroke();
//lights();
int index = 0;
PVector[] frame = new PVector[19200];
for(int x=0; x<w; x+=skip) {
for(int y=0; y<h; y+=skip) {
int offset = x+y*w;
// Convert kinect data to world xyz coordinate
int rawDepth = depth[offset];
boolean flt = true;
PVector v = depthToWorld(x,y,rawDepth);
if (flt && rawDepth > fltValue)
{
v = depthToWorld(x,y,2047);
}
frame[index] = v;
index++;
stroke(map(rawDepth,0,2048,0,256));
pushMatrix();
// Scale up by 200
float factor = 400;
translate(v.x*factor,v.y*factor,factor-v.z*factor);
//sphere(1);
point(0,0);
//line (0,0,1,1);
popMatrix();
}
}
if (write == true) {
recording.add(frame);
}
// Rotate
//a += 0.015f;
}
// These functions come from:http://graphics.stanford.edu/~mdfisher/Kinect.html
float rawDepthToMeters(int depthValue) {
if (depthValue < 2047) {
return (float)(1.0 / ((double)(depthValue) * -0.0030711016 + 3.3309495161));
}
return 0.0f;
}
PVector depthToWorld(int x, int y, int depthValue) {
final double fx_d = 1.0 / 5.9421434211923247e+02;
final double fy_d = 1.0 / 5.9104053696870778e+02;
final double cx_d = 3.3930780975300314e+02;
final double cy_d = 2.4273913761751615e+02;
PVector result = new PVector();
double depth = depthLookUp[depthValue];//rawDepthToMeters(depthValue);
result.x = (float)((x - cx_d) * depth * fx_d);
result.y = (float)((y - cy_d) * depth * fy_d);
result.z = (float)(depth);
return result;
}
void stop() {
kinect.quit();
super.stop();
}
int currentFile = 0;
void saveFile() {
}
void keyPressed() { // Press a key to save the data
if (key == '1')
{
fltValue += 50;
println("fltValue: " + fltValue);
}
else if (key == '2')
{
fltValue -= 50;
println("fltValue: " + fltValue);
}
else if (key=='4'){
if (write == true) {
write = false;
println( "recorded " + recording.size() + " frames.");
// saveFile();
// save
Enumeration e = recording.elements();
println("Stopped Recording " + currentFile);
int i = 0;
while (e.hasMoreElements()) {
// Create one directory
boolean success = (new File("out"+currentFile)).mkdir();
PrintWriter output = createWriter("out"+currentFile+"/frame" + i++ +".txt");
PVector [] frame = (PVector []) e.nextElement();
for (int j = 0; j < frame.length; j++) {
output.println(j + ", " + frame[j].x + ", " + frame[j].y + ", " + frame[j].z );
}
output.flush(); // Write the remaining data
output.close();
}
currentFile++;
}
}
else if (key == '3') {
println("Started Recording "+currentFile);
recording.clear();
write = true;
}
}
If the code works, then I wouldn't worry too much about it. Deprecated can just mean that a newer version is available, not that the older version stopped working.
However, if the code does not work, then updating to a newer library is probably a good idea anyway. Check out the library section of the Processing homepage, which lists several Kinect libraries.
In fact, one of those libraries is the updated version of the old library you're using: Open Kinect for Processing.
Edit: It looks like both of the errors you mentioned are due to missing import statements. You need to import both Vector and Enumeration to use them:
import java.util.Vector;
import java.util.Enumeration;

How to make waveform rendering more interesting?

I wrote a waveform renderer that takes an audio file and creates something like this:
The logic is pretty simple. I calculate the number of audio samples required for each pixel, read those samples, average them and draw a column of pixels according to the resulting value.
Typically, I will render a whole song on around 600-800 pixels, so the wave is pretty compressed. Unfortunately this usually results in unappealing visuals as almost the entire song is just rendered at almost the same heights. There is no variation.
Interestingly, if you look at the waveforms on SoundCloud almost none of them are as boring as my results. They all have some variation. What could be the trick here? I don't think they just add random noise.
I don't think SoundCloud is doing anything particularly special. There are plenty of songs I see on their front page that are very flat. It has more to do with the way detail is perceived and what the overall dynamics of the song are like. The main difference is that SoundCloud is drawing absolute value. (The negative side of the image is just a mirror.)
For demonstration, here is a basic white noise plot with straight lines:
Now, typically a fill is used to make the overall outline easier to see. This already does a lot for the appearance:
Larger waveforms ("zoomed out" in particular) typically use a mirror effect because the dynamics become more pronounced:
Bars are another way to visualize and can give an illusion of detail:
A pseudo routine for a typical waveform graphic (average of abs and mirror) might look like this:
for (each pixel in width of image) {
var sum = 0
for (each sample in subset contained within pixel) {
sum = sum + abs(sample)
}
var avg = sum / length of subset
draw line(avg to -avg)
}
This is effectively like compressing the time axis as RMS of the window. (RMS could also be used but they are almost the same.) Now the waveform shows overall dynamics.
That is not too different from what you are already doing, just abs, mirror and fill. For boxes like SoundCloud uses, you would be drawing rectangles.
Just as a bonus, here is an MCVE written in Java to generate a waveform with boxes as described. (Sorry if Java is not your language.) The actual drawing code is near the top. This program also normalizes, i.e., the waveform is "stretched" to the height of the image.
This simple output is the same as the above pseudo routine:
This output with boxes is very similar to SoundCloud:
import javax.swing.*;
import java.awt.*;
import java.awt.event.*;
import java.awt.image.*;
import java.io.*;
import javax.sound.sampled.*;
public class BoxWaveform {
static int boxWidth = 4;
static Dimension size = new Dimension(boxWidth == 1 ? 512 : 513, 97);
static BufferedImage img;
static JPanel view;
// draw the image
static void drawImage(float[] samples) {
Graphics2D g2d = img.createGraphics();
int numSubsets = size.width / boxWidth;
int subsetLength = samples.length / numSubsets;
float[] subsets = new float[numSubsets];
// find average(abs) of each box subset
int s = 0;
for(int i = 0; i < subsets.length; i++) {
double sum = 0;
for(int k = 0; k < subsetLength; k++) {
sum += Math.abs(samples[s++]);
}
subsets[i] = (float)(sum / subsetLength);
}
// find the peak so the waveform can be normalized
// to the height of the image
float normal = 0;
for(float sample : subsets) {
if(sample > normal)
normal = sample;
}
// normalize and scale
normal = 32768.0f / normal;
for(int i = 0; i < subsets.length; i++) {
subsets[i] *= normal;
subsets[i] = (subsets[i] / 32768.0f) * (size.height / 2);
}
g2d.setColor(Color.GRAY);
// convert to image coords and do actual drawing
for(int i = 0; i < subsets.length; i++) {
int sample = (int)subsets[i];
int posY = (size.height / 2) - sample;
int negY = (size.height / 2) + sample;
int x = i * boxWidth;
if(boxWidth == 1) {
g2d.drawLine(x, posY, x, negY);
} else {
g2d.setColor(Color.GRAY);
g2d.fillRect(x + 1, posY + 1, boxWidth - 1, negY - posY - 1);
g2d.setColor(Color.DARK_GRAY);
g2d.drawRect(x, posY, boxWidth, negY - posY);
}
}
g2d.dispose();
view.repaint();
view.requestFocus();
}
// handle most WAV and AIFF files
static void loadImage() {
JFileChooser chooser = new JFileChooser();
int val = chooser.showOpenDialog(null);
if(val != JFileChooser.APPROVE_OPTION) {
return;
}
File file = chooser.getSelectedFile();
float[] samples;
try {
AudioInputStream in = AudioSystem.getAudioInputStream(file);
AudioFormat fmt = in.getFormat();
if(fmt.getEncoding() != AudioFormat.Encoding.PCM_SIGNED) {
throw new UnsupportedAudioFileException("unsigned");
}
boolean big = fmt.isBigEndian();
int chans = fmt.getChannels();
int bits = fmt.getSampleSizeInBits();
int bytes = bits + 7 >> 3;
int frameLength = (int)in.getFrameLength();
int bufferLength = chans * bytes * 1024;
samples = new float[frameLength];
byte[] buf = new byte[bufferLength];
int i = 0;
int bRead;
while((bRead = in.read(buf)) > -1) {
for(int b = 0; b < bRead;) {
double sum = 0;
// (sums to mono if multiple channels)
for(int c = 0; c < chans; c++) {
if(bytes == 1) {
sum += buf[b++] << 8;
} else {
int sample = 0;
// (quantizes to 16-bit)
if(big) {
sample |= (buf[b++] & 0xFF) << 8;
sample |= (buf[b++] & 0xFF);
b += bytes - 2;
} else {
b += bytes - 2;
sample |= (buf[b++] & 0xFF);
sample |= (buf[b++] & 0xFF) << 8;
}
final int sign = 1 << 15;
final int mask = -1 << 16;
if((sample & sign) == sign) {
sample |= mask;
}
sum += sample;
}
}
samples[i++] = (float)(sum / chans);
}
}
} catch(Exception e) {
problem(e);
return;
}
if(img == null) {
img = new BufferedImage(size.width, size.height, BufferedImage.TYPE_INT_ARGB);
}
drawImage(samples);
}
static void problem(Object msg) {
JOptionPane.showMessageDialog(null, String.valueOf(msg));
}
public static void main(String[] args) {
SwingUtilities.invokeLater(new Runnable() {
#Override
public void run() {
JFrame frame = new JFrame("Box Waveform");
JPanel content = new JPanel(new BorderLayout());
frame.setContentPane(content);
JButton load = new JButton("Load");
load.addActionListener(new ActionListener() {
#Override
public void actionPerformed(ActionEvent ae) {
loadImage();
}
});
view = new JPanel() {
#Override
protected void paintComponent(Graphics g) {
super.paintComponent(g);
if(img != null) {
g.drawImage(img, 1, 1, img.getWidth(), img.getHeight(), null);
}
}
};
view.setBackground(Color.WHITE);
view.setPreferredSize(new Dimension(size.width + 2, size.height + 2));
content.add(view, BorderLayout.CENTER);
content.add(load, BorderLayout.SOUTH);
frame.pack();
frame.setResizable(false);
frame.setDefaultCloseOperation(JFrame.EXIT_ON_CLOSE);
frame.setLocationRelativeTo(null);
frame.setVisible(true);
}
});
}
}
Note: for the sake of simplicity, this program loads the entire audio file in to memory. Some JVMs may throw OutOfMemoryError. To correct this, run with increased heap size as described here.

Save image with Processing

I'm trying to save an image after certain time, the problem is that the image size is bigger than the display so when I use the save or saveFrame function it only saves the image that I can see in the display. There is any other way to save the whole image?
This is my code:
PImage picture, pictureFilter, img;
int total, cont, current;
ArrayList<ArrayList<Position>> columns;
String[] fontList;
public class Position {
public int x;
public int y;
}
void setup() {
fontList = PFont.list();
picture = loadImage("DSC05920b.JPG");
pictureFilter = loadImage("filtrePort2.jpg");
frame.setResizable(true);
size(picture.width, picture.height);
columns = new ArrayList<ArrayList<Position>>();
for(int i = 0; i < picture.width; i++) {
ArrayList<Position> row = new ArrayList<Position>();
for(int j = 0; j < picture.height; j++){
Position p = new Position();
p.x = i;
p.y = j;
row.add(p);
}
columns.add(row);
}
total = picture.width * picture.height;
cont = total;
current = 0;
img = createImage(picture.width, picture.height, RGB);
}
float randomLetter() {
float value = 23;
boolean found = false;
while(!found) {
value = random(48, 122);
if(value >48 && value <58) found = true;
if(value >65 && value <91) found = true;
if(value >97 && value <123) found = true;
}
return value;
}
void draw() {
int x = int(random(0, columns.size()));
ArrayList<Position> rows = columns.get(x);
int y = int(random(0, rows.size()));
Position p = rows.get(y);
color c = pictureFilter.get(p.x, p.y);
int r = (c >> 16) & 0xFF; // Faster way of getting red(argb)
if(r < 240) {
PFont f = createFont(fontList[int(random(0,fontList.length))],random(5, 24),true);
textFont(f);
fill(picture.get(p.x,p.y));
char letter = (char) int(randomLetter());
text(letter, p.x, p.y);
}
if(rows.size() == 1) {
if(columns.size() == 1) {
saveFrame("lol.jpg");
columns.remove(x);
} else {
columns.remove(x);
}
} else {
println(rows.size());
rows.remove(y);
}
--cont;
float percent = float(total-cont)/float(total)*100;
if(int(percent) != current) {
current = int(percent);
save("image_" + current + ".jpg");
}
println("DONE: " + (total-cont) + "/" + total + " Progress: " + percent + "%");
}
The code do a lot of stuff but the part that its not working well is at the final when I check if the percentage have been increased in order to save the image
You can write this into a PGraphics context - aka a graphics buffer.
The buffer can be as big as you need it to be, and you can choose whether to draw it on the screen or not..
// Create the buffer at the size you need, and choose the renderer
PGraphics pg = createGraphics(myImage.width, myImage.height, P2D);
// Wrap all your drawing functions in the pg context - e.g.
PFont f = createFont(fontList[int(random(0,fontList.length))],random(5, 24),true);
textFont(f);
pg.beginDraw();
pg.fill(picture.get(p.x,p.y));
char letter = (char) int(randomLetter());
pg.text(letter, p.x, p.y);
pg.endDraw();
// Draw your PG to the screen and resize the representation of it to the screen bounds
image(pg, 0, 0, width, height); // <-- this wont actually clip/resize the image
// Save it
pg.save("image_" + current + ".jpg");
The PImage class contains a save() function that exports to file. The API should be your first stop for questions like this.

Image gallery assistance if you may

I created an image gallery that works great, the only problem I have is that I don't know how to set up my program so when you open it it opens with the pictures as thumbnails and if you click on the thumbnails the image expands.
int maxImages;
int imageIndex;
// Declaring an array of images.
PImage[] images;
void setup() {
size(600,400);
images = new PImage[maxImages];
maxImages = 2;
imageIndex = 0; // Initial image to be displayed is the first
for (int i = 0; i < images.length; i ++ ) {
images[i] = loadImage( "changeling" + i + ".jpg" );
}
}
void draw() {
// Displaying one image
image(images[imageIndex],0,0);
}
Here it's the idea, to show the images i'm setting them whit a width and height of 100 you can use any value or your own algorithm to get the best sizes.
Knowing the space in the sketch that every image occupies I can know when the mouse was inside one of them when an mouse click event happen. Then I proceed to expand the image by doubling the size until it reaches a good size comparing whit the screen size, Again you can use you own value or algorithm to get the best expansion ratio.
Once the mouse it's clicked again when an image is expanded it goes back to the thumbnail again.
Hope this can be useful.
int maxImages;
int imageIndex;
boolean imageExpand;
// Declaring an array of images.
PImage[] images;
void setup() {
size(600,400);
maxImages = 2;
imageIndex = 0; // Initial image to be displayed is the first
boolean imageExpand;
images = new PImage[maxImages];
imageExpand = false;
for (int i = 0; i < images.length; i ++ ) {
images[i] = loadImage( "changeling" + i + ".jpg" );
}
}
void draw() {
if(!imageExpand){
showThumbnail();
}
}
void mouseClicked(){
if(!imageExpand){
for(int i=0; i < images.length; i++){
if((images[i].X > mouseX) && (images[i].X + images[i].width < mouseX)){
if((images[i].Y > mouseY) && (images[i].Y + images[i].height < mouseY)){
expandImage(images[i]);
imageExpand = true;
}
}
}
}
else{
imageExpand = false;
showThumbnail();
}
}
void expandImage(PImage myImage){
int largeWidth, largeHeight;
while((myImage.width * 2 < 600) && (myImage.height * 2 < 400)){
largeWidth = myImage.width * 2;
largeWidth = myImage.height * 2;
}
image(myImage, 0, 0, largeWidth, largeHeight);
}
void showThumbnail(){
int posX = 0, posY = 0, lastImage = 0;
for(int i=0; i < images.length; i++){
if(posX + 100 < 600){
image(images[i], posX, posY, 100, 100);
posX = posX + 100;
}
else{
posX = 0;
posY = posY + 100;
image(images[i], posX, posY, 100, 100);
}
}
}
Regards Jose

Resources