Processing, using texture inside the audio waveform - processing

Question

I've been trying to make the texture ( the img ) to be visible only where wave form active is. But so far my attempts failed. I didn't quite understand the usage of vertex.
PImage img;
import ddf.minim.*;
Minim minim;
AudioPlayer song;
void setup()
{
size(800, 600,P2D);
minim = new Minim(this);
song = minim.loadFile("song.mp3");
song.play();
img = loadImage("img.jpg");
}
void draw()
{
background(0);
stroke(255);
for (int i = 0; i < song.bufferSize() - 1; i++)
{
beginShape();
texture(img);
vertex(0,height/2);
vertex(i, height-100 - song.right.get(i)*50);
vertex(i+1, height-100 - song.right.get(i+1)*50);
vertex(width,height/2);
vertex(0,height/2);
vertex(0,height/2+100);
endShape();
}
}

Answer 1

You're almost there:
you are are passing the x,y values for the vertex position which renders the shape
you are not passing the texture mapping u,v coordinates
Be sure to read the vertex() reference:
This function is also used to map a texture onto geometry. The texture() function declares the texture to apply to the geometry and the u and v coordinates set define the mapping of this texture to the form. By default, the coordinates used for u and v are specified in relation to the image's size in pixels, but this relation can be changed with textureMode().
It's unclear what shape you are trying to draw, but one simple thing you could do is pass the x,y coordinates as u,v coordinates as well (instead of just vertex(x,y); use vertex(x,y,u,v);):
PImage img;
import ddf.minim.*;
Minim minim;
AudioPlayer song;
void setup()
{
size(800, 600,P2D);
noStroke();
minim = new Minim(this);
song = minim.loadFile("song.mp3");
song.play();
img = loadImage("img.jpg");
}
void draw()
{
background(0);
stroke(255);
for (int i = 0; i < song.bufferSize() - 1; i++)
{
beginShape();
texture(img);
vertex(0,height/2, //vertex 0,x,y
0,height/2); //vertex 0,u,v
vertex(i, height-100 - song.right.get(i)*50, //vertex 1,x,y
i, height-100 - song.right.get(i)*50); //vertex 1,u,v
vertex(i+1, height-100 - song.right.get(i+1)*50, //vertex 2,x,y
i+1, height-100 - song.right.get(i+1)*50); //vertex 2,u,v
vertex(width,height/2, //vertex 3,x,y
width,height/2); //vertex 3,u,v
vertex(0,height/2, //vertex 4,x,y
0,height/2); //vertex 4,u,v
vertex(0,height/2+100, //vertex 5,x,y
0,height/2+100); //vertex 5,u,v
endShape();
}
}
Not tested, but the comments should help spot the difference.
Here's a super basic example of using vertex() with texture():
PImage img;
void setup(){
size(100,100,P2D);
//make a texture
img = createImage(50,50,RGB);
for(int i = 0 ; i < img.pixels.length; i++) {
int x = i % img.width;
int y = i / img.height;
if(x % 4 == 0 && y % 4 == 0){
img.pixels[i] = color(255);
}else{
img.pixels[i] = color(0);
}
}
img.updatePixels();
}
void draw(){
background(0);
//sampling different u,v coordinates (vertex 1 is mapped to mouse) for same x,y
beginShape();
texture(img);
vertex(0,0,0,0);
vertex(50,0,mouseX,mouseY);
vertex(50,50,50,50);
vertex(0,50,0,50);
endShape();
text("u:"+mouseX+"v:"+mouseY,5,height);
translate(50,0);
//mapping u,v to x,y coordinates
beginShape();
texture(img);
vertex(0,0,0,0);
vertex(50,0,50,0);
vertex(50,50,50,50);
vertex(0,50,0,50);
endShape();
}
Notice how the texture distorts when you move the mouse, as it controls vertex 1's texture coordinates. vertex(x,y,u,v) is very similar to vertex(x,y), but in addition to the coordinate where the vertex gets rendered on screen you also can control the coordinate of where the texture is sampled from.
As the reference mentions, by default textureMode() is image, meaning the u,v coordinates are in the image coordinates (from 0,0 to texture image width,height). There's another mode available: NORMAL in which the u,v sampling coordinates are normalised (between 0.0 and 1.0) is closer to what you might have spotted in 3D applications UV mapping features