I want to implement a feature in which color of a particular area will be picked by 3d model. I am using vuforia and unity3d and successfully implemented the target detection. In next step I want to pick color of image and put that color on 3d Model.
Many people have already implemented this but I am not able to find a complete tutorial of that.
I have tired to use region Cature as well but no success.
I would take the area of the screen you are after, then place it in a Pixel array and average that array.
public Color GetColorFromScreen(int x, int y, int width, int height){
Texture2D tex = new Texture2D(1, 1);
tex.ReadPixels(new Rect(x, y, width, height), 0, 0);
tex.Apply();
Color [] pix = tex.GetPixels(x, y, width, height);
float r,g,b,a;
foreach (Color col in pix){
r += col.r;
g += col.g;
b += col.b;
a += col.a;
}
r /= pix.Length;
g /= pix.Length;
b /= pix.Length;
a /= pix.Length;
return new Color(r,g,b,a);
}
Then you grab the material of your model and apply that color
GetComponent<Renderer>().material.color = GetColorFromScreen(x,y,w,h);
Related
I'm trying to fill an image with gyroid lines with certain thickness at certain spacing, but math is not my area. I was able to create a sine wave and shift a bit in the X direction to make it looks like a gyroid but it's not the same.
The idea behind is to stack some images with the same resolution and replicate gyroid into 2D images, so we still have XYZ, where Z can be 0.01mm to 0.1mm per layer
What i've tried:
int sineHeight = 100;
int sineWidth = 100;
int spacing = 100;
int radius = 10;
for (int y1 = 0; y1 < mat.Height; y1 += sineHeight+spacing)
for (int x = 0; x < mat.Width; x++)
{
// Simulating first image
int y2 = (int)(Math.Sin((double)x / sineWidth) * sineHeight / 2.0 + sineHeight / 2.0 + radius);
Circle(mat, new System.Drawing.Point(x, y1+y2), radius, EmguExtensions.WhiteColor, -1, LineType.AntiAlias);
// Simulating second image, shift by x to make it look a bit more with gyroid
y2 = (int)(Math.Sin((double)x / sineWidth + sineWidth) * sineHeight / 2.0 + sineHeight / 2.0 + radius);
Circle(mat, new System.Drawing.Point(x, y1 + y2), radius, EmguExtensions.GreyColor, -1, LineType.AntiAlias);
}
Resulting in: (White represents layer 1 while grey layer 2)
Still, this looks nothing like real gyroid, how can I replicate the formula to work in this space?
You have just single ugly slice because I do not see any z in your code (its correct the surface has horizontal and vertical sin waves like this every 0.5*pi in z).
To see the 3D surface you have to raycast z ...
I would expect some conditional testing of actually iterated x,y,z result of gyroid equation against some small non zero number like if (result<= 1e-6) and draw the stuff only then or compute color from the result instead. This is ideal to do in GLSL.
In case you are not familiar with GLSL and shaders the Fragment shader is executed for each pixel (called fragment) of the rendered QUAD so you just put the code inside your nested x,y for loops and use your x,y instead of pos (you can ignore the Vertex shader its not important).
You got 2 basic options to render this:
Blending the ray casted surface pixels together creating X-Ray like image. It can be combined with SSS techniques to get the impression of glass or semitransparent material. Here simple GLSL example for the blending:
Vertex:
#version 400 core
in vec2 position;
out vec2 pos;
void main(void)
{
pos=position;
gl_Position = vec4(position.xy,0.0,1.0);
}
Fragment:
#version 400 core
in vec2 pos;
out vec3 out_col;
void main(void)
{
float n,x,y,z,dz,d,i,di;
const float scale=2.0*3.1415926535897932384626433832795;
n=100.0; // layers
x=pos.x*scale; // x postion of pixel
y=pos.y*scale; // y postion of pixel
dz=2.0*scale/n; // z step
di=1.0/n; // color increment
i=0.0; // color intensity
for (z=-scale;z<=scale;z+=dz) // do all layers
{
d =sin(x)*cos(y); // compute gyroid equation
d+=sin(y)*cos(z);
d+=sin(z)*cos(x);
if (d<=1e-6) i+=di; // if near surface add to color
}
out_col=vec3(1.0,1.0,1.0)*i;
}
Usage is simple just render 2D quad covering screen without any matrices with corner pos points in range <-1,+1>. Here result:
Another technique is to render first hit to surface creating mesh like image. In order to see the details we need to add basic (double sided) directional lighting for which surface normal is needed. The normal can be computed by simply partialy derivate the equation by x,y,z. As now the surface is opaque then we can stop on first hit and also ray cast just single period in z as anything after that is hidden anyway. Here simple example:
Fragment:
#version 400 core
in vec2 pos; // input fragmen (pixel) position <-1,+1>
out vec3 col; // output fragment (pixel) RGB color <0,1>
void main(void)
{
bool _discard=true;
float N,x,y,z,dz,d,i;
vec3 n,l;
const float pi=3.1415926535897932384626433832795;
const float scale =3.0*pi; // 3.0 periods in x,y
const float scalez=2.0*pi; // 1.0 period in z
N=200.0; // layers per z (quality)
x=pos.x*scale; // <-1,+1> -> [rad]
y=pos.y*scale; // <-1,+1> -> [rad]
dz=2.0*scalez/N; // z step
l=vec3(0.0,0.0,1.0); // light unit direction
i=0.0; // starting color intensity
n=vec3(0.0,0.0,1.0); // starting normal only to get rid o warning
for (z=0.0;z>=-scalez;z-=dz) // raycast z through all layers in view direction
{
// gyroid equation
d =sin(x)*cos(y); // compute gyroid equation
d+=sin(y)*cos(z);
d+=sin(z)*cos(x);
// surface hit test
if (d>1e-6) continue; // skip if too far from surface
_discard=false; // remember that surface was hit
// compute normal
n.x =+cos(x)*cos(y); // partial derivate by x
n.x+=+sin(y)*cos(z);
n.x+=-sin(z)*sin(x);
n.y =-sin(x)*sin(y); // partial derivate by y
n.y+=+cos(y)*cos(z);
n.y+=+sin(z)*cos(x);
n.z =+sin(x)*cos(y); // partial derivate by z
n.z+=-sin(y)*sin(z);
n.z+=+cos(z)*cos(x);
break; // stop raycasting
}
// skip rendering if no hit with surface (hole)
if (_discard) discard;
// directional lighting
n=normalize(n);
i=abs(dot(l,n));
// ambient + directional lighting
i=0.3+(0.7*i);
// output fragment (render pixel)
gl_FragDepth=z; // depth (optional)
col=vec3(1.0,1.0,1.0)*i; // color
}
I hope I did not make error in partial derivates. Here result:
[Edit1]
Based on your code I see it like this (X-Ray like Blending)
var mat = EmguExtensions.InitMat(new System.Drawing.Size(2000, 1080));
double zz, dz, d, i, di = 0;
const double scalex = 2.0 * Math.PI / mat.Width;
const double scaley = 2.0 * Math.PI / mat.Height;
const double scalez = 2.0 * Math.PI;
uint layerCount = 100; // layers
for (int y = 0; y < mat.Height; y++)
{
double yy = y * scaley; // y position of pixel
for (int x = 0; x < mat.Width; x++)
{
double xx = x * scalex; // x position of pixel
dz = 2.0 * scalez / layerCount; // z step
di = 1.0 / layerCount; // color increment
i = 0.0; // color intensity
for (zz = -scalez; zz <= scalez; zz += dz) // do all layers
{
d = Math.Sin(xx) * Math.Cos(yy); // compute gyroid equation
d += Math.Sin(yy) * Math.Cos(zz);
d += Math.Sin(zz) * Math.Cos(xx);
if (d > 1e-6) continue;
i += di; // if near surface add to color
}
i*=255.0;
mat.SetByte(x, y, (byte)(i));
}
}
I have this code that basically reads each pixel of an image and redraws it with different shapes. All shapes will get faded in using a sin() wave.
Now I want to rotate every "Pixelshape" around its own axis (shapeMode(CENTER)) while they are faded in and the translate function gives me a headache in this complex way.
Here is the code so far:
void setup() {
size(1080, 1350);
shapeMode(CENTER);
img = loadImage("loremipsum.png");
…
}
void draw() {
background(123);
for (int gridX = 0; gridX < img.width; gridX++) {
for (int gridY = 0; gridY < img.height; gridY++) {
// grid position + tile size
float tileWidth = width / (float)img.width;
float tileHeight = height / (float)img.height;
float posX = tileWidth*gridX;
float posY = tileHeight*gridY;
// get current color
color c = img.pixels[gridY*img.width+gridX];
// greyscale conversion
int greyscale = round(red(c)*0.222+green(c)*0.707+blue(c)*0.071);
int gradientToIndex = round(map(greyscale, 0, 255, 0, shapeCount-1));
//FADEIN
float wave = map(sin(radians(frameCount*4)), -1, 1, 0, 2);
//translate(HEADACHE);
rotate(radians(wave));
shape(shapes[gradientToIndex], posX, posY, tileWidth * wave, tileHeight * wave);
}
}
I have tried many calculations but it just lets my sketch explode.
One that worked in another sketch where I tried basically the same but just in loop was (equivalent written):
translate(posX + tileWidth/2, posY + tileHeight/2);
I think I just don't get the matrix right? How can I translate them to its meant place?
Thank you very much #Rabbid76 – at first I just pasted in your idea and it went of crazy – then I added pushMatrix(); and popMatrix(); – turned out your translate(); code was in fact right!
Then I had to change the x and y location where every shape is drawn to 0,0,
And this is it! Now it works!
See the code here:
float wave = map(sin(radians(frameCount*4)), -1, 1, 0, 2);
pushMatrix();
translate(posX + tileWidth/2, posY + tileHeight/2);
rotate(radians(wave*180));
shape(shapes[gradientToIndex], 0, 0, tileWidth*wave , tileHeight*wave );
popMatrix();
PERFECT! Thank you so much!
rotate defines a rotation matrix and multiplies the current matrix by the rotation matrix. rotate therefore causes a rotation by (0, 0).
You have to center the rectangle around (0, 0), rotate it and move the rotated rectangle to the desired position with translate.
Since translate and rotate multiplies the current matrix by a new matrix, you must store and restore the matrix by pushMatrix() respectively popMatrix().
The center of a tile is (posX + tileWidth/2, posY + tileHeight/2):
pushMatrix();
translate(posX + tileWidth/2, posY + tileHeight/2);
rotate(radians(wave));
shape(shapes[gradientToIndex],
-tileWidth*wave/2, -tileHeight*wave/2,
tileWidth * wave, tileHeight * wave);
popMatrix();
I'm trying to create a grid of an image (in the way one would tile a background with). Here's what I've been using:
PImage bgtile;
PGraphics bg;
int tilesize = 50;
void setup() {
int t = millis();
fullScreen(P2D);
background(0);
bgtile = loadImage("bgtile.png");
int bgw = ceil( ((float) width) / tilesize) + 1;
int bgh = ceil( ((float) height) / tilesize) + 1;
bg = createGraphics(bgw*tilesize,bgh*tilesize);
bg.beginDraw();
for(int i = 0; i < bgw; i++){
for(int j = 0; j < bgh; j++){
bg.image(bgtile, i*tilesize, j*tilesize, tilesize, tilesize);
}
}
bg.endDraw();
print(millis() - t);
}
The timing code says that this takes about a quarter of a second, but by my count there's a full second once the window opens before anything shows up on screen (which should happen as soon as draw is first run). Is there a faster way to get this same effect? (I want to avoid rendering bgtile hundreds of times in the draw loop for obvious reasons)
One way could be to make use of the GPU and let OpenGL repeat a texture for you.
Processing makes it fairly easy to repeat a texture via textureWrap(REPEAT)
Instead of drawing an image you'd make your own quad shape and instead of calling vertex(x, y) for example, you'd call vertex(x, y, u, v); passing texture coordinates (more low level info on the OpenGL link above). The simple idea is x,y would control the geometry on screen and u,v would control how the texture is applied to the geometry.
Another thing you can control is textureMode() which allows you control how you specify the texture coordinates (U, V):
IMAGE mode is the default: you use pixel coordinates (based on the dimensions of the texture)
NORMAL mode uses values between 0.0 and 1.0 (also known as normalised values) where 1.0 means the maximum the texture can go (e.g. image width for U or image height for V) and you don't need to worry about knowing the texture image dimensions
Here's a basic example based on the textureMode() example above:
PImage img;
void setup() {
fullScreen(P2D);
noStroke();
img = loadImage("https://processing.org/examples/moonwalk.jpg");
// texture mode can be IMAGE (pixel dimensions) or NORMAL (0.0 to 1.0)
// normal means 1.0 is full width (for U) or height (for V) without having to know the image resolution
textureMode(NORMAL);
// this is what will make handle tiling for you
textureWrap(REPEAT);
}
void draw() {
// drag mouse on X axis to change tiling
int tileRepeats = (int)map(constrain(mouseX,0,width), 0, width, 1, 100);
// draw a textured quad
beginShape(QUAD);
// set the texture
texture(img);
// x , y , U , V
vertex(0 , 0 , 0 , 0);
vertex(width, 0 , tileRepeats, 0);
vertex(width, height, tileRepeats, tileRepeats);
vertex(0 , height, 0 , tileRepeats);
endShape();
text((int)frameRate+"fps",15,15);
}
Drag the mouse on the Y axis to control the number of repetitions.
In this simple example both vertex coordinates and texture coordinates are going clockwise (top left, top right, bottom right, bottom left order).
There are probably other ways to achieve the same result: using a PShader comes to mind.
Your approach caching the tiles in setup is ok.
Even flattening your nested loop into a single loop at best may only shave a few milliseconds off, but nothing substantial.
If you tried to cache my snippet above it would make a minimal difference.
In this particular case, because of the back and forth between Java/OpenGL (via JOGL), as far as I can tell using VisualVM, it looks like there's not a lot of room for improvement since simply swapping buffers takes so long (e.g. bg.image()):
An easy way to do this would be to use processing's built in get(); which saves a PImage of the coordinates you pass, for example: PImage pic = get(0, 0, width, height); will capture a "screenshot" of your entire window. So, you can create the image like you already are, and then take a screenshot and display that screenshot.
PImage bgtile;
PGraphics bg;
PImage screenGrab;
int tilesize = 50;
void setup() {
fullScreen(P2D);
background(0);
bgtile = loadImage("bgtile.png");
int bgw = ceil(((float) width) / tilesize) + 1;
int bgh = ceil(((float) height) / tilesize) + 1;
bg = createGraphics(bgw * tilesize, bgh * tilesize);
bg.beginDraw();
for (int i = 0; i < bgw; i++) {
for (int j = 0; j < bgh; j++) {
bg.image(bgtile, i * tilesize, j * tilesize, tilesize, tilesize);
}
}
bg.endDraw();
screenGrab = get(0, 0, width, height);
}
void draw() {
image(screenGrab, 0, 0);
}
This will still take a little bit to generate the image, but once it does, there is no need to use the for loops again unless you change the tilesize.
#George Profenza's answer looks more efficient than my solution, but mine may take a little less modification to the code you already have.
I'm using processing, and I'm trying to create a circle from the pixels i have on my display.
I managed to pull the pixels on screen and create a growing circle from them.
However i'm looking for something much more sophisticated, I want to make it seem as if the pixels on the display are moving from their current location and forming a turning circle or something like this.
This is what i have for now:
int c = 0;
int radius = 30;
allPixels = removeBlackP();
void draw {
loadPixels();
for (int alpha = 0; alpha < 360; alpha++)
{
float xf = 350 + radius*cos(alpha);
float yf = 350 + radius*sin(alpha);
int x = (int) xf;
int y = (int) yf;
if (radius > 200) {radius =30;break;}
if (c> allPixels.length) {c= 0;}
pixels[y*700 +x] = allPixels[c];
updatePixels();
}
radius++;
c++;
}
the function removeBlackP return an array with all the pixels except for the black ones.
This code works for me. There is an issue that the circle only has the numbers as int so it seems like some pixels inside the circle won't fill, i can live with that. I'm looking for something a bit more complex like I explained.
Thanks!
Fill all pixels of scanlines belonging to the circle. Using this approach, you will paint all places inside the circle. For every line calculate start coordinate (end one is symmetric). Pseudocode:
for y = center_y - radius; y <= center_y + radius; y++
dx = Sqrt(radius * radius - y * y)
for x = center_x - dx; x <= center_x + dx; x++
fill a[y, x]
When you find places for all pixels, you can make correlation between initial pixels places and calculated ones and move them step-by-step.
For example, if initial coordinates relative to center point for k-th pixel are (x0, y0) and final coordinates are (x1,y1), and you want to make M steps, moving pixel by spiral, calculate intermediate coordinates:
calc values once:
r0 = Sqrt(x0*x0 + y0*y0) //Math.Hypot if available
r1 = Sqrt(x1*x1 + y1*y1)
fi0 = Math.Atan2(y0, x0)
fi1 = Math.Atan2(y1, x1)
if fi1 < fi0 then
fi1 = fi1 + 2 * Pi;
for i = 1; i <=M ; i++
x = (r0 + i / M * (r1 - r0)) * Cos(fi0 + i / M * (fi1 - fi0))
y = (r0 + i / M * (r1 - r0)) * Sin(fi0 + i / M * (fi1 - fi0))
shift by center coordinates
The way you go about drawing circles in Processing looks a little convoluted.
The simplest way is to use the ellipse() function, no pixels involved though:
If you do need to draw an ellipse and use pixels, you can make use of PGraphics which is similar to using a separate buffer/"layer" to draw into using Processing drawing commands but it also has pixels[] you can access.
Let's say you want to draw a low-res pixel circle circle, you can create a small PGraphics, disable smoothing, draw the circle, then render the circle at a higher resolution. The only catch is these drawing commands must be placed within beginDraw()/endDraw() calls:
PGraphics buffer;
void setup(){
//disable sketch's aliasing
noSmooth();
buffer = createGraphics(25,25);
buffer.beginDraw();
//disable buffer's aliasing
buffer.noSmooth();
buffer.noFill();
buffer.stroke(255);
buffer.endDraw();
}
void draw(){
background(255);
//draw small circle
float circleSize = map(sin(frameCount * .01),-1.0,1.0,0.0,20.0);
buffer.beginDraw();
buffer.background(0);
buffer.ellipse(buffer.width / 2,buffer.height / 2, circleSize,circleSize);
buffer.endDraw();
//render small circle at higher resolution (blocky - no aliasing)
image(buffer,0,0,width,height);
}
If you want to manually draw a circle using pixels[] you are on the right using the polar to cartesian conversion formula (x = cos(angle) * radius, y = sin(angle) * radius).Even though it's focusing on drawing a radial gradient, you can find an example of drawing a circle(a lot actually) using pixels in this answer
I'm trying to figure out how to convert the mouse position (screen coordinates) to the corresponding point on the underlying transformed image drawn on a direct2d surface.
the code here should be considered pseudo code as i'm using a modified c++/CLI wrapper around direct2d for c#, you won't be able to compile this in anything but my own project.
Render()
{
//The transform matrix combines a rotation, followed by a scaling then a translation
renderTarget.Transform = _rotate * _scale * _translate;
RectF imageBounds = new RectF(0, 0, _imageSize.Width, _imageSize.Height);
renderTarget.DrawBitmap(this._image, imageBounds, 1, BitmapInterpolationMode.Linear);
}
Zoom(float zoomfactor, PointF mousepos)
{
//mousePos is in screen coordinates. I need to convert it to image coordinates.
Matrix3x2 t = _translate.Invert();
Matrix3x2 s = _scale.Invert();
Matrix3x2 r = _rotate.Invert();
PointF center = (t * s * r).TransformPoint(mousePos);
_scale = Matrix3x2.Scale(zoomfactor, zoomfactor, center);
}
This is incorrect, the scale center starts moving around wildly when the zoomfactor increases or decreases smoothly, the resulting zoom function is not smooth and flickers a lot even though the mouse pointer is immobile on the center of the client surface. I tried all the combinations I could think of but could not figure it out.
If I set the scale center point as (imagewidth/2, imageheight/2), the resulting zoom is smooth but is always centered on the image center, so I'm pretty sure the flicker isn't due to some other buggy part of the program.
Thanks.
I finally got it right
this gives me perfectly smooth (incremental?, relative?) zooming centered on the client center
(I abandoned the mouse position idea since I wanted to use mouse movement input to drive the zoom)
protected float zoomf
{
get
{
//extract scale factor from scale matrix
return (float)Math.Sqrt((double)((_scale.M11 * _scale.M11)
+ (_scale.M21 * _scale.M21)));
}
}
public void Zoom(float factor)
{
factor = Math.Min(zoomf, 1) * 0.006f * factor;
factor += 1;
Matrix3x2 t = _translation;
t.Invert();
PointF center = t.TransformPoint(_clientCenter);
Matrix3x2 m = Matrix3x2.Scale(new SizeF(factor, factor), center);
_scale = _scale * m;
Invalidate();
}
Step1: Put android:scaleType="matrix" in ImageView XML file
Step 2: Convert screen touch points to Matrix value.
Step 3: Divide each matrix value with Screen density parameter to
get same coordinate value in all screens.
**XML**
<ImageView
android:id="#+id/myImage"
android:layout_width="match_parent"
android:layout_height="match_parent"
android:scaleType="matrix"
android:src="#drawable/ga"/>
**JAVA**
#Override
public boolean onTouchEvent(MotionEvent event) {
float[] point = new float[]{event.getX(), event.getY()};
Matrix inverse = new Matrix();
getImageMatrix().invert(inverse);
inverse.mapPoints(point);
float density = getResources().getDisplayMetrics().density;
int[] imagePointArray = new int[2];
imagePointArray[0] = (int) (point[0] / density);
imagePointArray[1] = (int) (point[1] / density);
Rect rect = new Rect( imagePointArray[0] - 20, imagePointArray[1] - 20, imagePointArray[0] + 20, imagePointArray[1] + 20);//20 is the offset value near to the touch point
boolean b = rect.contains(267, 40);//267,40 are the predefine image coordiantes
Log.e("Touch inside ", b + "");
return true;
}