direct2d image viewer How to convert screen coordinates to image coordinates? - matrix

I'm trying to figure out how to convert the mouse position (screen coordinates) to the corresponding point on the underlying transformed image drawn on a direct2d surface.
the code here should be considered pseudo code as i'm using a modified c++/CLI wrapper around direct2d for c#, you won't be able to compile this in anything but my own project.
Render()
{
//The transform matrix combines a rotation, followed by a scaling then a translation
renderTarget.Transform = _rotate * _scale * _translate;
RectF imageBounds = new RectF(0, 0, _imageSize.Width, _imageSize.Height);
renderTarget.DrawBitmap(this._image, imageBounds, 1, BitmapInterpolationMode.Linear);
}
Zoom(float zoomfactor, PointF mousepos)
{
//mousePos is in screen coordinates. I need to convert it to image coordinates.
Matrix3x2 t = _translate.Invert();
Matrix3x2 s = _scale.Invert();
Matrix3x2 r = _rotate.Invert();
PointF center = (t * s * r).TransformPoint(mousePos);
_scale = Matrix3x2.Scale(zoomfactor, zoomfactor, center);
}
This is incorrect, the scale center starts moving around wildly when the zoomfactor increases or decreases smoothly, the resulting zoom function is not smooth and flickers a lot even though the mouse pointer is immobile on the center of the client surface. I tried all the combinations I could think of but could not figure it out.
If I set the scale center point as (imagewidth/2, imageheight/2), the resulting zoom is smooth but is always centered on the image center, so I'm pretty sure the flicker isn't due to some other buggy part of the program.
Thanks.

I finally got it right
this gives me perfectly smooth (incremental?, relative?) zooming centered on the client center
(I abandoned the mouse position idea since I wanted to use mouse movement input to drive the zoom)
protected float zoomf
{
get
{
//extract scale factor from scale matrix
return (float)Math.Sqrt((double)((_scale.M11 * _scale.M11)
+ (_scale.M21 * _scale.M21)));
}
}
public void Zoom(float factor)
{
factor = Math.Min(zoomf, 1) * 0.006f * factor;
factor += 1;
Matrix3x2 t = _translation;
t.Invert();
PointF center = t.TransformPoint(_clientCenter);
Matrix3x2 m = Matrix3x2.Scale(new SizeF(factor, factor), center);
_scale = _scale * m;
Invalidate();
}

Step1: Put android:scaleType="matrix" in ImageView XML file
Step 2: Convert screen touch points to Matrix value.
Step 3: Divide each matrix value with Screen density parameter to
get same coordinate value in all screens.
**XML**
<ImageView
android:id="#+id/myImage"
android:layout_width="match_parent"
android:layout_height="match_parent"
android:scaleType="matrix"
android:src="#drawable/ga"/>
**JAVA**
#Override
public boolean onTouchEvent(MotionEvent event) {
float[] point = new float[]{event.getX(), event.getY()};
Matrix inverse = new Matrix();
getImageMatrix().invert(inverse);
inverse.mapPoints(point);
float density = getResources().getDisplayMetrics().density;
int[] imagePointArray = new int[2];
imagePointArray[0] = (int) (point[0] / density);
imagePointArray[1] = (int) (point[1] / density);
Rect rect = new Rect( imagePointArray[0] - 20, imagePointArray[1] - 20, imagePointArray[0] + 20, imagePointArray[1] + 20);//20 is the offset value near to the touch point
boolean b = rect.contains(267, 40);//267,40 are the predefine image coordiantes
Log.e("Touch inside ", b + "");
return true;
}

Related

Monogame - Rotate Sprite around centre of screen and itself

I have a problem and although I serached everywhere I couldn't find a solution.
I have a stacked sprite and I'm rotating this sprite around the center of the screen. So I iterate over a list of sprites (stacked) and increase the y-coordinate by 2 every loop (rotation is increased step by step by 0.01f outside of the loop):
foreach(var s in stacked)
{
Vector2 origin = new Vector2(Basic.width / 2, Basic.height / 2);
Rectangle newPosition = new Rectangle(position.X, position.Y - y, position.Width, position.Height);
float angle = 0f;
Matrix transform = Matrix.CreateTranslation(-origin.X, -origin.Y, 0f) *
Matrix.CreateRotationZ(rotation) *
Matrix.CreateTranslation(origin.X, origin.Y, 0f);
Vector2 pos = new Vector2(newPosition.X, newPosition.Y);
pos = Vector2.Transform(pos, transform);
newPosition.X = (int)pos.X;
newPosition.Y = (int)pos.Y;
angle += rotation;
s.Draw(newPosition, origin, angle, Color.White);
y += 2;
}
This works fine. But now my problem. I want not only to rotate the sprite around the center of the screen but also around itself. How to achieve this? I can only set one origin and one rotation per Draw. I would like to rotate the sprite around the origin 'Basic.width / 2, Basic.height / 2' and while it rotates, around 'position.Width / 2, position.Height / 2'. With different rotation speed each. How is this possible?
Thank you in advance!
Just to be clear:
When using SpriteBatch.Draw() with origin and angle, there is only one rotation: the final angle of the sprite.
The other rotations are positional offsets.
The origin in the Draw() call is a translation, rotation, translate back. Your transform matrix shows this quite well:
Matrix transform = Matrix.CreateTranslation(-origin.X, -origin.Y, 0f) *
Matrix.CreateRotationZ(rotation) *
Matrix.CreateTranslation(origin.X, origin.Y, 0f);
//Class level variables:
float ScreenRotation, ScreenRotationSpeed;
float ObjectRotation, ObjectRotationSpeed;
Vector2 ScreenOrigin, SpriteOrigin;
// ...
// In constructor and resize events:
ScreenOrigin = new Vector2(Basic.width <<1, Basic.height <<1);
// shifts are faster for `int` type. If "Basic.width" is `float`:
//ScreenOrigin = new Vector2(Basic.width, Basic.height) * 0.5f;
// In Update():
ScreenRotation += ScreenRotationSpeed; // * gameTime.ElapsedGameTime.Seconds; // for FPS invariant speed where speed = 60 * single frame speed
ObjectRotation+= ObjectRotationSpeed;
//Calculate the screen center rotation once per step
Matrix baseTransform = Matrix.CreateTranslation(-ScreenOrigin.X, -ScreenOrigin.Y, 0f) *
Matrix.CreateRotationZ(ScreenRotation) *
Matrix.CreateTranslation(ScreenOrigin.X, ScreenOrigin.Y, 0f);
// In Draw() at the start of your code snippet posted:
// moved outside of the loop for a translationally invariant vertical y interpretation
// or move it inside the loop and apply -y to position.Y for an elliptical effect
Vector2 ObjectOrigin = new Vector2(position.X, position.Y);
Matrix transform = baseTransform *
Matrix.CreateTranslation(-ObjectOrigin.X, -ObjectOrigin.Y, 0f) *
Matrix.CreateRotationZ(ObjectRotation) *
Matrix.CreateTranslation(ObjectOrigin.X, ObjectOrigin.Y, 0f);
foreach(var s in stacked)
{
Vector2 pos = new Vector2(ObjectOrigin.X, ObjectOrigin.Y - y);
pos = Vector2.Transform(pos, transform);
float DrawAngle = ObjectRotation;
// or float DrawAngle = ScreenRotation;
// or float DrawAngle = ScreenRotation + ObjectRotation;
// or float DrawAngle = 0;
s.Draw(pos, SpriteOrigin, DrawAngle, Color.White);
}
I suggest moving the Draw() parameter away from destinationRectangle and use the Vector2 position directly with scaling. Rotations within square rectangles can differ up to SQRT(2) in aspect ratio, i.e. stretching/squashing. Using Vector2 incurs a cost of higher collision complexity.
I am sorry for the ors, but without complete knowledge of the problem...YMMV
In my 2D projects, I use the vector form of polar coordinates.
The Matrix class requires more calculations than the polar equivalents in 2D. Matrix operates in 3D, wasting cycles calculating Z components.
With normalized direction vectors (cos t,sin t) and a radius(vector length),in many cases I use Vector2.LengthSquared() to avoid the square root when possible.
The only time I have used Matrices in 2D is display projection matrix(entire SpriteBatch) and Mouse and TouchScreen input deprojection(times the inverse of the projection matrix)

Let PShapes in an array rotate on its own axis in Processing

I have this code that basically reads each pixel of an image and redraws it with different shapes. All shapes will get faded in using a sin() wave.
Now I want to rotate every "Pixelshape" around its own axis (shapeMode(CENTER)) while they are faded in and the translate function gives me a headache in this complex way.
Here is the code so far:
void setup() {
size(1080, 1350);
shapeMode(CENTER);
img = loadImage("loremipsum.png");
…
}
void draw() {
background(123);
for (int gridX = 0; gridX < img.width; gridX++) {
for (int gridY = 0; gridY < img.height; gridY++) {
// grid position + tile size
float tileWidth = width / (float)img.width;
float tileHeight = height / (float)img.height;
float posX = tileWidth*gridX;
float posY = tileHeight*gridY;
// get current color
color c = img.pixels[gridY*img.width+gridX];
// greyscale conversion
int greyscale = round(red(c)*0.222+green(c)*0.707+blue(c)*0.071);
int gradientToIndex = round(map(greyscale, 0, 255, 0, shapeCount-1));
//FADEIN
float wave = map(sin(radians(frameCount*4)), -1, 1, 0, 2);
//translate(HEADACHE);
rotate(radians(wave));
shape(shapes[gradientToIndex], posX, posY, tileWidth * wave, tileHeight * wave);
}
}
I have tried many calculations but it just lets my sketch explode.
One that worked in another sketch where I tried basically the same but just in loop was (equivalent written):
translate(posX + tileWidth/2, posY + tileHeight/2);
I think I just don't get the matrix right? How can I translate them to its meant place?
Thank you very much #Rabbid76 – at first I just pasted in your idea and it went of crazy – then I added pushMatrix(); and popMatrix(); – turned out your translate(); code was in fact right!
Then I had to change the x and y location where every shape is drawn to 0,0,
And this is it! Now it works!
See the code here:
float wave = map(sin(radians(frameCount*4)), -1, 1, 0, 2);
pushMatrix();
translate(posX + tileWidth/2, posY + tileHeight/2);
rotate(radians(wave*180));
shape(shapes[gradientToIndex], 0, 0, tileWidth*wave , tileHeight*wave );
popMatrix();
PERFECT! Thank you so much!
rotate defines a rotation matrix and multiplies the current matrix by the rotation matrix. rotate therefore causes a rotation by (0, 0).
You have to center the rectangle around (0, 0), rotate it and move the rotated rectangle to the desired position with translate.
Since translate and rotate multiplies the current matrix by a new matrix, you must store and restore the matrix by pushMatrix() respectively popMatrix().
The center of a tile is (posX + tileWidth/2, posY + tileHeight/2):
pushMatrix();
translate(posX + tileWidth/2, posY + tileHeight/2);
rotate(radians(wave));
shape(shapes[gradientToIndex],
-tileWidth*wave/2, -tileHeight*wave/2,
tileWidth * wave, tileHeight * wave);
popMatrix();

Processing: Efficiently create uniform grid

I'm trying to create a grid of an image (in the way one would tile a background with). Here's what I've been using:
PImage bgtile;
PGraphics bg;
int tilesize = 50;
void setup() {
int t = millis();
fullScreen(P2D);
background(0);
bgtile = loadImage("bgtile.png");
int bgw = ceil( ((float) width) / tilesize) + 1;
int bgh = ceil( ((float) height) / tilesize) + 1;
bg = createGraphics(bgw*tilesize,bgh*tilesize);
bg.beginDraw();
for(int i = 0; i < bgw; i++){
for(int j = 0; j < bgh; j++){
bg.image(bgtile, i*tilesize, j*tilesize, tilesize, tilesize);
}
}
bg.endDraw();
print(millis() - t);
}
The timing code says that this takes about a quarter of a second, but by my count there's a full second once the window opens before anything shows up on screen (which should happen as soon as draw is first run). Is there a faster way to get this same effect? (I want to avoid rendering bgtile hundreds of times in the draw loop for obvious reasons)
One way could be to make use of the GPU and let OpenGL repeat a texture for you.
Processing makes it fairly easy to repeat a texture via textureWrap(REPEAT)
Instead of drawing an image you'd make your own quad shape and instead of calling vertex(x, y) for example, you'd call vertex(x, y, u, v); passing texture coordinates (more low level info on the OpenGL link above). The simple idea is x,y would control the geometry on screen and u,v would control how the texture is applied to the geometry.
Another thing you can control is textureMode() which allows you control how you specify the texture coordinates (U, V):
IMAGE mode is the default: you use pixel coordinates (based on the dimensions of the texture)
NORMAL mode uses values between 0.0 and 1.0 (also known as normalised values) where 1.0 means the maximum the texture can go (e.g. image width for U or image height for V) and you don't need to worry about knowing the texture image dimensions
Here's a basic example based on the textureMode() example above:
PImage img;
void setup() {
fullScreen(P2D);
noStroke();
img = loadImage("https://processing.org/examples/moonwalk.jpg");
// texture mode can be IMAGE (pixel dimensions) or NORMAL (0.0 to 1.0)
// normal means 1.0 is full width (for U) or height (for V) without having to know the image resolution
textureMode(NORMAL);
// this is what will make handle tiling for you
textureWrap(REPEAT);
}
void draw() {
// drag mouse on X axis to change tiling
int tileRepeats = (int)map(constrain(mouseX,0,width), 0, width, 1, 100);
// draw a textured quad
beginShape(QUAD);
// set the texture
texture(img);
// x , y , U , V
vertex(0 , 0 , 0 , 0);
vertex(width, 0 , tileRepeats, 0);
vertex(width, height, tileRepeats, tileRepeats);
vertex(0 , height, 0 , tileRepeats);
endShape();
text((int)frameRate+"fps",15,15);
}
Drag the mouse on the Y axis to control the number of repetitions.
In this simple example both vertex coordinates and texture coordinates are going clockwise (top left, top right, bottom right, bottom left order).
There are probably other ways to achieve the same result: using a PShader comes to mind.
Your approach caching the tiles in setup is ok.
Even flattening your nested loop into a single loop at best may only shave a few milliseconds off, but nothing substantial.
If you tried to cache my snippet above it would make a minimal difference.
In this particular case, because of the back and forth between Java/OpenGL (via JOGL), as far as I can tell using VisualVM, it looks like there's not a lot of room for improvement since simply swapping buffers takes so long (e.g. bg.image()):
An easy way to do this would be to use processing's built in get(); which saves a PImage of the coordinates you pass, for example: PImage pic = get(0, 0, width, height); will capture a "screenshot" of your entire window. So, you can create the image like you already are, and then take a screenshot and display that screenshot.
PImage bgtile;
PGraphics bg;
PImage screenGrab;
int tilesize = 50;
void setup() {
fullScreen(P2D);
background(0);
bgtile = loadImage("bgtile.png");
int bgw = ceil(((float) width) / tilesize) + 1;
int bgh = ceil(((float) height) / tilesize) + 1;
bg = createGraphics(bgw * tilesize, bgh * tilesize);
bg.beginDraw();
for (int i = 0; i < bgw; i++) {
for (int j = 0; j < bgh; j++) {
bg.image(bgtile, i * tilesize, j * tilesize, tilesize, tilesize);
}
}
bg.endDraw();
screenGrab = get(0, 0, width, height);
}
void draw() {
image(screenGrab, 0, 0);
}
This will still take a little bit to generate the image, but once it does, there is no need to use the for loops again unless you change the tilesize.
#George Profenza's answer looks more efficient than my solution, but mine may take a little less modification to the code you already have.

Different Processing rendering between native and online sketch

I get different results when running this sample with Processing directly, and with Processing.js in a browser. Why?
I was happy about my result and wanted to share it on open Processing, but the rendering was totally different and I don't see why. Below is a minimal working example.
/* Program that rotates a triange and draws an ellipse when the third vertex is on top of the screen*/
float y = 3*height/2;
float x = 3*width/2;
float previous_1 = 0.0;
float previous_2 = 0.0;
float current;
float angle = 0.0;
void setup() {
size(1100, 500);
}
void draw() {
fill(0, 30);
// rotate triangle
angle = angle - 0.02;
translate(x, y);
rotate(angle);
// display triangle
triangle(-50, -50, -30, 30, -90, -60);
// detect whether third vertex is on top by comparing its 3 successive positions
current = screenY(-90, -60); // current position of the third vertex
if (previous_1 < previous_2 && previous_1 < current) {
// draw ellipse at the extrema position
fill(128, 9, 9);
ellipse(-90, -60, 7, 10);
}
// update the 2 previous positions of the third vertex
previous_2 = previous_1;
previous_1 = current;
}
In processing, the ellipse is drawn when a triangle vertex is on top, which is my goal.
In online sketching, the ellipse is drawn during the whole time :/
In order to get the same results online as you get by running Processing locally you will need to specify the rendering mode as 3d when calling size
For example:
void setup() {
size(1100, 500, P3D);
}
You will also need to specify the z coordinate in the call to screenY()
current = screenY(-90, -60, 0);
With these two changes you should get the same results online as you get running locally.
Online:
Triangle Ellipse Example
Local:
The problem lies in the screenY function. Print out the current variable in your processing sketch locally and online. In OpenProcessing, the variable current grows quickly above multiple thousands, while it stays between 0 and ~260 locally.
It seems like OpenProcessing has a bug inside this function.
To fix this however, I would recommend you to register differently when you drew a triangle at the top of the circle, for example by using your angle variable:
// Calculate angle and modulo it by 2 * PI
angle = (angle - 0.02) % (2 * PI);
// If the sketch has made a full revolution
if (previous_1 < previous_2 && previous_1 < angle) {
// draw ellipse at the extrema position
fill(128, 9, 9);
ellipse(-90, -60, 7, 10);
}
// update the 2 previous angles of the third vertex
previous_2 = previous_1;
previous_1 = angle;
However, because of how you draw the triangles, the ellipse is at an angle of about PI / 3. To fix this, one option would be to rotate the screen by angle + PI / 3 like so:
rotate(angle + PI / 3);
You might have to experiment with the angle offset a bit more to draw the ellipse perfectly at the top of the circle.

Emgu CV draw rotated rectangle

I'm looking for few days a solution to draw rectangle on image frame. Basically I'm using CvInvoke.cvRectangle method to draw rectangle on image because I need antialiased rect.
But problem is when I need to rotate a given shape for given angle. I can't find any good solution.
I have tryed to draw rectangle on separate frame then rotate hole frame and apply this new image on top of my base frame. But in this solution there is a problem with antialiasing. It's not working.
I'm working on simple application that should allow draw few kinds of shape, resize them and rotation for given angle.
Any idea how to achive this?
The best way I found to draw a minimum enclosing rectangle on the contour is using the Polylines() function which uses vertices that are returned from MinAreaRect() function. There are surely other ways to do it as well. Here is the code walk down:
// Find contours
var contours = new Emgu.CV.Util.VectorOfVectorOfPoint();
Mat hierarchy = new Mat();
CvInvoke.FindContours(image, contours, hierarchy, RetrType.Tree, ChainApproxMethod.ChainApproxSimple);
// According to your metric, get an index of the contour you want to find the min enclosing rectangle for
int index = 2; // Say, 2nd index works for you.
var rectangle = CvInvoke.MinAreaRect(contours[index]);
Point[] vertices = Array.ConvertAll(rectangle.GetVertices(), Point.Round);
CvInvoke.Polylines(image, vertices, true, new MCvScalar(0, 0, 255), 5);
The result can be visualized in the image below, in red is the minimum enclosing rectangle.
I use C# and EMGU.CV(4.1), and I think this code will not be difficult to transfer to any platform.
Add function in the in your helper:
public static Mat DrawRect(Mat input, RotatedRect rect, MCvScalar color = default(MCvScalar),
int thickness = 1, LineType lineType = LineType.EightConnected, int shift = 0)
{
var v = rect.GetVertices();
var prevPoint = v[0];
var firstPoint = prevPoint;
var nextPoint = prevPoint;
var lastPoint = nextPoint;
for (var i = 1; i < v.Length; i++)
{
nextPoint = v[i];
CvInvoke.Line(input, Point.Round(prevPoint), Point.Round(nextPoint), color, thickness, lineType, shift);
prevPoint = nextPoint;
lastPoint = prevPoint;
}
CvInvoke.Line(input, Point.Round(lastPoint), Point.Round(firstPoint), color, thickness, lineType, shift);
return input;
}
This draws roteted rectangle by points. Here used rounding points by method Point.Round becose RotatedRect has points in float coordinates and CvInvoke.Line takes points as integer.
Use:
var mat = Mat.Zeros(200, 200, DepthType.Cv8U, 3);
mat.GetValueRange();
var rRect = new RotatedRect(new PointF(100, 100), new SizeF(100, 50), 30);
DrawRect(mat, rRect,new MCvScalar(255,0,0));
var brect = CvInvoke.BoundingRectangle(new VectorOfPointF(rRect.GetVertices()));
CvInvoke.Rectangle(mat, brect, new MCvScalar(0,255,0), 1, LineType.EightConnected, 0);
Result:
You should read the OpenCV documentation.
There is a RotatedRectangle class that you can use for your task. You can specify the angle by which the rectangle will be rotated.
Here is a sample code (taken from the docs) for drawing a rotated rectangle:
Mat image(200, 200, CV_8UC3, Scalar(0));
RotatedRect rRect = RotatedRect(Point2f(100,100), Size2f(100,50), 30);
Point2f vertices[4];
rRect.points(vertices);
for (int i = 0; i < 4; i++)
line(image, vertices[i], vertices[(i+1)%4], Scalar(0,255,0));
Rect brect = rRect.boundingRect();
rectangle(image, brect, Scalar(255,0,0));
imshow("rectangles", image);
waitKey(0);
Here is the result:

Resources