Related
I have a problem and although I serached everywhere I couldn't find a solution.
I have a stacked sprite and I'm rotating this sprite around the center of the screen. So I iterate over a list of sprites (stacked) and increase the y-coordinate by 2 every loop (rotation is increased step by step by 0.01f outside of the loop):
foreach(var s in stacked)
{
Vector2 origin = new Vector2(Basic.width / 2, Basic.height / 2);
Rectangle newPosition = new Rectangle(position.X, position.Y - y, position.Width, position.Height);
float angle = 0f;
Matrix transform = Matrix.CreateTranslation(-origin.X, -origin.Y, 0f) *
Matrix.CreateRotationZ(rotation) *
Matrix.CreateTranslation(origin.X, origin.Y, 0f);
Vector2 pos = new Vector2(newPosition.X, newPosition.Y);
pos = Vector2.Transform(pos, transform);
newPosition.X = (int)pos.X;
newPosition.Y = (int)pos.Y;
angle += rotation;
s.Draw(newPosition, origin, angle, Color.White);
y += 2;
}
This works fine. But now my problem. I want not only to rotate the sprite around the center of the screen but also around itself. How to achieve this? I can only set one origin and one rotation per Draw. I would like to rotate the sprite around the origin 'Basic.width / 2, Basic.height / 2' and while it rotates, around 'position.Width / 2, position.Height / 2'. With different rotation speed each. How is this possible?
Thank you in advance!
Just to be clear:
When using SpriteBatch.Draw() with origin and angle, there is only one rotation: the final angle of the sprite.
The other rotations are positional offsets.
The origin in the Draw() call is a translation, rotation, translate back. Your transform matrix shows this quite well:
Matrix transform = Matrix.CreateTranslation(-origin.X, -origin.Y, 0f) *
Matrix.CreateRotationZ(rotation) *
Matrix.CreateTranslation(origin.X, origin.Y, 0f);
//Class level variables:
float ScreenRotation, ScreenRotationSpeed;
float ObjectRotation, ObjectRotationSpeed;
Vector2 ScreenOrigin, SpriteOrigin;
// ...
// In constructor and resize events:
ScreenOrigin = new Vector2(Basic.width <<1, Basic.height <<1);
// shifts are faster for `int` type. If "Basic.width" is `float`:
//ScreenOrigin = new Vector2(Basic.width, Basic.height) * 0.5f;
// In Update():
ScreenRotation += ScreenRotationSpeed; // * gameTime.ElapsedGameTime.Seconds; // for FPS invariant speed where speed = 60 * single frame speed
ObjectRotation+= ObjectRotationSpeed;
//Calculate the screen center rotation once per step
Matrix baseTransform = Matrix.CreateTranslation(-ScreenOrigin.X, -ScreenOrigin.Y, 0f) *
Matrix.CreateRotationZ(ScreenRotation) *
Matrix.CreateTranslation(ScreenOrigin.X, ScreenOrigin.Y, 0f);
// In Draw() at the start of your code snippet posted:
// moved outside of the loop for a translationally invariant vertical y interpretation
// or move it inside the loop and apply -y to position.Y for an elliptical effect
Vector2 ObjectOrigin = new Vector2(position.X, position.Y);
Matrix transform = baseTransform *
Matrix.CreateTranslation(-ObjectOrigin.X, -ObjectOrigin.Y, 0f) *
Matrix.CreateRotationZ(ObjectRotation) *
Matrix.CreateTranslation(ObjectOrigin.X, ObjectOrigin.Y, 0f);
foreach(var s in stacked)
{
Vector2 pos = new Vector2(ObjectOrigin.X, ObjectOrigin.Y - y);
pos = Vector2.Transform(pos, transform);
float DrawAngle = ObjectRotation;
// or float DrawAngle = ScreenRotation;
// or float DrawAngle = ScreenRotation + ObjectRotation;
// or float DrawAngle = 0;
s.Draw(pos, SpriteOrigin, DrawAngle, Color.White);
}
I suggest moving the Draw() parameter away from destinationRectangle and use the Vector2 position directly with scaling. Rotations within square rectangles can differ up to SQRT(2) in aspect ratio, i.e. stretching/squashing. Using Vector2 incurs a cost of higher collision complexity.
I am sorry for the ors, but without complete knowledge of the problem...YMMV
In my 2D projects, I use the vector form of polar coordinates.
The Matrix class requires more calculations than the polar equivalents in 2D. Matrix operates in 3D, wasting cycles calculating Z components.
With normalized direction vectors (cos t,sin t) and a radius(vector length),in many cases I use Vector2.LengthSquared() to avoid the square root when possible.
The only time I have used Matrices in 2D is display projection matrix(entire SpriteBatch) and Mouse and TouchScreen input deprojection(times the inverse of the projection matrix)
Some time ago, I coded a little fidgetable logo based on CSS transforms alone.
You can fiddle with it over https://document.paris/
The result feels nice, it feels natural to click/touch and drag to rotate the logo.
I remember banging my head against the walls until I found out that I could chain CSS transforms quite easily just by chaining them.
transform: matrix3d(currentMatrix) rotate3d(x, y, z, angle);
And most importantly to get the currentMatrix I would simply do m = $('#logobackground').css('transform'); with jQuery, the browser would magically return the computed matrix instead of the raw "css" which actually avoided me to deal with matrices or to infinitely stack rotate3D() properties.
So the hardest part was then to calculate the rotate3D arguments (x, y, z, angle) based on mouse inputs. In theory shouldn't have problems transposing this part to java so i'll just skip over it.
Now
I'm trying to do the exact same thing with Processing and there is two problems :
There is no rotate3D() in processing.
There is no browser to apply/chain transformations and return me the current matrix state automatically.
Here's the plan/implementation I'm working on :
I need a "currentMatrix" to apply every frame to the scene
PMatrix3D currentMatrix = new PMatrix3D();
In the setup() I set it to the "identity matrix" which from what I understand is equivalent to "no transformation".
// set currentMatrix to identity Matrix
currentMatrix.set(1, 0, 0, 0, 0, 1, 0, 0, 0, 0, 1, 0, 0, 0, 0, 1);
Every frame I would calculate a transformation matrix and apply it to the currentMatrix.
Then I would apply this matrix to the scene.
// Apply Matrix to the currentMatrix
void mouseRotate() {
float diag = sqrt(pow(width,2)+pow(height,2));
float x = deltaX()/ diag * 10; // deltaX = difference between previous prevous MouseX and current mouseX)
float y = deltaY()/ diag * 10; // deltaY = same with Y axis
float angle = sqrt( pow(x, 2) + pow(y, 2) );
currentMatrix.apply( rotate3D(y,x,0,angle) );
}
// Apply Matrix to the scene
applyMatrix(currentMatrix);
PMatrix3D reference : https://processing.github.io/processing-javadocs/core/processing/core/PMatrix3D.html
ApplyMatrix() reference : https://processing.org/reference/applyMatrix_.html
All I need to do then is to implement the rotate3D css transform as a function which returns a transformation matrix.
Based on what I found on this page https://developer.mozilla.org/en-US/docs/Web/CSS/transform-function/rotate3d()
I implemented this first function :
PMatrix3D rotate3D(float x, float y, float z, float a) {
PMatrix3D rotationMatrix = new PMatrix3D();
rotationMatrix.set(
1+(1-cos(a))*(pow(x,2)-1), z*sin(a)+x*y*(1-cos(a)), -y*sin(a)+x*z*(1-cos(a)), 0,
-z*sin(a)+x*y*(1-cos(a)), 1+(1-cos(a))*(pow(y,2)-1), x*sin(a)+y*z*(1-cos(a)), 0,
y*sin(a)+x*z*(1-cos(a)), -x*sin(a)+y*z*(1-cos(a)), 1+(1-cos(a))*(pow(z,2)-1), 0,
0,0,0,1
);
return rotationMatrix;
}
and based on what I found on this page https://drafts.csswg.org/css-transforms-2/#Rotate3dDefined
I implemented this other function :
PMatrix3D rotate3Dbis(float getX, float getY, float getZ, float getA) {
float sc = sin(getA/2)*cos(getA/2);
float sq = pow(sin(getA/2),2);
float normalizer = sqrt( pow(getX,2) + pow(getY,2) + pow(getZ,2) );
float x = getX/normalizer;
float y = getY/normalizer;
float z = getZ/normalizer;
PMatrix3D rotationMatrix = new PMatrix3D();
rotationMatrix.set(
1-2*(pow(y,2)+pow(z,2))*sq, 2*(x*y*sq-z*sc), 2*(x*z*sq+y*sc), 0,
2*(x*y*sq+z*sc), 1-2*(pow(x,2)+pow(z,2))*sq, 2*(y*z*sq-x*sc), 0,
2*(x*z*sq-y*sc), 2*(y*z*sq+x*sc), 1-2*(pow(x,2)+pow(y,2)*sq), 0,
0, 0, 0, 1
);
return rotationMatrix;
}
When testing, they don't produce exactly the same result with the same inputs (although the differences are kind of "symmetric" which makes me think that they are kind of equivalent at least in some way ?) Also rotate3Dbis() has a tendency to produce NaN numbers, especially when i'm not moving the mouse (x & y = 0).
But most importantly, in the end it doesn't work. Instead of rotating, the drawing just zooms out progressively when I'm using rotate3D(), and rotate3Dbis() doesn't render correctly because of the NaNs.
The overall question :
I'm trying to get guidance from people who understand transformations Matrices and trying to narrow down where the issue is. Are my processing/java implementations of rotate3D() flawed ? Or would the issue come from somewhere else ? And are my rotate3D() and rotate3Dbis functions equivalent ?
You might get away with simply rotating on X and Y axis, as you already mentioned, using the previous and current mouse coordinates:
PVector cameraRotation = new PVector(0, 0);
void setup(){
size(900, 900, P3D);
rectMode(CENTER);
strokeWeight(9);
strokeJoin(MITER);
}
void draw(){
//update "camera" rotation
if (mousePressed){
cameraRotation.x += -float(mouseY-pmouseY);
cameraRotation.y += float(mouseX-pmouseX);
}
background(255);
translate(width * 0.5, height * 0.5, 0);
rotateX(radians(cameraRotation.x));
rotateY(radians(cameraRotation.y));
rect(0, 0, 300, 450);
}
The Document Paris example you've shared also uses easing. You can have a look at this minimal easing Processing example
Here's a version of the above with easing applied:
PVector cameraRotation = new PVector();
PVector cameraTargetRotation = new PVector();
float easing = 0.01;
void setup(){
size(900, 900, P3D);
rectMode(CENTER);
strokeWeight(9);
strokeJoin(MITER);
}
void draw(){
//update "camera" rotation
if (mousePressed){
cameraTargetRotation.x += -float(mouseY-pmouseY);
cameraTargetRotation.y += float(mouseX-pmouseX);
}
background(255);
translate(width * 0.5, height * 0.5, 0);
// ease rotation
rotateX(radians(cameraRotation.x -= (cameraRotation.x - cameraTargetRotation.x) * easing));
rotateY(radians(cameraRotation.y -= (cameraRotation.y - cameraTargetRotation.y) * easing));
fill(255);
rect(0, 0, 300, 450);
fill(0);
translate(0, 0, 3);
rect(0, 0, 300, 450);
}
Additionally there's a library called PeasyCam which can make this much simpler.
If you do want to implement your own version using PMatrix3D here are a couple of tips that could save you time:
When you instantiate PMatrix3D() it's the identity matrix. If you have transformations applied and you want to reset() to identity.
If you want to rotate a PMatrix3D() around and axis the rotate(float angleInRadians, float axisX, float axisY, float axisZ) override should help.
Additionally you could get away without PMatrix3D since resetMatrix() will reset the global transformation matrix and you can call rotate(float angleInRadians, float axisX, float axisY, float axisZ) directly.
Part of the answer is a fix added to the first rotate3D function.
I needed to normalize the x,y,z values to avoid the weird scaling.
I'm posting the current state of the code (i'm skipping a few parts for the sake of simplicity):
// Mouse movement since last fame on X axis
float deltaX() {
return (float)(mouseX-pmouseX);
}
// Mouse movement since last fame on Y axis
float deltaY() {
return (float)(mouseY-pmouseY);
}
// Convert user input into angle and amount to rotate to
void mouseRotate() {
double diag = Math.sqrt(Math.pow(width,2)+Math.pow(height,2));
double x = deltaX()/ diag * 50;
double y = -deltaY()/ diag * 50;
double angle = Math.sqrt( x*x + y*y );
currentMatrix.apply( rotate3D((float)y,(float)x,0,(float)angle) );
}
// Convert those values into a rotation matrix
PMatrix3D rotate3D(float getX, float getY, float getZ, float getA) {
float normalizer = sqrt( getX*getX + getY*getY + getZ*getZ );
float x = 0;
float y = 0;
float z = 0;
if (normalizer != 0) {
x = getX/normalizer;
y = getY/normalizer;
z = getZ/normalizer;
}
float x2 = pow(x,2);
float y2 = pow(y,2);
float z2 = 0;
float sina = sin(getA);
float f1cosa = 1-cos(getA);
PMatrix3D rotationMatrix = new PMatrix3D(
1+f1cosa*(x2-1), z*sina+x*y*f1cosa, -y*sina+x*z*f1cosa, 0,
-z*sina+x*y*f1cosa, 1+f1cosa*(y2-1), x*sina+y*z*f1cosa, 0,
y*sina+x*z*f1cosa, -x*sina+y*z*f1cosa, 1+f1cosa*(z2-1), 0,
0, 0, 0, 1
);
return rotationMatrix;
}
// Draw
draw() {
mouseRotate();
applyMatrix(currentMatrix);
object.render();
}
I thought that using this method would allow me to "stack" cumulative rotations relative to the screen and not relative to the object. But the result seems to always do the rotation relative to the object drawn.
I am not using a camera because I basically only want to rotate the object on itself. I'm actually a bit lost atm on what I should rotate and when to that the newly applied rotations are relative to the user, and the previously applied rotation are conserved.
I´d like to get the tilt of the device, so i can use this to mesure the tilt os some surface, laying down the device over the surface.
Right now i am using Device Motion Plugin for xamarin forms from here https://github.com/rdelrosario/xamarin-plugins
and the code below:
CrossDeviceMotion.Current.Start(MotionSensorType.Accelerometer);
CrossDeviceMotion.Current.SensorValueChanged += (s, a) =>
{
switch (a.SensorType)
{
case MotionSensorType.Accelerometer:
{
Debug.WriteLine("A: {0},{1},{2}", ((MotionVector)a.Value).X, ((MotionVector)a.Value).Y,
((MotionVector)a.Value).Z);
Exposicao.Inclinacao = ((MotionVector)a.Value).Z;
break;
}
case MotionSensorType.Compass:
{
// Debug.WriteLine("H: {0}", a.Value);
Exposicao.Bussola = (double)a.Value.Value;
break;
}
}
};
The compass part is ok, the accelerometer part is working but there are some but´s.
If i am not wrong, i get the tilt in Z axis, so z.Value.Value.
This value is diferent for android and ios, lets focus in android.
z values are from 10 when device is laying down on flat surface, to 0 if device is stand up, lets focus only in just one quadrant.
I am doing something wrong to achieve what i explained?
How can i convert those values to a Angle between 0 and 90? It seems not linear, so the 5 does not seem 45 degrees.
Thanks
I'd probably roll out my own platform implementation for the feature you're looking for. The DeviceMotion library looks a bit simple for your purposes, as can be seen from the answer below. I'm pretty sure you can use it as a good starting point but it needs to be extended a little.
Android
On Android, you should use the Rotation Vector Sensor which uses a Kalman filter (with accelerometer, magnetometer and gyroscope) to get accurate measurements of the device's rotation:
The rotation vector represents the orientation of the device as a combination of an angle and an axis, in which the device has rotated through an angle θ around an axis (x, y, or z).
Image from the official Android documentation
iOS:
For iOS, you have to do a bit more work yourself. The key is to make use of CMAttitude, which describes the attitude of the device relative to an initial attitude. I found a snippet I've saved to my collection from an unknown source (can't credit the original author) here:
public void CalculateLeanAngle ()
{
motionManager = new CMMotionManager ();
motionManager.DeviceMotionUpdateInterval = 0.02;
if (motionManager.DeviceMotionAvailable) {
motionManager.StartDeviceMotionUpdates(CMAttitudeReferenceFrame.XArbitraryZVertical, NSOperationQueue.CurrentQueue, (data, error) => {
CMQuaternion quat = motionManager.DeviceMotion.Attitude.Quaternion;
double x = quat.x;
double y = quat.y;
double w = quat.w;
double z = quat.z;
double degrees = 0.0;
//Roll
double roll = Math.Atan2 (2 * y * w - 2 * x * z, 1 - 2 * y * y - 2 * z * z);
degrees = Math.Round (-applyKalmanFiltering (roll) * 180.0 / Constants.M_PI);
});
}
public double applyKalmanFiltering (double yaw)
{
if (motionLastYaw == 0)
motionLastYaw = yaw;
float q = 0.1f; // process noise
float r = 0.1f; // sensor noise
float p = 0.1f; // estimated error
float k = 0.5f; // kalman filter gain
double x = motionLastYaw;
p = p + q;
k = p / (p + r);
x = x + k * (yaw - x);
p = (1 - k) * p;
motionLastYaw = x;
return motionLastYaw;
}
Image from the official Xamarin documentation
I'll try to look for the original source when I have more time but I'm pretty sure this will work out of the box for your purposes.
I'm trying to figure out how to convert the mouse position (screen coordinates) to the corresponding point on the underlying transformed image drawn on a direct2d surface.
the code here should be considered pseudo code as i'm using a modified c++/CLI wrapper around direct2d for c#, you won't be able to compile this in anything but my own project.
Render()
{
//The transform matrix combines a rotation, followed by a scaling then a translation
renderTarget.Transform = _rotate * _scale * _translate;
RectF imageBounds = new RectF(0, 0, _imageSize.Width, _imageSize.Height);
renderTarget.DrawBitmap(this._image, imageBounds, 1, BitmapInterpolationMode.Linear);
}
Zoom(float zoomfactor, PointF mousepos)
{
//mousePos is in screen coordinates. I need to convert it to image coordinates.
Matrix3x2 t = _translate.Invert();
Matrix3x2 s = _scale.Invert();
Matrix3x2 r = _rotate.Invert();
PointF center = (t * s * r).TransformPoint(mousePos);
_scale = Matrix3x2.Scale(zoomfactor, zoomfactor, center);
}
This is incorrect, the scale center starts moving around wildly when the zoomfactor increases or decreases smoothly, the resulting zoom function is not smooth and flickers a lot even though the mouse pointer is immobile on the center of the client surface. I tried all the combinations I could think of but could not figure it out.
If I set the scale center point as (imagewidth/2, imageheight/2), the resulting zoom is smooth but is always centered on the image center, so I'm pretty sure the flicker isn't due to some other buggy part of the program.
Thanks.
I finally got it right
this gives me perfectly smooth (incremental?, relative?) zooming centered on the client center
(I abandoned the mouse position idea since I wanted to use mouse movement input to drive the zoom)
protected float zoomf
{
get
{
//extract scale factor from scale matrix
return (float)Math.Sqrt((double)((_scale.M11 * _scale.M11)
+ (_scale.M21 * _scale.M21)));
}
}
public void Zoom(float factor)
{
factor = Math.Min(zoomf, 1) * 0.006f * factor;
factor += 1;
Matrix3x2 t = _translation;
t.Invert();
PointF center = t.TransformPoint(_clientCenter);
Matrix3x2 m = Matrix3x2.Scale(new SizeF(factor, factor), center);
_scale = _scale * m;
Invalidate();
}
Step1: Put android:scaleType="matrix" in ImageView XML file
Step 2: Convert screen touch points to Matrix value.
Step 3: Divide each matrix value with Screen density parameter to
get same coordinate value in all screens.
**XML**
<ImageView
android:id="#+id/myImage"
android:layout_width="match_parent"
android:layout_height="match_parent"
android:scaleType="matrix"
android:src="#drawable/ga"/>
**JAVA**
#Override
public boolean onTouchEvent(MotionEvent event) {
float[] point = new float[]{event.getX(), event.getY()};
Matrix inverse = new Matrix();
getImageMatrix().invert(inverse);
inverse.mapPoints(point);
float density = getResources().getDisplayMetrics().density;
int[] imagePointArray = new int[2];
imagePointArray[0] = (int) (point[0] / density);
imagePointArray[1] = (int) (point[1] / density);
Rect rect = new Rect( imagePointArray[0] - 20, imagePointArray[1] - 20, imagePointArray[0] + 20, imagePointArray[1] + 20);//20 is the offset value near to the touch point
boolean b = rect.contains(267, 40);//267,40 are the predefine image coordiantes
Log.e("Touch inside ", b + "");
return true;
}
I'm currently working on a library to generate synthetic fingerprints using the SFinGe method (by Maltoni, Maio and Cappelli) link :http://biolab.csr.unibo.it/research.asp?organize=Activities&select=&selObj=12&pathSubj=111%7C%7C12&
One of the steps requires me to apply different gabor filters to an image, each pixel in the image have an orientation and a frequency associated, so the convolution is not done with one kernel over the entire image but the filter must change during the process depending on those attributes of the pixels, that way each pixel on the image is altered in a different way.
If you apply the filters this way, and convolve the image several times(you also have to binarize the image after each convolution) you obtain this:
A master fingerprint, this image took about 20 seconds to be generated (which is way too slow, this is why I want to do it with FFT), since I had to perform 5 times the convolution to complete it (you start from a few random black points).
My filters are 30x30, and the image is 275x400. There are a total of 36000 filters, one for each degree and density (density goes from 0 to 100). I'm planing on reducing the number of filters from 36000 to 9000 since I can cover all the angles with those. Also all the filters are pre-calculated and stored in a filter bank.
This is the source code in C# of the gabor convolution implementation:
This two methods execute the convolution:
/// <summary>
/// Convolve the image with the different filters depending on the orientation and density of the pixel.
/// </summary>
/// <param name="image">The image to be filtered.</param>
/// <param name="directionalMap">The directional map.</param>
/// <param name="densityMap">The density map.</param>
/// <returns></returns>
public double[,] Filter(double[,] image, double[,] directionalMap, double[,] densityMap)
{
int midX = FILTER_SIZE / 2;
int midY = FILTER_SIZE / 2;
double[,] filteredImage = new double[image.GetLength(0), image.GetLength(1)];
double[,] filteredImageWithValuesScaled = new double[image.GetLength(0), image.GetLength(1)];
double[,] finalImage = new double[image.GetLength(0), image.GetLength(1)];
for (int i = 0; i < image.GetLength(0); i++)
for (int j = 0; j < image.GetLength(1); j++)
{
double pixelValue = GetPixelConvolutionValue(image, this.filterBank[(int)Math.Floor((directionalMap[i, j] * 180 / Math.PI))][Math.Round(densityMap[i, j], 2)], i - midX, j - midY);
filteredImage[i, j] = pixelValue;
}
filteredImageWithValuesScaled = this.RescaleValues(filteredImage, 0.0, 255.0);
return filteredImageWithValuesScaled;
}
/// <summary>
/// Gets the pixel convolution value.
/// </summary>
/// <param name="image">The image.</param>
/// <param name="filter">The filter.</param>
/// <param name="sourceX">The source X.</param>
/// <param name="sourceY">The source Y.</param>
/// <returns></returns>
private double GetPixelConvolutionValue(double[,] image, double[,] filter, int sourceX, int sourceY)
{
double result = 0.0;
int totalPixels = 0;
for (int i = 0; i < filter.GetLength(0); i++)
{
if(i + sourceX < 0 || i + sourceX >= image.GetLength(0))
continue;
for (int j = 0; j < filter.GetLength(1); j++)
{
if(j + sourceY < 0 || j + sourceY >= image.GetLength(1))
continue;
double deltaResult = image[sourceX + i,sourceY + j] * filter[i, j];
result += deltaResult;
++totalPixels;
}
}
double filteredValue = result / totalPixels;
return filteredValue;
}
This two methods generate the different gabor filters for the filter bank:
/// <summary>
/// Creates the gabor filter.
/// </summary>
/// <param name="size">The size.</param>
/// <param name="angle">The angle.</param>
/// <param name="wavelength">The wavelength.</param>
/// <param name="sigma">The sigma.</param>
/// <returns></returns>
public double[,] CreateGaborFilter(int size, double angle, double wavelength, double sigma)
{
double[,] filter = new double[size, size];
double frequency = 7 + (100 - (wavelength * 100)) * 0.03;
int windowSize = FILTER_SIZE/2;
for (int y = 0; y < size; ++y)
{
for (int x = 0; x < size; ++x)
{
int dy = -windowSize + y;
int dx = -windowSize + x;
filter[x, y] = GaborFilterValue(dy, dx, frequency, angle, 0, sigma, 0.80);
}
}
return filter;
}
/// <summary>
/// Gabor filter values generation.
/// </summary>
/// <param name="x">The x.</param>
/// <param name="y">The y.</param>
/// <param name="lambda">The wavelength.</param>
/// <param name="theta">The orientation.</param>
/// <param name="phi">The phaseoffset.</param>
/// <param name="sigma">The gaussvar.</param>
/// <param name="gamma">The aspectratio.</param>
/// <returns></returns>
double GaborFilterValue(int x, int y, double lambda, double theta, double phi, double sigma, double gamma)
{
double xx = x * Math.Cos(theta) + y * Math.Sin(theta);
double yy = -x * Math.Sin(theta) + y * Math.Cos(theta);
double envelopeVal = Math.Exp(-((xx * xx + gamma * gamma * yy * yy) / (2.0f * sigma * sigma)));
double carrierVal = Math.Cos(2.0f * (float)Math.PI * xx / lambda + phi);
double g = envelopeVal * carrierVal;
return g;
}
My goal is to reduce this time to under 1 second (There are several programs that do the exactly same thing in such time). So since the direct convolution approach is not working for me I decide to implement the Fast Fourier Transform Convolution, but the problem with this is that FFT applies the same kernel to the entire image at once, and I need to change the kernel per pixel, because each pixel must be altered depending on his attributes (density and orientation). In this post How to apply Gabor wavelets to an image? reve-etrange explains how to apply different gabor filters to an image, but the thing is that the way he does it, is applying the different filters to the whole image and then sum the responses , and what I need is the responses from different pixels to the different filters.
This is whats happening when I convolve one filter with the image (using FFT):
This was the filter used:
And this was the image it was convolved with:
This is the algorithm in C# of the FFT implementation:
/// <summary>
/// Convolve the image using FFT.
/// </summary>
/// <param name="image">The image to be filtered.</param>
/// <param name="directionalMap">The directional map.</param>
/// <param name="densityMap">The density map.</param>
/// <param name="FFT">if set to <c>true</c> [FFT].</param>
/// <returns></returns>
public double[,] Filter(double[,] image, double[,] directionalMap, double[,] densityMap, bool FFT)
{
double[,] filter = null;
double[,] paddedFilter = null;
double[,] paddedImage = null;
double[,] croppedImage = null;
double[,] filteredImage = new double[image.GetLength(0), image.GetLength(1)];
double[,] filteredImageWithValuesScaled = new double[image.GetLength(0), image.GetLength(1)];
double[,] finalImage = new double[image.GetLength(0), image.GetLength(1)];
filter = this.filterBank[70][0];
paddedFilter = PadImage(filter, 512, 512, 0, 0); // Pad the filter to have a potency of 2 dimensions.
paddedImage = PadImage(image, 512, 512, 0, 0); // Pad the image to have a potency of 2 dimensions.
FFT fftOne = new FFT(paddedImage);
FFT fftTwo = new FFT(paddedFilter);
fftOne.ForwardFFT();
fftTwo.ForwardFFT();
FFT result = fftOne * fftTwo;
result.InverseFFT();
filteredImage = result.GreyImage;
filteredImageWithValuesScaled = this.RescaleValues(filteredImage, 0.0, 255.0);
croppedImage = CropImage(filteredImageWithValuesScaled, image.GetLength(0), image.GetLength(1));
return croppedImage;
}
So what I'm asking is, how do you get the response from different pixels to different kernels with FFT? If this is not possible, is there a way to improve my direct convolution to make it at least 20 times faster?
Also would it be possible to make one kernel using all the filters, so I can apply those to the whole image?
I found a way to convolve the image with different gabor filters and gather the responses of the pixels base on their local characteristics using FFT.
This is call contextual filtering, usually when you filter an image you only apply a single kernel to the entire thing, but in contextual filtering the characteristics of the filter change according to the local context, in this case, density and orientation of the pixel.
In direct convolution the process is pretty straight forward, you simply change the kernel at each step of the convolution, but in FFT convolution since the kernel is applied to the image in the frequency domain you can't change the filter properties during the process. So the way you do it is by making the convolution of the image with each filter separately, this will give N number of filtered images where N is the numbers of filters in your filter bank, then you have to construct your final image taking information from the different filtered images based on the context of the pixel you are recreating.
So for each pixel, you look at his orientation and density properties and then grab the value of that pixel position from the filtered image that was generated by the convolving the original image with the filter with those same properties.
Here is an example of the process:
This is a graphic representation of the directional map.
I used the same density for all pixels to reduce the amount of filters generated.
This is the source image:
This is an example of three of the filters used (0 degrees, 45 degrees, 90 degrees):
These are three examples of the source image being convolve with the different filters at different degrees:
And finally this is the resulting image, the image was created taking the values of the pixels from the different filtered images base on the direction and density of the pixel.
This process is A LOT slower than the direct convolution =(, since you have to convolve the original image with all the filters first. The final image took about a minute to be generated.
So far I'm stuck with the direct convolution it seems =/.
Thanks for reading.
Did you try with CUDA? with the convolution and the FFT (in this case cufft) it will be faster! Try to see if there is a possibility of parallelization, actually I'm working on that and I can say that the improvement is huge. Currently I'm going to implement the Gabor filtering