int x = 31;
int y = 31;
int x_dir = 4;
int y_dir = 0;
void setup ()
{
size (800, 800);
}
void draw ()
{
background (150);
ellipse (x,y,60, 60);
if (x+30>=width)
{
x_dir =-4;
y_dir = 4;
}
if (y+30>=height)
{
x_dir=4;
y_dir = 0;
}
if (x+30>=width)
{
x_dir = -4;
}
x+=x_dir;
y+=y_dir;
println(x,y);
}
Hi,
I have to create this program in processing which produces an animation of a ball going in a Z pattern (top left to top right, diagonal top right to bottom left, and then straight from bottom left to bottom right) which then goes backwards along the same path it came.
While I have the code written out for the forward direction, I don't know what 2 if or else statements I need to write for the program so that based on one condition it goes forwards, and based on another condition it will go backwards, and it will continue doing so until it terminates.
If I am able to figure out which two if statements I need to write, all I need to do is copy and reverse the x_dir and y_dir signs on the forward loop.
There are a ton of different ways you can do this.
One approach is to keep track of which "mode" you're in. You could do this using an int variable that's 0 when you're on the first part of the path, 1 when you're on the second part of the path, etc. Then just use an if statement to decide what to do, how to move the ball, etc.
Here's an example:
int x = 31;
int y = 31;
int mode = 0;
void setup ()
{
size (800, 800);
}
void draw ()
{
background (150);
ellipse (x, y, 60, 60);
if (mode == 0) {
x = x + 4;
if (x+30>=width) {
mode = 1;
}
} else if (mode == 1) {
x = x - 4;
y = y + 4;
if (y+30>=height) {
mode = 2;
}
} else if (mode == 2) {
x = x + 4;
if (x+30>=width) {
mode = 3;
}
} else if (mode == 3) {
x = x - 4;
y = y - 4;
if (y-30 < 0) {
mode = 2;
}
}
}
Like I said, this is only one way to approach the problem, and there are some obvious improvements you could make. For example, you could store the movement speeds and the conditions that change the mode in an array (or better yet, in objects) and get rid of all of the if statements.
I have this method here which renders my players movement. It swaps between 3 images, standing, left leg forward, and right leg forward.
It swaps images very fast so how can I change the speed of the rendering?
public static void renderUpwardWalking() {
ImageIcon[] frames = { CharacterSheet.up, CharacterSheet.upLeftLeg,
CharacterSheet.upRightLeg };
if (Key.up && Character.direction == "up") {
currentFrame++;
if (currentFrame == 3)
currentFrame = 1;
Character.character.setIcon(frames[currentFrame]);
} else if (!Key.up && Character.direction == "up") {
currentFrame = 0;
}
}
This is usually done on timer.
Decide on a frame pattern and a frequency. You seem to have chosen the frame pattern CharacterSheet.up, CharacterSheet.upLeftLeg, CharacterSheet.upRightLeg. Let's say you want to swap frame every 400 ms.
Get the time from a clock with sufficient resolution. System.nanoTime() is usually accurate enough.
long frameTime = 400L * 1000000L; // 400 ms in nanoseconds Edit
currentFrame = (System.nanoTime() / frametime) % frames.length;
You can change the scale of your currentFrame counter, and use its range to control your frame rate:
//Let this go from 1...30
int currentFrameCounter;
.
.
.
currentFrameCounter++;
if(currentFrameCounter == 30) currentFrameCounter = 0;
//Take a fraction of currentframeCounter for frame index ~ 1/10 frame rate
//Note care to avoid integer division
currentFrame = (int) (1.0*currentFrameCounter / 10.0);
Putting it all together in a general model:
int maxCounter = 30; //or some other factor of 3 -- controls speed
int currentFrameCounter;
public static void renderUpwardWalking() {
ImageIcon[] frames = { CharacterSheet.up, CharacterSheet.upLeftLeg,
CharacterSheet.upRightLeg };
if (Key.up && Character.direction == "up") {
currentFrameCounter++; //add
if(currentFrameCounter == maxCounter) currentFrameCounter = 0;
currentFrame = (int) (1.0*currentFrameCounter / (maxCounter/3.0));
Character.character.setIcon(frames[currentFrame]);
} else if (!Key.up && Character.direction == "up") {
currentFrame = 0;
}
}
I am developing an XNA game that is using Kinect. The player seen on the screen is the real image of the person who is playing in front of Kinect sensor. For eliminating the background and getting only the player's image I am doing these operations in kinect.AllFramesReady:
using (ColorImageFrame colorVideoFrame = imageFrames.OpenColorImageFrame())
{
if (colorVideoFrame != null)
{
//Getting the image of the colorVideoFrame to a Texture2D named colorVideo
}
//And setting its information on a Color array named colors with GetData
colorVideo.GetData(colors);
}
using (DepthImageFrame depthVideoFrame = imageFrames.OpenDepthImageFrame())
{
if (depthVideoFrame != null){
//Copying the the image to a DepthImagePixel array
//Using only the pixels with PlayerIndex > 0 to create a Color array
//And then setting the colors of this array from the 'colors' array by using MapDepthPointToColorPoint method, provided by Kinect SDK
//Finally I use SetData function in order to set the colors to a Texture2D I created before
}
}
But the performance is very low unsurprisingly. Because I have to use GetData for a color array with 640*480 = 307200 length (because of the ColorImageFormat) and SetData for another color array with 320*480 = 76800 length (because of the DepthImageFormat) in every frame!
I wonder if there is any other solutions for this problem, any alternatives for SetData and GetData maybe. Because I know that these functions moving data between the GPU and CPU and that is an expensive operation for big data. Thanks for any help.
The Kinect for Windows Toolbox comes with a "GreenScreen-WPF" example, which should provide some insight into processing the information. Because you are working in XNA there may be some differences, but the overall concepts should work between the two examples.
The example works by extracting multiple players. Here is the business end of the processing function:
private void SensorAllFramesReady(object sender, AllFramesReadyEventArgs e)
{
// in the middle of shutting down, so nothing to do
if (null == this.sensor)
{
return;
}
bool depthReceived = false;
bool colorReceived = false;
using (DepthImageFrame depthFrame = e.OpenDepthImageFrame())
{
if (null != depthFrame)
{
// Copy the pixel data from the image to a temporary array
depthFrame.CopyDepthImagePixelDataTo(this.depthPixels);
depthReceived = true;
}
}
using (ColorImageFrame colorFrame = e.OpenColorImageFrame())
{
if (null != colorFrame)
{
// Copy the pixel data from the image to a temporary array
colorFrame.CopyPixelDataTo(this.colorPixels);
colorReceived = true;
}
}
// do our processing outside of the using block
// so that we return resources to the kinect as soon as possible
if (true == depthReceived)
{
this.sensor.CoordinateMapper.MapDepthFrameToColorFrame(
DepthFormat,
this.depthPixels,
ColorFormat,
this.colorCoordinates);
Array.Clear(this.greenScreenPixelData, 0, this.greenScreenPixelData.Length);
// loop over each row and column of the depth
for (int y = 0; y < this.depthHeight; ++y)
{
for (int x = 0; x < this.depthWidth; ++x)
{
// calculate index into depth array
int depthIndex = x + (y * this.depthWidth);
DepthImagePixel depthPixel = this.depthPixels[depthIndex];
int player = depthPixel.PlayerIndex;
// if we're tracking a player for the current pixel, do green screen
if (player > 0)
{
// retrieve the depth to color mapping for the current depth pixel
ColorImagePoint colorImagePoint = this.colorCoordinates[depthIndex];
// scale color coordinates to depth resolution
int colorInDepthX = colorImagePoint.X / this.colorToDepthDivisor;
int colorInDepthY = colorImagePoint.Y / this.colorToDepthDivisor;
// make sure the depth pixel maps to a valid point in color space
// check y > 0 and y < depthHeight to make sure we don't write outside of the array
// check x > 0 instead of >= 0 since to fill gaps we set opaque current pixel plus the one to the left
// because of how the sensor works it is more correct to do it this way than to set to the right
if (colorInDepthX > 0 && colorInDepthX < this.depthWidth && colorInDepthY >= 0 && colorInDepthY < this.depthHeight)
{
// calculate index into the green screen pixel array
int greenScreenIndex = colorInDepthX + (colorInDepthY * this.depthWidth);
// set opaque
this.greenScreenPixelData[greenScreenIndex] = opaquePixelValue;
// compensate for depth/color not corresponding exactly by setting the pixel
// to the left to opaque as well
this.greenScreenPixelData[greenScreenIndex - 1] = opaquePixelValue;
}
}
}
}
}
// do our processing outside of the using block
// so that we return resources to the kinect as soon as possible
if (true == colorReceived)
{
// Write the pixel data into our bitmap
this.colorBitmap.WritePixels(
new Int32Rect(0, 0, this.colorBitmap.PixelWidth, this.colorBitmap.PixelHeight),
this.colorPixels,
this.colorBitmap.PixelWidth * sizeof(int),
0);
if (this.playerOpacityMaskImage == null)
{
this.playerOpacityMaskImage = new WriteableBitmap(
this.depthWidth,
this.depthHeight,
96,
96,
PixelFormats.Bgra32,
null);
MaskedColor.OpacityMask = new ImageBrush { ImageSource = this.playerOpacityMaskImage };
}
this.playerOpacityMaskImage.WritePixels(
new Int32Rect(0, 0, this.depthWidth, this.depthHeight),
this.greenScreenPixelData,
this.depthWidth * ((this.playerOpacityMaskImage.Format.BitsPerPixel + 7) / 8),
0);
}
}
If you are interested in only a single player, you could look into using the player mask to more quickly extract the appropriate pixel set. You'd fi
using (SkeletonFrame skeletonFrame = e.OpenSkeletonFrame())
{
if (skeletonFrame != null && skeletonFrame.SkeletonArrayLength > 0)
{
if (_skeletons == null || _skeletons.Length != skeletonFrame.SkeletonArrayLength)
{
_skeletons = new Skeleton[skeletonFrame.SkeletonArrayLength];
}
skeletonFrame.CopySkeletonDataTo(_skeletons);
// grab the tracked skeleton and set the playerIndex for use pulling
// the depth data out for the silhouette.
this.playerIndex = -1;
for (int i = 0; i < _skeletons.Length; i++)
{
if (_skeletons[i].TrackingState != SkeletonTrackingState.NotTracked)
{
this.playerIndex = i+1;
}
}
}
}
You can then step through the depth data to extract the appropriate bits:
depthFrame.CopyPixelDataTo(this.pixelData);
for (int i16 = 0, i32 = 0; i16 < pixelData.Length && i32 < depthFrame32.Length; i16++, i32 += 4)
{
int player = pixelData[i16] & DepthImageFrame.PlayerIndexBitmask;
if (player == this.playerIndex)
{
// the player we are tracking
}
else if (player > 0)
{
// a player, but not the one we want.
}
else
{
// background or something else we don't care about
}
}
I'm pulling this code from a control I use to produce a silhouette, so it does not deal with the color stream. However, making a call to MapDepthFrameToColorFrame at the appropriate time should allow you to deal with the color stream data and extract the corresponding pixels to the player's mask.
Does anybody know how to find the local maxima in a grayscale IPL_DEPTH_8U image using OpenCV? HarrisCorner mentions something like that but I'm actually not interested in corners ...
Thanks!
A pixel is considered a local maximum if it is equal to the maximum value in a 'local' neighborhood. The function below captures this property in two lines of code.
To deal with pixels on 'plateaus' (value equal to their neighborhood) one can use the local minimum property, since plateaus pixels are equal to their local minimum. The rest of the code filters out those pixels.
void non_maxima_suppression(const cv::Mat& image, cv::Mat& mask, bool remove_plateaus) {
// find pixels that are equal to the local neighborhood not maximum (including 'plateaus')
cv::dilate(image, mask, cv::Mat());
cv::compare(image, mask, mask, cv::CMP_GE);
// optionally filter out pixels that are equal to the local minimum ('plateaus')
if (remove_plateaus) {
cv::Mat non_plateau_mask;
cv::erode(image, non_plateau_mask, cv::Mat());
cv::compare(image, non_plateau_mask, non_plateau_mask, cv::CMP_GT);
cv::bitwise_and(mask, non_plateau_mask, mask);
}
}
Here's a simple trick. The idea is to dilate with a kernel that contains a hole in the center. After the dilate operation, each pixel is replaced with the maximum of it's neighbors (using a 5 by 5 neighborhood in this example), excluding the original pixel.
Mat1b kernelLM(Size(5, 5), 1u);
kernelLM.at<uchar>(2, 2) = 0u;
Mat imageLM;
dilate(image, imageLM, kernelLM);
Mat1b localMaxima = (image > imageLM);
Actually after I posted the code above I wrote a better and very very faster one ..
The code above suffers even for a 640x480 picture..
I optimized it and now it is very very fast even for 1600x1200 pic.
Here is the code :
void localMaxima(cv::Mat src,cv::Mat &dst,int squareSize)
{
if (squareSize==0)
{
dst = src.clone();
return;
}
Mat m0;
dst = src.clone();
Point maxLoc(0,0);
//1.Be sure to have at least 3x3 for at least looking at 1 pixel close neighbours
// Also the window must be <odd>x<odd>
SANITYCHECK(squareSize,3,1);
int sqrCenter = (squareSize-1)/2;
//2.Create the localWindow mask to get things done faster
// When we find a local maxima we will multiply the subwindow with this MASK
// So that we will not search for those 0 values again and again
Mat localWindowMask = Mat::zeros(Size(squareSize,squareSize),CV_8U);//boolean
localWindowMask.at<unsigned char>(sqrCenter,sqrCenter)=1;
//3.Find the threshold value to threshold the image
//this function here returns the peak of histogram of picture
//the picture is a thresholded picture it will have a lot of zero values in it
//so that the second boolean variable says :
// (boolean) ? "return peak even if it is at 0" : "return peak discarding 0"
int thrshld = maxUsedValInHistogramData(dst,false);
threshold(dst,m0,thrshld,1,THRESH_BINARY);
//4.Now delete all thresholded values from picture
dst = dst.mul(m0);
//put the src in the middle of the big array
for (int row=sqrCenter;row<dst.size().height-sqrCenter;row++)
for (int col=sqrCenter;col<dst.size().width-sqrCenter;col++)
{
//1.if the value is zero it can not be a local maxima
if (dst.at<unsigned char>(row,col)==0)
continue;
//2.the value at (row,col) is not 0 so it can be a local maxima point
m0 = dst.colRange(col-sqrCenter,col+sqrCenter+1).rowRange(row-sqrCenter,row+sqrCenter+1);
minMaxLoc(m0,NULL,NULL,NULL,&maxLoc);
//if the maximum location of this subWindow is at center
//it means we found the local maxima
//so we should delete the surrounding values which lies in the subWindow area
//hence we will not try to find if a point is at localMaxima when already found a neighbour was
if ((maxLoc.x==sqrCenter)&&(maxLoc.y==sqrCenter))
{
m0 = m0.mul(localWindowMask);
//we can skip the values that we already made 0 by the above function
col+=sqrCenter;
}
}
}
The following listing is a function similar to Matlab's "imregionalmax". It looks for at most nLocMax local maxima above threshold, where the found local maxima are at least minDistBtwLocMax pixels apart. It returns the actual number of local maxima found. Notice that it uses OpenCV's minMaxLoc to find global maxima. It is "opencv-self-contained" except for the (easy to implement) function vdist, which computes the (euclidian) distance between points (r,c) and (row,col).
input is one-channel CV_32F matrix, and locations is nLocMax (rows) by 2 (columns) CV_32S matrix.
int imregionalmax(Mat input, int nLocMax, float threshold, float minDistBtwLocMax, Mat locations)
{
Mat scratch = input.clone();
int nFoundLocMax = 0;
for (int i = 0; i < nLocMax; i++) {
Point location;
double maxVal;
minMaxLoc(scratch, NULL, &maxVal, NULL, &location);
if (maxVal > threshold) {
nFoundLocMax += 1;
int row = location.y;
int col = location.x;
locations.at<int>(i,0) = row;
locations.at<int>(i,1) = col;
int r0 = (row-minDistBtwLocMax > -1 ? row-minDistBtwLocMax : 0);
int r1 = (row+minDistBtwLocMax < scratch.rows ? row+minDistBtwLocMax : scratch.rows-1);
int c0 = (col-minDistBtwLocMax > -1 ? col-minDistBtwLocMax : 0);
int c1 = (col+minDistBtwLocMax < scratch.cols ? col+minDistBtwLocMax : scratch.cols-1);
for (int r = r0; r <= r1; r++) {
for (int c = c0; c <= c1; c++) {
if (vdist(Point2DMake(r, c),Point2DMake(row, col)) <= minDistBtwLocMax) {
scratch.at<float>(r,c) = 0.0;
}
}
}
} else {
break;
}
}
return nFoundLocMax;
}
The first question to answer would be what is "local" in your opinion. The answer may well be a square window (say 3x3 or 5x5) or circular window of a certain radius. You can then scan over the entire image with the window centered at each pixel and pick the highest value in the window.
See this for how to access pixel values in OpenCV.
This is very fast method. It stored founded maxima in a vector of
Points.
vector <Point> GetLocalMaxima(const cv::Mat Src,int MatchingSize, int Threshold, int GaussKernel )
{
vector <Point> vMaxLoc(0);
if ((MatchingSize % 2 == 0) || (GaussKernel % 2 == 0)) // MatchingSize and GaussKernel have to be "odd" and > 0
{
return vMaxLoc;
}
vMaxLoc.reserve(100); // Reserve place for fast access
Mat ProcessImg = Src.clone();
int W = Src.cols;
int H = Src.rows;
int SearchWidth = W - MatchingSize;
int SearchHeight = H - MatchingSize;
int MatchingSquareCenter = MatchingSize/2;
if(GaussKernel > 1) // If You need a smoothing
{
GaussianBlur(ProcessImg,ProcessImg,Size(GaussKernel,GaussKernel),0,0,4);
}
uchar* pProcess = (uchar *) ProcessImg.data; // The pointer to image Data
int Shift = MatchingSquareCenter * ( W + 1);
int k = 0;
for(int y=0; y < SearchHeight; ++y)
{
int m = k + Shift;
for(int x=0;x < SearchWidth ; ++x)
{
if (pProcess[m++] >= Threshold)
{
Point LocMax;
Mat mROI(ProcessImg, Rect(x,y,MatchingSize,MatchingSize));
minMaxLoc(mROI,NULL,NULL,NULL,&LocMax);
if (LocMax.x == MatchingSquareCenter && LocMax.y == MatchingSquareCenter)
{
vMaxLoc.push_back(Point( x+LocMax.x,y + LocMax.y ));
// imshow("W1",mROI);cvWaitKey(0); //For gebug
}
}
}
k += W;
}
return vMaxLoc;
}
Found a simple solution.
In this example, if you are trying to find 2 results of a matchTemplate function with a minimum distance from each other.
cv::Mat result;
matchTemplate(search, target, result, CV_TM_SQDIFF_NORMED);
float score1;
cv::Point displacement1 = MinMax(result, score1);
cv::circle(result, cv::Point(displacement1.x+result.cols/2 , displacement1.y+result.rows/2), 10, cv::Scalar(0), CV_FILLED, 8, 0);
float score2;
cv::Point displacement2 = MinMax(result, score2);
where
cv::Point MinMax(cv::Mat &result, float &score)
{
double minVal, maxVal;
cv::Point minLoc, maxLoc, matchLoc;
minMaxLoc(result, &minVal, &maxVal, &minLoc, &maxLoc, cv::Mat());
matchLoc.x = minLoc.x - result.cols/2;
matchLoc.y = minLoc.y - result.rows/2;
return minVal;
}
The process is:
Find global Minimum using minMaxLoc
Draw a filled white circle around global minimum using min distance between minima as radius
Find another minimum
The the scores can be compared to each other to determine, for example, the certainty of the match,
To find more than just the global minimum and maximum try using this function from skimage:
http://scikit-image.org/docs/dev/api/skimage.feature.html#skimage.feature.peak_local_max
You can parameterize the minimum distance between peaks, too. And more. To find minima, use negated values (take care of the array type though, 255-image could do the trick).
You can go over each pixel and test if it is a local maxima. Here is how I would do it.
The input is assumed to be type CV_32FC1
#include <vector>//std::vector
#include <algorithm>//std::sort
#include "opencv2/imgproc/imgproc.hpp"
#include "opencv2/core/core.hpp"
//structure for maximal values including position
struct SRegionalMaxPoint
{
SRegionalMaxPoint():
values(-FLT_MAX),
row(-1),
col(-1)
{}
float values;
int row;
int col;
//ascending order
bool operator()(const SRegionalMaxPoint& a, const SRegionalMaxPoint& b)
{
return a.values < b.values;
}
};
//checks if pixel is local max
bool isRegionalMax(const float* im_ptr, const int& cols )
{
float center = *im_ptr;
bool is_regional_max = true;
im_ptr -= (cols + 1);
for (int ii = 0; ii < 3; ++ii, im_ptr+= (cols-3))
{
for (int jj = 0; jj < 3; ++jj, im_ptr++)
{
if (ii != 1 || jj != 1)
{
is_regional_max &= (center > *im_ptr);
}
}
}
return is_regional_max;
}
void imregionalmax(
const cv::Mat& input,
std::vector<SRegionalMaxPoint>& buffer)
{
//find local max - top maxima
static const int margin = 1;
const int rows = input.rows;
const int cols = input.cols;
for (int i = margin; i < rows - margin; ++i)
{
const float* im_ptr = input.ptr<float>(i, margin);
for (int j = margin; j < cols - margin; ++j, im_ptr++)
{
//Check if pixel is local maximum
if ( isRegionalMax(im_ptr, cols ) )
{
cv::Rect roi = cv::Rect(j - margin, i - margin, 3, 3);
cv::Mat subMat = input(roi);
float val = *im_ptr;
//replace smallest value in buffer
if ( val > buffer[0].values )
{
buffer[0].values = val;
buffer[0].row = i;
buffer[0].col = j;
std::sort(buffer.begin(), buffer.end(), SRegionalMaxPoint());
}
}
}
}
}
For testing the code you can try this:
cv::Mat temp = cv::Mat::zeros(15, 15, CV_32FC1);
temp.at<float>(7, 7) = 1;
temp.at<float>(3, 5) = 6;
temp.at<float>(8, 10) = 4;
temp.at<float>(11, 13) = 7;
temp.at<float>(10, 3) = 8;
temp.at<float>(7, 13) = 3;
vector<SRegionalMaxPoint> buffer_(5);
imregionalmax(temp, buffer_);
cv::Mat debug;
cv::cvtColor(temp, debug, cv::COLOR_GRAY2BGR);
for (auto it = buffer_.begin(); it != buffer_.end(); ++it)
{
circle(debug, cv::Point(it->col, it->row), 1, cv::Scalar(0, 255, 0));
}
This solution does not take plateaus into account so it is not exactly the same as matlab's imregionalmax()
I think you want to use the
MinMaxLoc(arr, mask=NULL)-> (minVal, maxVal, minLoc, maxLoc)
Finds global minimum and maximum in array or subarray
function on you image