I am aware that real time face detection is something that needs high cpu time, too much to implement it in a game(which is my goal). Therefore I am looking for a way to improve my FPS.
In the game, there should only be two faces. Those faces are nearly always on the same positions. One in the left lower middle of the screen, the other one in the right lower middle.
I CAN assume that there are ALWAYS exactly 2 Faces, which, like I said before, are roughly on the same positions as in the frame before.
My idea was to tell the algorithm WHERE he has to search.
First frame:
calculates where there are faces on the screen. Coordinates of Faces are stored for next frame.
Following frames:
use the coordinates of the frame before to start looking for faces in the area around the stored position. If nothing found, increase the distance from the position where it has to look for faces and search again.
Doing so would greatly improve my performance, however I didn't find any way to tell the algorithm where it has to look for faces.
Is there a way to do so?
Thanks.
If you want to use the OpenCV algorithm without modifying it, you can extract a sub-image around the location of the faces at the previous frame. In this way the OpenCV face detector performs a sliding window search on a much smaller region. Then you remap the face position in the full frame coordinate system. If your faces do not move too fast you can run this every n-frames and interpolate the position between the detection frames for a further speed-up.
To get the subImg you can use:
cv::Rect roi(xTl,yTl,w,h);
cv::Mat subImg = img(roi);
where xTl,yTl are the top left coordinates of the searching window and w,h the size.
Alternatively once you detect the faces, you can use MeanShift/CamShift tracker (or other trackers) to find the position in every frame:
http://docs.opencv.org/trunk/doc/py_tutorials/py_video/py_meanshift/py_meanshift.html .
Related
I'm sorry if this is a stupid question, but I want to make sure that I'm right or not.
suppose we have an 8x8 pixel screen and we want to represent a 2x2 square, a pixel can be black - 1 and white - 0. I would imagine this as an 8x8 matrix
[[0,0,0,0,0,0,0,0],
[0,0,0,0,0,0,0,0],
[0,0,0,0,0,0,0,0],
[0,0,0,1,1,0,0,0],
[0,0,0,1,1,0,0,0],
[0,0,0,0,0,0,0,0],
[0,0,0,0,0,0,0,0],
[0,0,0,0,0,0,0,0]]
using this matrix, we paint over the pixels and update them (for example) every second. we also have the coordinates of the pixels representing the square : (4,4) (4,5) (5,4) (5,5) and if we want to move the square we add 1 to x part of coordinate.
is it true or not?
Graphics Rendering is a complex mesh of art, mathematics, and hardware, assuming you're asking about how the screen actually works instead of a pet problem on simulating displays.
The buffer you described in the question is the interface which software uses to tell the hardware (video card) what to draw on the screen, and how it is actually done is in the realm of hardware. Hence, the logic for manipulating graphics objects (things you want drawn) is separate from the rendering process itself. Your program tells the buffer which pixels you want to update, and that's all; this can be done as often as you like, regardless of whether the hardware actually manages to flush its buffers onto the screen.
The software would be responsible for sorting out what exactly to draw on the screen; this is usually handled on multiple logical levels. Higher levels would construct a virtual worldspace for your objects and determine their interactions and attributes (position, velocity, collision, etc.), as well as a camera to determine the FOV the screen should display (if your world is 3D). Lower levels would then figure out the actual pixel values to write to the buffer, based on the camera FOV (3D), or just plain pixel coordinates after applying the desired transformations (rotation, shear, resize, etc.) to the associated image (2D).
It should be noted that virtual worldspace coordinates do not necessarily reflect pixel coordinates, even in 2D worlds. I'm not an expert on this subject, frankly, but I suspect it'll be easier if you first determine how far you want the object to move in virtual space first, and then apply the necessary transformations to show the results in a viewing window with customizable dimensions.
In short, you probably don't want to 'add 1 to x' when you want to move something on screen; you move it in a high abstraction layer, and then draw the results. This will save you a lot of trouble, especially if you have a complex scene with all kinds of stuff and a background.
Assuming you want to move a group of pixels to the right, then yes, all you need to do is identify the group of pixels and add 1 to their X coordinate. Of course you need to fill in the vacated spots with zeroes, otherwise that would have been a copy operation.
Keep in mind, my answer is a bit naive in the sense that when you reach the rightmost boundary, you have to wrap.
I am working on a CAD type system using threejs. I have thin objects next to other objects (think thin 2mm metal sheeting fixed to posts on a building measured in metres). When I am zoomed in it all looks fine. The objects do not intersect at all. As I zoom out the objects get smaller and I end up with cases where the post object 'glimmers' (sort of shows through) the metal sheet object as I rotate it around.
I understand it's the small numbers I am working with that is causing this effect. However, is there a way to set a priority such that one object (the metal sheeting) is more important than another object (post) so it doesn't get that sort of effect?
To answer the question from the title, it is possible to prioritize drawing orders with.
myMesh.renderOrder = 5
myOtherMesh.renderOrder = 7
It is then possible to apply different depth effects, turn off the test etc.
Another way is to group objects with, layers. Set the appropriate layer mask on the camera and then render (multiple times).
myMesh.layers.set(5)
camera.layers.set(1)
renderer.render(scene,camera)
camera.layers.set(5)
renderer.render(scene,camera)
This is called z-fighting, where two fragments are so close in the given depth space that their z-values are within the margin of error that their true depths might get inverted.
The easiest way to resolve this is to reduce the scale of your depth buffer. This is controlled by the near and far properties on your camera. You'll need to play with the values to determine what works best for your senario. If you can minimize the distance between the planes, you'll have better luck avoiding z-fighting.
For example, if (as a loose estimate) the bounding sphere of your entire model has a diameter of 100, then the distance between near and far need only be 100. However, their values are set as the distance into camera space. So as you zoom out, and your camera moves further away, you should adjust the values to maintain the minimum distance between them. If your camera is at z = 100, then set near = 50 and far = 150. When you pull your camera back to z = 250, then update near = 200 and far = 300.
Another option is to use the WebGLRenderer.logarithmicDepthBuffer option. (example)
Edit: There is one other cause: the faces of the shapes are actually co-planar. If two triangles are occupying the same space, then you're all but guaranteeing z-fighting.
The simple solution is to move one of the components such that the faces are no longer co-planar. You could also potentially apply a polygonOffset to the sheet metal material, but your use-case doesn't sound like that is appropriate.
I'm using Three.js and Physijs. I have a wall that should act as a boundary, but objects (especially boxes) often pass through it, if the force is sufficient. The collision is detected, as they do not do so cleanly, but they start spinning or bounce in some direction. Is there a way to increase the maximum force with which the wall can act on the colliding object?
All four of the wall's points are on the same plane, forming a rectangle. The mesh consists of two large triangular faces. I'm using a ConvexMesh.
Breaking the two triangles into many smaller ones does not alleviate the problem.
I can confirm the normals are fine, as the wall is shaded correctly.
How can I solve this without converting the wall into a BoxMesh?
I'll also appreciate an explanation of why this happens. I'm guessing that the engine limits the maximum force that collisions can apply.
I think it's Motion Clamping
https://github.com/chandlerprall/Physijs/wiki/Collisions
When an object has a high velocity, collisions can be missed if it
moves through and past other objects between simulation steps. To fix
this, enable CCD motion clamping. For a cube of size 1 try:
// Enable CCD if the object moves more than 1 meter in one simulation
frame mesh.setCcdMotionThreshold(1);
// Set the radius of the embedded sphere such that it is smaller than
the object mesh.setCcdSweptSphereRadius(0.2);
Hope this works ima try it now
OpenGL Question:I have something to ask about clip space transformation. I am reading an online tutorial and it says that everything you draw outside the clip space will be clipped. When it come to this, does the elements outside the clip space affects the performance or not? Because it will not be drawn and thus it doesn't affect.
Assuming that it will affect performance and in case of 2d game like super mario, I am thinking about not to draw the elements outside the clip space to achieve better performance. Please clarify. Thanks.
OpenGL has only a certain amount of knowledge about your scene and will clip very late in the pipeline. It can't apply a broad phase test. Assuming you can, you should.
Supposing you had a model with 30,000 triangles, OpenGL would transform each and every one of those 30,000 triangles before considering clipping. If you know something as simple as the bounding sphere for the model it's possible you could see that the whole thing is completely outside of the frustum in a single test and save almost 30,000 extra bits of effort.
In a 2d game like Mario what this usually means is using the scroll position to index into the map and to generate geometry only for potentially visible tiles and sprites that are within the visible area.
For the map that will generally just men figuring out the (x, y) of one corner and then generating geometry for the known width and height of the screen so it means discarding the vast majority of the geometry with zero processing.
For the sprites, this is generally why in those sort of games you often see enemies reset to their starting position if you walk a little way from them and then walk back: they're added to the active list based on a map location trigger and removed when you walk far enough away. While not active, no mutable storage is afforded to them.
When several objects overlap on the same plane, they start to flicker. How do I tell the renderer to put one of the objects in front?
I tried to use .renderDepth, but it only works partly -
see example here: http://liveweave.com/ahTdFQ
Both boxes have the same size and it works as intended. I can change which of the boxes is visible by setting .renderDepth. But if one of the boxes is a bit smaller (say 40,50,50) the contacting layers are flickering and the render depth doesn't work anymore.
How to fix that issue?
When .renderDepth() doesn't work, you have to set the depths yourself.
Moving whole meshes around is indeed not really efficient.
What you are looking for are offsets bound to materials:
material.polygonOffset = true;
material.polygonOffsetFactor = -0.1;
should solve your issue. See update here: http://liveweave.com/syC0L4
Use negative factors to display and positive factors to hide.
Try for starters to reduce the far range on your camera. Try with 1000. Generally speaking, you shouldn't be having overlapping faces in your 3d scene, unless they are treated in a VERY specific way (look up the term 'decal textures'/'decals'). So basically, you have to create depth offsets, and perhaps even pre sort the objects when doing this, which all requires pretty low-level tinkering.
If the far range reduction helps, then you're experiencing a lack of precision (depending on the device). Also look up 'z fighting'
UPDATE
Don't overlap planes.
How do I tell the renderer to put one of the objects in front?
You put one object in front of the other :)
For example if you have a camera at 0,0,0 looking at an object at 0,0,10, if you want another object to be behind the first object put it at 0,0,11 it should work.
UPDATE2
What is z-buffering:
http://en.wikipedia.org/wiki/Z-buffering
http://msdn.microsoft.com/en-us/library/bb976071.aspx
Take note of "floating point in range of 0.0 - 1.0".
What is z-fighting:
http://en.wikipedia.org/wiki/Z-fighting
...have similar values in the z-buffer. It is particularly prevalent with
coplanar polygons, where two faces occupy essentially the same space,
with neither in front. Affected pixels are rendered with fragments
from one polygon or the other arbitrarily, in a manner determined by
the precision of the z-buffer.
"The renderer cannot reposition anything."
I think that this is completely untrue. The renderer can reposition everything, and probably does if it's not shadertoy, or some video filter or something. Every time you move your camera the renderer repositions everything (the camera is actually the only thing that DOES NOT MOVE).
It seems that you are missing some crucial concepts here, i'd start with this:
http://www.opengl-tutorial.org/beginners-tutorials/tutorial-3-matrices/
About the depth offset mentioned:
How this would work, say you want to draw a decal on a surface. You can 'draw' another mesh on this surface - by say, projecting a quad onto it. You want to draw a bullet hole over a concrete wall and end up with two coplanar surfaces - the wall, the bullet hole. You can figure out the depth buffer precision, find the smallest value, and then move the bullet hole mesh by that value towards the camera. The object does not get scaled (you're doing this in NDC which you can visualize as a cube and moving planes back and forth in the smallest possible increment), but does translate in depth direction, ending up in front of the other.
I don't see any flicker. The cube movement in 3D seems to be super-smooth. Can you try in a different computer (may be faster one)? I used Chrome on Macbook Pro.