I recently came across a product called Kolibree on kickstarted, which is a smart toothbrush. From what they say on their website, it seems that Kolibree can detect each tooth. I have some exposure to gesture recognition and flight dynamics (roll angle, pitch angle, heading angle, ...) the technologies I believe need be used in this product, but I'm confused how it can accurately detect EACH tooth ? I think we can detect the left, right, up and down region using roll and pitch angle, maybe a little more precisely by using the heading angle. but accurate to each tooth is beyond my understanding. Could someone shed light on this ?
thanks,
Ted
from the kickstarter video it has:
Accelerometers
Gyroscopes
Magnetometers
These provide relative position and absolute direction of the device
So how to detect tooths? I would start with this:
tooth shape
by brushing you can collect surface data of close proximity to brush
but only when no significant surface movement is detected then
this can differentiate tooth types by curvature shape/size
so you have an idea in what part of jaw you are
vibrations
spinning brush creates noise pulses in accelerometer readings
these should be dependent on the movement and surface shape
when linear movement is detected (you move brush from side to side)
then the gaps between tooths will create measurable readings in acceleration
this can be used to recognize relative tooth position
angular constraints
when we brush teeth on the left/right side or up down of the mouth
we hold the brush differently
this can be also measured
if overall angular position is within certain borders
then we can assume which side of mouth are actually brushing
when you put all these data together
then we can improve the accuracy of tooth scan to better numbers
also if some kind of calibration is used that can improve it more
for example hold/click some button to start calibration
and move around the mouth by specific calibration movement ...
[notes]
some things that have to be taken in mind
left/right handed people hold the brush differently
this goes also for motoric dis-functions (disabled people)
missing or curved tooth anomalies (can be later used as mark point)
my guess is by adding camera info (for example from the linked device)
for head/jaw position detection can improve detection even more
Related
I would like to make a game where I use a camera with infrared tracking, so that I can track peoples heads (from top view). For example each player will get a helmet so that the camera or infrared sensor can track him/her.
After that I need to know the exact positions of that person in unity, to place a 3D gameobject at the players position.
Maybe there is another workaround to get peoples positions in unity. I know I could use a kinect, but I need to track at least 10 people at the same time.
Thanks
Note: This is not really a closed answer, just a collection of my thoughts regarding your question on how to transfer recorded positions into unity.
If you really need full 3D positions, I believe you won't be happy when using only one sensor. In order to obtain depth information, which can further be used to calculate 3D positions in a reference coordinate system, you would have to use at least 2 sensors.
Another thing you could do is fixing the camera position and assuming, that all persons are moving in the same plane (e.g. fixed y-component), which would allow you to determine 3D positions utilizing the projection formula given the camera parameters (so camera has to be calibrated).
What also comes to my mind is: You could try to simulate your real camera with a virtual camera in unity. This way you can use the virtual camera to project image coordinates (coming from the real camera) into unity's 3D world. I haven't tried this myself, but there was someone who tried it, you can have a look at that: https://community.unity.com/t5/Editor/How-to-simulate-Unity-Pinhole-Camera-from-its-intrinsic/td-p/1922835
Edit given your comment:
Okay, sticking to your soccer example, you could proceed as follows:
Setup: Say you define your playing area to be rectangular with its origin in the bottom left corner (think of UVs). You set these points in the real world (and in unitys representation of it) as (0,0) (bottom left) and (width, height) (top right), choosing whichever measure you like (e.g. meters, as this is unitys default unit). As your camera is stationary, you can assign the corresponding corner points in image coordinates (pixel coordinates) as well. To make things easier, work with normalized coordinates instead of pixels, thus bottom left is (0,0) ans top right is (1,1).
Tracking: When tracking persons in the image, you can calculate their normalized position (x,y) (with x and y in [0,1]). These normalized positions can be transferred into unitys 3D space (in unity you will have a playable area of the same width and height) by simply calculating a Vector3 as (x*widht, 0, y*height) (in unity x is pointing right, y is pointing up and z is pointing forward).
Edit on Tracking:
For top-view tracking in a game, I would say you are on the right track with using some sort of helmet, which enables you to use some sort of marker based tracking (in my opinion markerless multi-target tracking is not reliable enough for use in a video game) (if you want learn more about object tracking, there are lots of resources in the field of computer vision).
Independent of the sensor you are using (IR or camera), you would go create some unique marker for each helmet, thus enabling you to identify each helmet (and also the player). A marker in that case is some sort of unique pattern, that can be recognized by an algorithm for each recorded frame. In IR you can arrange quadratic IR markers to form a specific pattern and for normal cameras you can use markers like QR codes (there are also libraries for augmented reality related content, that offer functionality for creating and recognizing markers, e.g. ArUco or ARToolkit, although I don't know if they offer C# libraries, I have only used ArUco with c++ a while ago).
When you have your markers of choice, the tracking procedure is then pretty straightforward, for each recorded image:
- detect all markers in the current image (these correspond to all players currently visible)
- follow the steps from my last edit using the detected positions
I hope that helps, feel free to contact me again.
A quick introduction:
We're developing a positioning system that works the following way. Our camera is situated on a robot and is pointed upwards (looking at the ceiling). On the ceiling we have something like landmarks, thanks to whom we can compute the position of the robot. It looks like this:
Our problem:
The camera is tilted a bit (0-4 degrees I think), because the surface of the robot is not perfectly even. That means, when the robot turns around but stays at the same coordinates, the camera looks at a different position on the ceiling and therefore our positioning program yields a different position of the robot, even though it only turned around and wasn't moved a bit.
Our current (hardcoded) solution:
We've taken some test photos from the camera, turning it around the lens axis. From the pictures we've deduced that it's tilted ca. 4 degrees in the "up direction" of the picture. Using some simple geometrical transformations we've managed to reduce the tilt effect and find the real camera position. On the following pictures the grey dot marks the center of the picture, the black dot is the real place on the ceiling under which the camera is situated. The black dot was transformed from the grey dot (its position was computed correcting the grey dot position). As you can easily notice, the grey dots form a circle on the ceiling and the black dot is the center of this circle.
The problem with our solution:
Our approach is completely unportable. If we moved the camera to a new robot, the angle and direction of tilt would have to be completely recalibrated. Therefore we wanted to leave the calibration phase to the user, that would demand takings some pictures, assessing the tilt parameters by him and then setting them in the program. My question to you is: can you think of any better (more automatic) solution to computing the tilt parameters or correcting the tilt on the pictures?
Nice work. To have an automatic calibration is a nice challenge.
An idea would be to use the parallel lines from the roof tiles:
If the camera is perfectly level, then all lines will be parallel in the picture too.
If the camera is tilted, then all lines will be secant (they intersect in the vanishing point).
Now, this is probably very hard to implement. With the camera you're using, distortion needs to be corrected first so that lines are indeed straight.
Your practical approach is probably simpler and more robust. As you describe it, it seems it can be automated to become user friendly. Make the robot turn on itself and identify pragmatically which point remains at the same place in the picture.
I am developing a 3D game for Windows Phone that includes terrains and volcanoes at infinite distance similar to Battle Zone (1980) by Atari Inc. The player can never touch the terrains no matter how far player drives. Currently, to implement this I am mapping a 2D texture inside the wall of cylinder. The cylinder is also moving with the player so that the player can never reach terrains. I am not sure whether this is a good method to implement terrains as I am facing problems like distortion of texture when mapping it on the wall of cylinder.
Please suggest me methods to implement a view of terrains in XNA similar to Battle Zone?
normally instead of cylinder developers use box (so-called SkyBox)
It has less polygons and in general less distortion (could be some at edges)
To make it look more real some devs like Valve use off-screen render in first pass that include skybox + some distant models with low details and moving cloud sprites or textured ring with alpha. Both points of view are synchronised (main camera and off-screen camera) then (without clearing colour buffer) they render final scene on top. Thanks to that far building will move a bit and scene surrounding will look less plain. To avoid z-buffer cleaning between passes they simply doing first pass under the floor(literally) of the scene of main pass.
I want to implement a physical raytracer (i.e. with actual photons with a given wavelength), restricting myself to small scenes (like two spheres and an enclosing box), to do experiments. It's not meant to be fast but I'll optimize it later.
I'm currently gathering all I know about how photons interact with surfaces, i.e. they either reflect (get absorbed, then emitted again) or refract with a probability based on the surface's absorption spectrum and reflectivity/refractivity indices, and refraction is dependent on the wavelength (which naturally results in dispersion) etc...
I understand how shooting photons out of emissive materials (like "lights") and making them bounce around the scene until they happen to land into the camera produces an accurate result, but is unacceptably slow, thus the need to do it backwards (shoot photons from the camera)
But I'm having trouble understanding how surface interactions can be modelled "backwards" - for instance, if a photon coming from the camera hits the side of a red box, if the photon has a wavelength corresponding to red, it will be reflected, and all other wavelengths will be absorbed, which will produce a red color. But is the intensity of the color decided by taking many samples of very close photons, and checking which of them eventually collide with a light, and which don't? Because ultimately, either a photon hits a light or it doesn't (after a given number of bounces) - there is no notion of partial collision.
So basically my question is - is the intensity of the light received by a pixel a function of the number of photon samples for that pixel that actually make it to a light source, or is there something else involved?
It sounds like you want to do something called http://en.wikipedia.org/wiki/Path_tracing which is like raytracing, except it does not directly sample light sources when a direct ray from the camera hits a surface (causing it to be quite slow, but not as slow as shooting rays "forwards" from the light sources).
However you seem to confuse yourself by thinking of "reverse photons" coming from the camera which you assume to already have the properties ("the photon has a wavelength corresponding to red") you are actually trying to decide in the first place. To wrap your mind around this, you might want to read up on "regular" raytracing first. So think of rays from the camera that bounce through a scene up to a certain bounce depth or until they hit an object, at which point they directly sample light sources to see if they illuminate the object.
About your final question "Is the intensity of the light received by a pixel a function of the number of photon samples for that pixel that actually make it to a light source, or is there something else involved?" I'll refer you to http://en.wikipedia.org/wiki/Rendering_equation where you will find the rendering equation (the general mathematical problem all 3D graphics algorithms like raytracing try to solve) and a list with its limitations, which answers your question in the negative (i.e. other than the light source these effects are also involved in deciding the ultimate colour and intensity of a pixel):
phosphorescence, which occurs when light is absorbed at one moment in time and emitted at a different time,
fluorescence, where the absorbed and emitted light have different wavelengths,
interference, where the wave properties of light are exhibited, and
subsurface scattering, where the spatial locations for incoming and departing light are different. Surfaces rendered without accounting for subsurface scattering may appear unnaturally opaque.
I'd like to implement a dragging feature where users can drag objects around the workspace. That of course is the easy bit. The hard bit is to try and make it a physically correct drag which incorporates rotation due to torque moments (imagine dragging a book around on a table using only one finger, how does it rotate as you drag?).
Does anyone know where I can find explanations on how to code this (2D only, rectangles only, no friction required)?
Much obliged,
David
EDIT:
I wrote a small app (with clearly erroneous behaviour) that I hope will convey what I'm looking for much better than words could. C# (VS 2008) source and compiled exe here
EDIT 2:
Adjusted the example project to give acceptable behaviour. New source (and compiled exe) is available here. Written in C# 2008. I provide this code free of any copyright, feel free to use/modify/whatever. No need to inform me or mention me.
Torque is just the applied force projected perpendicular to a vector between the point where the force is applied and the centroid of the object. So, if you pull perpendicular to the diameter, the torque is equal to the applied force. If you pull directly away from the centroid, the torque is zero.
You'd typically want to do this by modeling a spring connecting the original mouse-down point to the current position of the mouse (in object-local coordinates). Using a spring and some friction smooths out the motions of the mouse a bit.
I've heard good things about Chipmunk as a 2D physics package:
http://code.google.com/p/chipmunk-physics/
Okay, It's getting late, and I need to sleep. But here are some starting points. You can either do all the calculations in one coordinate space, or you can define a coordinate space per object. In most animation systems, people use coordinate spaces per object, and use transformation matrices to convert, because it makes the math easier.
The basic sequence of calculations is:
On mouse-down, you do your hit-test,
and store the coordinates of the
event (in the object coordinate
space).
When the mouse moves, you create a
vector representing the distance
moved.
The force exterted by the spring is k * M, where M is the amount of distance between that initial mouse-down point from step 1, and the current mouse position. k is the spring constant of the spring.
Project that vector onto two direction vectors, starting from the initial mouse-down point. One direction is towards the center of the object, the other is 90 degrees from that.
The force projected towards the center of the object will move it towards the mouse cursor, and the other force is the torque around the axis. How much the object accelerates is dependent on its mass, and the rotational acceleration is dependent on angular momentum.
The friction and viscosity of the medium the object is moving in causes drag, which simply reduces the motion of the object over time.
Or, maybe you just want to fake it. In that case, just store the (x,y) location of the rectangle, and its current rotation, phi. Then, do this:
Capture the mouse-down location in world coordinates
When the mouse moves, move the box according to the change in mouse position
Calculate the angle between the mouse and the center of the object (atan2 is useful here), and between the center of the object and the initial mouse-down point. Add the difference between the two angles to the rotation of the rectangle.
This would seem to be a basic physics problem.
You would need to know where the click, and that will tell you if they are pushing or pulling, so, though you are doing this in 2D, your calculations will need to be in 3D, and your awareness of where they clicked will be in 3D.
Each item will have properties, such as mass, and perhaps information for air resistance, since the air will help to provide the motion.
You will also need to react differently based on how fast the user is moving the mouse.
So, they may be able to move the 2 ton weight faster than is possible, and you will just need to adapt to that, as the user will not be happy if the object being dragged is slower than the mouse pointer.
Which language?
Here's a bunch of 2d transforms in C