A project I've been working on for the past few months is calculating the top area of an object taken with a 3D depth camera from top view.
workflow of my project:
capture a group of objects image(RGB,DEPTH data) from top-view
Instance Segmentation with RGB image
Calculate the real area of the segmented mask with DEPTH data
Some problem on the project:
All given objects have different shapes
The side of the object, not the top, begins to be seen as it moves to the outside of the image.
Because of this, the mask area to be segmented gradually increases.
As a result, the actual area of an object located outside the image is calculated to be larger than that of an object located in the center.
In the example image, object 1 is located in the middle of the angle, so only the top of the object is visible, but object 2 is located outside the angle, so part of the top is lost and the side is visible.
Because of this, the mask area to be segmented is larger for objects located on the periphery than for objects located in the center.
I only want to find the area of the top of an object.
example what I want image:
Is there a way to geometrically correct the area of an object located on outside of the image?
I tried to calibrate by multiplying the area calculated according to the angle formed by Vector 1 connecting the center point of the camera lens to the center point of the floor and Vector 2 connecting the center point of the lens to the center of gravity of the target object by a specific value.
However, I gave up because I couldn't logically explain how much correction was needed.
fig 3:
What I would do is convert your RGB and Depth image to 3D mesh (surface with bumps) using your camera settings (FOVs,focal length) something like this:
Align already captured rgb and depth images
and then project it onto ground plane (perpendicul to camera view direction in the middle of screen). To obtain ground plane simply take 3 3D positions of the ground p0,p1,p2 (forming triangle) and using cross product to compute the ground normal:
n = normalize(cross(p1-p0,p2-p1))
now you plane is defined by p0,n so just each 3D coordinate convert like this:
by simply adding normal vector (towards ground) multiplied by distance to ground, if I see it right something like this:
p' = p + n * dot(p-p0,n)
That should eliminate the problem with visible sides on edges of FOV however you should also take into account that by showing side some part of top is also hidden so to remedy that you might also find axis of symmetry, and use just half of top side (that is not hidden partially) and just multiply the measured half area by 2 ...
Accurate computation is virtually hopeless, because you don't see all sides.
Assuming your depth information is available as a range image, you can consider the points inside the segmentation mask of a single chicken, estimate the vertical direction at that point, rotate and project the points to obtain the silhouette.
But as a part of the surface is occluded, you may have to reconstruct it using symmetry.
There is no way to do this accurately for arbitrary objects, since there can be parts of the object that contribute to the "top area", but which the camera cannot see. Since the camera cannot see these parts, you can't tell how big they are.
Since all your objects are known to be chickens, though, you could get a pretty accurate estimate like this:
Use Principal Component Analysis to determine the orientation of each chicken.
Using many objects in many images, find a best-fit polynomial that estimates apparent chicken size by distance from the image center, and orientation relative to the distance vector.
For any given chicken, then, you can divide its apparent size by the estimated average apparent size for its distance and orientation, to get a normalized chicken size measurement.
I am trying to solve a question related to transformation of coordinates in 3-D space but not sure how to approach it.
Lets a vertex point named P is drawn at the origin with a 4x4 transformation matrix. It's then views through a camera that's positioned with a model view matrix and then through a simple projective transform matrix.
How do I calculate the new screen coordinates of P' (x,y,z)?
Before explain of pipeline, you need to know is how pipeline do process to draw on screen.
Everything between process is just matrix multiplication with vector
Model - World - Camera - Projection(or Nomalized Coordinate) - Screen
First step, we call it 'Model Space' because of (0,0,0) is based in model.
And we need to move model space to world space because of we are gonna place model to world so
we need to do transform will be (translate, rotation, scale)TRS * Model(Vector4) because definition of world transform will be different
After do it, model place in world.
Thrid, need to render on camrea space because what we see is through the camera. in world, camera also has position, viewport size and
rotation.. It needs to project from the camera. see
General Formula for Perspective Projection Matrix
After this job done, you will get nomalized coordinate which is Techinically 0-1 coordinates.
Finaly, Screen space. suppose that we are gonna make vido game for mobile. mobile has a lot of screen resolution. so how to get it done?
Simple, scale and translate to get result in screen space coordinate. Because of origin and screen size is different.
So what you are tring to do is 'step 4'.
If you want to get screen position P1 from world, formula will be "Screen Matrix * projection matrix * Camera matrix * P1"
If you want to get position from camera space it would be "Screen Matrix * projection matrix * P1".
There are useful links to understand matrix and calculation so see above links.
https://answers.unity.com/questions/1359718/what-do-the-values-in-the-matrix4x4-for-cameraproj.html
https://www.google.com/search?q=unity+camera+to+screen+matrix&newwindow=1&rlz=1C5CHFA_enKR857KR857&source=lnms&tbm=isch&sa=X&ved=0ahUKEwjk5qfQ18DlAhUXfnAKHabECRUQ_AUIEigB&biw=1905&bih=744#imgrc=Y8AkoYg3wS4PeM:
Given an EPSG projection (say, this Alabama one: [http://spatialreference.org/ref/epsg/26729/][1])
How can you take the given WGS84 projection bounds in such a way that you can use them in a D3.js projection.
For example, how would you know what projection, degree of rotation or bounding box to use to show the map?
This is a fairly complex question. The answer will differ based on the spatial reference (SRS, or coordinate reference system(CRS)) system you are looking at and what your ultimate goal is.
I am using d3.js v4 in this answer
Short Answer:
For example, how would you know what projection, degree of rotation or
bounding box to use to show the map?
There is no hard and fast set of rules that encompasses all projections. Looking at the projection parameters can usually give you enough information to create a projection quickly - assuming the projection comes out of the box in d3.
The best advice I can give on setting the parameters, as when to rotate or when to center, what parallels to use etc, is to zoom way out when refining the projection so you can see what each parameter is doing and where you are looking. Then do your scaling or extent fitting. That and use a geojson validator for your bounding box, like this one.
Lastly, you could always use projected data and drop d3.geoProjection altogether (this question), if all your data is already projected in the same projection, trying to define the projection is a moot point.
Datums
I'll note quickly that the question could be complicated further if you look at differences between datums. For example, the SRS you have referenced used the NAD27 datum. A datum is a mathematical representation of the earth's shape, NAD27 will differ from NAD83 or WGS84, though all are measured in degrees, as the datum represents the three dimensional surface of the earth. If you are mixing data that uses conflicting datums, you could have some precision issues, for example the datum shift between NAD27 and NAD83 is not insignificant depending on your needs (wikipedia screenshot, couldn't link to image):
If shifts in locations due to use of multiple datums is a problem, you'll need more than d3 to convert them into one standard datum. D3 assumes you'll be using WGS84, the datum used by the GPS system. If these shifts are not a problem, then ignore this part of the answer.
The Example Projection
So, let's look at your projection, EPSG:26729:
PROJCS["NAD27 / Alabama East",
GEOGCS["NAD27",
DATUM["North_American_Datum_1927",
SPHEROID["Clarke 1866",6378206.4,294.9786982138982,
AUTHORITY["EPSG","7008"]],
AUTHORITY["EPSG","6267"]],
PRIMEM["Greenwich",0,
AUTHORITY["EPSG","8901"]],
UNIT["degree",0.01745329251994328,
AUTHORITY["EPSG","9122"]],
AUTHORITY["EPSG","4267"]],
UNIT["US survey foot",0.3048006096012192,
AUTHORITY["EPSG","9003"]],
PROJECTION["Transverse_Mercator"],
PARAMETER["latitude_of_origin",30.5],
PARAMETER["central_meridian",-85.83333333333333],
PARAMETER["scale_factor",0.99996],
PARAMETER["false_easting",500000],
PARAMETER["false_northing",0],
AUTHORITY["EPSG","26729"],
AXIS["X",EAST],
AXIS["Y",NORTH]]
This is a pretty standard description of a projection. Each type of projection will have parameters that are specific to it, so these won't always be the same.
The most important parts of this description are:
NAD27 / Alabama East Projection name, not needed but a good reference as it's a little easier to remember than an EPSG number, and references/tools may only use a common name instead of an EPSG number.
PROJECTION["Transverse_Mercator"] The type of projection we are dealing with. This defines how the 3d coordinates representing points on the surface of the earth are translated to 2d coordinates on a cartesian plane. If you see a projection here that is not listed on the d3 list of supported projections (v3 - v4), then you have a bit of work to do in defining a custom projection. But, generally, you will find a projection that matches this. The type of projection changes whether a map is rotated or centered on each axis.
PARAMETER["latitude_of_origin",30.5],
PARAMETER["central_meridian",-85.83333333333333],
These two parameters set the center of the projection. For a transverse Mercator, only the central meridian is important. See this demo of the effect of choosing a central meridian on a transverse Mercator.
The latitude of origin is chiefly used to set the a reference point for the northnigs. The central meridian does this as well for the eastings, but as noted above, sets the central meridian in which distortion is minimized from pole to pole (it is equivalent to the equator on a regular Mercator). If you really need to have proper northings and eastings so that you can compare x,y locations from a paper map and a web map sharing the same projection, d3 is probably not the best vehicle for this. If you don't care about measuring the coordinates in Cartesian coordinate space, these parameters do not matter: D3 is not replicating the coordinate system of the projection (measured in feet as false eastings/northings) but is replicating the same shape in SVG coordinate space.
So based on the relevant parameters in the projection description, a d3.geoProjection centered on the origin of this projection would look like:
d3.geoTransverseMercator()
.rotate([85.8333,0])
.center([0,30.5])
Why did I rotate roughly 86 degrees? This is how a transverse Mercator is built. In the demo of a transverse Mercator, the map is rotated along the x axis. Centering on the x axis will simply pan the map left and right and not change the nature of the projection. In the demo it is clear the projection is undergoing a change fundamentally different than panning, this is the rotation being applied. The rotation I used is negative as I turn the earth under the projection. So this projection is centered at -85.833 degrees or 85.8333 degrees West.
Since on a Transverse Mercator, distortion is consistent along a meridian, we can pan up down and not need to rotate. This is why I use center on the y axis (in this case and in others, you could also rotate on the y axis, with a negative y, as this will spin the cylindrical projection underneath the map, giving the same result as panning).
If we are zoomed out a fair bit, this is what the projection looks like:
It may look pretty distorted, but it is only intended to show the area in and near Alabama. Zooming in it starts to look a lot more normal:
The next question is naturally: What about scale? Well this will differ based on the size of your viewport and the area you want to show. And, your projection does not specify any bounds. I'll touch on bounds at the end of the answer, if you want to show the extent of a map projection. Even if the projection has bounds, they may very well not align with the area you want to show (which is usually a subset of the overall projection bounds).
What about centering elsewhere? Say you want to show only a town that doesn't happen to lie at the center of the projection? Well, we can use center. Because we rotated the earth on the x axis, any centering is relative to the central meridian. Centering to [1,30.5], will center the map 1 degree East of the central meridian (85.8333 degrees West). So the x component will be relative to the rotation, the y component will be in relation to the equator - its latitude).
If adhering to the projection is important, this odd centering behavior is needed, if not, it might be easier to simply modify the x rotation so that you have a projection that looks like:
d3.geoTransverseMercator()
.center([0,y])
.rotate([-x,0])
...
This will be customizing the transverse Mercator to be optimized for your specific area, but comes at the cost of departing from your starting projection.
Different Projections Types
Different projections may have different parameters. For example, conical projections can have one (tangent) or two (secant) lines, these represent the points where the projection intersects the earth (and thus where distortion is minimized). These projections (such as an Albers or Lambert Conformal) use a similar method for centering (rotate -x, center y) but have the additional parameter to specify the parallels that represent the tangent or secant lines:
d3.geoAlbers()
.rotate([-x,0])
.center([0,y])
.parallels([a,b])
See this answer on how to rotate/center an Albers (which is essentially the same for all conical projections that come to mind at the moment).
A planar/azimuthal projeciton (which I haven't checked) is likely to be centered only. But, each map projection may have a slightly different method in 'centering' it (usually a combination of .rotate and .center).
There are lots of examples and SO questions on how to set different projection types/families, and these should help for most specific projections.
Bounding Boxes
However, you may have a projection that specifies a bounds. Or more likely, an image with a bounds and a projection. In this event, you will need to specify those bounds. This is most easily done with a geojson feature using the .fitExtent method of a d3.geoProjection():
projection.fitExtent(extent, object):
Sets the projection’s scale and translate to fit the specified GeoJSON object in the center of the given extent. The extent is specified as an array [[x₀, y₀], [x₁, y₁]], where x₀ is the left side of the bounding box, y₀ is the top, x₁ is the right and y₁ is the bottom. Returns the projection.
(see also this question/answer)
I'll use the example in the question here to demonstrate the use of a bounding box to help define a projection. The goal will be to project the map below with the following knowledge: its projection and its bounding box (I had it handy, and couldn't find a good example with a defined bounding box quick enough):
Before we get to the bounding box coordinates however, let's take a look at the projection. In this case it is something like:
PROJCS["ETRS89 / Austria Lambert",
GEOGCS["ETRS89",
DATUM["European_Terrestrial_Reference_System_1989",
SPHEROID["GRS 1980",6378137,298.257222101,
AUTHORITY["EPSG","7019"]],
AUTHORITY["EPSG","6258"]],
PRIMEM["Greenwich",0,
AUTHORITY["EPSG","8901"]],
UNIT["degree",0.01745329251994328,
AUTHORITY["EPSG","9122"]],
AUTHORITY["EPSG","4258"]],
UNIT["metre",1,
AUTHORITY["EPSG","9001"]],
PROJECTION["Lambert_Conformal_Conic_2SP"],
PARAMETER["standard_parallel_1",49],
PARAMETER["standard_parallel_2",46],
PARAMETER["latitude_of_origin",47.5],
PARAMETER["central_meridian",13.33333333333333],
PARAMETER["false_easting",400000],
PARAMETER["false_northing",400000],
AUTHORITY["EPSG","3416"],
AXIS["Y",EAST],
AXIS["X",NORTH]]
As we will be letting d3 choose the scale and center point based on the bounding box, we only care about a few parameters:
PARAMETER["standard_parallel_1",49],
PARAMETER["standard_parallel_2",46],
These are the two secant lines, where the map projection intercepts the surface of the earth.
PARAMETER["central_meridian",13.33333333333333],
This is the central meridian, the number we will use for rotating the projection along the x axis (as one will do for all conical projections that come to mind).
And most importantly:
PROJECTION["Lambert_Conformal_Conic_2SP"],
This line gives us our projection family/type.
Altogether this gives us something like:
d3.geoConicConformal()
.rotate([-13.33333,0]
.parallels([46,49])
Now, the bounding box, which is defined by these limits:
East: 17.2 degrees
West: 9.3 degrees
North: 49.2 degrees
South: 46.0 degrees
The .fitExtent (and .fitSize) methods take a geojson object and translate and scale the projection appropriately. I'll use .fitSize here as it skips margins around the bounds (fitExtent allows provision of margins, that's the only difference). So we need to create a geojson object with those bounds:
var bbox = {
"type": "Polygon",
"coordinates": [
[
[9.3, 49.2], [17.2, 49.2], [17.2, 46], [9.3, 46], [9.3,49.2]
]
]
}
Remember to use the right hand rule, and to have your end point the same as your start point (endless grief otherwise).
Now all we have to do is call this method and we'll have our projection. Since I'm using an image to validate my projection parameters, I know the aspect ratio I want. If you don't know the aspect ratio, you may have some excess width or height. This gives me something like:
var projection = d3.geoConicConformal()
.parallels([46,49])
.rotate([-13.333,0])
.fitSize([width,height],bbox)
And a happy looking final product like (keeping in mind a heavily downsampled world topojson):
Here is the deal, I'm programming a 2D framework/game engine with opengl ES. I am using VBOs and an ortho projection to draw an arrangement of sprites throughout the screen (as part of the testing), and everything was going nice and smooth until I had to play with translations and rotations. The specific problem I am having is that when I apply a translation with glTranslatef() prior to the rotation, the function does not only move the sprite, but also my origin, messing up my whole transformation. I am 100% sure it is working this way, because I used glTranslatef() to move to the right and bottom the sprite half of the size of the screen (yes, my origin is in the top left) and then apply a constant rotation and the thing just keeps mooving in a circular path around the center of the screen (actually rotating, but not as I expect.
If you want some code, here we go:
gl.glTranslatef(-(x+width/2), -(y+height/2), -layer);
gl.glRotatef(angle, 0.0f, 0.0f, -1.0f);
gl.glTranslatef(x+width/2, y+height/2, layer);
In this fragment of code, x and y are the position of the sprite, height and width are the size of the sprite, angle the angle of rotation, and layer just a form of organizing the sprites into several layers, pretty straight forward, right?
Again, my problem is that glTranslatef(); is moving both, the sprite and the origin, am I doing something wrong or misunderstanding something about the translation?
Thanks in advance.
you might need to use glPushMatrix and glPopMatrix since anything you do after those translations and rotations will be affected by them
but what you are describing is actually how it works, if you use a translate, that sort of becomes your new origin because once you do a translate, everything after that is affected by that translate, thats why you need to push and pop, so that you can go, push -> translate object and/or rotate -> pop, and then you can go about with whatever other translations you need to do without having that previous translation affecting everything else
its a bit confusing at first but google around and you'll see how to use them properly
http://www.khronos.org/opengles/sdk/1.1/docs/man/glPushMatrix.xml
I think you misunderstood how matrices work in openGL. When you do a matrix operation such as glRotatef and glTranslatef the matrices are being multiplied, resulting in affecting the base vectors.. For instance, let's say we are only drawing a point that starts at (0,0,0). If you call translate(1,0,0) the point will be in (1,0,0), after that you call rotate(90, 0, 0, 1) and your point will be on the same place as before but rotated. Now the last call is translate(-1,0,0) and your point is at (1,-1,0) (and not where you started)!
And that is what you did in your "fragment of code". The thing is you did not specify what you really want to do and how do you define your verices is relative as well.. If you want something like a view with some image that you want to control in sense of changing the position and rotation, you might want to create a square vertex buffer with values from -1 to 1 in both dimensions (or (-width/2, -height/2) to (width/2, height/2)). In this case the base center of your object is in (0,0,0) and that is probably the point you want to rotate it around (or am I wrong here?). So when you want to define the position of the object with origin point, you will need to write translatef(x+width/2,y+height/2,..).
As for the whole process of drawing in this case: If you want the origin to be at (x,y,z), with a (width, height) and rotated by (angle) here is the sequence
glTranslatef(x,y,z)
glTranslatef(width/2,height/2,0)
glScalef(width/2,height/2, 1) //only if verices defined at (-1,1)
glRotatef(angle, 0, 0, 1)
Do note in this case that since you rotate the object around its center its origin will not be at (x,y,z) anymore.
In general I would suggest to stay away from glRotate, glTranslate and glScale if possible. They tend to make things very nasty. So another way is to construct a matrix directly from base vectors: With little math you can compute all 4 points of your "square view" based on parameters such as origin, width, height and rotation.. The 4 points being (A-origin), (B-lower left point), (C-lower right point), (D-upper right point) your base vectors are (B-A), (D-A) and normalized(dotProduct((B-A), (D-A))) this 3 vectors can be inserted int top left 3x3 matrix of the GL matrix (witch is 4x4 or float[16]) and they represent both, rotation and scale so all you need to add is the translation part (just google around a bit for this approach).
I'm a fresh in cocos3d, now I have a problem.
In cocos3d, I want to rotate a node. I got the angles in x axis, y axis, z axis, then I used the property:rotation to rotate, like this:
theNodeToBeRotated.rotation = cc3v(x,y,z);
But I found out it didn't rotate as I expected, because the document said the rotate order is y-x-z.
I want to change the order to x-y-z. Can anyone let me know how?
You might need to clarify further regarding the following: "it didn't rotate as I expected"
OpenGL ES (and ergo, cocos3D) uses the y-axis as up so the rotation order is still x-y-z. If you are importing a model, you then need to take into account the 3D editor's co-ordinate system and adapt accordingly.
If you are not used to working with three-dimensional representations, the leap from 2D to 3D can be a significant hurdle. Within Cocos3D:
the x-axis is positive on the right and negative on the left
the y-axis is positive upwards and negative downwards
the z-axis is positive moving towards you and negative moving away from you
Envisage those three lines of axis, or even better, a piece of string.
If you are rotating around the x-axis, hold the string horizontally from left to right: the object would rotating towards you or away from you.
If you are rotating around the y-axis, hold the string vertically from feet to head: the object would rotate as if like a revolving door.
If you are rotating around the z-axis, hold one end close to your chest and the other end as far away as possible: the object would rotate similar to a clock face.
-- Update
I heavily wouldn't recommend changing the rotation order as it is the OpenGL standard to use Y-X-Z. If you wish to modify it, take a look at CC3GLMatrixMath and look for kmMat4RotationYXZ - there is also kmMat4RotationZYX. If you want to have X-Y-Z, you would need to construct your own rotation matrix and update accordingly in CC3GLMatrix and CC3GLMatrixMath.
As a reference, you also have the OpenGL Red book - it should have some suggestions for you.