I'm working on Augmented reality using Project Tango. After loading adf file I place a 3d object to each positions those are marked during area learning process, and it adds 3d objects
I expected 3d object to be static as those objects were placed exactly on particular place using Coordinate object, but those get misplaced and even oscillating when I move camera
What I found to be the reason is whenever Tango gets connected, current position is taken as origin (0,0,0) and objects get placed (after recognising loaded adf, of course) relative to this origin
Is there any other way to precisely place 3d objects and making them static?
First, the position of object is relative to ADF that being saved. In order to get the object restores to the original location, you will need to get relocalized. That requires: 1. loading the ADF. 2. walk around the area and relocalize the device.
Tango Service start as (0, 0, 0) for the device with respect to start of service pair. But once the device is relocalized, app will start to receive the device with respect to area description frame pair's callback.
If you are using Unity, Area Learning example is doing exactly the same thing you are looking for.
Probably if you are placing good the scene, is up to tangoCoordinateFramePairs: Normally if you are using an ADF file you should set :(in java)
tangoCoordinateFramePairs.add(new TangoCoordinateFramePair(
TangoPoseData.COORDINATE_FRAME_AREA_DESCRIPTION,
TangoPoseData.COORDINATE_FRAME_START_OF_SERVICE));
Then in your render has to place correctly the scene.
Related
My question builds up on this thread: Computer Vision / Augmented Reality: how to overlay 3D objects over vision? and on its first answer. I want to build an application that projects on real time the position of a fictional 3D object into a video feed, but the first step I have to take is: How can I do this over a single image?
What I am going for at the moment is having some kind of function that given a picture, its 6D pose (position + orientation), a 3D object (on fbx, 3ds, or something easily convertable to or from others), and its own position and orientation, returns me the projection of the 3D object over the image. Once I have that, I should be able to apply it over every frame of the video feed (how will I get the 6D information of the camera is a problem I'll deal with later)
My problem is that I am unsure where to find such a function, if it even exists. It should be offered like some kind of script or API so an external program can make use of it. Where should I look? Unity? Some kind of OpenCL functionality? So far my reading has not given me any conclusive answers, and as I am a novice in the topic, I'm sure a steep learning curve is ahead and I'd rather put my efforts on the right direction. Thank you
Indeed there's an API for that.
https://developer.vuforia.com
read the GetStarted page.
On this site, there is a "Target Manager", you'll want to upload your target images. Those will allow you to display the 3D object that you want.
On the same "page" you can have several target images.
Example : One that display your 3D object when visible, one that makes it rotates when hided. etc ...
For the real time projection video part, I will make the assumption that, on Unity, you can have a movie texture running on a plane in background and sort your layers in a way that your 3D object is above.
Please update the topic whenever you find a way.
Bye
So I know about setSurface, and have no problem using it as an overlay or whatever - its on a surfacecontrol. That said, I am stumped about getting pixel data
1) I've tried everything I can think of (the control, the root, etc) to use the drawing cache functions to get the bits for the camera surface. Yah, no. The cached bitmap is always zerod out.
2) I've used both SurfaceView and GLSurfaceView successfully as a setSurface taget. I cannot use any other class, such as TextureView.
3) I've investigated the C API and I see the camera exposes connectOnFrameAvailable, which will give me access to the pixels
My guess is that the internal tango logic is just using the surface in java to gain access to the underlying bit transfer channel - in the C API it requires a texture ID, which makes me suspect at the end of the day, the camera data is shipped in to the GPU pretty quickly, and I bet that CUDA lib operates on it - given the state of things, I can't see how to get the bits on the Java side without rooting the device - just cause I have a texture or simple surface view rendering raw bits on the screen doesn't mean I can get to them.
I don't want to peel the image data back out of the GPU. I'd need to switch my busy animation from a watch to a calendar for that.
Before I dive down into the C API, is there any way I can get the camera bits in Java ? I really want to be able to associate them with a specific pose, but right now I can't even figure out how to get them at all. I really want to know the location and color of a 3D point. Camera intrinsics, the point cloud, and the 2d image that generated the point cloud are all I need. But I can't do anything if I can't get the pixels, and the more questionable the relationship between an image and a (pose and a pointcloud) the sketchier any efforts will become.
If I do dive into C, will the connectOnFrameAvailable give me what I need ? How well synced is it with the point cloud generation ? Oh, and have I got this right ? Color camera is used for depth, fisheye is used for pose ?
Can I mix Java and C, i.e. create a Tango instance in Java and then just use C for the image issue ? Or am I going to have to re-realize everything in C and stop using the tango java jar ?
will the connectOnFrameAvailable give me what I need?
Yes, it indeed returns the YUV byte buffer.
How well synced is it with the point cloud generation?
The Tango API itself doesn't provide synchronization between the color image and depth point cloud, however, it does provide the timestampe which allow you to sync at the application level.
Color camera is used for depth, fisheye is used for pose?
Yes, you are right.
Can I mix Java and C (i.e. create a Tango instance in Java and then just use C for the image issue)
Starting two Tango instance is really not the way Tango supported, even though it works, it will be extremely hacky..
As the temp walk-around, you could probably try to use the drawing cache of the view?
I am writing a program in visual basic 2010 to create a 2D scrolling map. I am using Pictureboxes at the moment, all 50 by 70 in size. It starts with 1, and depending on what is needed may easily end up with 1000 - 2000 of them. They all need to be clickable. I am concerned it might use too many resources and run too slow. Can anyone tell me what the best approach to make something like would be.
Thankyou.
A common method is to only draw the tiles (or pictureboxes in your case) when they can be seen by the player's camera. So you should first determine how many boxes fill the screen and then calculate whether or not they are within the bounds of the player camera as represented by a rectangle. Additionally, you should draw boxes slightly outside the player's view (1 or 2 additional boxes per edge, this is explained below).
When the player moves, move the position of the world, not the player. So if the player moves left, scroll all of the tiles to the right while the player sprite remains stationary. Now when a player's position changes you should check again which tiles are visible in the player's screen. Since you are drawing additional rows slightly outside the view of the player, the edges will smoothly scroll in and will not 'pop' in as is a common problem when starting out.
Since you are only drawing tiles which are visible to the player, your game should run a lot more efficiently. It is OK to store this data in a 2D array in memory for now. In the future when you have huge maps, it may be a good idea to load sections of your map which are far outside of the player's view from disk to memory. You don't want to load in map data from disk which the player can navigate to before it is fully loaded.
Exactly how much map data to store in memory is up to you and depends on what platforms you want to release on and other constraints.
Additionally, you may want to look into different rendering libraries. Common low-level libraries are DirectX and OpenGL. These libraries work directly with your graphics card to dramatically speed up rendering. There are libraries built on top of those for various languages, but I don't know any for Visual Basic. An example for JavaScript is PixiJS. Additionally, there are full featured game creation libraries such as Unity or the Unreal Engine 3.
if i do a human model and import him to game engine. does game engine knows all point cordinates on model and rotates each ones? all models consists million points and and if i rotate a model 90 degree , does game engine calculates millions point new location and rotate? how does it works. Thanks
This is a bit of a vague question since each game engine will work differently, but in general the game engine will not touch the model coordinates.
Models are usually loaded with model space (or local space) coordinates - this simply means that each vertex is defined with a location relative to the origin of that model. The origin is defined as (0,0,0) and is the point around which rotations take place.
Now the game engine loads and keeps the model in this coordinate space. Then you provide your transformations (such as translation and rotation matrices) to place that model somewhere in your "world" (i.e. the global coordinate space shared by all objects). You also provide the way you want to view this world with various other transforms such projection and view matrices.
The game engine then takes all of these transformations and passes them to the GPU (or software renderer, in some cases) - it will also setup other stuff such as textures, etc. These are usually set once per frame (or per object for a frame).
Finally, it then passes each vertex that needs to be processed to the renderer. Each vertex is then transformed by the renderer using all the transformations specified to get a final vertex position - first in world space and then in screen space - which it can use to render pixels based on various other information (such as textures and lighting).
So the point is, in most cases, the engine really has nothing to do with the rotation of the model/vertices. It is simply a way to manage the model and the various settings that apply to it.
Of course, the engine can rotate the model and modify it's vertices, but this is usually only done during loading - for example if the model needs to be converted between different coordinate spaces.
There is a lot more going on, and this is a very basic description of what actually happens. There are many many sources that describe this process in great detail, so I won't even try to duplicate it. Hopefully this gives you enough detail to understand the basics.
I am still shiny new to XNA, so please forgive any stupid question and statements in this post (The added issue is that I am using Visual Studio 2010 with .Net 4.0 which also means very few examples exist out on the web - well, none that I could find easily):
I have two 2D objects in a "game" that I am using to learn more about XNA. I need to figure out when these two objects intersect.
I noticed that the Texture2D objects has a property named "Bounds" which in turn has a method named "Intersects" which takes a Rectangle (the other Texture2D.Bounds) as an argument.
However when you run the code, the objects always intersect even if they are on separate sides of the screen. When I step into the code, I noticed that for the Texture2D Bounds I get 4 parameters back when you mouse over the Bounds and the X, and Y coordinates always read "X = 0, Y = 0" for both objects (hence they always intersect).
The thing that confuses me is the fact that the Bounds property is on the Texture rather than on the Position (or Vector2) of the objects. I eventually created a little helper method that takes in the objects and there positions and then calculate whether they intersect, but I'm sure there must be a better way.
any suggestions, pointers would be much appreciated.
Gineer
The Bounds property was added to the Texture2D class to simplify working with Viewports. More here.
You shouldn't think of the texture as being the object itself, it's merely what holds the data that gets drawn to the screen, whether it's used for a Sprite or RenderTarget. The position of objects or sprites and how position/moving is handled is entirely up to you, so you have to track and handle this yourself. That includes the position of any bounds.
The 2D Rectangle Collision tutorial is a good start, as you've already found :)
I found the XNA Creator Club tutorials based on another post to stackoverflow by Ben S. The Collision Series 1: 2D Rectangle Collision tutorial explains it all.
It seems you have to create new rectangles, based on the original rectangles moving around in the game every time you try to run the intersection method, which then will contain the updated X and Y coordinates.
I am still not quite sure why the original object rectangles position can not just be kept up to date, but if this is the way it should work, that's good enough for me... for now. ;-)