Transforming Kinect coordinates to character model - animation

I have the latest Kinect SDK, XNA 4.0, and a model with a skeleton and I'm looking into working with the kinect to create some simple games. I heard that various engines, such as the DigitalRune engine provide an easy way to map the skeleton generated from the Kinect sensor to the skeleton of the model.
Which engine is the best for this task and why? I'd prefer if you could list both the best free engine and the one you have to pay for, as well as any additional suggestion, should you have them.
I really need help with this. While I did manage to find about five engines that might suit my needs (Ploobs, Engine Nine, DigitalRune, Hilva, Xen, XNA Final Engine) I have no way of telling which is better than the other, as there is very little activity in the related forums and I really don't have the time (or, more importantly, the knowledge) to test all these engines.

I think XNA is better, because it gives more transparency when it comes to game programming, there is no doubt it is the most used for game programming and as a plus point you will have tons of tutorials. xna framework will be best even you can have e-books too.

Related

How a marker-based augmented reality algorithm (like ARToolkit's one) works?

For my job i've been using a Java version of ARToolkit (NyARTookit). So far it proven good enough for our needs, but my boss is starting to want the framework ported in other platforms such as web (Flash, etc) and mobiles. While i suppose i could use other ports, i'm increasingly annoyed by not knowing how the kit works and beyond that, from some limitations. Later i'll also need to extend the kit's abilities to add stuff like interaction (virtual buttons on cards, etc), which as far as i've seen in NyARToolkit aren't supported.
So basically, i need to replace ARToolkit with a custom mark detector (and in case of NyARToolkit, try to get rid of JMF and use a better solution via JNI). However i don't know how these detectors work. I know about 3D graphics and i've built a nice framework around it, but i need to know how to build the underlying tech :-).
Does anyone know any sources about how to implement a marker-based augmented reality application from scratch? When searching in google i only find "applications" of AR, not the underlying algorithms :-/.
'From scratch' is a relative term. Truly doing it from scratch, without using any pre-existing vision code, would be very painful and you wouldn't do a better job of it than the entire computer vision community.
However, if you want to do AR with existing vision code, this is more reasonable. The essential sub-tasks are:
Find the markers in your image or video.
Make sure they are the ones you want.
Figure out how they are oriented relative to the camera.
The first task is keypoint localization. Techniques for this include SIFT keypoint detection, the Harris corner detector, and others. Some of these have open source implementations - i think OpenCV has the Harris corner detector in the function GoodFeaturesToTrack.
The second task is making region descriptors. Techniques for this include SIFT descriptors, HOG descriptors, and many many others. There should be an open-source implementation of one of these somewhere.
The third task is also done by keypoint localizers. Ideally you want an affine transformation, since this will tell you how the marker is sitting in 3-space. The Harris affine detector should work for this. For more details go here: http://en.wikipedia.org/wiki/Harris_affine_region_detector

When should I break into GUI/game development?

I am a hobbyist console C++ developer. I have worked with pointers, arrays, std::vectors, std::strings, classes, and several data structures, including stacks and binary trees. I have some experience in linear algebra and geometry, and know the basics of physics. I do NOT have experience with win32, QT, openGL, DX9, OGRE, etc. I am still learning about the more valuable parts of OOP, like polymorphism.
I started C++ as a first language, and do not have experience with other languages. I could probably work with C, but I'd need to get used to manipulating char*'s and regular arrays (and not initing variables).
My question is, with my experience, when should I break into the development of GUI applications/game applications? Do I need to ground myself more firmly in certain areas of math, become comfortable with win32, get used to SDK?
If this question is too subjective for you to comfortably give advice, then when did you break into GUI/game development, and what steps did you take to make yourself comfortable with it?
Editing this so it will get bumped. Does anyone else have any opinions?
Caveat: I am a very "learn-by-doing" type of person, so take this with a grain of salt.
Sounds like you know enough programming basics to jump into something more realistic, and have enough background to justify that realistic project being a game.
I'd recommend downloading Visual C# Express and Microsoft's XNA Game Studio 3.0.
XNA is a game framework that has a lot of stuff done for you (sound, sprites, 3D support, etc.) built on a professional-quality C# platform and it would be a good starting point. Create a new XNA project and play around. Get some stuff to appear on the screen, then learn to manipulate it with user input. If you are interested in 3D, make a 3D shape such as a triangle. Then, make it spin. Then, make it spin based on user input. Then, add other objects and collisions.
Surely, there will be things in the framework that you don't understand. Tackle them as they come - use Google and ask questions here until you do understand them. Take it one step at a time and you should be just fine.
I'd personally recommend you to start out with Win32; try creating a basic window & move on from that point. Try making a simple 2D game engine in which you are able to make a game like chess or so. This could also serve as project for which you could write an AI; which is another part of Game Development!
After you finish that, the next step should be 3D. You could use the engine you wrote before and modify it from 2D to 3D. Pick a 3D API; OpenGL or DirectX. Once you have a basic engine, start writing a game. Need extra functionality? Then add it to the engine!
Math-wise you should know what matrices are. Trigonometry can come in handy as well.
I wouldn't waste my time with Xna, it's just a hype. :P
It seems you already have gained the basic knowledge of a programming language to start game programming. I'm with you in building on what you have already gained, such as learning OOP, and practicing more with pointers. I recommend you move on and don't turn to learning another tool "programming language" to achieve your goals.
So if you are interested in game programming, I recommend you pick a C++ framework and work on it, you'll definitely learn more advanced programming by just using it.
I recommend Gosu. It's not full of advanced features, which can be an advantage, but it has a very clean design and uses C++ in an elegant and modern way. Which makes it very friendly especially for beginners.
Also HGE is another good 2D engine.
To sum up, dive into programming more by actually "doing it" with what you have now. That's how you'll progress, and you'll be amazed with the results. And when "doing it" don't get disturbed with other languages and tools you already know something similar to it, and at the same time when learning a tool that helps you to build on your current knowledge, in your case I mean the C++ engine, don't choose very complicated ones (IMO, like OpenGL, DirectX, Win32...etc) because you'll end up spending time on learning the tool not using it and there is a great chance you'll get frustrated. You can always learn the low level things later, and it will make a lot more sense then.
as this question is kind of subjective, because every programmer has a favourite library to start with, I will recommend SDL as it is simple, well structured, and very complete, there are a lot of tutorials out there to guide you step-by-step from making a simple window to complex 3D manipulation. Everything can be implemented with ease.
As a side note, if you want to start programming games, I would recommend, also, that you read some tuts or books about a Game basics (initialization, game loop, update cycles), so that you know how to put your knowledge to the good work.

What is a good game involving coding?

I remember the days of Shadowrun that got me excited about hacking. There is CodeWar and LightBot which are both fun (though CoreWar is a little dated). What other games are there involving coding that are fun and challenging that can be used to get someone excited about coding or flex their chops or even learn the basics?
How about RoboCode
You code your tank in Java and let it loose in the 'ring' with other coded tanks. People got pretty into coding strategy, targeting, etc. IBM sponsored it and came up with some nice introductory programming tutorials to get you started.
Here's a great article to get the feel for it:
Rock 'em, sock 'em Robocode!
(source: sourceforge.net)
Uplink isn't so much a coding game, but it is a great game that makes you feel like a hacker.
There's a whole bunch of "drag-and-drop" coding games, where you make a little thing (usually a robot) solve some puzzle by giving it a list of instructions. They're only vaguely similar to actual coding, but they are still pretty fun.
RoboZZle
The Codex of Alchemical Engineering
light-Bot
Not sure if it's considered a "game", but the TopCoder Competitions are fun, and come in various sizes and commitment levels. You can also work on puzzles from the archives for some good programming practice.
The Python Challenge is like those "look at the html source" riddles, but requires a bit of programming to get the answers.
When I was a kid I played "Rocky's Boots", where you had to hook up logic gates to solve puzzles. That had a big impact on my thinking.
Core Wars.
Here's something that allows you to make games and animations: Alice
If you're looking for a board game, you might want to have a look at Robo Rally. In this game 2-8 people are trying to maneuver their robots over the board as quickly as possible, dodging deadly obstacles and trying to shove other people robots into obstacles on the way.
Each game round all players have to "code" the program the robot is going to execute in the next round and then the robots just follow their program. The programs are just five instructions long, but still creating an optimal program can be quite tricky. There usually is very little luck involved, which is why I really like this game.
Similar to Uplink is HackWars. Instead of point and click hacking though, it's multiplayer and you can write your own attack scripts. There's actually an included runtime for writing 2d/3d games and there's a bunch of different places to hook in scripts (for defense, banking, in game website, etc).
Scripting language looks similar to Java.
How about Ai-Board
You play it on your phone/tablet.
It's IDE is built into the game.
It has a built-in node-based visual programming language, whose code-behind is a python-like language.
You write the code, that drives the Ai, that moves the pawns, that plays the game, all still on your mobile device.
YouTube Video: Visual Programming Time-lapse on a mobile device
It comes with quite a few tutorials that introduce the player to programming, genetic algorithms etc, and you get a step-by-step walk-through of all these methods.
It also comes with ready-made scripts that work right out of the box, and are ready for you to copy into your free 'Dev' & 'Test' envs,...
...so that you can tweak them to your heart's content, knowing that you can always revert to the original at any time.
The in-built machine learning engine allows you to
train your AiBot to play the board-game,
play your AiBot against their in-built Ai
play against your own AiBot
breed your AiBot (Genetic Algorithm)
fine-tune your AiBot (Back-Propagation)
...debug your AiBot, and so on.
YouTube Video: Machine Learning is mobile!
Its currently in BETA testing, but soon to be released, and everything described comes for free.
In addition, there are single and multi-player modes as well, but it is primarily a game about coding, coming complete "...with batteries included!"
Project Euler

Porting Wii and/or PSOne Games to OpenGL ES

I have been asked to investigate porting Wii games and some (Sony) PSOne games to OpenGL ES (can you guess what platform?).
I have never undertaken a game port like this before (and will be hiring someone to do it) but I'd like to understand the process.
Does the Wii use OpenGL? If not what does it use and how easy is it to port to OpenGL / OpenGL ES?
Are there any resources/books/blogs that will help me in understanding the process?
Will my company have to become an official Wii developer? If so where do I start that process?
Porting from the Wii or the PSOne is a complex and involved task that can be broken down into multiple separate engineering efforts working in parallel to produce a working end product. The best possible thing you can do before moving to the target hardware is to compartmentalize all of the non-portable code while ensuring that the game continues to run as expected. When you commit to moving to the new platform, your effort switches to reimplementing the non-portable compartmentalized parts.
So, to answer your question, yes, you will need to become or work with a Sony and Nintendo licensed developer in order to take this approach. In the case of Sony, I don't even know if they offer a PSOne development program anymore which presents issues. Your Sony account rep can help clarify.
The major subsystems that are likely to be the focus of your porting effort are:
Rendering Graphics code contains fundamental assumptions about the hardware it is being run on in order to perform optimally. API-level compatibility is superficial compatibility and does not get you as much as you may hope it does. Plan on finding the entry point to the renderer and determining what data you need to render a scene and rewriting all the render code from there for your target hardware.
Game Saving Game state serialization and archival will need to be separated out. Older games often fwrite() structs with #pragma packed fields. Is that still going to work for you?
Networking Wii games write to high level services that are unavailable on your target hardware. At the low level, sockets are still sockets. What network services do your Wii games rely on?
Controls From where you are coming from to where you are going, anything short of a full redesign or reimagining of input will result in poor reviews of the software.
Memory Management Console games often make fundamental assumptions about the rate the system software returns memory from the heap, how much fragmentation it will cause and the duration the game needs to operate under these conditions. These memory management assumptions are obsolete on the new platform. It is wise to write your own memory manager that provides a cushion from the operating system. Also, console games compiled for release are stripped of most error handling and don't gracefully handle running out of memory-- just a heads up.
Content Your bottleneck will be system memory. Can you fit the necessary assets into memory? With textures, you can reduce mip where necessary and with graphics hardware timing, you can pull in the far clipping plane. With assets resident in memory, you may need a technical artist to go through and reduce the face density of your models or an animation programmer to implement a more size-friendly animation codec. This is very game specific.
You also run into the standard set of problems with things like bit compatibility (though the Wii and PSOne are both 32-bit), compiler idiosyncrasies, build script incompatibilities and proprietary compiler extensions.
Games are relatively challenging to test. A good rule of thumb is you want to have enough testers on your team to run through the game in a maximum of two days, covering all major aspects of play. In games that take a long time to beat (RPGs with 30+ hours of gameplay), your testing team needs to be quite large to offer full coverage. Because you are just doing a port, you can come up with a testing plan that maximizes coverage of your new code without having a testing team punch every wall in your game to make sure it (still) has clipping. The game shipped once.
Becoming a licensed developer requires you to apply. The turnaround time, from experience, is not good. Generally speaking, priority is given to studios with shipped titles and organized offices with reasonably good security and the ability to buy the (relatively) expensive development kits. You may be better off working with a licensed developer if you do not meet these criteria.
Console and game development is challenging for people already experienced in it. There is no book that covers it all. My recommendation is to attempt to recruit an expert who has experience shipping titles in a position of systems or engine programmer. What types of programmers and skillsets exist in games is a whole different question for Stack, though.
Games consoles don't use OpenGL but their own, custom libraries. The main reason is that they are pretty slow and have little RAM. So you need to squeeze out every drop of performance you can get. And that means: Custom code. Usually, you get a framework with the developer kit which gets you started and then, you build your code from that. Eventually, you'll start replacing parts from the developer kit with your own special code to get all the speed and special effects you need.
There is a reason why PSOne games are so ugly on the PS3 despite the fact that the developers have access to the sources: Revenue just doesn't justify to touch the code.
Which is one reason why game development is so expensive: Every game is (more or less) a completely new product. Sometimes, game companies can reuse a bit of code from the last version but more often than not, they have to develop everything again. They also don't talk much with each other.
In recent years, kits have become more complex and powerful and you can get complete game engines (with all kinds of effects and 3D support) but each engine is a completely different kind of beast, so you can't even copy code from engine A to B.
Today, media content (video, audio and render sequences) are so expensive that the actual game engine is often a minor detail, so this isn't going to change any time soon.
Net result: If you want to port a game, write an emulator for the hardware (which is usually pretty simple and allows you to run all kinds of games).
[EDIT] To develop software for the Wii, see here: http://www.warioworld.com/
For a Wii emulator, see http://wiiemulator.net/
I ported a couple of games, when I was a new game programmer, from working with one version of our engine to a newer version (where backwards compatibility was neither ignored nor pursued). Even copying (and possibly renaming) the files and placing them in a home in the new project was a bit of work. Following that, the procedure was:
recompile
fix many of the hundreds of errors [in many places, with the same error occurring over and over again]
and
"wire up" calls from the new game engine to the appropriate calls in the old code
"wire up" function calls from the old code into the new game engine
deal with other oddities (ex. in the old game engine, the 2d game would "swizzle" textures itself; in the new version, the engine did it (on specific platforms))
and, while I don't recall this clearly, it was probably mixed in with a bunch of #ifdeffing out portions of code so the thing would actually compile, and possibly creating function stubs to be filled in later.
As I recall, it was three or four days until I had something that compiled. (But, it did help when we ported other games from the old version to the new one!)
The magnitude of the task will come down to what the code you are getting is like. If it has generic 3D calls that you can intercept -- add a thunking layer to -- then you are in business. It depends on the level of abstraction in the code. If it is well-behaved and has things like "RenderModel" and "RenderWorld" calls, you can replace those functions, and even the structures that they work with. If drawing is occurring all over the place, and calls are more like "Draw Polygon" and "Draw Line" or "Draw using this highly optimised data structure", then you are likely in for a long slog.
You shouldn't need a Wii dev kit. Sometimes it is nice to verify that the code you are given does indeed compile in the original environment (and matches the shipping code!), but sometimes you can just take it on faith and make it work in its new environment.
Lastly, I don't think the Wii uses OpenGL, and I really don't know where to point you for further help.
What you may want to do is to start with designing the architecture of the game, write up a detailed specification for what the new game is like.
Once you have this, since you will be rewriting the code, you may find that some of the business logic that doesn't deal with the console can be ported over. But, anything dealing with I/O, user interaction or graphics/sounds will be rewritten, so you might as well do that from scratch.
A specification is very important, to make certain that you know how the current game is working so that the new port will give the same user experience, if that is what is desired.
You may want to keep the same bugs, if that is part of the experience, as, if I know that in the Wii I can jump down and bounce off the wall to safely land, then if I can't do that in the new version then that may be bothersome.
Well porting a PS1 game to an iPhone would be quite a task they work in very different ways. I'm sure its doable but it will be a LOT of work to replace all the fixed point maths and lack of Z-Buffer based rendering to a real graphics chip.
Wii would be a lot easier. The Wii API is very similar to OpenGL. However the Wii has some very nice fixed function features that just are not available on any other GL based platform. Should be doable, though ...
I'm not really sure I can say anything more than that. Have signed far too many NDAs over the years to be 100% sure of what I can and cannot say ;)
Still if you want to hire someone to do some porting work and are prepared to supply the required hardware then I might be free ;)

Depicting a graphical demo of a car lapping a track

I am planning to design an application which will provide a video of a car on a track (just like a demo of a game like RACE Pro), and depict its characteristics.
For example, if the user selects a poor handling car, then a graphical demo will be shown of that car completing a lap around a modelled track with the car's behaviour in relation to its characteristics (e.g. prone to skidding, understeer).
I know this is a fairly complex task, and I don't intend to code this straight away. What issues/topics does this requirement fall under? I have seen plenty of books on physics in programming and game engine/3d modelling and rendering engines. Are these relevant to this?
This is certainly a significant undertaking, but as a starting point you could take a look at the Racer project. It's a very detailed car racing simulation, highly customisable for different vehicle setups, and has publicly available source code for its engine (see the downloads page).
Physics and game programming books are absolutely relevant to this. Fortunately there are some great free and Open Source projects out there which could be a great help to you with this project. I've listed some below.
Physics:
Open Dynamics Engine
Tokamak
Bullet
3D graphics:
Ogre
Crystal Space
Crystal Entity Layer (CEL), which provides a game engine interface for Crystal Space.
You can tie any of the above together in whatever combination suits you.
You could also use Blender's built-in game engine, which includes the Bullet physics engine. I would recommend this as a starting point since it will give you a fully integrated interactive 3D simulation creation kit. The game engine is scriptable with Python (you can do all your script editing in Blender with its internal text editor).
In addition to the Racer project you could take a look at TORCS - The Open Racing Car Simulator, which is also Open Source as the name suggests.

Resources