I have been working Augmented Reality for quite a few months. I have used third party tools like Unity/Vuforia to create augmented reality applications for android.
I would like to create my own framework in which I will create my own AR apps. Can someone guide me to right tutorials/links to achieve my target. On a higher level, my plan is to create an application which can recognize multiple markers and match it with cloud stored models.
That seems like a massive undertaking: model recognition is not an easy task. I recommend looking at OpenCV (which has some standard algorithms you can use as a starting point) and then looking at a good computer vision book (e.g., Richard Szeliski's book or Hartley and Zisserman).
But you are going to run into a host of practical problems. Consider that systems like Vuforia provide camera calibration data for most Android devices, and it's hard to do computer vision without it. Then, of course, there's efficiently managing the whole pipeline which (again) companies like Qualcomm and Metaio invest huge amounts of $$ in.
I'm working on a project that does framemarker tracking and I've started exporting bits of it out to a project I'm calling OpenAR. Right now I'm in the process of pulling out unpublishable pieces and making Vuforia and the OpenCV versions of marker tracking interchangeable. You're certainly welcome to check out the work as it progresses. You can see videos of some of the early work on my YouTube channel.
The hard work is improving performance to be as good as Vuforia.
Related
Our company is planning to start building some AR apps for Android and iOS. As the first step we need to decide whether we are going to use a Opensource SDK like ARToolKit or to go for a commercialized product like Vuforia, Wikitude, CraftAR, KudanAR etc or whether we should start writing our own AR sdk based on libraries like OpenCV/OpenGL etc..
I have read many articles and comparisons about different SDKs available and have a good idea about what each of them can do and how much they are going to cost. e.g.
http://socialcompare.com/en/comparison/augmented-reality-sdks
https://www.linkedin.com/pulse/dozens-more-augmented-reality-sdks-than-you-think-here-offermann
In the past we have used Vuforia and we have it at the top of our list. But the main issue is the pricing.
So I would like to know if any of you have written or tried to build your own AR sdk based on OpenCV and what type of an effort it will be. To support features like image and 3D object tracking and augmenting 2D and 3D objects. And need to support iOS and Android devices.
This Augmented Reality SDK with OpenCV has some basic guidelines on how to start.
Mainly what I would like to know is if a software Engineer with about 5+ years of good programming skills try to do this, how much of an effort will that be? Will it be like 1 month work or a 6 months work or even with 12 months of work will it be difficult to get closer to what Vuforia SDK can does?
I am newbie with drones. I would like to develop a program to manage a drone using opencv to fly indoor over a line.
I am searching a lot of languages but most all of them are GPS based. I saw there is an alternative which calls SLAM to detect the position using the sensors.
Well I have a line in the floor and a camera on my drone. I like mission planner but I am not quite sure if it is the best choice. I will be using Parrot AR, but I would like to use any drone.
So I would like to use mission planner but I am not sure if it is the best choice.
What would be the best SDK you would recommend me to use in order to manage the drone not using the GPS points but relative locations or SLAM?
Well, you have the Parrot API ,and a couple of wrappers in different languages. Node-AreDrone for nodeJs, PyArdrone for python, and there is a wrapper coded in C# which I have used AR.Drone. It has a good user interface which you can see the both cameras, record and replay the videos, control the drone by clicking on buttons, you can see the metrics and configuration of the drone and you have also a way to send commands in a queue. Because I love c# and the features I've mentioned you have already in a user interface, I prefer this. Most of them are quite the same as they use the Parrot API inside by sending udp messages. I couldn't try others, so, there are a lot, and anybody could tell me which one is the best. For mission planner I couldn't find a good solution for indoors. So, for anyone who is lost and do not know here to start as I was. I recommend to select the language you want and search for the corresponding wrapper. If you like c# as me, so AR.Drone is a good choice.
Also if you want to do something with OpenCV. Copterface is a good example. You could implement it in any language with OpenCV.
I'm developing this project where I'm trying to create a distributed version of Tensorflow (the actual open source version is single node) and where the cluster is entirely composed by mobile devices (e.g. smartphones).
In your opinion, what is a possible application or use case where this could be useful? Can you give me some example please?
I know that this is not a "standard" Stack Overflow question, but I didn't know where to post it (if you know a better place where to post it, please let me know it). Thanks so much for your help!
http://www.google.com.hk/search?q=teonsoflow+android
TensorFlow can be used for image identification and there is an example using the camera for Android.
There could be many distributed uses for this. Face recognition, 3D space construction from 2D images.
TensorFlow can be used for a chat bot. I am working towards using it for a personal assistant. The Ai on one phone could communicate with the Ai on other phones.
It could use vision and GPS to 'reserve' a lane for you on the road. Intelligent crowd planned roads and intersections would be safer.
I am also interested in using it for distributed mobile. Please contact me with my user name at gmail or Skype.
https://boinc.berkeley.edu
I think all my answers could run on individual phones with communication between them. If you want them to act like a cluster as #Yaroslav pointed out there is Seti#home and other projects running in the BOINC client.
TensorFlow could be combined with a game engine. You could have a proceduraly generated Ai learning augumented reality game generating the story as multiple players interact with it. I have seen research papers for each of these components.
I am using an XBox Kinect with the Kinect for Windows SDK. I want to make an application that will augment a 3D mask (a 3D model of a mask made in 3DS Max) onto the face of anyone using the application. The application will be used in an exhibit locally. I have not tried much because I don't know where to start. So what I want to know is, is it currently possible to augment a 3DS Max model onto a live video stream using the facial recognition and skeletal tracking features in the newest Kinect for Windows SDK, and if so, how/where should I start trying to do/implement this? Any point in the right direction would be great. Thank you! PS And yes, I have read the UI guidelines and the facial documentation. My problem is one of not knowing where to start programming, not one of not understanding the fundamental concepts. Thanks!
If you are serious about getting into developing for the Kinect I would recommend getting this book:
http://www.amazon.com/Programming-Kinect-Windows-Software-Development/dp/0735666814
This goes through developing with the Kinect for Windows SDK from the ground up. There is a face tracking and an augmented reality example so I'm pretty sure you will be able to achieve your goal quite easily.
All the code from the book is here:
http://kinecttoolbox.codeplex.com/
Alternatively, there is an example here which pretty much is what you want to achieve:
http://www.codeproject.com/Articles/213034/Kinect-Getting-Started-Become-The-Incredible-Hulk
It is developed using the Beta version of the SDK, but the same priciples apply.
You can also check out the quick start videos here:
http://channel9.msdn.com/Series/KinectQuickstart
In summary, based on my own experience, I would spend some time going through the beginner examples either in the vides or the book (I found the book very good) just to get familiar with how to setup a simple Kinect project and how the different parts of the SDK work.
When you have developed some throwaway apps with the Kinect, I would then try tackling your project (although, the Incredible Hulk project above should get you most the way there!)
Best of luck with your project
I have used PyGame for a while now, and quite enjoyed doing so, but only 2D experience is not going to cut it when I apply for video game jobs, and besides, I have a number of ideas I want to try out in 3D. What 3D library should I go into first? I was considering Irrlicht, which will also force me to work on my C++ again, but I was also looking towards XNA, as it would be cool to write 360 games. Any suggestions?
You can try Unity3D http://unity3d.com/. In the fall they released their engine for free that is cross platform (Mac, Win and possible Linux) and it can also be targeted for iPhone and web browser and is starting to build an open community and looks promising.
I've done some XNA stuff and the Unity tools look pretty good in comparison making it easier to get content into your games.
In my opinion, going with Irrlicht or OGRE is a good idea. It will help you get back in touch with C++ and experiment with 3D without making it too easy. It is a good idea to learn XNA as well and build a few 3D game prototypes there.
Summary: Do both, eventually. Learn OpenGL/DirectX later.
All the best.
XNA Game Studio is a fantastic framework and let's users jump into creating games quite quickly if you are comfortable with the .NET Framework.
However, if you decide that you want to move to the 3D world, you have 1 more dimension to take into consideration and it makes things exponentially harder. Simple collision detection, isn't so simple anymore. However, with the vast amount of information available for XNA Game Studio, you should find yourself getting to grips with the 3rd dimension quite quickly.
If you haven't done the whole Nehe type tutorials, it's probably best to play with the API, so that you get comfortable with it.
Create a Triangle
Spin the Triangle
Color the Triangle
Load some assets (models and textures) and render them
Try your hand at Alpha Blending
Take into consideration that XNA Game Studio does not have a fixed function pipeline and you will have to use Shaders (HLSL).
I have personally used Irrlicht, Ogre3d, Unity & Xna. Because you want to do this to help for when you apply for jobs in the game industry in my opinion I would suggest Irrlicht as you will want to work on your c++ skills. I also suggest Irrlicht as it is a 3D engine not a specific game engine which will allow you to work on your physics and maths skills as well which are highly sought after in the games industry.
I would suggest avoiding Unity if you are looking for experience to take to industry as it is a package that handles a lot for you and as such could limit your learning.
Ogre3D is a viable option as it is similar to Irlicht, maybe look into the tools available to help you make a choice between those two.
Xna is another interesting choice, it will give you a lot of chances to learn as Irrlicht could and there are a huge amount of resources to help you learn but at the same time you lose the c++ experience that comes with irrlicht.