I am working with the telepresence robot of the Ohmnilabs (here a picture of the robot). I set up the navigation stack and tuned the parameters and everything worked fine for the map creation phase.
When I try autonomous navigation, after a few goals, the lidar data showed on rviz is misaligned with respect to the map.
I thought the problem was in the odometry and my thoughts were confirmed when I followed the step 1.2 in this guide. In particular, the robot failed the rotation test.
Can you please tell me what can I do to avoid or reduce the misalignment? Are there tutorials to solve this problem?
Related
We are developing an Unity/ARKit app for outside exterior use. While testing inside, everything works fine and environment occlusion works as expected. But outside, when reaching a certain distance to the AR-objects, they start flickering until they disappear completely. Please see attached video.
https://www.youtube.com/watch?v=paJ92c1kNPE
I've tried to visualize the depth map, but i don't see any change in the values.
If i switch off environment depth, the models appear again.
What causes this and why is the occlusion not happening according to the depth map?
For example at second 0:25 in the video you see occlusion is working, but the depth map is completely red?
Or is the depth map only visualizing what the LIDAR sees?
Or is this just a limitation of ARKit?
What is the best way to find the intrinsic parameters of a Digital Camera?
I have tried using the opencv Camera Calibration tool but it gives me different values after every calibration. So Im not sure if it is correct or not. Whereas the same method works perfectly fine for a USB camera.
I am not sure if the process to get intrinsic parameters for digital camera is slightly different and if im missing something.
If someone has any clue regarding this... please help!
Changes in ISO and white balance happen "behind" the lens+sensor combo, so they cannot directly affect the calibrated geometry model of the camera.
However, they may affect the results of a calibration procedure by biasing the locations of the detected features on whatever rig you are using ("checkerboard" or similar) - anything that affects the appearance of grayscale gradients can potentially do that.
The general rule is: "Find the lens and exposure settings that work for your application (not the calibration), then lock them down and disable any and all auto-gizmo features that may modify them". If those settings don't work for the particular calibration setup you are using, change it (for example, dim the lights or switch the target pattern to gray/black from white/black if the application lighting causes the target to be overexposed/saturated).
Think if someone in real life waved their hand and hit the 3D object in AR, how would I detect that? I basically want to know when something crosses over the AR object so I can know that something "hit" it and react.
Another example would be to place a virtual bottle on the table and then wave your hand in the air where the bottle is and then it gets knocked over.
Can this be done? If so how? I would prefer unity help but if this can only be done via Xcode and ARKit natively, I would be open to that as well.
ARKit does solve a ton of issues with AR and make them a breeze to work with. Your issue just isn't one of them.
As #Draco18s notes (and emphasizes well with the xkcd link 👍), you've perhaps unwittingly stepped into the domain of hairy computer vision problems. You have some building blocks to work with, though: ARKit provides pixel buffers for each video frame, and the projection matrix needed for you to work out what portion of the 2D image is overlaid by your virtual water bottle.
Deciding when to knock over the water bottle is then a problem of analyzing frame-to-frame differences over time in that region of the image. (And tracking that region's movement relative to the whole camera image, since the user probably isn't holding the device perfectly still.) The amount of of analysis required varies depending on the sophistication of effect you want... a simple pixel diff might work (for some value of "work"), or there might be existing machine learning models that you could put together with Vision and Core ML...
You should take a look at ManoMotion: https://www.manomotion.com/
They're working on this issue and suppose to release a solution in form of library soon.
I recently managed to get my augmented reality application up and running close to what is expected. However, I'm having an issue where, even though the values are correct, the augmentation is still off by some translation! It would be wonderful to get this solved as I'm so close to having this done.
The system utilizes an external tracking system (Polaris Spectra stereo optical tracker) with IR-reflective markers to establish global and reference frames. I have a LEGO structure with a marker attached which is the target of the augmentation, a 3D model of the LEGO structure created using CAD with the exact specs of its real-world counterpart, a tracked pointer tool, and a camera with a world reference marker attached to it. The virtual space was registered to the real world using a toolset in 3D Slicer, a medical imaging software which is the environment I'm developing in. Below are a couple of photos just to clarify exactly the system I'm dealing with (May or may not be relevant to the issue).
So a brief overview of exactly what each marker/component does (Markers are the black crosses with four silver balls):
The world marker (1st image on right) is the reference frame for all other marker's transformations. It is fixed to the LEGO model so that a single registration can be done for the LEGO's virtual equivalent.
The camera marker (1st image, attached to camera) tracks the camera. The camera is registered to this marker by an extrinsic calibration performed using cv::solvePnP().
The checkerboard is used to acquire data for extrinsic calibration using a tracked pointer (unshown) and cv::findChessboardCorners().
Up until now I've been smashing my face against the mathematics behind the system until everything finally lined up. When I move where I estimate the camera origin to be to the reference origin, the translation vector between the two is about [0; 0; 0]. So all of the registration appears to work correctly. However, when I run my application, I get the following results:
As you can see, there's a strange offset in the augmentation. I've tried removing distortion correction on the image (currently done with cv::undistort()), but it just makes the issue worse. The rotations are all correct and, as I said before, the translations all seem fine. I'm at a loss for what could be causing this. Of course, there's so much that can go wrong during implementation of the rendering pipeline, so I'm mostly posting this here under the hope that someone has experienced a similar issue. I already performed this project using a webcam-based tracking method and experienced no issues like this even though I used the same rendering process.
I've been purposefully a little ambiguous in this post to avoid bogging down readers with the minutia of the situation as there are so many different details I could include. If any more information is needed I can provide it. Any advice or insight would be massively appreciated. Thanks!
Here are a few tests that you could do to validate that each module works well.
First verify your extrinsic and intrinsic calibrations:
Check that the position of the virtual scene-marker with respect to the virtual lego scene accurately corresponds to the position of the real scene-marker with respect to the real lego scene (e.g. the real scene-marker may have moved since you last measured its position).
Same for the camera-marker, which may have moved since you last calibrated its position with respect to the camera optical center.
Check that the calibration of the camera is still accurate. For such a camera, prefer a camera matrix of the form [fx,0,cx;0,fy,cy;0,0,1] (i.e. with a skew fixed to zero) and estimate the camera distortion coefficients (NB: OpenCV's undistort functions do not support camera matrices with non-zero skews; using such matrices may not raise any exception but will result in erroneous undistortions).
Check that the marker tracker does not need to be recalibrated.
Then verify the rendering pipeline, e.g. by checking that the scene-marker reprojects correctly into the camera image when moving the camera around.
If it does not reproject correctly, there is probably an error with the way you map the OpenCV camera matrix into the OpenGL projection matrix, or with the way you map the OpenCV camera pose into the OpenGL model view matrix. Try to determine which one is wrong using toy examples with simple 3D points and simple projection and modelview matrices.
If it reprojects correctly, then there probably is a calibration problem (see above).
Beyond that, it is hard to guess what could be wrong without directly interacting with the system. If I were you and I still had no idea where the problem could be after doing the tests above, I would try to start back from scratch and validate each intermediate step using toy examples.
I am thinking of creating a robot that can navigate using a map. It is controlled from a PC. An 8-bit controller performs low level tasks and the PC is doing the image processing. I plan to implement it in a single room where the robot is placed and the robot and environment are tracked by a camera from a height or from the ceiling of the room. First, the robot needs to be mapped, like this http://www.societyofrobots.com/programming_wavefront.shtml
To do:
Track the robot from some height using camera Following the wavefont algorithim to locate robot and obstacles.
Procedure:(just my idea)
The camera will give image of the robot surrounded by obstacles in
the random places. using some opencv technique draw some grind over
the image.
Locating the grid which contain robot(by having some colored symbol
over the robot) and locating the grids containing the obstacle.
Now the grids with obstacle is thought as wall and the remaining is
the free space for the robot to navigate.
robot is going to get the goal place which should be reached is given
from the pc(may be like point the place to reach in the image by
mouse click).
firstly finding the obstacle from the video stream is to be done for that iam going to catch a image of the robot which is in the room now the image is edit by manually in M.S paint by filling the grid with stationary obstacle by some color say red.
now the edited image is going to be the reference and it is compared with video stream to tell the free space available to robot.
each grid is given some value using that place value of the robot should be determined
now i should be able to give some grid value in the free space to make it as goal for the robot
now the Pc should calculate and tell the robot which grid should be traveled to reach the goal. the only thing the robot need to figure out itself is dynamic obstacle avoidance like cat walking across the robot
comparing the area of the room with the image of some dimension to find how far the robot moved and how far to go
is it possible to do? can anybody help me to do this?
\thanks in advance
Not sure if this will help you, but I did some work related to your post. Though I'd share.
http://letsmakerobots.com/node/38208