How to merge stereo video? - opencv

I recently purchased a stereo USB camera to capture some footage through my AR headset. Now the camera records through 2 lenses and gives 2 outputs: a left and a right video feed. My goal is to combine the left and right video feed into a single mono output.
Here is a picture of the camera and the link to the exact model.
Here is a screenshot of the video output:
As you can see I want to somehow interlace those 2 videos into 1 (I guess similar to anaglyph) so it can be viewed on a 2d screen (without a vr headset). Does anyone know any software programs that can do this? Or has anyone written a custom script?

Related

Can I feed ARKit facial capture pre-recorded videos?

I know a similar question has been asked before, but I have a very specific use case and thought I might put more details here.
I'm not an iOS dev, but very curious about the ARKit -- specifically want to test out the facial capture (tracking facial expressions) using ARKit ARFaceAnchor.
I want to know if it is possible to feed pre-recorded videos to ARKit instead of the camera feed.
I did find an article with sample code here about using "sensor replay".
https://github.com/ittybittyapps/ARRecorder/tree/master/ARRecorder
https://www.ittybittyapps.com/blog/2018-09-24-recording-arkit-sessions/
Unfortunately it is not possible ship an app this way.
I know that the facial capture doesn't necessarily require depth sensor data (I've tried it by holding up the camera to a pre-recorded face on a monitor).
So I'm curious to know if anyone knows of a way to feed a static, pre-recorded video to the ARKit?
Thanks in advance for the help :)

From a video file (.mp4) to live video (usbcamera) - OpenCV

Recenty I made a proyect with OpenCV, dectects cars, that inputs video file and output another video file.
I want, with the help of a raspberry pi or similar, turn the input video(mp4) to a live webcam input(usb camera or similar), and output the video processed in livetime to my screen, to basicly mount a rasppbery pi (or similar) to my car with a camera, and output in live the detection in a screen.
How to do that? any documentation?

Is there front facing camera support with ARKit?

How can we access Front Facing Camera Images with ARCamera or ARSCNView and is it possible to record ARSCNView just like Camera Recording?

Regarding the front-facing camera: in short, no.
ARKit offers two basic kinds of AR experience:
World Tracking (ARWorldTrackingConfiguration), using the back-facing camera, where a user looks "through" the device at an augmented view of the world around them. (There's also AROrientationTrackingConfiguration, which is a reduced quality version of world tracking, so it still uses only the back-facing camera.)
Face Tracking (ARFaceTrackingConfiguration), supported only with the front-facing TrueDepth camera on iPhone X, where the user sees an augmented view of theirself in the front-facing camera view. (As #TawaNicolas notes, Apple has sample code here... which, until iPhone X actually becomes available, you can read but not run.)
In addition to the hardware requirement, face tracking and world tracking are mostly orthogonal feature sets. So even though there's a way to use the front facing camera (on iPhone X only), it doesn't give you an experience equivalent to what you get with the back facing camera in ARKit.
Regarding video recording in the AR experience: you can use ReplayKit in an ARKit app same as in any other app.
If you want to record just the camera feed, there isn't a high level API for that, but in theory you might have some success feeding the pixel buffers you get in each ARFrame to AVAssetWriter.

As far as I know, ARKit with Front Facing Camera is only supported for iPhone X.
Here's Apple's sample code regarding this topic.

If you want to access the UIKit or AVFoundation cameras, you still can, but separately from ARSCNView. E.g., I'm loading UIKit's UIImagePickerController from an IBAction and it is a little awkward to do so, but it works for my purposes (loading/creating image and video assets).

Reverse Engineering Google Cardboard

I'm building a 360 video viewer for iOS in order to better understand the nuances of monoscopic and stereoscopic 360 video. Effectively, I'm trying to reverse engineer what the Google Cardboard SDK does.
My in-progress code is here: https://github.com/alfiehanssen/ThreeSixtyPlayer
I understand monoscopic video and the basics of stereoscopic video. I would like to better understand the client-side configuration/transformations required to display stereoscopic video.
Specifically, in two parts:
Cardboard can display monoscopic video files in stereoscopic mode. Although the same exact video is being presented to each eye, each eye's video view is clearly different. What transformations are being made to each eye to get this to work?
For stereoscopic videos (let's assume top/bottom layout), it also appears that transformations are applied to each eye's video view. What are the transformations being applied to each eye's video view?
It looks as though the video is being skewed. There are black masks applied to all sides of each eye's video, where are these coming from / are they the result of transformations?
A sample screenshot from a cardboard view:

hand detection in c using opencv capturing video from kinect device

I am working on a project in which I have to detect the hand in a video. Kinect is being used to capture the video. I have already tried skin segmentation using hsv colour scheme. It works well when I get the video from cam of my laptop but does not work with Kinect. I have also tried colour segmentation and thresholding but it is also not working well. I am using opencv in c. I will be grateful if someone can give any type of suggestions or steps to detect the hand.

You can use OpenNI to obtain a skeleton of the person which robustly tracks the hand in your video. E.g. http://www.youtube.com/watch?v=pZfJn-h5h2k

Resources