I'm currently working on app that records video of user, binds particle emitters to hands landmarks on preview. In result effects are shown on camera preview, but they aren't captured on video.
I saw a great tutorial on AVVideoCompositionCoreAnimationTool https://www.raywenderlich.com/6236502-avfoundation-tutorial-adding-overlays-and-animations-to-videos, but that way it only is possible to rendere animations on already recoded video.
I wonder if there is any chance to use AVVideoCompositionCoreAnimationTool for recording video form camera and animations in real time. Or if you know other method to do it, without diving deep in metal and so on.
Thanks in advance!
Related
at a bit of a loss here.
Basically, I need to create a frame server that will give my app an image of each frame contained in the video at said video's frame rate. I need to do this while also playing its audio.
I'll be using these frames as a texture source for certain geometries on an SceneKit
Have never used AVFoundation, any pointers, tutorials or suggestions are welcomed
That's a very general question.
Since you've never used AVFoundation, the AVFoundation Programming Guide should be your first stop.
In particular, the Still and Video Media Capture section shows how to set a delegate for the capture process.
In my app I have to play an alpha channel video as an overlay over the current view (I'm planning to achieve this alpha channel video using GPUImageAlphaBlendFilter or GPUImageChromaKeyBlendFilter), so I wanted to know if the output video after applying these filters can be played using GPUImage? If we can, then can I get some sample code for the same.
I know AVAnimator is an option but I want to apply filters to these overlay videos i.e.brightness,saturation etc which has to be visible while video is being played because of which I can't use AVAnimator. But this being the next step for now I want to know how to play video using GPUImage.
Thanks in advance! :]
Well, even though I like telling people about AVAnimator, Brad Larson's GPUImage is specifically designed to be a GPU based filtering framework for iOS apps. Application of real time effects like the ones you describe is exactly what GPUImage was designed for. See GPUImage Chroma Key filter
What I'm trying to achieve is grabbing a still frame from a AVCaptureSession for something like a QR Code scanner. I have a view with a AVCaptureVideoPreviewLayer sublayer.
I already tried using AVCaptureStillImageOutput, which worked, but that function makes the shutter sound. Since there's no way to mute that I can't use it. After that I tried to make a screenshot of the entire screen, which also failed, because it can't capture a AVCaptureVideoPreviewLayer. Now I'm kinda lost, the only real way to do this would be to feed a video feed into my OCR library but that would lag to much/be a lot of work.
Are there any other options?
Here's a tech note describing exactly what you want to achieve:
How to capture video frames from the camera as images using AV Foundation on iOS
I'm looking for a tips to develop an application for iPhone/iPad that will be able to process video (let's consider only local files stored on the device for simplicity) and play it in real-time. For example you can choose any movie and choose "Old movie" filter and want it like on old lamp TV.
In order to make this idea real i need to implement two key features:
1) Grab frames and audio stream from a movie file and get access to separate frames (I'm interested in raw pixel buffer in BGRA or at least YUV color space).
2) Display processed frames somehow. I know it's possible to render processed frame to OpenGL texture, but i would like to have more powerful component with playback controls. Is there any class of media player that supports playing custom image and audio buffers?
The processing function is done and it's fast (less than duration on one frame).
I'm not asking for ready solution, but any tips are welcome!
Answer
Frame grabbing.
It seems the only way to grab video and audio frames is to use AVAssetReader class. Although it's not recommended to use for real-time grabbing it does the job. In my tests on iPad2 grabbing single frame needs about 7-8 ms. Seeking across the video is a tricky. Maybe someone can point to more efficient solution?
Video playback. I've done this with custom view and GLES to render a rectangle texture with a video frame inside of it. As far as i know it's the fastest way to draw bitmaps.
Problems
Need to manually play a sound samples
AVAssetReader grabbing should be synchronized with a movie frame rate. Otherwise movie will go too fast or too slow.
AVAssetReader allows only continuous frame access. You can't seek forward and backward. Only proposed solution is to delete old reader and create a new with trimmed time range.
This is how you would load a video from the camera roll..
This is a way to start processing video. Brad Larson did a great job..
How to grab video frames..
You can use AVPlayer+ AVPlayerItem, it provide you a chance to apply a filter on the display image.
I would like to create a game for XBox360 which is mostly full-screen HD videos. The player will be given choices during the game to determine which video is to be played.
I need very fine-grained control over the video such as controlling playback speed, seeking to video frames and possibly applying simple effects to the videos.
I also want to be able to use augmented reality to add elements to the videos so I need to be able to render 3d objects over the video.
It would be great if this could be done in XNA however there is only basic video playback functionality there. What other options do I have?
Your options for decoding videos are limited. The VideoPlayer class provides functionality for playing videos from the start, pausing and resuming them, looping them, and setting their audio volume.
As far as displaying videos goes - you have a huge degree of freedom. You basically get each frame of the video as a texture that you can draw as a sprite, or apply to any 3D object. This includes using it as an input to a pixel shader, allowing you to apply all kinds of effects to the video.
The only alternative to the built-in player is to create your own. If you want to target the Xbox 360 this will limit you to managed code only. I am not aware of any suitable video decoder libraries.
For Windows, a little Googling revealed this library, which may be a good starting point.