I have an app (enterprise, distributed OTA) that among other things records video clips. All of a sudden, we started getting video uploads that were missing audio, and this issue now seems to be totally reproducible. I've been using the PBJVision library, which had seemed to work great, but I have also tested this with SwiftyCam (another AVFoundation-based library) with same results. It's unclear exactly when this was introduced, but I've checked the following:
Ensure that a NSMicrophoneUsageDescription is set in the target .plist
Ensure that camera and microphone permissions are showing as granted in system settings
Try disabling microphone permissions in settings (app correctly prompts the user to re-enable permissions)
Try earlier releases of the video capture library in case of regression
Try different video capture library
Explicitly set audio enabled and bitrate for PBJVision/SwiftyCamera, and ensure that the session is at least reporting that it has audio in the logs (that is, the library and AVFoundation think there's an input set up, with an input stream that's being handled)
Take a video with the system camera, and upload through the app — in this case, audio does work (it's not a problem with the hardware)
Reset all content and permissions on a device, to make sure there isn't some kind of cached permission hanging out
Make sure volume is not muted
The copy that gets saved to the camera roll is also silent, so it's not happening when the video gets uploaded. I also started to implement recording using just AVFoundation, but don't want to waste time if that will produce the same results. What could be causing a particular app not to record audio with video? I have looked at related questions, and none of the solutions provided address the problem I'm having here.
EDIT:
Here are the logs that appear when starting, recording, and stopping a PBJVision session:
[5411:1305718] VISION: camera setup
[5411:1305718] VISION: switchDevice 1 switchMode 1
[5411:1305718] VISION: capture session setup
[5411:1305291] VISION: session was started
[5411:1305718] VISION: capture session running
[5411:1305291] VISION: starting video capture
[5411:1305718] VISION: ready for video (1)
[5411:1305718] VISION: audio stream setup, channels (1) sampleRate (44100.000000)
[5411:1305718] VISION: ready for audio (1)
[5411:1305291] VISION: ending video capture
[5411:1305963] VISION: capture session stopped
[5411:1305963] VISION: session was stopped
[5411:1305291] CMTimeMakeWithSeconds(8.396 seconds, timescale 24): warning: error of -0.021 introduced due to very low timescale
It turns out that this was actually due to using another library to play a sound after starting the video recording. This apparently preempts the audio channel for the recording, as that ends up being empty (see Record Audio/Video with AVCaptureSession and Playback Audio simultaneously?). It does not appear to matter whether or not the other sound playback is started before or after starting video recording. This is a good warning case around using multiple libraries that all touch the same system APIs — in some cases, like this one, they interact in undesirable ways.
In this case, the solution is to make sure that the two sources aren't using the same AVAudioSessionCategory, so they don't conflict.
Related
I need the same behaviour as seen in SnapChat, but for a longer duration.
I already have an AVCaptureSession that records video and audio from the built-in microphone.
What I need is to also record music running in the background, i.e. from Spotify, Apple Music etc.
I assume there is two parts to this:
1) Setting the correct category on AVAudionSession (in order to allow background music while recording)
2) Setting some (custom?) input AVCaptureDevice to grap audio playing in the background.
But how do I do it?
Spotify, iTunes etc generally have the music encrypted and the keys protected via a DRM mechanism, so in theory you can't capture or record the music.
Even if you do find a workaround, without the rights for the music you will face copyright issues if this is for an app you want to make available to others, unfortunately.
I'm on the final steps of finalizing a video.js heavy website and am running into one problem that I really don't want to leave to experimentation. I was wondering if there were any recommendations for streaming over 4G/3G.
The problem occurs specifically when streaming HLS using the hls-contrib tech over 4G/3G. Both Android (Lollipop) and iOS 9 phones immediately pick up audio but never get video, or do several minutes later (a normal user would stop watching by that point). When I plug in an Android or iOS device for console debugging, there are no console errors (symptoms persist), and turning on WiFi gets me both audio and video. Which leads me to believe I'm just dealing with an encoder settings problem here. Their signal chain is encoder (setting below) to a Wowza server, and out to ScaleEngine as a CDN. They're of course using the CDN HLS and RTMP links for public playback.
Encoder Settings:
Video: H.264 # 1,250Kbps CBR, 720x480#30i
Audio: AAC # 96Kbps Stereo
Profile: Main
Level: 3.1
B-Frames: 0
Don't see any other pertinent info.
Appreciate the help/opinions/general information. Thanks.
I'm having some trouble working with the iOS Audio frameworks to create a simple app. I would like to record audio through the Microphone and play it back to the user while recording.
I have tried each of the audio framework layers(AVFoundation, Audio Queue API, and RemoteIO), but have only found old documentation and broken examples.It seems like a simple request that AVFoundation should handle, but I have explored the following other SO questions and still find myself circling for hours to get the hang of this. Here is what I have reviewed:
iOS: Sample code for simultaneous record and playback (Other SO Users also state the accepted answer is not concrete and difficult to implement even with a delay of ~70ms.)
Record and play audio Simultaneously (From 2010 and very high level, I have downloaded the sample code and can't find a working example that does simultaneous playback and recording).
Adjust the length of an AudioUnit Buffer (RemoteIO is so confusing to me right now, is this really required?)
I have also downloaded and reviewed both the SpeakHere and AurioTouch sample projects from Apple. I promise I wouldn't post up without hours of googling and struggling. You can see "record audio and playback iOS simultaneously" returns many dated and non-working examples.I know myself and the community could really benefit from some updated documentation and examples in the audio section. RemoteIO seems to be too advanced for such a simple task. Thanks again for your help and consideration.
The appropriate way to do this is via AudioUnit APIs, even though it seems like a common scenario which should be handled by higher level APIs.
I wrote a small demo app using AudioUnit. You're free to try it our and modify it for suiting your purpose. The demo app does record audio and play it simultaneously, but it's recommended to use a ear phone to see the effect.
The RemoteIO Audio Unit is the only way to play back what is being recorded with low latency. RemoteIO is low latency because it runs audio callbacks in a separate dedicated real-time thread which is why it is fast, but also why it is a bit more complex to code. All the other iOS audio APIs are built on top of RemoteIO and thus add latency.
You will also need to configure the app's Audio Session APIs to request low latency with the appropriate audio session type. The foreground app can request and get audio input and output latencies as low as 5.6 milliseconds on most iOS devices most of the time.
I'm using MobileVLCKit to stream video and audio from Wowza RTMP server. At the same time I'm using VideoCore]1 to stream audio to Wowza RTMP server (I closed off the video channel in VideoCore). Now I'm attempting to make this sort of a teleconferencing solution. Now I'm limited to RTMP or RTSP, not teleconferencing solution (WebRTC or SIP or what not...I am not familiar with these at the moment) because of the limitation on the other end of the line.
The above setup doesn't work. Turning the both functions (video and audio streaming down and audio streaming up) individually runs fine. But not when run simultaneously as audio cannot be heard on the other end. In fact, when app started with VideoCore streaming audio upstream, as soon as I started to downstream via MobileVLCKit, audio cannot be heard on the other end, even though the stream is open. It appears that microphone is somehow wrested away from VideoCore, even though MobileVLC should not need the microphone.
However, when I made the two into two apps and allow them to run in the background (audio & airplay background mode), the two runs fine with one app stream down video & audio and the other picking up microphone voices and stream to the other end.
Is there any reason why the two functions appear to apparently be in conflict within the same app, and any ideas how to resolve the conflict?
I encountered the same problem. Say I have two objects, a vlc player and the other audio processor which listens the microphone. It works fine in simulator for operating both functions in the same time. But conflict in the iPhone device. I think the root cause is that there is only one position or right for listening the microphone. And vlc occupies the right so that my audio processor cannot work. But for some reasons, I cannot modify the vlc code. So I'd to figure out the workaround resolution. And I found one.
The problem comes from vlc which occupies the right but doesn't event use the microphone, and my audio processor did. So the way appears clearly. That is, vlc player plays first and then we new the other object instance, audio processor in my case, which needs to listen the microphone. Since audio processor comes after vlc player, it takes back the right of microphone listening. And they both work properly.
For your reference and hope it can help you.
I want to process the stereo output from iOS devices, no matter what application causes them and visualize it in real-time.
Is it possible to use the generic output device (or anything else) to get at the audio data which are currently being played? Maybe as an input to a remoteIO unit?
In other words: I want to do what aurioTouch2 does (FFT only) but instead of using the microphone as input source, I want to process everything which is coming out of the speakers at a given time.
Kind regards
If your own app is playing using the RemoteIO Audio Unit, you can capture that content. You can not capture audio your app is playing using many of the other audio APIs. The iOS security sandbox will prevent your app from capturing audio that any other app is playing (unless that app explicitly exports audio via the Inter-App Audio API or equivalent).