I'm in the process of creating an iPhone app to record/playback user dictations with the resulting file being uploaded to a server for speech recognition and transcription. I have everything working apart being able append or insert dictation into a recording thats being created. For example when a user creates a dictation they will record some speech, then play back some or all of that speech recording, then switch back into recording. The subsequent speech recording should append itself if the payback reached its end or overwrite the original record at the point the user stopped the playback.
I have been looking for a solution to this for a couple of weeks now and found nothing but vague suggestions, I also got a copy of the Learning Core Audio book but I'm no further forward. If anyone can shed some light on how to proceed I would be most grateful. Thanks in advance.
Related
I have been trying to track this down and there doesn't seem to be a consistent answer. If I have a website that tries to play multiple songs in a row (think playlist) using the HTML 5 audio element, can it continue to work on the iOS lock screen?
For some background, this answer seems to indicate it may be possible. But then this article suggests it is not.
I tried following the closest example of what Apple recommends, as found here, to replicate this. I am using plain, vanilla javascript and HTML, nothing fancy. I copied their example exactly and just substituted the audio tag for the video one, picking two random mp3 songs. All it does is wait for one song to end, then switch the src, load, and play the next track.
When I hit Play on the website, I then lock the iPhone. The first song will play, then stop. It does not continue to the next song.
If the website is open to the page, it will properly transition to the next song. On Android, it will continue to the next song even if the phone is locked.
I tried this with iOS 11 and 12. Neither worked. I have read many differing answers about how javascript is stopped when the website isn't in the foreground, and how iOS requires user interaction to play audio (even going from one song to the next). So it doesn't seem like this would work. But then other answers out there seem to indicate this is possible.
Is there a definitive yes or no? Am I doing something wrong here? Or is it just not possible?
There are multiple issues, which is causing some of the confusion.
First track plays, second does not due to user permission
If the first track was started with user permission, then the only way you can switch to a new track via script is with Apple's recommendation. Handle the ended event and swap in a new src, and call .play() before the callback completes. Otherwise, your code will not have permission to start the new audio.
Cannot get to ended event due to time-constrained source (#t=5,10)
Say you have a 30-second audio file and you tell the browser to only load seconds 5 through 10, via something like #t=5,10 at the end of the URL. When you reach 10, a paused event will fire, not ended. Therefore, it isn't possible to go on to the next track normally, as this event is not "blessed" by Apple to count as a relay of the previous user interaction.
I got around this by having my JavaScript check the currentTime repeatedly, and when it crossed a threshold, seek it to the duration (end) of the file myself. This allows ended to fire normally.
iOS doesn't allow scripts to run after the screen is locked or other apps are in use
This one is a real problem to debug, since it doesn't turn up in any of the simulators. In any case, you're right, your JavaScript is going to get suspended so even if you follow Apple's recommendations, you're out of luck. Or, are you?
A hack... set up a ScriptProcessorNode on a Web Audio context. Set the buffer size to like 512 samples or something. Then, on script process, force a timeupdate event for your Audio Element. This keeps your JavaScript there ticking away. ScriptProcessorNode has to have special privileges here, by its nature.
Apple: If you read this, please fix your browser. It would be great if we didn't have to hack around things to make something as simple as an audio playlist to work. Other browsers don't have nearly this level of problems.
Here's the needs: People record their video via my App and upload stream to our website, you know, like a live show.
When they are recording, the network goes bad, then record audio instead of video.
And when audience watch, first comes the video, in a certain time comes the audio with a static picture.
How am I supposed to do that? Thanks.
I got this after thinking. So I'll answer by my self.
If you wanna record video and audio separately. Use lower level Library to handle this. Then upload it. And you can get the upload speed, When the speed goes low, you can shut down the video upload process.
I'm researching to start developing an audio streaming apps. First thing I'd like to ask is the framework to use for it. I've tested DOUAudioStreamer , tumtumtum/StreamingKit , RadioKit and etc.
The framework needs to be able to seek a song from certain seconds with low latency. Spotify has that amazing seek feature i.e. find a song you've never listened before and play it, then slide the time slider to the middle, the song continues to play with low latency as if it has downloaded the whole song before.
I guess you could do some predictive analysis as to what songs the user will listen to next, i.e. start caching songs before they actually start to play. That will at least improve the experience somewhat.
I'm building an app in which I need to steam multiple tracks of audio that make up a song. They all need to be synchronized so the song plays back naturally.
I've been able to play back local multitrack audio very well with the solution on this thread: multi track mp3 playback for iOS application, but it looks like AVAudioPlayer isn't able to stream.
I've been looking into working with DOUAudioStreamer because I've read that it's the best solution for streaming audio on iOS without going pretty low-level, but it seems to lack the equivalent of a -playAtTime: method, which is how the tracks were synced up using `AVAudioPlayer.
Does anyone know a workaround for this using DOUAudioStreamer, or have any advice on another way I should approach this? Thanks.
I'd like to stream video from the camera on an iOS device to a receiver via wifi, in effect turning the device into a wireless webcam. Is there a way to build a small app that captures video input on an iOS app and sends it via an RTSP stream or similar?
As this is an ad hoc experiment, I'm not concerned about App Store guidelines and can jailbreak if necessary.
If I interpret your question correctly you more or less need to solve four problems:
Get the camera feed.
Convert/encode this to the right format.
Stream the data.
Prevent the phone from locking itself and going into deep sleep.
The first one is fairly simple and Apple has as always provided good documentation and examples -> API link. Make sure you check out their example in the end as you will get a CMSampleBufferRef data object back.
For the second and third part, you should check out the CFNetwork framework and specially CFFTPStream for streaming using FTP.
If your are only building this for yourself then you can always turn off the Auto-Lock feature in the settings. If you on the other hand would like to distribute this to other users you could use a trick to play a mute sound every 10 seconds. This is more or less how all the alarm clocks work in the App Store. Here's a tutorial. =)
I hope I helped a little bit at least.
Good luck and best regards!
I'm 70% of the way to doing the same thing. Here's how I did it:
Capture content from video input
Chop video into files for use in HTML Live Streaming.
Spin up a web server on the iPhone and make the video files available.
Connect to the IP address of the phone and viola! you've got live streaming video.
Last time I touched the code I was trying to debug my Live Streaming not working. I'll try and get my source code posted on github this weekend, if you'd like to take a look.