Basically i'm gonna be working on an iOS music app which when a song is being played, it shows the fancy Equalizer meter, Something like this but with all the animation of bars going up and down:
After looking into this and not finding enough resource, I really want to carry this as a project perhaps making a web version using j query.
I'm not really asking for specific code, i just want to know how the animation works in general ?
Thanks a million !!!
Checkout the Cocoa Waveform Audio Player Control project. It's a cocoa audio player component which displays the waveform of the audio file.
Also, there is already a lot of questions on this topic:
iOS FFT Accerelate.framework draw spectrum during playback
Using the Apple FFT and Accelerate Framework
iOS FFT Draw spectrum
Animation would be pretty straight forward. It is just animating changes of the height of rectangles.
Related
As an introduction and context, I'm currently a novice iOS app developer and I want to make sure I'm not reinventing the wheel too much as I make this app (reinventing wheels can get very expensive.)
The app will allow the user to download our videos off the internet and will allow storage for offline usage. The problem with storing these videos on the device is that many of them will be too long and thus too big to be practical to store.
The videos are quite simple however, consisting of a couple short "real" video clips at the beginning and end, with the bulk of the video being still images animated around the screen. The animations would consist solely of opacity and simple transformation keyframes (translate, scale, rotate around static anchor point), and would require a variety of easing functions for each transition.
The hardest part likely would be that the "video" player will also have to be able to track with an audio player's timecode, and will have to support seeking to any arbitrary point like a normal video player.
So, now that I've described the problem, here's the solution I've come up with so far. Hopefully doing it this way will reduce the probability of XY problems. :)
The idea is to basically do a dumbed-down version of what Final Cut and other editing programs do with animations—have a bunch of clips, sometimes overlapping, and be able to animate the position, scale, rotation, and opacity of each using keyframes.
My first instinct as far as implementation goes is to use some of iOS's game engine stuff to do animations (maybe SceneKit because it seems to allow animations to use scene time as opposed to real time, despite the fact that it's primarily 3d and I am doing 2d animations) and manually handle syncing time with the audio player, as well as manually handling the adding and removing of nodes from the scene when seeking through the video and when clips begin/end.
What are some built-in systems, plugins, etc. that I can take advantage of to make this easier and faster to develop and maintain? Double points if I don't have to transcode the animations by hand to some custom format.
As I mentioned in my comment your question is rather broad and contains multiple questions in one, I will address what you mentioned to be likely the hardest part:
https://developer.apple.com/documentation/avfoundation/avplayeritem
https://developer.apple.com/documentation/avfoundation/avasset
Instead of SceneKit, take a look at SpriteKit and its SKVideoNode.
Also, research Metal video processing. There are quit a few example projects available you could use as a starting point.
Basically I have an SKScene, and I want to play a video over the scene. The video is confetti falling with an alpha background. It will play when the player gets a high score. I am using an SKScene with shapes and images drawn with shape nodes and image nodes. I just was wondering if anyone could please tell me how to play the video over the screen and still see the game in the back, and be able to touch the buttons through the video. It is supposed to look like an animation playing.
I am using a video because I was just thinking that playing a video would be more processor efficient than having the game generate particles.
There is no built-in iOS solution. You can play 24BPP (fully opaque) movies under iOS, but the only built-in way to display alpha channel video would be to load a series of PNG images with alpha. Downside is that takes up a huge amount of memory and it bloats the app download. If you want to have a look at some working examples of this kind of functionality with a 3rd party app then see Alpha Channel Examples. You might also be interested in this blog post which shows example code of how to impl Alpha channel textures in OpenGL would could be implemented on top of SpriteKit too. The cube example shows rendering an alpha channel movie onto a cube, it was adapted from a Ray Wenderlich tutorial.
Here as an answer how to do that with GPUImageView. Also project on GitHub here and similar question from stackoverflow
The video stack doesn't yet support formats with alpha. For confetti, you should use SKEmitterNode. Size it to the area you envisioned for your video, and see Creating Particle Effects, i.e., its link to Add a particle emitter to your project and try out the "Snow" effect. It looks more like confetti when you give it a different color than white. Click the dot under "Color Ramp" to set the color.
I'm looking for a way to create a audio bars visualizer similar to this in iOS.
Every white bar will move up and down depending of audio wave. I'm really lost because haven't much experience dealing with audio in Objective-c.
EDIT: What i'm seeking is what Overcast's app does on its visualizer (the group of vertical orange bars on the lower part of the podcast's image)
Anyone can help?
Thanks
EDIT: Thanks to Tomer's answer I finally made it. First I did this tutorial in order to make it all clear. Then I created my own VisualizerView for my project, you can find it in this gist. Maybe is not perfect but it does what I needed to do.
Generally, you have a few options if you want to get an idea of what something sounds like in iOS:
Use the simple AVAudioPlayer audio player, and then use the [audioPlayer averagePowerForChannel:] method to get the avarage audio level for the current moment. Check out this tutorial.
Use the Audio Queue API, which lets you send whatever audio you want to the speaker: You would read audio from your source and fill the buffers with it every time. (If you're reading from a file, use AVAssetReader) This way you always know exactly what waveform you're playing, so you can, for example, calculate its avarage power or process it in other ways like FFT. Then you'd update the bars accordingly.
EDIT: The standard way of doing such a thing is to use the Fast Fourier Transform (FFT) - it extracts frequency information from a sound. Here's a good example of using it on iOS (Apple's guide here). But, of course, to use it you have to know exactly what waveform you're playing every time, so you'd probably want to use a lower-level API such as Audio Queue.
I'm going to ask this at the risk of being too vague or asking too many things in one question, but I'm really just looking for a point in the right direction.
In my app I want to record audio, show a waveform while recording, and scroll through the waveform to record and playback from a specified time. For example, if I have 3 minutes of audio, I should be able to scroll back to 2:00 and start recording from there to fix a mistake.
In Voice Memos, this is accomplished instantaneously, without any delay or loading time. I'm trying to figure out how the did this, if anyone has a clue.
What I've tried:
EZAudio - This library is great, but doesn't do what I want. You can't scroll through the waveform. It deletes the waveform data at the beginning and begins appending it to the end once it reaches a certain length.
SCWaveformView - This waveform is nice, but it uses images. Once the waveform is too long, putting it in a scroll view causes really jittery scrolling. Also you can't build the waveform while recording, only afterward.
As far as appending, I've used this method: https://stackoverflow.com/a/11520553/1391672
But there is significant processing time, even when appending two very short clips of audio together (in my experience).
How does Voice Memos do what it does? Do you think the waveform is drawn in OpenGL or CoreGraphics? Are they using Core Audio or AVAudioRecorder? Has anyone built anything like this that can point me in the right direction?
When zoomed-in, a scrollview only needs to draw the small portion of the waveform that is visible. When zoomed-out, a graph view might only drawn every Nth point of the audio buffer, or use some other DSP down-sampling algorithm on the data before rendering. This likely has to be done using your own custom drawing or graphics rendering code inside a UIScrollView or similar custom controller. The waveform rendering code during and after recording don't have to be the same.
The recording API and the drawing API you use can be completely independent, and can be almost anything, from OpenGL to Metal to Core Graphics (on newer faster devices). On the audio end, Core Audio will help provide the lowest latency, but Audio Queues and the AVAudioEngine might also be suitable.
I am interested in a way to play sounds from specific points in space relative to the user.
Basically I would like to say the user is at point (0,0) and a sound came from (10,10) and then take a sound and send it through some library that plays it, sounding as though it came from the source (10,10). Performance in doing this would be very important.
If it wasn't painfully obvious from reading the question, I have very little experience with audio on any device.
After doing a little research, it seems the options are to use the OpenAL framework which is supported by apple, or essentially roll your own on top of Audio Unit.
There is a 3D Mixer Audio Unit that apple provides, which requires you to develop a lot of understanding of Audio Units.
Then there is OpenAL which is a cross platform audio framework where you can position a "source" and a "listener" and it will compute attenuation and stereo for you.
Both require low level understanding of playing Audio and are not very fun. So I figured I might as well jump all the way in the water and learn about the Audio Units, since I may want to do some more specialized stuff in the future.
This is an easy wrapper for the iOS OpenAL functionality: ObjectAL-for-iPhone
Play around with the example and see if it does what you want.