I have a question regarding the synchronization of 2 Directsound streams.
To record and play sound I currently use Portaudio to open 2 Directsound streams.
There are 2 callback functions which are called every time the input buffer is filled and the output buffer needs data.
Now here`s my problem...
The input stream is running at 48kHz samplerate (#1024 samples). The output stream is running at 192kHz samplerate (#4096 samples). Every time the input buffer is filled and the callback is called I do some DSP and after that I convert the result to 192kHz. The output stream takes the result and outputs the data. Now the 2 streams are running completely out of sync.
I have looked through the entire Portaudio API but I cant`t find a sync option to lock the 2 streams together.
Is there any way to lock 2 Directsound streams? I really need 48kHz input and 192kHz output.
Br,
Vincent Bruinink.
The thing is that you can't really open two streams "at the same time", nor can you open two devices (or even one device at two different sample rates) and expect them to stay truly in sync, even if they were, at one time, in sync. To understand why, you may want to read something about how audio works on a computer. You may also want to read this document, which is specific to PortAudio.
As an alternative, you may want to consider opening a single device in a single stream and using software sample-rate conversion.
Related
I am trying to create a RTSP client which live broadcast Audio and Video. I modified the iOS code at link http://www.gdcl.co.uk/downloads.htm and able to broadcast the Video to server properly. But now i am facing issues in broadcasting the audio part. In the link example the code is written in such a way that it writes the Video data to file and than reads the data from the file and upload the NALU's video packets to RTSP server.
For Audio part i am not sure how to proceed on it. Right now what i have tried is that get the audio buffer from mic and than broadcast it to the server directly by adding RTP headers and ALU.. but This approach is not properly working as Audio starts lagging behind and lag increases with time. Can someone let me know if there is some better approach to achieve this and with lip sycn audio/video.
Are you losing any packets on the client? If so, you need to leave "space." If you receive packet 1,2,3,4,6,7, You need to leave space for the missing packet (5).
The other possibility is a what is known as a clock drift problem. The clock (crystal) on your client and server are not perfectly in sync with each other.
This can be caused by environment, temperature changes, etc.
Let's say in a perfect world your server is producing audio samples 20ms audio samples at 48000 hz. Your client is playing them back using a sample rate of 48000 hz. Realistically your client and server are not exactly 48000hz. Your server might be 48000.001 and your client might be 47999.9998. So your server might be delivering faster than your client or vise versa. You would either consume packets too fast and under run the buffer or lag too far behind and overflow the client buffer. In your case, it sounds like the client is playing back too slow and slowly lagging behind the server. You might only lag a couple milliseconds per minute but the issue will keep continuing and it will look like a 1970s lip synced Kung Fu movie.
In other devices, there is often a common clock line to keep things in sync. For example, Video camera clocks, midi clocks. multitrack recorder clocks.
When you deliver data over IP, there is no common clock shared between a client and server. So your issue concerns syncing clocks between disparate devices with no. I have successfully solved this problem using this general approach:
A) Let the client count the rate of packets that come in over a period of time.
B) Let the client count the rate that the packets are consumed (played back).
C) Adjust the sample rate of the client based on A and B.
So your client requires that you adjust the sample rate of the playback. So yes you play it faster or slower. Note that the playback rate change will be very very subtle. You might set the sample rate to be 48000.0001 hz instead of 48000 hz. The difference in pitch would be undetectable by humans as it would only cause a fraction a cent difference in pitch. I gave an explanation of a very simplified approach. There many other nuances and edge cases that must be considered when developing such a control system. You don't just set it and forget it. You need a control system to manage the playback.
An interesting test to demonstrate this is to take two devices with the exact same file. A long recording (say 3 hours) is best. Start them at the same time. After 3 hours of playback, you will notice that one is ahead of the other.
This post explains that it is NOT a trivial task to stream audio and video.
I have a PCM stream incoming from the microphone. I am analyzing short chunks (Java language) of it to detect short spikes in sound loudness (amplitude). I have a determined sound that plays periodically and I need to know if detected spike is in fact this sound recorded. I have the PCM for sound played, it's completely determined.
I have no clue where to start, should I perform some comparison in time domain or frequency domain? Would be great if someone could give me some insight on how this is done and where should I dig.
Thanks.
It sounds like you want to compare an incoming set of pulses to a references set of pulses. Cross-correlation is probably what you want to use. You may need to precondition your data first, eg create an envelope instead of using raw data, or the cross-correlation may fail unless the match is perfect.
I'm trying to send music over bluetooth from one iOS device to another. I've been using this to build packets like in Ray Wenderlich's SNAP tutorial, but I've been having trouble reconstructing the packet information on the receiving phone. I have tried using https://github.com/abbood/iphoneAudioSyncer but I think it is too complicated for my needs (since I do not need synced playing). What is the simplest buffer approach that accounts for things like lost/out of order packets? I have read through a lot of CoreAudio stuff but it is very dense, so I would appreciate help from someone who has tackled this type of problem.
when you talk about los/out of order packets.. you're talking about the topic of Packet Loss Concealment.. which is a very dense topic (I mean if you think core audio is dense.. wait till you dive into PLC).
In a nutshell, there are many ways to deal with packet loss.. but the simplest way (which I advise you to do) is to replace the lost packets with silence (same goes with out of order packets.. if a packet is out of order.. just discard it).
that being said.. you are dealing with audio that is streamed to you (ie sent via the bluetooth/wifi network).. which means in almost 100% of the time it's compressed audio you're getting (ie Variable Bit Rate audio VBR).. if you simply try to substitute lost VBR packets with silence.. you'll run into this problem. You'll either have to insert silence packets in the same compression format as the VBR audio you're dealing with, or you will have to convert your VBR compressed audio into non-compressed audio (Lossless PCM), then insert zeros in place of the missing packets.
I have a DirectShow push source filter and a DirectShow simple audio mixer filter both written in Delphi 6 with the help of the DSPACK component library. In my app, I build a filter graph manually and for the pin connections I use IFilterGraph.ConnectDirect() to avoid any interference from DirectShow's "intelligent connection" technology. I am using both of those filters as private/unregistered filters internal to my program.
The graph I build has a capture filter and my push source audio filter sharing the head position of the graph. Their output pins are connected to my simple audio mixer, the latter supporting multiple input connections. The mixer forces all connections to its input and output pins to be the exact same media format type that is preset in its constructor. In this case the format setting I'm using is WAV format with a sample rate of 8000, 16 bits per sample, and one channel. Note, I am using DecideBufferSize() to set all filters to a buffer size of 50 milliseconds. This results in buffers being delivered that are 400 bytes (200 samples) large.
The capture filter is an external COM object that I find using the DirectShow API. Currently I am assigning my VOIP phone as the device (Moniker). For some strange reason my push source filter is pumping out buffers at a rate of exactly 7 times that of the capture filter. In other words, my mixer filter is getting 7 buffers from my push source filter for each buffer it receives from the capture filter. I know this because I debug print a line every time the mixer filter gets a buffer and I identify the filter that is the source of the buffer.
I don't know how the capture filter is forming its timestamps since it is external code, but I would expect its the usual scheme. My push source filter starts at zero and with each FillBuffer() call increments the timestamp in DirectShow reference time format by the amount of time the buffer represents.
Here are my questions:
1) Should the timestamps even matter if I am building the graph manually? Does DirectShow get in-between the filters and can somehow affect the timing of pin writes (Receive calls) even if you build the graph completely manually?
2) What common mistake could cause a filter to push out buffers too fast, despite a homogeneous media format all around the graph?
In DirectShow source/push fitlers are normally either live or non-live. Both inject data into pipeline, and the important difference is that a live filter streams data as soon as possible, as soon as it generates, receives from outside of pipeline (such as from network) etc.
A non-live filter pushes as much data as it can. A fitler that plays 5 minutes long MP3 file? It is prepared to inject all five minutes at once. It is a task of a renderer filter to block streaming when no more buffers available and to honor presentation time. So when source filter loads 100% of buffers, it just cannot push anything any more until buffers are released by playback.
The important part of this behavior is to timestamp media samples correctly. If one fails to time stamp, the renderer would not be able to present data on time, and could be showing/playing media too slow, or too fast.
Could someone explain in terms of Audio Unit connections how to modify the iPhone microphone data stream visible to other processes with gain or EQ? I understand how to use a remote I/O unit to grab mic data and do my processing. I want this new data to replace the original mic data stream, not go to speakers or a file. "Audio Unit Hosting Fundamentals" Figure 1-3 is close.
I have read everything out there on Audio Units and used several of the online examples (Tim B, Play It Loud, Tasty Pixel) but don't see how to do this yet.
Any help?
Thanks
This doesn't seem to be clearly explained or illustrated in the documentation. However, if you look at the AURIOTOUCH sample code, you will see how within the remote I/O render callback, it makes a call to retrieve data from the microphone. then it optionally processes this data, and returns it.
this is kind of doubly useful because this call to retrieve microphone data returns already created buffesr. this means you don't have to create your own buffers, which is great becaues that is a bit of a hassle.