we´ve created a custom push source / parser filter that is expected to work in a directshow
editing services timeline.
Now everything is great except that the filter does not stop to deliver samples when the current
cut has reached it´s end. The rendering stops, but the downstream filter continues to consume
samples. The filter delivers samples until it reaches EOF. This causes high cpu load, so the application
is simply unusable.
After a lot of investigation I’m not able to find a suitable mechanism that can inform my filter
that the cut is over so the filter needs to be stopped :
The Deliver function on the connected decoder pins always returns S_OK, meaning the attached decoder
is also not aware the IMediaSamples are being discarded downstream
there’s no flushing in the filter graph
the IMediaSeeking::SetPositions interface is used but only the start positions are set –
our is always instructed to play up to the end of the file.
I would expect when using IAMTimelineSrc::SetMediaTimes(Start, Stop) from the application
that this would set a stop time too, but this does not happen.
I’ve also tried to manipulate the XTL timeline adding ‘mstop’ attributes to all the clip in the
hope that this would imply a stop position being set, but to no avail
In the filters point of view, the output buffers are always available (as the IMediaSamples are being discarded downstream),
so the filter is filling samples as fast as it can until the source file is finished.
Is there any way the filter can detect when to stop or can we do anything from the application side ?
Many thanks
Tilo
You can try adding a custom interface to your filter and call a method externally from your client application. See this SO question for a bit more of details on this approach. You should be careful with thread safety while implementing this method, and it is indeed possible that there is a neater way of detecting that the capturing should be stopped.
I'm not that familiar with DES, but I have tried my demux filters in DES and the stop time was set correctly when there was a "stop=" tag for the clip.
Perhaps your demux does not implement IMediaSeeking correctly. Do you expose IMediaSeeking through the pins?
I had a chance to work with DES and custom push source filter recently.
From my experience;
DES actually does return error code to Receive() call, which is in turn returned to Deliver() of the source, when the cut reaches the end.
I hit the similar situation that source does not receive it and continues to run to the end of the stream.
The problem I found (after a huge amount of ad-hoc trials) is that the source needs to call DeliverNewSegment() method at each restart after seek. DES seems to take incoming samples only after that notification. It looks like DES receives the samples as S_OK even without that notification, but it just throws away.
I don't see DES sets end time by IMediaSeeking::SetPositions, either.
I hope this helps, although this question was very old and I suppose Tilo does not care this any more...
Related
I have a really specific use-case where in prod I have to play back a continously appended wav file in my java application (I have no way of modify this scenario).
My problem is that when I open this wav file for playback libvlc handles the duration calculation, and after a while it detects EOF, despite the file actual length is much larger since we opened it (my guess is because of buffered playback). This causes the player to stop (raise the finished event). As of today I restart the playback and set the time to the end position from this finished event. This causes little pauses and unacceptable in the final product.
I try to implement a logic in libvlc (vlc 3.0.6 version) that will handle this problem, but I can't really find out the proper way of doing it.
My approach would be to recalculate the duration of the wav file in case it's detecting EOF. If the new duration equals the old one than it is really EOF else it can continue playback.
I have tied to modify the VLC_DEMUXER_EOF handling in input.c, follow the file end trace, modify the demux-run.c and wav.c process, and to play around with the event handling (finished/stopped), but cannot get much closer to a valid solution.
I would really appreciate some help with this one, because I loosing my hair rapidly in the last couple of days. (I'm open for alternatives too, if you have some idea.)
I'm not sure which binding you are using, but I'm assuming vlcj since you mentioned Java.
Anyway, one solution could be to use libvlc_media_new_callbacks. Doc: https://www.videolan.org/developers/vlc/doc/doxygen/html/group__libvlc__media.html#ga591c3cbe56444f1949165b2b9b75d8e2
Implementing these custom callbacks will allow you to tell libvlc explicitly to wait. You can do this in the libvlc_media_read_cb callback, where the documentation states:
If no data is immediately available, then the callback should sleep.
You should find how this API is exposed through whichever binding you use and then use it from Java code.
My solution was to modify the wav file handling in the libvlc source, and build a new vlc player for this specific problem.
By updating the i_data_size in the wav.c Control method, with the stearm_Size(p_demux->s) the player was able to manage the appending, and plays back the files like they were streams (this possibly generates issues with the lenght_change event).
And secondly I have to manage occasional collision, when the appender allocates the file, and stream block operations cannot be executed. I managed this problem by implementing a retry mechanism to the stream.c vlc_stream_ReadRaw method. This will retry the s->pf_read(s, buf, len) call x times with some microsleep, if it returns 0 (it means EOF in usual playbacks, but here it can indicate failed operation).
This is NOT a proper solution by any means, but I was in a hurry, and had to make it work. I will accept the solution described by mfkl.
Is it OK to create multiple ExecutionEnvironments in a Flink program? More specifically, create one ExecutionEnvironment and one StreamExecutionEnvironment in the same main method, so that one can work with batch and later transit to streaming without problems?
I guess that the other possibility would be to split the program in two, but for my testing purposes this seems better. Is Flink prepared for this scenario?
All seems to work fine, except I am currently having problems with no output when joining two streams on a common index and using window(TumblingProcessingTimeWindows.of(Time.seconds(1))). I have already called setStreamTimeCharacteristic(TimeCharacteristic.EventTime) on the StreamExecutionEnvironment and even tried assigning custom watermarks on both joined streams with assignTimestampsAndWatermarks where I just return System.currentTimeMillis() as the timestamp of each record.
Since it finishes really quickly, both streams should fit in that 1-second window, no? Both streams print just fine right before the join. I can try supplying the important parts of code (it's rather lengthy) if anyone's interested.
UPDATE: OK, so I separated the two environments (put each inside a main method) and then I simply call the first main from the second main method. The described problem no longer occurs.
No, this not supported, and won't really work.
At least up through Flink 1.9, a given application must either have an ExecutionEnvironment and use the DataSet API, or a StreamExecutionEnvironment and use the DataStream API. You cannot mix the two in one application.
There is ongoing work to more completely unify batch and streaming, but that's a work in progress. To understand this better you might want to watch the video for this recent Flink Forward talk when it becomes available.
I am using julius speech recognition for my application. I have one doubt regarding julius:
I have downloaded the latest version and was successful in using its lib and making it work. the problem I am facing is..once the app starts and I call the voice recognition function in my application...it takes the input from mic and displays whatever is said in the mic, but the function still continues to do so again and again. The control will never come out of that function. Here I am facing problem since the control is not returning back I am not able to proceed further. What I want is once the engine gets input from mic it should recognize and stop there.. which I tried to do by deleting the callback function but was unsuccessful.
Can anyone please guide me in this matter, what I need to do to get the desired output. It will be helpful for me.
As discussed in the same post on VoxForge:
You have a couple of choices: first to use the Julius -input control to get the sound data from a list of files (see the .jconf sample file), so that when the list (even if only length one) is exhausted then Julius stops. It is quite easy to record the voice input to a file and then feed the file into Julius. Second you can put a dialog manager in control. If you need more information on what a dialog manager does there are many posts on this forum on that subject accessible by a search.
The basic function of Julius is to start up and then keep on decoding input. When you get more experience you can run Julius as a server, and then tell the server to respond, not respond or shut down as required. It's more efficient than having Julius start and stop all the time.
When an avenue exists for a complex application to yield the required result by using an effective combination of options at run time, editing the application, while possible, might involve a lot of unnecessary work. The emphasis then shifts to passing the options correctly in whatever script is being used to access Julius.
I am trying to create an app using a combination of AVAudioPlayerNode instances and other AUAudioUnits for EQ and compression etc. Everything connects up well and using the V3 version of the API certainly makes configuration easier for connecting nodes together. However during playback I would like to be able to automate parameter changes such a the gain on a mixer so that the changes are ramped (eg. fade out or fade in.) and feel confident that the changes are sample accurate.
One solution I have considered to install a tap on a node (perhaps the engine's mixer node) and within that adjust the gain for a given unit but since the tap is on the output of a unit this is always going to be too late to have the desired effect (I think) without doing of offset calculations and then delaying my source audio playback to match up to the parameater changes. I have also looked at the scheduleParameterBlock property on AUAudioUnit but it seems I would need to implement my own custom unit to make use of that rather than use built-in units even though it was mentioned in
WWDC session 508: " ...So the first argument to do schedule is a sample
time, the parameter value can ramp over time if the Audio Unit has
advertised it as being rampable. For example, the Apple Mixer does
this. And the last two parameters, of course, are function parameters
are the address of the parameter to be changed and the new parameter
value.... "
Perhaps this meant that internally the Apple Mixer uses it and not that we can tap into any rampable capabilities. I can't find many docs or examples other than implementing a custom audio unit as in Apple's example attached to this talk.
Other potential solutions I have seen include using NSTimer, CADisplayLink or dispatchAfter... but these solutions feel worse and less sample accurate than offsetting from the installed tap block on the output of a unit.
I feel like I've missed something very obvious since there are other parts of the new AVAudioEngine API that make a lot of sense and the old AUGraph API allowed more access to sample accurate sequencing and parameter changing.
This is not as obvious as you'd hope it would be. Unfortunately in my tests, the ramp parameter on scheduleParameterBlock (or even the underlying AudioUnitScheduleParameters) simply doesn't do anything. Very odd for such a mature API.
The bottom line is that you can only set a parameter value within a single buffer, not at the sample level. Setting a parameter value at a sample time, will automatically ramp from the current value to the new value by the end of the containing buffer. There seems to be no way to disable this automatic ramping.
Longer fades have to be done in sections by setting fractional values across multiple buffers and keeping track of the fade's relative progress. In reality, for normal duration fades, this timing discrepancy is unlikely to be a problem because sample-accuracy would be overkill.
So to sum up, sample-level parameter changes seem to be impossible, but buffer-level parameter changes are easy. If you need to do very short fades (within a single buffer or across a couple of buffers) then this can be done at the sample-level by manipulating the individual samples via AURenderCallback.
I am transmitting and receiving data using aux cable and GNU RADIO between two laptops.
I have implemented DQPSK using PSK mod block.
The problem is that while receiving I have to provide a delay, some integer value e.g 0,1,2 etc.
It is different every time.
Is there a way to dynamically check for the right delay value or any other workaround to this situation?
I have written 'start.' at the start of data being transmitted and 'end.' at the end.
I have to give a demo for this project and I dont want to manually change the delay at runtime.
I cannot find the .cc file of file sink in GNU RADIO, I can change the C++ code according my requirement but there is no such file.
Below is the screenshot of the grc file on the receive side.
Any help will be appreciated.
Since there's no way for the receiver to know when the transmitter started transmitting, it decodes stuff before there's actually anything to decode.
In essence, you need some kind of preamble or so to tell your receiver when to start – side effect of having something like that would be that you could correct some things (the two sound cards don't share the same oscillator, which leads to a symbol rate offset, and a center frequency offset).
You basically added that framing - your start. and end. strings.
I cannot find the .cc file of file sink in GNU RADIO, I can change the C++ code according my requirement but there is no such file.
It's in gr-blocks/lib; however, you shouldn't modify the file sink. Really,
I'd recommend you take the time to go through the guided tutorials, use gr_modtool to generate a general block which has a state machine that looks for the bits of your start string and drops everything before and including those, and then passes everything till it sees the stop string. That all can be done with a single state machine, and a bit of python or C++ code.