I have an IP camera that i can view in VLC via the link rtsp://admin:admin#192.168.1.199:554/mpeg4/ch0/main/av_stream but i noticed there is a significant delay to the video in vlc compared to when the camera is viewed in the NVR. The vlc has a delay of 4-6 seconds while in the nvr its barely noticeable at all less than 1 second of delay.
I need to know why that is so i can then plan out what methods/libraries to use in the program im going to make. It helps to know why so that a possible work aroung maybe explored.
Is this a problem inherent to vlc or a limitation to rtsp?
Is there any way i can reduce this delay?
First get sure that your camera has no issue with getting multiple streams. Deactivate your camera on NVR and check if you have a better latency.
VLC use per default rtsp/rtp over TCP so force vlc to use rtsp/rtp over UDP just google about the vlc argument.
And verify if u have better latency.
As BijayRegmi wrote be aware of the default buffering.
Also you can try ffplay from ffmpeg libary and open the rtsp stream with it. There u have more informations about the health of the stream like package loss etc. Also this gives u an second option to verify your stream/latency, then u should know wich part produce the latency.
Related
This took me too long to figure out, so in the hopes of helping anyone out there dealing with any of these issues, I wanted to post the solution. But first, the problems:
I bought one of those cheap HDMI->USB dongles and connected my PS3 as a video source. On vlc the image looked crisp, but I was getting no sound, and the video was really choppy. Checking the codec tab in the "info" section, I saw I was getting 1080p at 5 fps. I thought I got a defective dongle, but decided to check with other apps. tvtime/xawtv gave me great framerate, but low resolution that I couldn't change, cheese allowed me to set all the options, and I was getting good framerate, and good resolution (but no sound), and then I finally tried obs which gave me a perfect result. So clearly the dongle is fine, and the problem was with vlc.
See my answer below for the solution to all those problems (and more!)
I found, through much research and experimentation, that the reason the video was choppy in vlc was because it was using the default "chroma" of YUV2, which if I am not mistaken is uncompressed. (You can check your webcam/dongle's capabilities by running: v4l2-ctl --list-formats-ext -d /dev/video0 where /dev/video0 is your device)
The correct setting to overcome this is mjpg. However, that results in a flood of errors saying:
[mjpeg # 0x7f4e0002fcc0] No JPEG data found in image
This is caused by the fact that the default resolution and framerate (1080p#60fps) overwhelm what I guess is the mjpeg decoder. Setting it to 720p, or lowering the framerate to 30fps prevents the errors.
Next, the sound was missing, and this is due to the fact that I am using pulseaudio and vlc cannot figure out which source to use.
I found the pulse source by running:
pactl list short sources
which yielded:
alsa_input.usb-MACROSILICON_USB_Video-02.multichannel-input
You can test that this is the correct source by running:
vlc pulse://alsa_input.usb-MACROSILICON_USB_Video-02.multichannel-input
I found that to combine the v4l2 video source with the correct pulseaudio sink, you have to set the audio via the input-slave parameter to vlc, but unfortunately, that did not work for me as specified in the guides, and instead I had to set the video source as the slave. The final commands that worked for me were either of:
720p:
vlc pulse://alsa_input.YOUR-SOURCE-HERE-input --input-slave=v4l2:///dev/video0:chroma=mjpg:width=1280:height=720
1080p#30fps:
vlc pulse://alsa_input.YOUR-SOURCE-HERE-input --input-slave=v4l2:///dev/video0:chroma=mjpg:fps=30
I'm trying to get a livestream working on youtube. I want to stream 360° content with H264 video and AAC audio. The stream is started with the youtube live api from my mobile app and librtmp is used to deliver video and audio packets. I easily get to the point where the livestream health is good and my broadcast and stream are bound successfully.
However, when I try to transition to "testing" like this:
YoutubeManager.this.youtube.liveBroadcasts().transition("testing", liveBroadcast.getId(), "status").execute();
I get stuck on the "startTesting" status every time (100% reproducible) while I expect it to change to testing after few seconds to allow me to change it to live.
I don't know what's going on as in the youtube live control room everything seems to be fine but the encoder won't start.
Is it a common issue? Is there a mean to access the encoder logs? If you need more information feel free to ask me.
Regards.
I found a temporary fix !
I noticed 2 things :
When the autostart option was on, the stream changed its state to startLive as soon as I stopped sending data. It suggested that the encoder was trying to start but it was too slow to do it before some other data paket was received (I guess)
When I tried to stream to the "Stream now" URL, as #noogui suggested, it worked ! So I checked out what was the difference in the stream now & event configurations.
It turned out I just had to activate the low latency option as it's done by default in the stream now configuration.
I consider it as a temporary fix because I don't really know why the encoder isn't starting otherwise and because it doesn't work with the autostart option... So I hope it wont break again if Youtube does another change on their encoder.
So, if you have to work with the Youtube api, good luck guys !
I am using a proprietary RTSP server (I don't have access to the source code) running on a Linaro based embedded system. I am connecting to the device using WiFi and use VLC player to watch the stream. Every often, VLC player's window resizes to different sizes.
Is this a normal behavior in RTSP stream (resizing the video)?
-If yes, what is causing this change? Is it my WiFi bandwidth?
-If not, what are the suggested steps to find the root cause of this problem.
Thank you
Ahmad
Is this a normal behavior in RTSP stream (resizing the video)?
Yes, the RTSP DESCRIBE Request should give info about the resolution. (See this discussion)
-If yes, what is causing this change? Is it my WiFi bandwidth?
Most probably not. However I guess more info would be needed on your bandwidth and network setup.
-If not, what are the suggested steps to find the root cause of this problem.
Option 1: Try to disable (uncheck) VLC's preference to resize the interface to native video size, and see what happens.
Also see the following post over at superuser discussing about automatic resizing options
Option 2: Enable VLC's verbose mode (console log) and see what errors or messages come up. This often helps, and points into new directions to look for solutions.
Option 3: It could be a problem with how information is encoded in the stream concerning the resolution. You would need to get in touch with the vendor of your RTSP server software in order to dig deeper.
Open the VLC player press (Ctrl + P) or go to
Tools -> Prefrences -> Interface (look for below options)
Integrated video in interface [Check]
Resize interface to video size [UnCheck]
Must close and open again the VLC player
I am trying to create a RTSP client which live broadcast Audio and Video. I modified the iOS code at link http://www.gdcl.co.uk/downloads.htm and able to broadcast the Video to server properly. But now i am facing issues in broadcasting the audio part. In the link example the code is written in such a way that it writes the Video data to file and than reads the data from the file and upload the NALU's video packets to RTSP server.
For Audio part i am not sure how to proceed on it. Right now what i have tried is that get the audio buffer from mic and than broadcast it to the server directly by adding RTP headers and ALU.. but This approach is not properly working as Audio starts lagging behind and lag increases with time. Can someone let me know if there is some better approach to achieve this and with lip sycn audio/video.
Are you losing any packets on the client? If so, you need to leave "space." If you receive packet 1,2,3,4,6,7, You need to leave space for the missing packet (5).
The other possibility is a what is known as a clock drift problem. The clock (crystal) on your client and server are not perfectly in sync with each other.
This can be caused by environment, temperature changes, etc.
Let's say in a perfect world your server is producing audio samples 20ms audio samples at 48000 hz. Your client is playing them back using a sample rate of 48000 hz. Realistically your client and server are not exactly 48000hz. Your server might be 48000.001 and your client might be 47999.9998. So your server might be delivering faster than your client or vise versa. You would either consume packets too fast and under run the buffer or lag too far behind and overflow the client buffer. In your case, it sounds like the client is playing back too slow and slowly lagging behind the server. You might only lag a couple milliseconds per minute but the issue will keep continuing and it will look like a 1970s lip synced Kung Fu movie.
In other devices, there is often a common clock line to keep things in sync. For example, Video camera clocks, midi clocks. multitrack recorder clocks.
When you deliver data over IP, there is no common clock shared between a client and server. So your issue concerns syncing clocks between disparate devices with no. I have successfully solved this problem using this general approach:
A) Let the client count the rate of packets that come in over a period of time.
B) Let the client count the rate that the packets are consumed (played back).
C) Adjust the sample rate of the client based on A and B.
So your client requires that you adjust the sample rate of the playback. So yes you play it faster or slower. Note that the playback rate change will be very very subtle. You might set the sample rate to be 48000.0001 hz instead of 48000 hz. The difference in pitch would be undetectable by humans as it would only cause a fraction a cent difference in pitch. I gave an explanation of a very simplified approach. There many other nuances and edge cases that must be considered when developing such a control system. You don't just set it and forget it. You need a control system to manage the playback.
An interesting test to demonstrate this is to take two devices with the exact same file. A long recording (say 3 hours) is best. Start them at the same time. After 3 hours of playback, you will notice that one is ahead of the other.
This post explains that it is NOT a trivial task to stream audio and video.
I have programed a voice recognition program and I am have problems with the mic hearing me, over the computer playing music. I need software that can filter out the sound leaving the speakers from the sound entering the mic.
Is there software or a component (for Delphi) that would solve my problem?
You need to capture:
computer output
mic. input
Then you need to find two parameters, depending of your mic. location and sound system delay. This two parameter is n-delay and k-amplify.
Stream1[t+n]*k=Stream2[t]
Where t = time. When you find this parameter then your resulting Stream, only speek mic. input will be
Stream2[t]-Stream1[t+n]*k=MusicReductionStream[t]
I think you want to do what noise canceling microphones do. These systems use at least one extra microphone to calculate the difference between "surrounding noise" and the noise that is aimed directly at the microphone (the speech it has to register). I don't think you can reliably obtain the same effect with a software-only solution.
A first step would obviously be to turn music down :-)
Check out the AsioVST library.
100% open source Delphi code
Free
Very complete
Active (support for xe2 / x64 is being added for example)
Under Examples\Plugins\Crosstalk Cancellation\ you'll find the source code for a plugin that probably does what you're looking for.
The magic happens in DAV_DspCrosstalkCancellation.pas.
I think the speex pre-processor has an echo-cancellation feature. You'll need to feed it the audio data you recorded, and the audio you want to cancel, and it'll try to remove it.
The main problem is finding out what audio your computer plays. Not sure if there is a good API for that.
It also has a noise reduction feature, and voice activity detection. You can compile it as a dll, and then write a delphi header.
You need to estimate the impulse response of the speaker and room, etc., which can change with exact speaker and mic positioning and the size and contents of the room, etc., as well as knowing/estimating the system delay.
If the person or the mic are moveable, the impulse response and delay will need to be continually re-estimated.
Once you have estimated the impulse response, you can convolve it with the output signal and try subtract delayed versions of the result from the mic input until you can null silent portions of the speech input. Cross correlation might be useful for estimating the delay.