I have a raspberry pi setup with the uv4l driver and the native uv4l-WebRTC module. So far, I can see the video stream work fine on my browser, but what I want to do now is to be able to simultaneously stream the video to the browser and also pass some of the frames to my opencv-python program.
I was able to test if I can get some data on a video by using the following python code:
import numpy as np
import cv2
imageMat = np.array((3,4,3), np.uint8)
cap = cv2.VideoCapture()
cap.open('https://<IP ADDRESS:8080>/stream/video.mjpeg')
cap.read(imageMat)
which works if I put the URL in the sample code above on my browser. This URL is provided by the people who made the uv4l driver, but the problem is that I actually want to be able to use my custom webpage's video instead of the one being streamed from this default URL.
I've seen from other posts that I can pass the frames by drawing them on a
canvas element and then turning this into a Blob and then sending it over a websocket, but this would mean that I have to open another websocket (using python this time) but I'm not too sure if this is the correct approach. I thought that by using UV4L, I can easily obtain the frames while still be able to stream the video.
Related
How do i stream live video from my raspberry pi to my phone on vlc?
I am aware of tools such as raspivid, but that only gives streams the raw output of the camera, I want to process the frames using opencv, add a few text boxes, and facial recognition, etc...
EDIT: I think i was unclear about what i wanted.
I want to capture video from a camera to a raspberry pi, process it, add boxes, from YOLO/PyTesseract, etc, and stream all those processed frames in real time to my phone's VLC client
You definitely can do this using opencv.
cap = cv2.VideoCapture(cam, cv2.CAP_DSHOW)
cap = cv2.VideoCapture(cam)
test, frame = cap.read()
process the frame
I didnt really written the whole code but i am confidently it works if you do this. I assume you webcam is connected to PC or another raspberry pi.
Capture the image (referring to code above), write and established socket connection and send the image over to raspberry pi.
At your raspberry pi receive the image from socket connection, do whatever processing you need and send through VLC stream. You may refer to this link
Your phone can access the image using http:ip:port
I purchased an IP camera, and I was planning to use its video stream as input for OpenCV (with Python) to run some machine learning algorithms on the video frames.
I opened port 554 for RTSP on my router, and now I'm trying to access the stream by using (11 identifies the main video stream):
cap = cv2.VideoCapture("rtsp://user_name:password#camera_IP:554/11")
while(True):
ret, frame = cap.read()
...
It works without any problems from within the local network, but not from outside...in this case, frame is returned as a 'NoneType' object.
In the camera settings, ports 80 and 1935 are indicated for HTTP and RTMP, respectively. I tried to use them as well, but without any success.
If I simply open the camera IP in a browser, I get to the configuration page, and there I have the possibility to watch the live stream. It's embedded in a Flash object, and I'm trying to figure out if I can extract the video stream URL from it.
I viewed the page source, and there seems to be a reference to the source of the stream:
flashvars="&src=rtmp://camera_IP:1935/flash/11:YWRtaW46YWRtaW4=&
but I wasn't able to use it to fetch the stream in OpenCV.
Any suggestion, or should I just go for another camera?
I'm trying to create an overlay of two security camera views onto one video stream, and streaming it out as RTSP protocol. There are two IP camera streams on input, and only one video stream combining both views in overlay - as output.
To create overlay effect and to stream video out, I use VLC player v3.0.6 on Windows 10. I run it from command line to setup everything (correct overlay, accepting input streams and creating output stream). I can receive my inputs and create overlay, and then either display it on screen, or stream it out using http protocol. HTTP steam works great, I can open it on another computer and watch it. Hovewer I cannot change output from HTTP to RTSP and make it work.
This is a VLM config file, setting inputs and outputs. This one outputs HTTP stream.
del all
new channel1 broadcast enabled
setup channel1 input rtsp://xxx:xxx#192.168.xx.xx/profile2/media.smp
setup channel1 output #duplicate{dst=mosaic-bridge{id=1,height=720},select=video}
new channel2 broadcast enabled
setup channel2 input rtsp://xxx:xxx#192.168.x.x/profile2/media.smp
setup channel2 output #duplicate{dst=mosaic-bridge{id=4,height=340},select=video}
new background broadcast enabled
setup background input "file:///C:\Program Files\VideoLAN\VLC\pic.jpg"
setup background option image-duration=-1
setup background option image-fps=10
setup background option mosaic-width=1280
setup background option mosaic-height=720
setup background option mosaic-keep-picture=1
setup background output #transcode{sfilter=mosaic,vcodec=mpeg,vb=2000,fps=10}:bridge-in{delay=0,id-offset=0}:standard{access=http,mux=ogg,dst=192.168.xx.xx:18554}
control channel1 play
control channel2 play
control background play
To run it, I call VLC using this command:
vlc "--vlm-conf=C:\Projekty\mosaic\mosaic4.vlm" "--clock-jitter=0" "--mosaic-width=1280" "--mosaic-height=720" "--mosaic-keep-picture" "--mosaic-row=2" "--mosaic-cols=2" "--mosaic-position=1" "--mosaic-order=1,2,3,4" "--ttl=12" "--udp-caching=800" --verbose=2
It sets the mosaic view and resolution.
Now, the problem lies in the VLM file, when setting output. I use :standard module for output, but this module doesn't support RTSP.
Ok, let's try and let VLC configure everything for me. There is option of streaming using regular VLC GUI. You choose what to steam (file/your screen/single input stream), then you choose output format, and that's it. At the end of process, VLC even shows you the commands it uses to stream. It looks like this:
:sout=#transcode{vcodec=h264,vb=56,venc=x264{profile=baseline},fps=12,scale=Automaticky,width=176,height=144,acodec=mp3,ab=24,channels=1,samplerate=44100,scodec=none}:rtp{sdp=rtsp://:8554/} :no-sout-all :sout-keep
It's a bunch of video transcoding settings, and then the output - :rtp{sdp=rtsp://:8554/} . But it works great, the other side receives working RTSP stream.
Naturally, I try to replace my :standard(http) module with this :rtp setting, but for some reason, it just doesn't work - the other side can't open stream.
setup background output #transcode{sfilter=mosaic,vcodec=mpeg,vb=2000,fps=10}:bridge-in{delay=0,id-offset=0}:rtp{sdp=rtsp://:8554/} :no-sout-all :sout-keep
Any suggestions? I can receive my streams, I can merge them together, I just can't get them out. VLC documentation doesn't help much at this point.
Any help would be greatly appreciated.
I've got experience with building iOS apps but don't have experience with video. I want to build an iPhone app that streams real time video to a server. Once on the server I will deliver that video to consumers in real time.
I've read quite a bit of material. Can someone let me know if the following is correct and fill in the blanks for me.
To record video on the iPhone I should use the AVFoundation classes. When using the AVCaptureSession the delegate method captureOutput:didOutputSampleBuffer::fromConnection I can get access to each frame of video. Now that I have the video frame I need to encode the frame
I know that the Foundation classes only offer H264 encoding via AVAssetWriter and not via a class that easily supports streaming to a web server. Therefore, I am left with writing the video to a file.
I've read other posts that say they can use two AssetWritters to write 10 second blocks then NSStream those 10 second blocks to the server. Can someone explain how to code the use of two AVAssetWriters working together to achieve this. If anyone has code could they please share.
You are correct that the only way to use the hardware encoders on the iPhone is by using the AVAssetWriter class to write the encoded video to a file. Unfortunately the AVAssetWriter does not write the moov atom to the file (which is required to decode the encoded video) until the file is closed.
Thus one way to stream the encoded video to a server would be to write 10 second blocks of video to a file, close it, and send that file to the server. I have read that this method can be used with no gaps in playback caused by the closing and opening of files, though I have not attempted this myself.
I found another way to stream video here.
This example opens 2 AVAssetWriters. Then on the first frame it writes to two files but immediately closes one of the files so the moov atom gets written. Then with the moov atom data the second file can be used as a pipe to get a stream of encoded video data. This example only works for sending video data but it is very clean and easy to understand code that helped me figure out how to deal with many issues with video on the iPhone.
I have a video stream coming in on rtp to ffmpeg and I want to pipe this to my OpenCV tools for live streaming processing. The rtp linkage is working because I am able to send the incoming data to a file and play it (or play if via ffplay). My OpenCV implementation is functional as well because I am able to capture video from a file and also a webcam.
The problem is the streaming to OpenCV. I have heard that this may be done using a named pipe. First I could stream the ffmpeg output to the pipe and then have OpenCV open this pipe and begin processing.
What I've tried:
I make a named-pipe in my cygwin bash by:
$ mkfifo stream_pipe
Next I use my ffmpeg command to pull the stream from rtp and send it to the pipe:
$ ffmpeg -f avi -i rtp://xxx.xxx.xxx.xxx:1234 -f avi -y out.avi > stream_pipe
I am not sure if this is the right way to go about sending the stream to the named pipe but it seems to be accepting the command and work because of the output from ffmpeg gives me bitrates, fps, and such.
Next I use the named pipe in my OpenCV capture function:
$ ./cvcap.exe stream_pipe
where the code for cvcap.cpp boils down to this:
cv::VideoCapture *pIns = new cv::VideoCapture(argv[1]);
The program seems to hang when reaching this one line, so, I am wondering if this is the right way of going about this. I have never used named pipes before and I am not sure if this is the correct usage. In addition, I don't know if I need to handle the named pipe differently in OpenCV--change code around to accept this kind of input. Like I said, my code already accepts files and camera inputs, I am just hung up on a stream coming in. I have only heard that named pipes can be used for OpenCV--I haven't seen any actual code or commands!
Any help or insights are appreciated!
UPDATE :
I believe named pipes may not be working in the way I intended. As seen on this cygwin forum post:
The problem is that Cygwin's implementation of fifos is very buggy. I wouldn't recommend using fifos for anything but the simplest of applications.
I may need to find another way to do this. I have tried to pipe the ffmpeg output into a normal file and then have OpenCV read it at the same time. This works to some extent, but I imagine in can be dangerous to read and write from a file concurrently--who knows what would happen!
hope it's not too late to answer, but I have tried the same thing some time ago, and here is how I did it.
The video-decoding backend for OpenCV is actually ffmpeg, so all its facitilites are available in OpenCV as well. Not all the interface is exposed, and that adds some difficulties, but you can send the rtp stream address to OpenCV.
cap.open("rtp://xxx.xxx.xxx.xxx:1234");
Important: OpenCV is not able to access password-protected rtp streams. To do that, you would need to provide the username and the password, there is no API exposed for it.