I have ruby on rails app that allows users to upload videos. When a video is added, I have a before_save filter that uses ffmpeg to generate a series of thumbnails. The problem is that ffmpeg is producing tons of console output when I'm saving a video item in the rails console, and when I run my tests.
My environment:
Host Machine: OS X 10.9.2
Vagrant Box: Ubuntu 10.04.4
ffmpeg version: SVN-r0.5.9-4:0.5.9-0ubuntu0.10.04.3
ruby version: 1.9.3-p194
Command I'm running:
`ffmpeg -v 0 -ss #{timestamp} -i #{video_file.path} -y -f image2 -vcodec mjpeg -vframes 1 -s 640*360 #{thumbnail_path}/thumbnail#{i}.jpg`
This version of ffmpeg on my VM doesn't seem to care about the "-v 0" option. I've also tried "-loglevel quiet" which causes ffmpeg to error, indicating that the option isn't recognized (both loglevel and v work on my host machine's ffmpeg).
Tried using both exec() and system(), which both caused execution to hang. Tried to redirecting output to a file by doing:
`ffmpeg -v 0 -ss #{timestamp} -i #{video_file.path} -y -f image2 -vcodec mjpeg -vframes 1 -s 640*360 #{thumbnail_path}/thumbnail#{i}.jpg > #{thumbnail_path}/output.txt`
Still see output. Next I tried:
`ffmpeg -v 0 -ss #{timestamp} -i #{video_file.path} -y -f image2 -vcodec mjpeg -vframes 1 -s 640*360 #{thumbnail_path}/thumbnail#{i}.jpg &> dev/null`
Still seeing output! Finally I tried:
$stdout.reopen("#{thumbnail_path}/output.txt", "w")
$stderr.reopen("#{thumbnail_path}/error.txt", "w")
`ffmpeg -v 0 -ss #{timestamp} -i #{video_file.path} -y -f image2 -vcodec mjpeg -vframes 1 -s 640*360 #{thumbnail_path}/thumbnail#{i}.jpg`
$stdout = STDOUT
$stderr = STDERR
Holy cow, that worked! Well, sort of. No more verbose output when running tests, BUT somehow anytime this runs I get kicked out of the rails console.
Does anyone have a more elegant solution?
You can try :
ffmpeg ... >output.txt 2>&1
Which insert stdout and stderr in output.txt
Related
I'm trying to get GStreamer + OpenCV RTSP video capture working in a Docker container based on a NVIDIA PyTorch image. I had to end up building OpenCV from source to enable GStreamer integration, which I do in my Dockerfile like so:
FROM nvcr.io/nvidia/pytorch:19.12-py3
# OpenCV custom build instructions from:
# https://medium.com/#galaktyk01/how-to-build-opencv-with-gstreamer-b11668fa09c
# https://github.com/junjuew/Docker-OpenCV-GStreamer/blob/master/opencv3-gstreamer1.0-Dockerfile
# Install base dependencies + gstreamer
RUN pip uninstall -y opencv-python
RUN apt-get update
RUN apt-get -y install build-essential
RUN apt-get -y install pkg-config
RUN apt-get install -y libgstreamer1.0-0 \
gstreamer1.0-plugins-base \
gstreamer1.0-plugins-good \
gstreamer1.0-plugins-bad \
gstreamer1.0-plugins-ugly \
gstreamer1.0-libav \
gstreamer1.0-doc \
gstreamer1.0-tools \
libgstreamer1.0-dev \
libgstreamer-plugins-base1.0-dev \
cmake \
protobuf-compiler \
libgtk2.0-dev \
ocl-icd-opencl-dev
# Clone OpenCV repo
WORKDIR /
RUN git clone https://github.com/opencv/opencv.git
WORKDIR /opencv
RUN git checkout 4.2.0
# Build OpenCV
RUN mkdir /opencv/build
WORKDIR /opencv/build
RUN ln -s /opt/conda/lib/python3.6/site-packages/numpy/core/include/numpy /usr/include/numpy
RUN cmake -D CMAKE_BUILD_TYPE=RELEASE \
-D INSTALL_PYTHON_EXAMPLES=ON \
-D INSTALL_C_EXAMPLES=OFF \
-D PYTHON_EXECUTABLE=$(which python) \
-D BUILD_opencv_python2=OFF \
-D CMAKE_INSTALL_PREFIX=$(python -c "import sys; print(sys.prefix)") \
-D PYTHON3_EXECUTABLE=$(which python3) \
-D PYTHON3_INCLUDE_DIR=$(python -c "from distutils.sysconfig import get_python_inc; print(get_python_inc())") \
-D PYTHON3_PACKAGES_PATH=$(python -c "from distutils.sysconfig import get_python_lib; print(get_python_lib())") \
-D WITH_GSTREAMER=ON \
-D BUILD_EXAMPLES=ON ..
RUN make -j$(nproc)
# Install OpenCV
RUN make install
RUN ldconfig
This builds successfully, and if I print OpenCV's build information from within the Docker container, GStreamer shows as available:
python -c 'import cv2; print(cv2.getBuildInformation());'
/* snip */
Video I/O:
DC1394: NO
FFMPEG: NO
avcodec: NO
avformat: NO
avutil: NO
swscale: NO
avresample: NO
GStreamer: YES (1.14.5)
v4l/v4l2: YES (linux/videodev2.h)
However, as soon as I try to use a GStreamer pipeline with cv2.VideoCapture() within the Docker container, it immediately fails:
import cv2
video = cv2.VideoCapture('gst-launch-1.0 rtspsrc location=<<rtsp URL>> latency=0 ! queue ! rtph264depay ! h264parse ! avdec_h264 ! videoconvert ! appsink', cv2.CAP_GSTREAMER)
I get this "warning" (e.g., error). I'm not able to pull frames from the RTSP feed.
[ WARN:0] global /opencv/modules/videoio/src/cap_gstreamer.cpp (713) open OpenCV | GStreamer warning: Error opening bin: unexpected reference "gst-launch-1" - ignoring
[ WARN:0] global /opencv/modules/videoio/src/cap_gstreamer.cpp (480) isPipelinePlaying OpenCV | GStreamer warning: GStreamer: pipeline have not been created
If I do this outside of the Docker container, it works like a charm. Also note, if I run the GStreamer pipeline from the command line within the Docker container, I get reasonable output that's identical to running the same command outside of the Docker container:
root:/# gst-launch-1.0 rtspsrc location=<<rtsp URL>> latency=0 ! queue ! rtph264depay ! h264parse ! avdec_h264 ! videoconvert ! appsink
Setting pipeline to PAUSED ...
Pipeline is live and does not need PREROLL ...
Progress: (open) Opening Stream
Progress: (connect) Connecting to <<rtsp URL>>
Progress: (open) Retrieving server options
Progress: (open) Retrieving media info
Progress: (request) SETUP stream 0
Progress: (request) SETUP stream 1
Progress: (open) Opened Stream
Setting pipeline to PLAYING ...
New clock: GstSystemClock
Progress: (request) Sending PLAY request
Progress: (request) Sending PLAY request
Progress: (request) Sent PLAY request
Redistribute latency...
Redistribute latency...
I'm not sure what to do next in terms of debugging the issue with GStreamer not working with OpenCV's VideoCapture - any suggestions?
I don't think you are supposed to include the gst-launch-1.0 command line tool into the cv2 pipeline description.
Instead of a console command it wants the sole GStreamer pipeline only. E.g.:
gst_str = ('v4l2src device=/dev/video{} ! '
'video/x-raw, width=(int){}, height=(int){} ! '
'videoconvert ! appsink').format(dev, width, height)
return cv2.VideoCapture(gst_str, cv2.CAP_GSTREAMER)
So in your case try:
import cv2
video = cv2.VideoCapture('rtspsrc location=<<rtsp URL>> latency=0 ! queue ! rtph264depay ! h264parse ! avdec_h264 ! videoconvert ! appsink', cv2.CAP_GSTREAMER)
I have a few suggestions. But do note that I don't have much experience with Gstreamer.
Could it be related to the resolution of the feeds? From here:
It was nothing docker related. I'm using my Android phone as webcam (using adb-ffmpeg-v4l2loopback) and I've just used the wrong resolution of 640x360 instead of 1280x720 (pick the wrong line from my bash history).
Have you tried installing opencv_contrib? It might be irrelevant, but I see a lot of opencv problems fixed by installing it.
Perhaps try adding the parameter format=(string)NV12, source: OpenCV VideoCapture not working with GStreamer plugin
Here is the first example from Python cv2.CAP_GSTREAMER Examples:
def open_cam_rtsp(uri, width, height, latency):
"""Open an RTSP URI (IP CAM)."""
gst_elements = str(subprocess.check_output('gst-inspect-1.0'))
if 'omxh264dec' in gst_elements:
# Use hardware H.264 decoder on Jetson platforms
gst_str = ('rtspsrc location={} latency={} ! '
'rtph264depay ! h264parse ! omxh264dec ! '
'nvvidconv ! '
'video/x-raw, width=(int){}, height=(int){}, '
'format=(string)BGRx ! videoconvert ! '
'appsink').format(uri, latency, width, height)
elif 'avdec_h264' in gst_elements:
# Otherwise try to use the software decoder 'avdec_h264'
# NOTE: in case resizing images is necessary, try adding
# a 'videoscale' into the pipeline
gst_str = ('rtspsrc location={} latency={} ! '
'rtph264depay ! h264parse ! avdec_h264 ! '
'videoconvert ! appsink').format(uri, latency)
else:
raise RuntimeError('H.264 decoder not found!')
return cv2.VideoCapture(gst_str, cv2.CAP_GSTREAMER)
They seem to check the output of subprocess.check_output('gst-inspect-1.0') for Gstreamer elements, and set gst_str accordingly.
Of the 13 examples in the link above, I don't see the gst-launch-1.0 command. Could it be causing the problem?
Would this link help? Docker Error Could not capture frame on Ubuntu 18.04
Sorry I none of these helped. I hope you find a solution soon!
I do my first step in developing on the PX4 using Docker.
Therefore I extend the px4io/px4-dev-nuttx image to px4dev with some extra installations.
Dockerfile
FROM px4io/px4-dev-nuttx
RUN apt-get update && \
apt-get install -y \
python-serial \
openocd \
flex \
bison \
libncurses5-dev \
autoconf \
texinfo \
libftdi-dev \
libtool \
zlib1g-dev
RUN useradd -ms /bin/bash user
ADD ./Firmware /src/firmware/
RUN chown -R user:user /src/firmware/
Than I run the image/container:
docker run -it --privileged \
--env=LOCAL_USER_ID="$(id -u)" \
-v /dev/serial/by-id/usb-3D_Robotics_PX4_FMU_v2.x_0-if00:/dev/serial/by-id/usb-3D_Robotics_PX4_FMU_v2.x_0-if00:rw \
px4dev \
bash
I also tried:
--device=/dev/ttyACM0 \
--device=/dev/serial/by-id/usb-3D_Robotics_PX4_FMU_v2.x_0-if00 \
Than I switched to /src/firmware/, build the code. But the upload always results in this error:
make px4fmu-v2_default upload
ninja: Entering directory `/src/firmware/build/nuttx_px4fmu-v2_default'
[0/1] uploading px4
Loaded firmware for board id: 9,0 size: 1028997 bytes (99.69%), waiting for the bootloader...
I use a Pixhawk 2.4.8, my host is an Ubuntu 18.04 64bit. Doing the same at the host will work.
What is going wrong here? Does maybe a reboot of the PX4 during flashing it cause the problem?
If it is generally not possible, what is the output file of the build and is it possible to upload this using QGroundControl?
Kind regards,
Alex
run script:
#!/bin/bash
docker run -it --rm --privileged \
--env=LOCAL_USER_ID="$(id -u)" \
--device=/dev/ttyACM0 \
--device=/dev/serial/by-id/usb-3D_Robotics_PX4_FMU_v2.x_0-if00 \
--name=dev01 \
px4dev \
bash
for any reason sometimes the upload ends differently:
user#7d6bd90821f9:/src/firmware$ make px4fmu-v2_default upload
...
[153/153] Linking CXX executable nuttx_px4io-v2_default.elf
[601/602] uploading /src/firmware/build/px4fmu-v2_default/px4fmu-v2_default.px4
Loaded firmware for 9,0, size: 1026517 bytes, waiting for the bootloader...
If the board does not respond within 1-2 seconds, unplug and re-plug the USB connector.
but even if I do so. It stucks here.
regarding the default device, I grep through the build folder:
user#7d6bd90821f9:/src/firmware$ grep -r "/dev/serial" ./build/
./build/px4fmu-v2_default/build.ninja: COMMAND = cd /src/firmware/build/px4fmu-v2_default && /usr/bin/python /src/firmware/Tools/px_uploader.py --port "/dev/serial/by-id/*_PX4_*,/dev/serial/by-id/usb-3D_Robotics*,/dev/serial/by-id/usb-The_Autopilot*,/dev/serial/by-id/usb-Bitcraze*,/dev/serial/by-id/pci-3D_Robotics*,/dev/serial/by-id/pci-Bitcraze*,/dev/serial/by-id/usb-Gumstix*" /src/firmware/build/px4fmu-v2_default/px4fmu-v2_default.px4
there is px_uploader.py --port "...,/dev/serial/by-id/usb-3D_Robotics*,...". So I would say it looks out for /dev/serial/by-id/usb-3D_Robotics_PX4_FMU_v2.x_0-if00!
Looking with ls /dev/ inside the container for the devices available, neither /dev/ttyACM0 nor /dev/serial/by-id/usb-3D_Robotics_PX4_FMU_v2.x_0-if00 is listed. Here may is the problem. Something is wrong with --device=...
But ls shows that /dev/usb/ is available. So I checked it with lsusb and the PX4 is listed next to the others:
user#3077c8b483f8:/$ lsusb
Bus 003 Device 018: ID 26ac:0011
Maybe there is not correct driver inside the container for this USB device?
On my host I got the major:minor no 166:0:
user:~$ ll /dev/
crw-rw---- 1 root dialout 166, 0 Jan 2 00:40 ttyACM0
The folder /sys/dev/char/166:0 is identical at host and container as far as I can see. And at the container seems to be a link to someting with */tty/ttyACM0 like on the host:
user#3077c8b483f8:/$ ls -l /sys/dev/char/166\:0
lrwxrwxrwx 1 root root 0 Jan 1 23:44 /sys/dev/char/166:0 -> ../../devices/pci0000:00/0000:00:14.0/usb3/3-1/3-1.3/3-1.3.1/3-1.3.1.3/3-1.3.1.3:1.0/tty/ttyACM0
At the host I got this information about the devices - but this is missing inside the container:
user:~$ ls -l /dev/ttyACM0
crw-rw---- 1 root dialout 166, 0 Jan 2 00:40 ttyACM0
user:~$ ls -l /dev/serial/by-id/
total 0
lrwxrwxrwx 1 root root 13 Jan 2 00:40 usb-3D_Robotics_PX4_FMU_v2.x_0-if00 -> ../../ttyACM0
Following this post I changed my run script to (without the privileged flag)
#!/bin/bash
DEV1='/dev/serial/by-id/usb-3D_Robotics_PX4_FMU_v2.x_0-if00'
docker run \
-it \
--rm \
--env=LOCAL_USER_ID=0 \
--device=/dev/ttyACM0 \
--device=$DEV1 \
-v ${PWD}/Firmware:/opt/Firmware \
px4dev_nuttx \
bash
Than I see the devices. But they are not accessible.
root#586fa4570d1c:/# setserial /dev/ttyACM0
/dev/ttyACM0, UART: unknown, Port: 0x0000, IRQ: 0
root#586fa4570d1c:/# setserial /dev/serial/by-id/usb-3D_Robotics_PX4_FMU_v2.x_0-if00
/dev/serial/by-id/usb-3D_Robotics_PX4_FMU_v2.x_0-if00, UART: unknown, Port: 0x0000, IRQ: 0
I have container which in logs sometimes write key word which is for me important, and I want to highlight this word in color in my terminal, but also important is still see all content logs in real time (--follow). I just tried command
docker logs -f my_app --tail=100 | grep --color -E '^myWord'
but not working.
So exist some way to do this ?
I use ccze. as #aimless said, grc is the great utility also. It easy to install by sudo apt install ccze for debian/ubuntu-like OS
But if you want to colorize stderr, you need to redirect stderr output to stdout. For example:
docker logs -f my-app 2>&1 | ccze -m ansi
arg -m ansi helps if you want to scroll output normally
UPD:
ccze can be very slow. If you encounter this, try running ccze with the nolookups option: ccze -o nolookups.
originally answered - https://unix.stackexchange.com/a/461390/83391
Try this.
docker logs -f my_app --tail=100 | grep --color=always -E '^myWord'
Note the "--color=always" argument.
Another option would be to use something like https://github.com/jlinoff/colorize. I wrote it to specifically address situations like this. For example it has the ability to specify different colors for each pattern (see the help for details).
Here is an example of how to use it for your case.
$ curl -L https://github.com/jlinoff/colorize/releases/download/v0.8.1/colorize-linux-amd64 --out colorize
$ chmod a+x colorize
$ ./colorize -h
$ docker logs -f my_app --tail=100 | ./colorize '^myWord'
$ # really make it standout.
$ docker logs -f my_app --tail=100 | ./colorize -c red+greenB+bold '^myWord'
try grc. Follow the instruction to install and just pipe the logs output:
docker logs -app | grc
I'm trying to create a thumbnail 3 seconds into a mp4 video I've transcoded.
Running....ffmpegthumbnailer -c jpg -q 10 -s 175 -t 00:00:03.0 -a -i /<FILEPATH>/thumb_20160816182847.MP4 -o /<FILEPATH>/tmpfile.jpg
The thumbnail is still showing second 0...what am I doing incorrectly?
The command is successful.
I just upgrade my Nagios server to the latest version (4.0.1) on my Debian 7 system. When i start the daemon, i have the following error:
# /etc/init.d/nagios start
/etc/init.d/nagios: 20: .: Can't open /etc/rc.d/init.d/functions
The /etc/rc.d/init.d/functions did not exist on my Debian system (and also on my Ubuntu 12.04 workstation).
What can i do to solve this issue ?
===
Update:
Just hack the startup script with the following command line:
sudo apt-get install daemon
sudo sed -i 's/^\.\ \/etc\/rc.d\/init.d\/functions$/\.\ \/lib\/lsb\/init-functions/g' /etc/init.d/nagios
sudo sed -i 's/status\ /status_of_proc\ /g' /etc/init.d/nagios
sudo sed -i 's/daemon\ --user=\$user\ \$exec\ -ud\ \$config/daemon\ --user=\$user\ --\ \$exec\ -d\ \$config/g' /etc/init.d/nagios
sudo sed -i 's/\/var\/lock\/subsys\/\$prog/\/var\/lock\/\$prog/g' /etc/init.d/nagios
sudo service nagios start
Works fine on my Debian server.
You can simply write your own init script. Copy /etc/init.d/skeleton to /etc/init.d/nagios and fill in the values in that file:
DESC="Nagios"
NAME=nagios
DAEMON=/usr/local/nagios/bin/$NAME
DAEMON_ARGS="-d /usr/local/nagios/etc/nagios.cfg"
PIDFILE=/usr/local/nagios/var/$NAME.lock
I also commented these lines:
#[ -r /etc/default/$NAME ] && . /etc/default/$NAME
and
#start-stop-daemon --start --quiet --pidfile $PIDFILE --exec $DAEMON --test > /dev/null \
# || return 1
Don't forget to chmod +x /etc/init.d/nagios.
Have fun.
Little add for ubuntu 12.04 [desktop] :
'runuser' program doesn't exist for debianLike, but 'su' instead,
'service' program is not located in /sbin but in /usr/sbin
Then Nicolargo's mods + some of mine :
sudo apt-get install daemon
sudo sed -i 's/^\.\ \/etc\/rc.d\/init.d\/functions$/\.\ \/lib\/lsb\/init-functions/g' /etc/init.d/nagios
sudo sed -i 's/status\ /status_of_proc\ /g' /etc/init.d/nagios
sudo sed -i 's/daemon\ --user=\$user\ \$exec\ -ud\ \$config/daemon\ --user=\$user\ --\ \$exec\ -d\ \$config/g' /etc/init.d/nagios
sudo sed -i 's/\/var\/lock\/subsys\/\$prog/\/var\/lock\/\$prog/g' /etc/init.d/nagios
sudo sed -i 's/\/sbin\/service\ /\/usr\/sbin\/service\ /g' /etc/init.d/nagios
sudo sed -i 's/runuser/su/g' /etc/init.d/nagios
sudo service nagios start
I also removed the '-d 10' option applied on killproc in the stop sequence (around line 94) to avoid error message on 'service nagios stop' call.
$Stopping nagios: Illegal option -d
/sbin/start-stop-daemon: signal value must be numeric or name of signal (KILL, INT, ...)
Try '/sbin/start-stop-daemon --help' for more information.
'joy!
You've probably found a solution, but to answer the question:
One possible solution is installing Nagios 3.x from your package manager and then updating to 4 by compiling it from source. The new init script seems to be messed up, but the older one still works.
Source(german): http://www.monitoring-portal.org/wbb/index.php?page=Thread&threadID=29431&pageNo=2