I'm trying to send AirPlay audio from shairport-sync over a VLC HTTP stream. The problem I'm having is the VLC stream disconnects whenever the AirPlay source disconnects, the VLC stream clients also disconnect. I would like it to just stream silence when an AirPlay device is not connected.
Here's my command:
sudo shairport-sync --stuffing=soxr -t=0 | sox -traw -r 44.1k -es -b16 -c2 -V1 - -t flac -r 192k - gain +2 | cvlc - --sout '#standard{access=http,mux=ogg,dst=10.0.0.50:8085/airplay.ogg}'
I have a Ruby on Rails web application that allow users to upload images which then automatically get resized as small thumbnails using libvips and the ImageProcessing ruby gem. Sometimes users legitimately need to upload 100MP+ images. These large images break our server that only has 1GB of RAM. If it's relevant, these images are almost always JPEGs.
What I'm hoping is to use libvips to first scale down these images to a size that my server can handle--maybe like under 8,000x8,000 pixels--without using lots of RAM. Then I would use that image to do the other things we already do, like change the colorspace to sRGB and resize and strip metadata, etc.
Is this possible? If so can you give an example of a vips or vipsthumbnail linux CLI command?
I found a feature in Imagemagick that should theoretically solve this issue, mentioned in the two links below. But I don't want to have to switch the whole system to Imagemagick just for this.
https://legacy.imagemagick.org/Usage/formats/#jpg_read
https://github.com/janko/image_processing/wiki/Improving-ImageMagick-performance
P.S.: I'm using Heroku so if the RAM usage peaks at up to 2GB the action should still work.
(I've always been confused about why image processing seems to always require loading the entire image in RAM at once...)
UPDATE:
I'm providing more context because jcupitt's command is still failing for me.
This is the main software that is installed on the Docker container that is running libvips, as defined in the Dockerfile:
FROM ruby:3.1.2
RUN apt-get update -qq && apt-get install -y postgresql-client
# uglifier requires nodejs -- `apt-get install nodejs` only installs older version by default
RUN apt-get install -y curl
RUN curl -sL https://deb.nodesource.com/setup_14.x | bash -
RUN apt-get install -y nodejs
RUN apt-get install -y libvips libvips-dev libvips-tools
# install pdftotext
RUN apt-get install -y xpdf
I am limiting the memory usage of the sidekiq container to 500MB to be more similar to production server. (I also tried this when limiting memory and reserved memory to 1GB and the same thing happens.) This is the config as specified in docker-compose.yml
sidekiq:
depends_on:
- db
- redis
build: .
command: sidekiq -c 1 -v -q mailers -q default -q low -q searchkick
volumes:
- '.:/myapp'
env_file:
- '.env'
deploy:
resources:
limits:
memory: 500M
reservations:
memory: 500M
This is the exact command I'm trying, based on the command that jcupitt suggested:
first I run docker stats --all to see the sidekiq container's memory usage after booting up, not running libvips:
CONTAINER ID NAME CPU % MEM USAGE / LIMIT MEM % NET I/O BLOCK I/O PIDS
4d7e9ff9c7c7 sidekiq_1 0.48% 210.2MiB / 500MiB 42.03% 282kB / 635kB 133MB / 0B 7
I also check docker-compose exec sidekiq top and get a higher RAM limit, which I think is normal for Docker
top - 18:39:48 up 1 day, 3:21, 0 users, load average: 0.01, 0.08, 0.21
Tasks: 3 total, 1 running, 2 sleeping, 0 stopped, 0 zombie
%Cpu(s): 1.2 us, 1.5 sy, 0.0 ni, 97.1 id, 0.2 wa, 0.0 hi, 0.0 si, 0.0 st
MiB Mem : 3929.7 total, 267.4 free, 1844.1 used, 1818.1 buff/cache
MiB Swap: 980.0 total, 61.7 free, 918.3 used. 1756.6 avail Mem
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
1 root 20 0 607688 190620 12848 S 0.3 4.7 0:10.31 ruby
54 root 20 0 6984 3260 2772 R 0.3 0.1 0:00.05 top
39 root 20 0 4092 3256 2732 S 0.0 0.1 0:00.03 bash
then I run the command
docker-compose exec sidekiq bash
root#4d7e9ff9c7c7:/myapp# vipsheader /tmp/shrine20220728-1-8yqju5.jpeg
/tmp/shrine20220728-1-8yqju5.jpeg: 23400x15600 uchar, 3 bands, srgb, jpegload
VIPS_CONCURRENCY=1 vipsthumbnail /tmp/shrine20220728-1-8yqju5.jpeg --size 500x500
Then in another Terminal window I check docker stats --all again
In maybe 0.5s the memory usage quickly shoots to 500MB and the vipsthumbnail process dies and just returns "Killed".
libvips will almost always stream images rather than loading them in memory, so you should not see high memory use.
For example:
$ vipsheader st-francis.jpg
st-francis.jpg: 30000x26319 uchar, 3 bands, srgb, jpegload
$ ls -l st-francis.jpg
-rw-rw-r-- 1 john john 227612475 Sep 17 2020 st-francis.jpg
$ /usr/bin/time -f %M:%e vipsthumbnail st-francis.jpg --size 500x500
87412:2.57
So 87MB of memory and 2.5s. The image is around 3gb uncompressed. You should get the same performance with ActiveRecord.
In fact there's not much useful concurrency for this sort of operation, so you can run libvips with a small threadpool.
$ VIPS_CONCURRENCY=1 /usr/bin/time -f %M:%e vipsthumbnail st-francis.jpg --size 500x500
52624:2.49
So with one thread in the threadpool it's about the same speed, but memory use is down to 50MB.
There are a few cases when this will fail. One is with interlaced (also called progressive) images.
These represent the image as a series of passes of increasingly higher detail. This can help when displaying an image to the user (the image appears in slowly increasing detail, rather than as a line moving down the screen), but unfortunately this also means that you don't get the final value of the first pixel until the entire image has been decompressed. This means you have to decompress the whole image into memory, and makes this type of file extremely unsuitable for large images of the sort you are handling.
You can detect an interlaced image in ruby-vips with:
if image.get_typeof("interlaced") != 0
error "argh! can't handle this"
end
I would do that test early on in your application and block upload of this type of file.
I'm trying to run this command: sudo tcpdump "ether proto 0x888e and ether host <BSSID>" -I -w -U -vvv -I <INTERFACENAME> -w ~/Desktop/handshake.cap which works perfectly in terms of its function, however, when I run the command, I get the following output:
tcpdump: listening on en0, link-type IEEE802_11_RADIO (802.11 plus radiotap header), capture size 262144 bytes
Got 0
Where Got 0 counts the number of packets captured. Furthermore when stopping the command, I get the following:
tcpdump: listening on en0, link-type IEEE802_11_RADIO (802.11 plus radiotap header), capture size 262144 bytes
^C0 packets captured
3526 packets received by filter
0 packets dropped by kernel
I'm trying to integrate this command into a script and would simply like everything but Got 0 to be omitted from the output.
I have experienced this sort of problem before but have simply used 2> /dev/null to get rid of the output I don't want. However, it seems that Got 0 is included in this blocked output and as a result, I get no output at all. Similarly, &>/dev/null removes all output as well. I have also tried piping it to sed -n '1!p' to ignore the first line but this has no effect and would not be preferable because in theory it would not remove 0 packets captured
3526 packets received by filter
0 packets dropped by kernel
Is anyone aware of how to resolve this issue?
Thank you in advance for any help,
Kind regards, Rocco
P.S. I am running macOS
I have a target board where there is a can interface driven by peak drivers. On the host side I installed can-utils. I have a can cable which gets detected as /dev/ttyUSBx when I plugin.
The target board receives candata at 500000 baud rate
So on the host I executed,
sudo modprobe can
sudo modprobe can-raw
sudo modprobe slcan
sudo slcand -o -s8 -t hw -S 500000 /dev/ttyUSBx
sudo ip link set up slcan0
then I sent the candata from the host using:
cansend slcan0 600#ffffffffffffff
I can see the same using candump slcan0 on the host.
But I don't receive anything on the target.
I tried the same with other cable in which I used (on the host)
sudo modprobe can
sudo ip link set can0 type can bitrate 500000 triple-sampling on
sudo ifconfig can0 up
then I was able to send can messages. If I try the same on the previous cable it throws error saying Cannot find device "can0".
So How do I send through slcan interface?
Your CAN bitrate is wrong. To setup 500kbit/s you need to invoke following commands:
sudo slcand -o -s6 -t hw -S 3000000 /dev/ttyUSBx
sudo ip link set up slcan0
-s6 means CAN bitrate 500kbit/s, -s8 means 1Mbit/s. -S parameter is used to setup serial speed to the USB-to-serial controller in the USB-to-CAN cable. The full CAN bitrate table for slcan can be found here.
source
ASCII Command, CAN Bitrate
s0 10 Kbit/s
s1 20 Kbit/s
s2 50 Kbit/s
s3 100 Kbit/s
s4 125 Kbit/s
s5 250 Kbit/s
s6 500 Kbit/s
s7 800 Kbit/s
s8 1000 Kbit/s
When capturing H460 data on wireshark (on multiplexed mode), wireshark does not parse RTP data correctly. it should discard first 4 bytes on any RTP packet. looking for hints how to do that
Thanks
Amit
shark (packaged with wireshark) has this functionality built in.
Make sure that wireshark/tshark is in your PATH variable, and open a new command line window if you've just set it. Let me know if you want me to be more clear there.
If you want to discard the first 4 packets of rtp data on the fly :
tcpdump -i eth0 port ! 5060 and dst 192.168.1.101 -T rtp -n -s0 -w- | editcap -F libpcap -C 4 - - | tcpdump -nlvvv -r - -w output.pcap
For already captured file (capture.pcap):
tcpdump -r capture.pcap | editcap -F libpcap -C 4 - - | tcpdump -nlvvv -r - -w output.pcap
or
editcap capture.pcap output.pcap -C 4
I didn't test these exact examples myself, but I think tshark's "chop" (-C) option might be what you're looking for.