Mosquitto increasing maximum connection limit - mqtt

Question

I am trying to increase the maximum open file connection of Mosquitto broker. But I read that increasing concurrent connections are not controlled by Mosquitto only.
As per our study we decided for 100k concurrent connection, we are targeting 1.6 GB RAM. But for testing I have to increase from default 1024 connections to 20000
Testing environment configurations:
t2. micro AWS server with 64 MB 14.04 ubuntu operating system. Changing connection limit in the mosquitto configuration is not reflecting. What can be the reason?
Do we need to change any configuration related to AWS Server?
My configurations:
Our system wide open connections is configured on /etc/sysctl.conf:
fs.file-max =99905
Running the command sysctl -p or cat /proc/sys/fs/file-max is reflecting the changes
In /etc/security/limits.conf:
ubuntu hard nofile 45000
ubuntu soft nofile 35000
Mosquitto is installed under the user 'Ubuntu' .
We also added below line of code on /etc/pam.d/common-session
session required pam_limits.so
Running ulimit -a is giving the below result:
core file size (blocks, -c) 0
data seg size (kbytes, -d) unlimited
scheduling priority (-e) 0
file size (blocks, -f) unlimited
pending signals (-i) 7859
max locked memory (kbytes, -l) 64
max memory size (kbytes, -m) unlimited
open files (-n) 35000
pipe size (512 bytes, -p) 8
POSIX message queues (bytes, -q) 819200
real-time priority (-r) 0
stack size (kbytes, -s) 8192
cpu time (seconds, -t) unlimited
max user processes (-u) 7859
virtual memory (kbytes, -v) unlimited
file locks (-x) unlimited
My init configuration file for mosquitto in /etc/init/mosquitto.conf
description "Mosquitto MQTTv3.1 broker"
author "Roger Light <roger#atchoo.org"
start on net-device-up
respawn
exec /usr/sbin/mosquitto -c /etc/mosquitto/mosquitto.conf
#limit nofile 4096 4096
limit nofile 24000 24000
Below is the configuration in /etc/mosquitto/mosquitto.conf:
# change user to root
user ubuntu
set_ulimit () {
ulimit -f unlimited
ulimit -t unlimited
ulimit -v unlimited
ulimit -n 24000
ulimit -m unlimited
ulimit -u 24000
}
start)
...
# Update ulimit config in start command
set_ulimit
...
;;
stop)
But running cat /proc/4957/limits is still showing default value 1024 open files:
Limit Soft Limit Hard Limit Units
Max cpu time unlimited unlimited seconds
Max file size unlimited unlimited bytes
Max data size unlimited unlimited bytes
Max stack size 8388608 unlimited bytes
Max core file size 0 unlimited bytes
Max resident set unlimited unlimited bytes
Max processes 7859 7859 processes
Max open files 1024 4096 files
Max locked memory 65536 65536 bytes
Max address space unlimited unlimited bytes
Max file locks unlimited unlimited locks
Max pending signals 7859 7859 signals
Max msgqueue size 819200 819200 bytes
Max nice priority 0 0
Max realtime priority 0 0
Max realtime timeout unlimited unlimited us
4957 -is the process id of Mosquitto

Answer 1

The number of open files is limited by the user limits, see ulimit man page.
I set the ulimit -n to 20000, and run mosquitto broker and it shows
% ps ax | grep mosquitto
9497 pts/44 S+ 0:00 ./mosquitto -c mosquitto.conf
9505 pts/10 S+ 0:00 grep --color=auto mosquitto
% cat /proc/9497/limits
Limit Soft Limit Hard Limit Units
Max cpu time unlimited unlimited seconds
Max file size unlimited unlimited bytes
Max data size unlimited unlimited bytes
Max stack size 8388608 unlimited bytes
Max core file size 0 unlimited bytes
Max resident set unlimited unlimited bytes
Max processes 63084 63084 processes
Max open files 20000 20000 files
Max locked memory 65536 65536 bytes
Max address space unlimited unlimited bytes
Max file locks unlimited unlimited locks
Max pending signals 63084 63084 signals
Max msgqueue size 819200 819200 bytes
Max nice priority 0 0
Max realtime priority 0 0
Max realtime timeout unlimited unlimited us
Regardless, since mosquitto is single-threaded, we have not found it
useable for anything more than about 1000 publisher clients with a
reasonable payload rate at 1 / 10 seconds.

Answer 2

changing limits in /etc/sysctl.conf or /etc/security/limits.conf seems to have no effect for process launched as service: The limit has to be set in the file starting up the daemon.
At the beginning of /etc/init.d/mosquitto:
ulimit -n 20000 #or more if need more....
in /etc/mosquitto/mosquitto.conf:
max_connections -1 #or the max number of connection you want

Answer 3

Till now I have achieved 74K concurrent connections on a broker. I have configured the ulimit of broker server by editing sysctl.conf and limit.conf file.
# vi /etc/sysctl.conf
fs.file-max = 10000000
fs.nr_open = 10000000
net.ipv4.tcp_mem = 786432 1697152 1945728
net.ipv4.tcp_rmem = 4096 4096 16777216
net.ipv4.tcp_wmem = 4096 4096 16777216
net.ipv4.ip_local_port_range = 1000 65535
# vi /etc/security/limits.conf
* soft nofile 10000000
* hard nofile 10000000
root soft nofile 10000000
root hard nofile 10000000
After this reboot your system.

Answer 4

If you are using ubuntu 16.04, we need to make change in system.conf
# vim /etc/system/system.conf
DefaultLimitNOFILE=65536
Reboot, this will increase the connection limit

Answer 5

For me none of the provided solutions worked with Ubuntu 18.04. I had to add LimitNOFILE=10000 to /lib/systemd/system/mosquitto.service under Service:
[Unit]
Description=Mosquitto MQTT Broker
Documentation=man:mosquitto.conf(5) man:mosquitto(8)
After=network.target
Wants=network.target
[Service]
LimitNOFILE=5000
Type=notify
NotifyAccess=main
ExecStart=/usr/sbin/mosquitto -c /etc/mosquitto/mosquitto.conf
ExecReload=/bin/kill -HUP $MAINPID
Restart=on-failure
ExecStartPre=/bin/mkdir -m 740 -p /var/log/mosquitto
ExecStartPre=/bin/chown mosquitto:mosquitto /var/log/mosquitto
ExecStartPre=/bin/mkdir -m 740 -p /run/mosquitto
ExecStartPre=/bin/chown mosquitto:mosquitto /run/mosquitto
[Install]
WantedBy=multi-user.target
Then run systemctl daemon-reload to reload the changes and restart mosquitto with systemctl restart mosquitto.