I have a scenario wherein the MQTT client publishes a message with a topic . Post this I bring down my mosquitto broker (by killing the process) . When I restart my broker and try consuming the message with the topic , the message is not available . Is there any way I can consume the message which was published before the broker is restarted ?
Note : This is possible if RabbitMQ is used as the messages are stored in the queue . I want to implement the same using MQTT-Mosquitto.
If you enable persistence and specify a directory in your mosquitto.conf file then retained messages will be saved to disk and restored when the broker is restarted.
e.g.
persistence true
persistence_location /path/to/store/data/
Also look at the autosave_interval and autosave_on_changes options in the docs
Related
I have been using following command to see published messages on the Mosquitto broker:
mosquitto_sub -h IP_ADDRESS -t TOPIC_NAME
However, when I run this command I can see only the recent messages not all published messages. In Mqtt client, I can see messages delivered.
Is there a command to see all messages that were delivered to the broker?
The command only shows messages when we start subscribing?
I have used value 2 for Quality of Service in message properties.
MQTT as a protocol doesn't store all messages, no matter what QOS they were published/subscribed at.
It's also important to remember that QOS is only for one leg at a time. E.g. a message published at QOS2 is ensured to arrive at the broker, but makes no promises about it's delivery to any clients, that is down to what QOS those clients subscribed to the topic at.
MQTT will queue messages for a known client if it has previously subscribed at QOS 1 or 2 and reconnects with the cleanSession flag set to false. This connection needs to exactly the same client id as the previous session.
So if you run the following command:
mosquitto_sub -h [ip addr] -t [topic] -c -i [clientid] -q 2
This will create a persistent session, if you then disconnect (kill mosquitto_sub) when you run the same command again (making sure to keep [clientid] the same the broker will deliver all the messages sent while the client was disconnected. But there is no way to get messages published before the session was established by running the command the first time.
-c tells the client to set the cleanSession flag to false.
-i [clientid] sets the client id
-q 2 sets the subscription QOS to 2
If this is to debug a problem then one possible option would be to ramp up the mosquitto instances logging level then it will log absolutely everything, but this will produce a LOT of output.
My mosquitto.conf is (This is the entire conf file since everything else is just commented out)
log_dest file /mosquitto/log/mosquitto.log
log_type all
connection_messages true
log_timestamp true
log_timestamp_format [%H:%M:%S]
persistence true
persistence_location /mosquitto/data/
listener 1883
listener 9001
protocol websockets
Then i run
docker exec -it ecs-Eclipse_mosquitto_MQTT-11-Eclipse-mosquitto-MQTT-eafxxxxxxx mosquitto_pub -t presence -m hellothere -q qos=2
(NOTE I dont have any active subscribers yet online)
Navigated to /mosquitto/data and there was indeed a file called mosquitto.db
So i could deduce that my message hellothere is stored in mosquitto.db
Then i ran the command
docker exec -it ecs-Eclipse_mosquitto_MQTT-11-Eclipse-mosquitto-MQTT-eafxxxxxxx mosquitto_sub -t presence -q qos=2
What i expect
For the MQTT broker to send the stored message since subscriber is now online
What actually happens
indefinite waiting for the message
Maybe i am missing something ?
I think the relevant bit of the mqtt spec is:
When a Server takes ownership of an incoming Application Message it
MUST add it to the Session state of those clients that have matching
Subscriptions.
In your case when the message is published there are no clients with matching subscriptions (because you have not run mosquitto_sub at that point) so it does nothing further.
If you want messages to be stored for a client you must first connect, as that client, and subscribe to the relevant topic (with QOS>0 and CleanSession=0). After doing that any messages received whilst the client is offline will be stored and delivered when it reconnects (as long as CleanSession=0).
Note: To do the above with mosquitto_sub you will need to use the --id parameter to set the client id (messages will only be queued for the specific clients that have subscribed to the relevant topics). You will also need the -c flag to disable the 'clean session' flag.
So the steps will be as follows (I have removed the extra docker bits - you can add these back in):
mosquitto_sub -c --id subscriber1 -t presence -q 2
This will subscribe and then wait for any messages; you can safely close it (it's the subscription bit that needs to happen). Next publish your messgae:
mosquitto_pub -t presence -m hellothere -q 2
Now you can start mosquitto_sub again to retrieve the stored messages:
mosquitto_sub -c --id subscriber1 -t presence -q 2
Warning: When you subscribe like this the broker will store any messages received whilst you are offline. This can become an issue if you are testing a broker that processes a lot of messages (because storing all of the messages takes space). Probably not an issue here as you are only testing but something to bear in mind... Running mosquitto_sub without the -c will clear any subscriptions - there are also options in the mosquitto config that allow you to limit the number of messages held and how long they are held.
MQTT doesn't work that way.
Messages are only queued for clients that have previously been connected, it is a pub/sub system not a message queuing system.
A client needs to have been connected in the past and subscribed to the topic the message in question. It then needs to reconnect after the message has been published with the cleanSession flag set to false and using the same client id.
You can read more about this on the HiveMQ blog here
I have one remote broker (cloudmqtt) and one local broker on my board. Both are connected as bridge. I have one paho client connected to local broker. I want to know the status of bridge in order to publish message. I know I can publish message to local broker without knowing status and broker will take care. But I want to design my application like I will Publish message only if bridge is Up.
I am using paho client library in C and mosquitto broker v1.6.
Below is my conifg file :
pid_file /var/run/mosquitto.pid
persistence true
persistence_location /var/lib/mosquitto/
log_dest file /var/log/mosquitto/mosquitto.log
log_timestamp true
log_timestamp_format %Y-%m-%dT%H:%M:%S
log_type all
user root
connection cloudmqtt
address xxxx.cloudmqtt.com:13287
remote_username xxxxxxx
start_type automatic
try_private true
remote_password xxxxxx
notifications true
notification_topic /broker/connection/state
restart_timeout 20
max_queued_messages 0
topic # both 2
The short answer is you can't at a pure MQTT protocol level or specific to the Paho client (and you shouldn't care for the reasons you mentioned).
Now having said all that you can actually get messages about the bridge status from the $SYS/broker/connection/<remote-clientid>/# topic tree on mosquitto. To enable this you need to set the notification true flag in the bridge config. The doc for configuring bridges is here
How to monitor mosquitto server through HTTP? I am looking something like https://my.mosquitto.server.com/health or https://my.mosquitto.server.com/info.
Mosquitto Server: v3
OS: CentOS 7
Mosquitto Web: Enabled
Thanks
You are always going to need a MQTT client to get access to the information in the $SYS topic space.
When MQTT over Websockets is enabled you can have mosquitto serve a page that will connect to the broker with the Paho Javascript client with MQTT over Websockets. The page can subscribe to what ever topics you want.
e.g. You can use the http_dir option to specify a directory that moquitto will server static files from. Here you could deploy a slightly modified (The modification would be to change the topic from # to %SYS/#) version of my D3 MQTT topic tree viewer (code on github).
From the mosquitto.conf man page:
http_dir directory
When a listener is using the websockets protocol, it is possible to
serve http data as well. Set http_dir to a directory which contains
the files you wish to serve. If this option is not specified, then no
normal http connections will be possible.
Not reloaded on reload signal.
I recently upgraded and updated my Mosquitto Broker to 1.4.15 on Raspberry-pi3. It works as expected. However, every time I enable and set cleansession to true in the Mosquitto.Conf file, Mosquitto broker refuses to start. As soon as I comment out or disable cleansession, Mosquitto broker starts up immediately by its service. The reason I need to set this to true is because I want the Mosquitto Broker to clean up any disconnected clients' session. So that the same client can reconnect again to the broker. Am I doing this right? or Is there another Mosquitto Broker's feature that I can use in place of cleansession?
The cleansession flag in the mosquitto.conf is to control what options mosquitto uses when it connects to remote brokers as a client when setting up a bridge.
If you want your clients to connect with a clean session then you need to set that option in your client library when setting up the connection not on the broker.