I'm programming chat app with MQTT protocol
You can see here mqtt.org about MQTT
My used broker is mosquito
What struck my mind is that
How many users can connect to Mosquito broker?
I saw 100k in some website but i am not sure
This is dependent on a number of factors:
What OS your running on
The size of the machine(s) you run the broker on
What broker you choose to use
How much and what type of load you are generating:
How many clients subscribed to each topic
How big the messages are
How many retained messages you are generating
What the message rates is
Are you queuing message for offline clients
You also need to configure the broker/OS to get the most out of it, e.g. for mosquitto you need to set the number of open file handles on Linux to the maximum.
For a large scale app you may want to look at one of the brokers that supports federation/clustering to spread the load and allow fail over.
Related
I'm running mosquttio as an MQTT broker and I have multiple devices sending sensory data periodically. I want to collect all the messages and store them. My question is, is there any advantage to having multiple connections (each connection has a unique id and subscribed to subset of the topics.) to the broker or is it preferable to have a single connection gathering all the data.
Note: the subscribers will be on the same machine as the broker.
It probably depends on the messages in question and what processing the client is going to do on those messages.
Having a single client subscribed to '#' will mean that there will only be a single entry on the subscribed topic to search for matches when processing a message. But this is probably a negligible amount of overhead under most situations.
If the message rate is high enough and there is any overhead to the storage then using something called Shared Subscriptions to allow a pool of client to all subscribe to the same topic (or wildcard) and ensure that any message is only delivered to a single client in the pool. This means that the processing of messages can be load balanced over the pool of clients.
Using Shared Subscriptions means that you can dynamically add or remove clients from the pool without having to repartition the topic space across the clients.
I implemented a MQTT message broker using mosquitto on my network. I have one web app publishing things to the broker and several servers that subscribed the same topic. So i have a redundancy scenario.
My question is, using mosquitto alone, is there any way to configure it to publish data only on the first subscriber? Otherwise, all of them will do the same thing.
I don't think that is possible.
But you can do this.
Have the first subscriber program respond with an ack on the channel as soon as it gets the message, and have the redundancy program look for the ack for a small time after the initial message.
IF the ack is received the redundancy should not do anything.
So if the first subscriber gets and uses the message, the others wont do anything even if they get the message.
No this is not possible with mosquitto at the moment (without communication between the 2 subscribers as described in the other answer).
For the new release of the MQTT spec (v5)* there is a new mode called "Shared Subscriptions". This allow s multiple clients to subscribe to a single topic and messages will be delivered by round robin to each client. This is more for load balancing rather than master/slave fail over.
*There are some brokers (HiveMQ, IBM MessageSight) that already support some version of Shared Subscriptions at MQTT v3.1.1, but they implement it in slightly different ways (different topic prefixes) so they are not cross compatible.
MQTT is a publish/subscribe protocol. Whenever a publisher publishes to a topic, all the subscribers that have subscribed to that topic will get the message via an MQTT broker. I would like to know the maximum number of clients an MQTT broker can handle. Is there any upper limit for that?
The only way to work this out is to test depending on your specific workload.
It will be entirely dependent on the following:
The size of the machine you run the broker on.
The size of the messages you send.
The rate of messages.
The number of clients (both subscribers and publishers).
The performance characteristics you need to meet.
Which broker you are using.
And possibly many more factors.
How many clients an MQTT broker can serve depends on the MQTT broker software you're using. Most MQTT brokers will likely only be limited by the amount of memory available (each socket uses a chunk of memory) and it therefore becomes a question of which broker software utilizes the memory (and other resources) in the most efficient manner. Of course some brokers might have other limitations.
In practice you'd also have to look at what you can do with the connected clients - some brokers may behave differently (performance wise) depending on how many clients are connected etc.
In the context on the MQTT protocol, is there a way to make a client not send publish messages when there are no subscribers to that topic?
In other words, is there a standard way to perform subscriber-aware publishing, reducing network traffic from publishing clients to the broker?
This important in applications where we have many sensors capable of producing huge amounts of data, but most of the time nobody will be interested in all of that data but on a small subset, and we want to save battery or avoid network congestion.
In the upcoming MQTT v5 specification the broker can indicate to a client that there are no subscribers for a topic when the client publishes to that topic. This is only possible for QoS 1 or QoS 2 publishes because a QoS 0 message does not result in a reply.
No, the publisher has absolutely no idea how many subscribers to a given topic there are, there could be zero or thousands.
This is a key point to pub/sub messaging, the near total decoupling of the information producer and consumer.
Presumably you can design your devices and applications so that the device as well as publishing data to a 'data topic', it also subscribes to another device-specific 'command topic' which controls the device data publishing. If the application is interested in data from a specific device it must know which device that is to know which data topic to subscribe to, so it can publish the 'please publish data now' command to the corresponding command topic.
I suppose there might be a somewhere-in-between solution where devices publish data less frequently when no apps are interested, and faster when at least one app is asking for data to be published.
Seems to me that one thing about MQTT is that you should ideally design the devices and applications as a system, not in isolation.
I'm using mosquitto (http://mosquitto.org/) as an MQTT broker and am looking for advice on load balancing subscribers (to the same topic). How is this achieved? Everything I've read about the protocol states that all subscribers to the same topic will be given a published message.
This seems inefficient, and so I'm looking for a way for a published message to be given to one of the connected subscribers in a round-robin approach that would ensure a load balanced state.
If this is not possible with MQTT, how does a subscriber avoid being overwhelmed with messages?
Typically you design MQTT applications in a way that you don't have overwhelmed subscribers. You can achieve this by spreading load to different topics.
If you really can't do that, take a look at the shared subscription approach sophisticated MQTT brokers like MessageSight and HiveMQ have. This is exactly the feature you're looking for but is broker dependant and is not part of the official MQTT spec.
MQTT v5 has the support of shared subscriptions and mosquitto version 1.6 added support of MQTT v5.
Check release notes
Good article on shared subscriptions here
MQTT is a Pub/Sub protocol, the basis is for a 1 to many distribution of messages not a 1 to 1 (of many) you describe. What you describe would be more like a Message Queuing system which is distinctly different from Pub/Sub.
Mosquitto as a pure implementation of this protocol does not support the delivery as you describe it. One solution is to us a local queue with in the subscriber which incoming messages are added to and then consumed by a thread pool.
I do believe that the IBM Message Sight appliance may offer the type of message delivery you are looking for as an extension to the protocol called Shared Subscriptions, but with this enabled it is deviating from the pure MQTT spec.