I'm new to MQTT: but I have got some basic Python programs working where sensor readings can be published to a particular topic: and other clients can then subscribe to get the temperature on a event-driven basis.
But when it comes to sending commands; I'm a little stuck on the best to do this.
So for example: take a 'countdown timer' connected to mqtt.
This timer has two states: 'stopped' and 'started'.
It will initialize itself into the 'stopped' state and wait for a 'start' command; and then will count down; publishing the current countdown to a topic.
When the countdown reaches zero; it will switch its state to 'stopped' again, and wait for another 'start' command.
If it receives a 'stop' command (over mqtt); it should also go into the 'stopped' state.
So perhaps I could create topics something like:
countdown_timer/command
countdown_timer/state
countdown_timer/value
And the countdown device could subscribe to 'command' and react by publishing to 'state'. ('stopped' or 'started'?)
But should the client somehow 'consume' the 'command' topic value once it has processed it ?
Or would it better to have something like:
countdown_timer/send_command
countdown_timer/command_result
Where the controller would send a command, the subscribed-device would carry-out the command and put 'ok' or 'error' on the 'command_result' topic ?
In general, both approaches that you describe are valid MQTT patterns. You choose what is most appropriate for your application. Here are some comments:
For your countdown timer, I would go with your first suggestion. But for other applications, other approaches may make more sense.
If you write to countdown/state and countdown/value, you may want to make these publish messages retained. This will ensure that newly subscribed clients will immediately receive the latest value.
If your countdown timer process is always running, then you don't need the retained flag for countdown_timer/command --- but sometimes it makes sense when a server process can fail, restart and reconnect to just continue with the last command.
The send_command and command_result pattern is common for MQTT when one client speaks to one server and receives one answer for each question. This doesn't seem to fit this current example well: You don't have one specific answer to respond to for each command.
Here is another pattern for client-server applications: The server subscribes to one channel server/command and each client subscribes to a separate channel: client/1, client/2, client/3 etc. When a client sends a command to the server, it includes its client id --- and the server responds on the corresponding channel.
A modification of this pattern is to use independent channels for command queries: service/1, service/2 etc. The first clients publishes to service/1 and subscribes to client/1. The second client publishes to service/2 and subscribes to client/2. The server subscribes to service/#, extracts the client id from the topic name of the received message and responds to the corresponding client channel.
You see: There are many valid patterns for MQTT --- at one hand, this flexibility is an advantage. At the other hand it puts the responsibility on you to choose wisely.
Related
I implemented a MQTT message broker using mosquitto on my network. I have one web app publishing things to the broker and several servers that subscribed the same topic. So i have a redundancy scenario.
My question is, using mosquitto alone, is there any way to configure it to publish data only on the first subscriber? Otherwise, all of them will do the same thing.
I don't think that is possible.
But you can do this.
Have the first subscriber program respond with an ack on the channel as soon as it gets the message, and have the redundancy program look for the ack for a small time after the initial message.
IF the ack is received the redundancy should not do anything.
So if the first subscriber gets and uses the message, the others wont do anything even if they get the message.
No this is not possible with mosquitto at the moment (without communication between the 2 subscribers as described in the other answer).
For the new release of the MQTT spec (v5)* there is a new mode called "Shared Subscriptions". This allow s multiple clients to subscribe to a single topic and messages will be delivered by round robin to each client. This is more for load balancing rather than master/slave fail over.
*There are some brokers (HiveMQ, IBM MessageSight) that already support some version of Shared Subscriptions at MQTT v3.1.1, but they implement it in slightly different ways (different topic prefixes) so they are not cross compatible.
I am using rabbitmq to communicate between microservices written in ruby on rails. Each service subscribes to a topic. All services are scaled and run as multiple instances based on need.
During subscription bunny moves all the messages from the queue into unacked state. This makes other scaled instances to be just idle, since there is no message in ready state.
Is there a way to limit the number of messages a subscription can fetch, so that other instances can take the remaining messages from the queue.
Based on the information you made available, I'm assuming you're using rubybunny. If this assumption is incorrect (there are other ruby clients available for rabbitmq) let me know and/or check the documentation related to your client.
Back to rubybunny, link provided points to necessary information, quoting it:
For cases when multiple consumers share a queue, it is useful to be
able to specify how many messages each consumer can be sent at once
before sending the next acknowledgement.
In AMQP 0.9.1 parlance this is known as QoS or message prefetching.
Prefetching is configured on a per-channel basis.
To configure prefetching use the Bunny::Channel#prefetch method like so:
ch1 = connection1.create_channel
ch1.prefetch(10)
we are implementing (or more reimplementing) a distributed software system. What we have are different processes (possibly running on different computers) that should communicate with each other (let's call these clients). We don't want them to directly communicate with each other, but instead use some kind of message broker.
Since we like to avoid implementing the message broker ourselves we would like to use an existing implementation. But we don't find a protocol or system that fully fulfilles our requirements.
MQTT with its publish-subscribe-mechanism seems nice and could even be used for point-to-point communication (where some specific topics are only subscribed by certain clients).
But it is (like JSM, STOMP, etc.) asynchronous. The sender sends a message into the broker and doesn't know whether it is ever delivered to it's recipient. We want that the sender gets informed about a successful delivery or an elapsed timeout (when no one is receiving the message).
Is there some protocol/implementation available that provides such synchronous messaging functionality?
(It would be nice however if asynchronous delivery would be possible, too)
The messaging by default is ( usually ) asynchronous .
You can considerer RabbitMQ, it contains the following features:
Publisher-confirms (in asynchronous way):
http://www.rabbitmq.com/blog/2011/02/10/introducing-publisher-confirms/
Transaction Commit:
https://www.rabbitmq.com/semantics.html
Messages TTL (to handle time out)
https://www.rabbitmq.com/ttl.html
With this features you can handle the time-out situations and the successful delivery.
If this is not enough you can use the RPC:
https://www.rabbitmq.com/tutorials/tutorial-six-java.html
Let me know if you need more information.
I am using TIdCmdTCPClient and TIdCmdTCPServer. Suddenly I find that I might like to have bi-directional communication.
What would be best? Should I possibly use some other components? If so, which? Or should I kludge and have the 'client' poll the 'server' to ask if it wishes to communciate anything?
This is a very small system. Two clients and ten servers, with a burst of one tarnscation every 30 to 60 seconds for a few minutes once a day, so overhead for polling is inconsequential.
I'm just woder if there is a 'correct' way.
Update: this really is an incredibly simple system. Very little traffic and all of it simple. All transmissions are an indication of even type an an optional single parameter.
<event type> [ <parameter>] e.g. "HERE_IS_SOME_DATA 42"
This can be sent in both directions, hover here is no "reply" as such. Just fire off a message (and hope that it got there)? Receive an Ack with no data? Non-catching of an exception indicates that message was successfully sent?)
Would it be possible (would it be overkill) to use two TIdCmdTCPServer?
Both TIdCmdTCPClient and TIdCmdTCPServer continuously poll their socket endpoints for inbound data during the lifetime of the connection. You do not have to do anything extra for that. So, as soon as a TIdCmdTCPClient connects to the TIdCmdTCPServer, both components will initially be in a reading state until one of them sends a command to the other.
Now, there is a problem with doing that - as soon as either component sends that first command, the receiving component will interpret it as a command and send back a reply, which the other component will interpret as a command and send back a reply, which will be interpretted as a command and send back a reply, and so on, causing an endless cycle of replies back and forth. For that reason, it is not wise to use TIdCmdTCPClient and TIdCmdTCPServer together. You should either use TIdTCPClient with TIdCmdTCPServer, or use TIdCmdTCPClient with TIdTCPServer. Depending on what exactly your protocol looks like, you may have to forgo using TIdCmdTCPClient and TIdCmdTCPServer altogether and just use TIdTCPClient with TIdTCPServer so you have more control over reading and writing on both ends. It is hard to answer with actual code without first knowing what the communication protocol should look like.
A single TCP socket connection can be used in two directions. The server can send data asynchronously to the client at any time. It is up to the client however to read the socket, for asynchronous processing this is done in a listener thread which reads from the socket and synchronizes incoming data operations with the main worker thread.
An example use case in the Indy components is the Telnet client component (TIdTelnet) which has a receive thread listening for server messages.
But you also asked about the 'correct' way - and then the answer depends on other factors such as network stability, guaranteed delivery and how to handle temporary server outages. In enterprise environments, one central messaging hub is preferred in many use cases, so that all parties connect only to this central server which is only responsible for reliable message delivery, and keeps messages until the recipient is available.
You can download the INDY 10 TCP server demo sample code here.
I'm looking for the best way for my organization to implement some BizTalk monitoring that will notify us when messages are being suspended.
How can this be done?
I would take a look at System Center Operations Manager or BizTalk 360.
Did you look at this option, it helps to notify not just suspended instances, you can look out for any states like active, ready-to-run, dehydrated etc
http://blogs.biztalk360.com/what-is-biztalk-service-instances-suspended-active-schedule-etc-and-how-to-monitor-it/
If your project is fairly simple, it is also possible to do it entirely in Biztalk, using BAM and BAM Alerts. A good book is 'Pro Business Activity Monitoring in Biztalk 2009' which has a chapter 'Monitoring Biztalk Status using BAM'.
Briefly, this is what you do for a simple routing based scenario, where you have a receive port, orchestration/map to transform the incoming message and a send port.
Define a BAM Activity for Inbound/Outbound properties like filename, received (timestamp), sent(timestamp), messageid etc.
Create views and then deploy your BAM Activity.
Configure Tracking Profile Editor to start monitoring the ports and apply your changes.
Create a BAM alert, where the send port timestamp (sent) is empty and the receive port timestamp (received) is in the past 1 hour.
Add subscribers to the newly created Alert.
This will alert any situation, where you have received a file and did not send it out even after 1 hour ( configurable), which most probably will end up as a suspended message.
You can use power shell script to get email alert for suspended messages for BizTalk.
https://gallery.technet.microsoft.com/scriptcenter/BizTalk-Suspended-Service-12f5342e
Attached script will be use for monitoring suspended message.