I have the doubt if a MQTT Broker also can be a client? Or I must need to separate and get a dispositive to act as a broker and another as a client. I'm not finding information on internet.
If I understand the question properly, what you are looking for is MQTT bridging.
This is where one broker acts as a client to a second (or multiple) broker and based on config copies messages on topics between the 2 brokers.
How this is configured depends on which broker you are using, but the concept is part of the spec (I don't think it's optional off the top of my head) so all brokers should support it
Related
I have an application when I'm sending MQTT messages to an IoT platform, the IoT platform has their own broker. The problem arose when the broker went down for 2-3 days, with that I lost 2-3 days worth of data.
I was wondering if there was a way to ensure that all data points are stored, and then sent when the broker come back online in order. I've been testing this with Mosquitto, but I can't seem to get it to work.
Is it a matter of using Quality of Service (QoS)? Does this work even the broker is down, or does it need the broker to communicate with? Or do I need to use persistence or retain?
Yes, you are on the right track, it requires QoS and must be used with the other settings together, you can test under the following conditions:
Initialize your MQTT client with clean session flag set to False and a unique client ID;
Here is an example using Paho python library,
mqttc = mqtt.Client("specify_a_unique_client_id", clean_session=False)
Subscribe to a topic with QoS >= 1;
Publish to a topic with QoS >= 1;
NOTICE: you must specify a unique client ID, so that your broker can still recognize the previous client session in case it reconnects. Leave the client ID as empty will auto generate a new one.
Bonus, Here is a good series of articles to explain all the configurations in MQTT, in case you want to understand the details.
I'm trying to create a reasonable setup for client-client-communication for our existing infrastructure. I've been reading the docs for Spring, Websocket, STOMP, SockJS and ActiveMQ for over a week now and I'm not not sure whether what I am trying to do is feasible or sensible. Spring server and JavaScript client were up and running relatively quickly and sending messages between clients just works (direct connection from JS client to Spring server). This setup won't suffice for our needs so we decided to put dedicated brokers in between. Configuring ActiveMQ is a nightmare probably because I don't really know where to start. I have not worked with a dedicated broker so far.
Environment
170 independent servers (Tomcat, Spring, SockJS, STOMP)
2 ActiveMQ (Artemis) brokers (load balance, failure safety)
a few thousand clients (JavaScript/.NET, SockJS, STOMP)
Requirement
I need every client to be able to talk to every other client. Every message has to be curated by one of the servers. I'd like the clients to connect to one of the ActiveMQ brokers. The ActiveMQ brokers would hold a single connection to every single server. The point is to avoid that all my clients would have to open 170 WebSocket connections to all the servers. The servers do not need to talk to each other (yet/necessarily) since they are independent with different responsibilities.
Question
Is ActiveMQ or any other dedicated broker viable as transparent proxy/relay i.e. can it handle this situation and are there ways to dynamically decide the correct recipients or should I go another route like rolling my own Spring-based relay?
In a traditional messaging use-case (e.g. using ActiveMQ Artemis with STOMP) the broker manages "destinations" and any messages sent to those destinations. Messages are only dispatched to clients if the client specifically creates a consumer on a destination.
In your use-case all of your 170 "servers" would actually be messaging clients. They would need to create a consumer on the broker in order to receive messages. To be clear, once the consumer is created then the broker would dispatch messages to it as soon as they arrived.
I'm not sure what exactly you mean by "transparent," but if it means that the process(es) receiving the message(s) don't have to do anything then no message broker will handle your use-case.
I am willing to create a mediator which is subscribed and published to 2 separate broker who have no access to topics of each other. The aim is to updates and create a logic of the message published by broker 1 and send it to broker 2 according to the set of rules
Do I need 2 separate ports ? As the topic level might be different in both brokers
Any help is much appreciated!!!
There is no MQTT standard (note, can only speak for 3.1.1) defined features that would allow a client maintain two concurrent connections. Therefore, this is entirely broker implementation-dependent and necessitates a bridge.
For example, the Eclipse Mosquitto broker can be configured as a bridge to another broker and even remap topics from itself to a different topic structure of the other. Please refer to the Mosquitto man page section Configuring Bridges for the specifics.
As far as creating a bespoke application, you can always write a simple Python program that is running two instances of an MQTT client (Eclipse Paho for example, which has a lightweight asyncio wrapper to facilitate concurrency), each connected to different brokers. The glue logic between them just has to re-publish an incoming subscribed topic message from Broker A to some topic, with or without a remapping step, to Broker B.
If the two brokers are both running locally on a single NIC, then you would need to use different ports.
I'm trying to help a customer connect their Mosquitto bridge to Azure IoT Edge. They have some legacy equipment that speaks MQTT, but because it can't do TLS and the topics can't be changed, we are trying to run the messages through the Mosquitto MQTT Broker, and over to IoT Edge via the Mosquitto bridge...
I've had no problems getting the actual connection made from the bridge to IoT Edge and I have messages flowing to the bridge. That connectivity works fine. The problem comes in the topics. I really can't change the topic structure that the client publish on. However, IoT Edge requires messages to be published on a specific MQTT topic (devices//messages/events). Where device_id is the name of my broker, let's say 'mymqttbroker' just for fun.
So, what I'm trying to do is to take the messages that some in on pretty much any topic, and resend those messages through the bridge on the devices/mymqttbroker/messages/events topic to IoT Edge.
I know the topic line in the bridge config has the remote_prefix and local_prefix parameters, but that won't cut it. Per this article, it says you can't do this..
"E.g A broker would receive messages to topic sensor1 and remap them to new_sensor1. Currently this form of remapping is not available,"
Any idea how to do something like this? is it possible? Essentially, is there any way in the bridge to accept messages from any topic, and republish them on a specific fixed topic?
The quick and dirty way is to write a little helper app that subscribes to the old topics and republishes to the new topics, then just bridge the new topics.
It does add another point of failure, but it's the only option for mosquitto.
If you are not wedded to mosquitto, you can build your own custom broker with something link mosca and add the remapping into the broker.
With an MQTT broker, is it possible to set up multiple consumers for a topic such that for any given message on that topic only one consumer will receive the message?
The short answer is no, not with any broker that purely implements the MQTT spec.
I suppose it would be possible to write a broker that talked to the clients using MQTT and only delivered messages to a single subscriber. (It would have to deliver with QOS2 to ensure that every message was consumed)
By coincidence I was talking to a colleague about something similar to this sort of thing earlier in the week, he had found a way to do it using IBM* MQ Light and something called 'Shared Destinations'. (MQ Light uses AMPQ not MQTT)
https://developer.ibm.com/messaging/mq-light/
full disclosure, I work for IBM
UPDATE:
I've since been informed that the IBM MessageSight v1.2 appliance can actually do shared destinations using MQTT (http://www-03.ibm.com/software/products/en/messagesight)
UPDATE 2:
Shared subscriptions is an optional part of the MQTT v5 spec so worth checking any v5 brokers for the option.
Look at Shared Subscriptions https://issues.oasis-open.org/browse/MQTT-234
some MQTT servers support it.
EMQTT (open source):
https://github.com/emqtt/emqttd/issues/639#issuecomment-247851593
HiveMQ:
http://www.hivemq.com/blog/mqtt-client-load-balancing-with-shared-subscriptions/
IBM MessageSight:
http://www.ibm.com/support/knowledgecenter/SSCGGQ_1.2.0/com.ibm.ism.doc/Developing/devsharedsubscriptions.html
VerneMQ:
https://vernemq.com/docs/configuration/balancing.html
That is not possible. In MQTT all subscribers to a particular topic receive messages published to said topic. In order to direct a message to a particular subscriber, both publisher and subscriber would have to use a particular topic different to that used by other subscribers.
Independent of the broker that you're are using, you can use Apache Camel to implement a route that copies all messages from Topic A to Topic B.
Or copy only specific messages that match an specific rule such as user, content pattern, QoS.
Other solution is using a multi-protocol broker such as ActiveMQ and copy specific message topics to a Queue (queues only can have one consumer) and consume the queue with another protocol such as JMS or STOMP.