how to sanitize mqtt message payload in server side? - mqtt

I made an instant messaging app using MQTT protocol.
I want to add some extra data about messages in payload like sent time ( server time not client time ) and also provide kind of server side payload sanitizing.
Is it a good idea to add a third party client with superuser privileges between message sender and message receiver on broker's local machine to do this job ?
or is there any better idea ?
by the way I'm using EMQTT as message broker.

From a pure security view having direct peer to peer traffic (without filtering and sanitising) sounds like a dangerous idea. (At least in the Internet-of-things domain I would clearly object against it.)
Why? Because the clients are outside of your control (i.e. a hacker can re-engineer) and inject any traffic to exploit security holes on the receiving side of other clients.
So sanitising on the server side sounds like a very good idea.
I would suggest two topics: One (inbound) topic the clients use to publish messages, and another (outbound) topic used by clients to subscribe to messages. A server side component would then read the messages from inbound topic, sanitize it and publish to the outbound topic.
This de-coupeling makes it also easier to introduce MQTT payload changed: If you update the payload in a non-compatible way, introduce a new inbound topic and keep the old inbound topic too. This allows you to support old and new clients during the transition phase.

Related

How to get all messages sent to a Smooch appid without using webhooks?

Using the Smooch API, I am trying to obtain all of the messages sent to my Facebook appid in the past few minutes or hours.
The Get Messages REST method does exactly what I need, except for that it only returns messages from a particular appUserId. This isn't useful unless you already know what users have sent you messages. I cannot use a webhook as the application resides behind a corporate firewall. Opening the firewall to connections that originate from the outside is not an option (even with white-listing).
Is there a way to invoke the Get Messages REST method such that it will ignore the appUserId filter? Perhaps some sort of wildcard character?
GET {{url}}/{{apiVersion}}/apps/{{appId}}/appusers/{{appUserId}}/messages
Unfortunately you do need to have the appUserId (or userId) on hand in order to query user messages.
Webhooks are a pretty essential part of building a Smooch integration. If you can't receive them through your firewall, then you might consider building an intermediary service outside of your corporate network for receiving Smooch webhooks. For each webhook event you receive it would either:
Forward it through a secure tunnel into your coprorate network
Store the appUserId (or the whole event) in its own database, and provide a secure endpoint that allows your corporate network service to query that data
I'm curious to know more about your use case, e.g which Smooch channels are you integrating? With more details I might be able to improve this answer.
#alavers We would like to leverage nearly every messaging integration you offer.
#alavers You may want to consider providing a Get Messages variant that is better suited for use within a corporate firewall environment. An excellent example is the http long poll implementation provided by APIs such Amazon's SQS API. Their receiveMessage method waits for up to the specified time period but returns as soon as a message is received. This provides nearly the same performance of a webhook but eliminates the need for a customer to open their corporate firewall to connections that originate from outside the corporation. Most IT departments will approve connections that originate from within the corporation, but permitting connections that originate from the outside becomes a very difficult sell.

Does MQTT Support User to User Messages

I know that using MQTT topics devices can subscribe to them. But is there any way that a IoT device can send some message to a target IoT device (by device id or something) without using a topic or is there any standard topic for this scenario?
There is no way to communicate without a topic, but you can create a topic for any purpose. So typically if you wanted to send a message to another client, you would publish it somewhere in the hierarchy of topics to which that client is subscribed.
That could be as simple as something like device/12345/inbound or whatever you prefer. And because topics can have hierarchy, in addition to whatever detail you put in the body, you can also encode categorization of your message into the topic, much as RESTful APIs often do in a URL.
A good reason for using target-specific (or owner-account-specific) topics is that the most easy solutions for MQTT security compartmentalization are topic-scope.

Can mosquitto forward the ClientID of a messages sender?

I am using the mosquitto MQTT Broker.
Also, I have multiple (currently 10, but the number will increase) clients that publish some sensor data periodically to topic A. These clients are technically identical, but do have a unique identifier (serial number).
I also have a client that subscribes topic A in order to receive the published messages and persist the sensor valus in a database.
I certainly need to know which Sensor (i.e. client) has sent a particular value.
As a solution, one could just append some Sensor ID to the payload of each published message. But since the sensors access the broker via GSM, I need to keep the traffic low, so I am trying to avoid that.
I assume, the Broker itself knows which message comes from which client, especially when using perisistent connections, i.e. clean_session=False. Is that correct?
If yes, is there any chance that the subscribing clients can obtain the client_id when receiving the message?
Can it be configured in mosquitto? Or is it default behavior and I am missing something?
I am using paho-mqtt 1.3.1 for all clients.
No, the client id is not part of a published message. It is only used to identify the client to the broker when the connection is established in order to determine if stored messages and persistent subscriptions should be honoured.
The easiest solution is to use a separate topic for each sensor but with a shared root. e.g.
sensor 1 publishes to A/1
sensor 2 published to A/2
The client would then subscribe to A/+ this would then receive all the messages and can use the second half of the topic to determine which sensor it came from.
The other options is as you suggested which is to include the id in the payload.
Sending the client-id with payload(message) is possible. But you need to use delimiters in payload(message) at publisher side . Example: Publisher sends the payload as "client-ID=3 - temperature = 29 " . At the subscriber side , you remove the delimiters using strtok() .
There is no configuration available at broker side.
Per my experience with mosquitto, I don't think there is an option for mosquitto to change either the topic or the payload when re-publishing a received message.
However, I think it is just an implementation issue.
Theoretically, I think it is OK and good to support such kind of feature, since it does not violate MQTT specification at all.
(http://docs.oasis-open.org/mqtt/mqtt/v3.1.1/csprd02/mqtt-v3.1.1-csprd02.html#_Toc385349773, Section 3.3.2.1)
However, since the Server is permitted to override the Topic Name,
it might not be the same as the Topic Name in the original PUBLISH Packet.
The pratical solution for your current problem is, as pointed by #hardillb, either publishing using different topics but receiving using a topic with wildcard (+ or #), or, containing publisher information in payload.

How do I differentiate between MQTT Publish events originating from Users vs internal messages meant to broadcast only

I am using mosquitto mqtt client.
For example, you have users that publish and subscribe to topics. The topic actually correlates to a REsT endpoint.
Scenario 1 (typical pub/sub usage)
UserA subscribes to topic /device/123/meta
UserB publishes some data to topic /device/123/meta
by definition, this publish is broadcasted out to the subscribers
we have a script subscribed to /devices/# which knows how to save the payload for the topic /device/123/meta when it receives publish'ed data. This data is saved to the database.
Scenario 2
Someone updates data /device/123/meta via a ReST interface (or a direct DB update, the key is it's not a MQTT publish).
database is updated
a publish message is sent to the MQTT broker so that all subscribers get the updates as a payload
Scenario 2 is what I'm trying to wrap my head around. This creates a nasty feedback loop. When internal messages are broadcasted out, my script to deal with publish events from users can't differentiate between publish events originating from a 3rd party user or an internal publish event only meant to broadcast out some data (with no saving of data needed).
How should I handle this? The MQTT message is very simplistic and I'm not finding anything I can base logic off of. I'm trying to explore using the origin somehow, but no luck this far. I realize I can write plugins, but this is quite the task for mosquitto.
There is no way to distinguish where a message originated from the subscriber at a pure MQTT protocol level. Part of the point of a pub/sub protocol is to decouple publishers from subscribers.
The most portable way to do this would be to add a flag to the actual message payload to indicate that message originated from somewhere other than the actual device.
Or assuming the message is being published as a result a trigger in the database have the trigger check if the incoming message actually changed the database stored value, if the incoming messages matches the existing state of the DB then there is no need to republish it.
Mosquitto's plugin mechanism is currently only for writing authentication and authorisation solutions, but the JavaScript mosca or Java HiveMQ brokers support plugins that may be able to do what you want.

Need to poll Mqtt broker from a java REST API for already published messages to MQTT

I am looking for a solution to poll messages from MQTT broker. I will describe the solution briefly here.
We have a Spring based Controller class which exposes REST APIs to handle certain vehicle related diagnostics data. Through one of these API-s Notify3P() I create a MQTT java client and publish messages based on some input data to the MQTT broker on a given topic. My requirement is to notify a third-party system every time the client publishes a message on MQTT.
The 3P system is going to pick up the message from MQTT once it receives the notification. It then needs to get the message from the MQTT broker through a getMessage() REST API (which we need to expose on the above controller class). The getMessage() API needs to poll MQTT for the messages that have already been published and hand it to the 3P system. The 3P system would then do some processing and send back a response to our system through another REST API postMessage() exposed on our controller class. The postMessage() should post the message on the response topic on MQTT. I need another REST API checkResponse() which then polls the response topic of MQTT and send back the response to the client.
What I have done so far: On application start up I have a start up bean which listens to MQTT request and response topics. Now I publish data to request topic using the REST API Notify3P(). I have attached a callback with the startup bean which gets the message. The problem comes when the 3P needs to call my controller to poll message from MQTT.
I am not clear how to hold back messages on MQTT and consume it on demand. Is there a mechanism to do it in MQTT? Also once the 3P system post messages on the response then again how do I poll the response topic to pick up the response from MQTT and send to clients of my Controller?
I hope the problem description makes sense. If there is any solution from anyone please post it. Any sample code would of great help.
Thanks in advance!!
You may have got the idea of MQTT a bit confused. One of the key points is that there is no polling.
You subscribe to your response topic and publish to the request topic. As soon as a response is available you will be sent it by the broker. You can't hold back messages.
It sounds like your controller also needs to talk MQTT. If it is subscribed to the response topic from the start then it will receive the messages and you can do with them what you will, no need for polling.
To achieve exactly what you want, where the third party notifies the controller to read messages from MQTT then the controller would need to be able to use MQTT anyway. At that point you might as well do it "properly". If you don't want to integrate MQTT into your controller, then you can't do what you describe and you will have to come up with another means of communicating between the two components.
Summary - get your controller to talk MQTT if you can.

Resources