I am very new with mqtt design.
As I see from some tutorials in the internet, common mqtt topic has this format: /home/room/device_type/device_id
I could not see the benefit to do that. And have no idea how to use this kind of design.
From my point of view, the device (dev) might subscribe (sub) to control topic and publish (pub) to status topic. Like this:
pub: clients/dev/devid/stat
sub: clients/dev/devid/ctrl
In this way, it seems sub,pub logic is very simple for both clients and devices
Could someone please tell me some good way to design mqtt topic ?
(!) Please do not start topic with '/' (This one has been recommended by HiveMQ Team)
EDIT:
I just figured out that for whatever kind of design, the model must serve-able at least:
Individual control: send control command to a particular device.
Group control: send control command to a group of devices: type, defined group
Able to recieve the status of device.
Thank you very much
I found that the following topic split scheme works very well in multiple applications
protocol_prefix / src_id / dest_id / message_id / extra_properties
protocol_prefix is used to differentiate between different protocols / application that can be used at the same time
src_id is the ID of the mqtt client that publishes the message. It is expected to be the same as "client ID" used to connect to MQTT broker. It allows quick ACL control to check whether the client is allowed to publish specific topic.
dest_id is client ID of the "destination" unit, i.e. to whom the message is intended. Also allows quick ACL control on the broker of whether client is allowed to subscribe to a particular topic. There can be reserved "destination" strings to specify that the message is broadcasted to anyone who is interested. For example all.
message_id is actual ID of the message within used protocol. I usually use numeric value (as string of course), because the IOT or other embedded system that is connected to MQTT broker can have other I/O links and I would like to use the same protocol (but with different transport framing) to control the device using these other I/O links. I usually use numeric message IDs in such communication links.
extra_properties is an optional subtopic which can be used to communicate other MQTT specific extra information (comma separated key=value pairs for example). Good example would be reporting timestamp of the message when it was actually sent by the client. In case of "retained" messages it can help to identify the relevance of the received message. With MQTTv5 protocol that is expected to arrive soon, the need for this subtopic may disappear because there will be other way to communicate extra properties.
Hope it helps.
We have done some work on that for the domain of manufacturing (Industrial-IoT, not IoT!).
In our scenario there are lots of serverside apps of different companies communicating through MQTT. Thus we needed some overall structure. We call this "Manufacturing Message Stack".
The bottom layer is MQTT, then there is the "Messaging Layer". It consists mainly of
basic topic specs
basic payload specs
On top of the messaging layer there are domain message layers covering various domain specific topics as system messages, alerting, physical device / digital twin messages or other manufacturing related messages.
Topics
The topics are roughly defined as <senderapp>/<app-id>/<message-name>/<args> e.g. pacman/pacman-1/gameover (this is a sample for illustration only!)
The developer of an app which publishes MQTT messages defines <message-name> and <args> depending on the semantics of the payload.
The <senderapp> and <app-id> refers to the sending App and allows to quickly select messages from a defines origin (publisher). We deploy Apps in a microservice environment built with Docker, Rancher - and soon - Kubernetes.
Payload
The payload is specified in JSON format. There is a JSON schema reference URL in each
build an URL to an API of the publishing app which holds further information (e.g. JSON schema) of the message sent. Thus a subscriber can get meta data of a MQTT message on demand. Static meta data is not sent with a message to reduce the payload size.
Payload sample:
{
"$schema": "http://app/api/messages/message1.json",
"score": 1234,
"highscore": false
}
Publisher's message meta data
The publishing app holds an index of all messages which can be sent in an API at
http://<app>/api/messages/index.json:
{
"message1": "message1.json",
...
}
Each message is described by its JSON schema message1.json:
{
"$schema": "http://json-schema.org/draft-06/schema#",
"title": "Pacman end of game",
"properties": {
"score": {
"description": "Players score at the end of game",
"type": "integer"
},
...
}
}
Unfortunately we did not publish our manufacturing message stack yet. Publishing is planned in the next months. Feedback is welcome.
I think if topics are to reflect the physical world, we should look at something like Signal K.
EDIT:
That spec is also still maturing, but it includes concepts like "self" for the server/broker, and a tree that can start at the current vessel/home, but easily extends upwards to other vessels/aircraft/things.
My two cents:
All topics are read-only unless they end in "/set"
Ideally, topics are reasonably normalized and granular. I can understand grouping values up into a group topic. IMHO, this kind of decision should be application-specific.
Payloads should be strings, to avoid endian-ness issues
Here's one suggested tree:
broker = information of this specific broker
broker/clients
broker/clients/count
broker/clients/0/name or broker/clients[0]/name
broker/topics
home = this current location (could also be "here" or something)
home/kitchen/temperature "19C"
home/kitchen/temperature/hardware/type "ESP8266"
home/garage/maindoor/set "closed"
locations = list of all known locations
locations/0/uuid
locations/0/name
locations/0/address
Related
In my current IOT project. I will be using multiple ESP8266s (3) to send data and receive actions. Each MCU will be in charge of monitoring different aquariums around the house. I have thought of structuring my topics like the following:
"Data" topics will follow the same structure, for example to retrieve temperature data:
esp8266/aquarium/aquarium_id/temperature/dht11
"Action topics", the topics the MCU subscribes to receive commands, for example:
aquariumcontroller/aquarium_id/action/water
The topic which the aquarium subscribes to to updte the MCU's params:
aquariumcontroller/aquarium/aquarium_id/params
The aquariumcontroller is the MQTT client written in Python. This is the entity that will be sending actions and handling messages received. I have two questions, are my topics correctly named and structured to handle multiple aquariums? Also as I will have a Controller, isn't it better to also have a database which will contain the aquariums info like the topics for this specific aquarium and the params or will i run into problems if i persist topics when i change the aquarium ID?
Thank you
What you mentioned will work.
You can also have a common topic that everyone subscribes to and use JSON format to send data. In JSON format you can mention necessary identifiers and actions.
{
"aquarium_id": "xxx",
"operation": "temperature_read",
"value": "24.5"
}
For every aquarium_id you can have an individual topic. The advantage of this scheme is that aquarium controller will only receive message that are associated with it however their will be a added complexity where by you have to pre-populate aquarium_id in your server.
{
"operation": "temperature_read",
"value": "24.5"
}
You should ideally have a database to store aquarium_id and its relevant properties (something like master record). With database you can also store you readings and action to get an historical view of your data. You can use postgres as DBMS.
I am trying to set up the Cumulocity smartREST response templates to supply info from the device object stored by an app. I am thinking it is not possible but I want to confirm because the documentation is brief and not clear.
I have successfully implemented MEASUREMENT POST templates previously but I am trying to do INVENTORY GET now. I have set some values in the device object and I can see them from tenant.cumulocity.com/inventory/managedObjects/id/. This could look like:
"custom_values": {
"val1": "abcde",
"val2": "fghij"
}
Now I go to the smartREST template web interface editor and create a template (eg t1) and set a message (eg m001) and set it to 'inventory' and 'get' and tick 'includes response'. I'm not sure if I need custom fields here. Then I create a response (eg r001) and I now have to fill in base pattern with a condition and some number of other patterns. I assume in one of those fields I have $.custom_values.val1 etc but all permutations I try result in 'no access to object' error from topic 's/e'.
My ideal result (from the above example) would be to publish to '/s/uc/t1' with 'm001' and receive 'r001,abcde' (ie custom_values.val1) and publish 'm002' and receive 'r001,fghij' (ie custom_values.val2).
Thanks for taking the time to read.
Your approach is correct but your MQTT user probably is just missing the rights to the object as it is created by the app (I guess the mqtt client is a device user).
The best approach here is to give this object you are creating from the app the fragment c8y_Global
{
"name": "testObject3",
"c8y_Global": {},
"custom_values": {
"val1": "abcde",
"val2": "fghij"
}
}
This fragment will make this object available to all registered users. This is common practice for app meta data.
I am pretty new to Cumulocity and I am trying to get data into the platform from my own device using mqtt and the smartrest templates. I can get data in using the static templates but they only support certain data types. I am struggling to create the appropriate smartrest template in the UI and the documentation doesn't go into much detail.
I get that the template name goes in the MQTT topic (or selected on login as part of the username) in s/ut/template_name and the messageId of the messages in the template get matched to the first CSV field of the MQTT publish payload. What I don't get is the template terminology. In the UI I choose API->Measurement and Method->POST and I am presented with required values $.type and $.time. My questions:
Is $.type the "measurement fragment type" name or do I have to make it "c8y_CustomMeasurement"? Can I call it whatever I want?
$.time has a value field. Is this the default value if one is not supplied in the publish?
I assume I need to add a numerical value in the optional API values. To link it to the value of the data point should I make the key "c8y_CustomMeasurement.custom.value"?
Am I way off base here?
Every time I publish to my own smartrest template the server drops the connection so I assume its an error in my template setup but I don't see a way of accessing debug messages (also nothing is published back to me on s/e or s/dt).
For the sake of an example, lets say I wish to publish a unitless, timestamped pulse count with payload format "mId,ts,value" with example data "p01,'2017-07-17 12:34:00',1234"
What you wrote so far is mostly correct just to be a bit more precise:
The topic is s/uc/template_id (not the template name, this is just a label)
The $.type refers to the 'type' fragment in the measurement JSON. It is a free text field
In 99% of cases you want to leave the $.time empty. If you set something here it is not the default but fixed to that timestamp and you cannot change it when using the template. If you leave it empty and still not send something in
Example: p01,2017-07-17T12:34:00,1234 (no quotes arounf timestamp and ISO8601 format
Example without sending time: p01,,1234 (sending empty string as time results in server time beeing set. The template is the same)
Hope these points help you to find you issue
I'm using System Center Orchestrator 2012, and I have a generic error handler which is connected to all the activities in my runbook.
I have subscribed to a list of published data as a parameter in this error handler, which looks like this:
Activity Name : {Activity Name from "Monitor File"}{Activity Name from "Move file"}.......
What I need to do is copy this list of subscribed data and do a search and replace to change the Activity Name parameter to something else, say Error Message. Whenever I copy this list of variables, I get gibberish when I paste it
\`d.T.~Ed/{598EBDFA-BF5B-4B77-8156-E6FA6ECD0CE1}.Object.Name\`d.T.~Ed/\`d.T.~Ed/{6838D741-DF8E-4C25-8C28-D06A52F67D36}.Object.Name\`d.T.~Ed/\`d.T.~Ed/{D1D9BBE6-5AAF-4D8F-A98A-1A8BDD977E7E}.Object.Name\`d.T.~Ed/\`d.T.~Ed/{9EC92323-1B9B-4D06-88E9-A97BA525CF5A}.Object.Name\`d.T.~Ed/\`d.T.~Ed/{FAD32B4C-92CC-40BD-837A-4C5F22C2E018}.Object.Name\`d.T.~Ed/\`d.T.~Ed/{DFF7A110-ACFD-4377-AFEC-16B5BEC8BFF4}.Object.Name\`d.T.~Ed/\`d.T.~Ed/{139ACC4E-CF6D-4EEE-BD88-9DC1E0FC2038}.Object.Name\`d.T.~Ed/\`d.T.~Ed/{1484789C-BB37-4507-AD21-E367665E0BE6}.Object.Name\`d.T.~Ed/\`d.T.~Ed/{01EA8BD0-69C5-4959-86DB-29FAD34D144A}.Object.Name\`d.T.~Ed/\`d.T.~Ed/{A60C582B-5DD0-41F7-BB0A-B5D71C3B9ECB}.Object.Name\`d.T.~Ed/\`d.T.~Ed/{1FA1E2D6-813D-4A4A-A5CD-07EB2AD4AC9B}.Object.Name\`d.T.~Ed/
Is there a way to simply copy the text? If not, I will have to click on all the activities over and over again to subscribe to different types of data.
Thanks in advance
The gibberish you're seeing is how Orchestrator translates the published data and variables to the database guids. Copying this out will always translate to the DB guids, so there's no way to do a copy/replace with "activity name" for "Error Summary". You'd need to subscribe to each activity's data or look up all of the guids which probably wouldn't be easier.
Unfortunately, setting up the error handling process in SCOrch becomes tedious to be effective.
I use a separate email linked to each activity to give more information based on the error message. I don't like to use generic error emails with published data from multiple activities since it doesn't necessarily give all the useful information you need (or an end user will be able to understand).
What I'm trying to do:
Be able to have users subscribed to a number of different 'chat rooms' and use reverse AJAX / comet to send messages from a chat room to everyone logged into that room. (a bit more complicated but this is a similar use case).
What I'm doing:
Using Grails with JMS and Atmosphere. When a message is sent, I'm using JMS to send the message object which is received by a Grails service which is then broadcasted to the atmosphere URL (i.e. atmosphere/messages).
Obviously JMS is a bit redundant there but I though I could use it to help me filter who should retrieve the message although that doesn't really look it'll work (given that the subscriber is basically a singleton service...).
Anyway, what I need to be able to do is only send out a message to the correct subset of people listening to atmosphere/messages. A RESTful-type URL will be perfect here (i.e. atmosphere/messages/* where * is the room ID) however I have no idea how to do that with Atmosphere.
Any ideas / suggestions on how I can achieve what I want? Nothing is concrete at all here so feel free to suggest almost anything. I've even been thinking (based on the response to another question), for example, if I could do something like send out messages to a Node.js server and have that handle the reverse AJAX / comet part.
If I understand your requirements correctly the following should work (jax-rs + scala code):
1) Everyone who wants to get messages from a chat room registers for it:
#GET
#Path(choose/a/path)
def register(#QueryParam("chatroomId") chatroomId: Broadcaster) {
// alternatively, the Suspend annotation can be used
new SuspendResponse.SuspendResponseBuilder[String]()
.resumeOnBroadcast(false).broadcaster(chatroomId).scope(SCOPE.REQUEST)
.period(suspendTimeout, TimeUnit.MINUTES)
.addListener(new AtmosphereEventsLogger()).build
}
2) To broadcast a message for all the registered users, call the following method:
#POST
#Broadcast
#Path(choose/a/path/{chatroomId})
def broadcast(#PathParam("chatroomId") id: String) {
// first find your broadcaster with the BroadcasterFactory
BroadcasterFactory.getDefault().lookupAll() // or maybe there is a find by id?
broadcaster = ...
broadcaster.broadcast(<your message>)
}
I also recommend reading the atmosphere whitepaper, have a look at the mailing list and at Jeanfrancois Arcand's blog.
Hope that helps.
There is a misunderstaning of the concept of comet. Its just another publish/subscribe implementation. If you have multiple chat-rooms, then you need to have multiple "topics", i.e. multiple channels the user can register to. E.g.:
broadcaster['/atmosphere/chatRoom1'].broadcast('Hello world!')
broadcaster['/atmosphere/chatRoom2'].broadcast('Hello world!')
So I would advance you to creaet multiple channels and do not filter manually the set of users, which should retrieve messages (which is definitely not the way it should be done). You do not need to create anything on the server side on this, since the user will just register for a specific channel and receive messages, which anyone is putting into it.
I would recommend you create an AtmosphereHandler for one URL like /atmosphere/chat-room and then use the AtmosphereResource and bind an BroadcastFilter with it, lets say name it ChatRoomBroadcastFilter.
Whenever a user subscribes to a new chat room, a message would be sent to the server (from the client) telling the server about the subscription. Once subscribed, maintain the list of users <> chat room bindings somewhere on the server.
Whenever a message is broadcasted, broadcast it with the chat room id with it. The in the ChatRoomBroadcastFilter (You probably need to make this a PerRequestBroacastFilter) propagate the message to the user only if the user subscribed to the chat room. I am not sure if this clears it out. If you need code example please mention in the comments. I'll put that but that needs some time so ain't putting it right now ;).