How can I prevent receiving the same message in MQTT what the same client has published?
Problem:
Client1 subscribes: site1/feeds/#
Client1 publishes: site1/feeds/one
Client1 receives: site1/feeds/one <- how can I prevent this on the local client?
Client2 publishes: site1/feeds/two
Client1 receives: site1/feeds/two <- this I want to keep
As Bubbafat said you can't prevent this at the protocol level and I'm not aware of any brokers that implement this as a feature.
Your only real option is to add a filter in the onMessage callback
Related
I borrowed code from Toms Hardware on how to use MQTT and subscribe. JRMI is the publisher of the messages and it keeps repeating them over and over again. Is there anyway to have the message sent only once? I dont have this problem when I subscribe to MQTT via http://www.hivemq.com/demos/websocket-client/ The MQTT service I'm using is broker.hivemq.com
For those not familiar with JRMI, it is the JAVA program that model railroads use to control tracks,lighting, DCC etc. Ref: https://www.jmri.org/
The link to Tom's is here https://www.tomshardware.com/how-to/send-and-receive-data-raspberry-pi-pico-w-mqtt
The code adapted from Tom's is
import network
import time
from machine import Pin
from umqtt.simple import MQTTClient
wlan = network.WLAN(network.STA_IF)
wlan.active(True)
wlan.connect("whatever","pwd")
time.sleep(5)
print(wlan.isconnected())
mqtt_server = 'broker.hivemq.com'
client_id = 'bigles'
topic_sub = b'/trains/track/turnout/#'
def sub_cb(topic, msg):
print("New message on topic {}".format(topic.decode('utf-8')))
msg = msg.decode('utf-8')
print(msg)
def mqtt_connect():
client = MQTTClient(client_id, mqtt_server, keepalive=60)
client.set_callback(sub_cb)
client.connect()
print('Connected to %s MQTT Broker'%(mqtt_server))
return client
def reconnect():
print('Failed to connect to MQTT Broker. Reconnecting...')
time.sleep(5)
machine.reset()
try:
client = mqtt_connect()
except OSError as e:
reconnect()
while True:
client.subscribe(topic_sub)
time.sleep(1)
The setup inside JRMI for MQTT (edit->preferences) is as follows:
JMRI, by default, publishes with "the retain option on". When you subscribe to a topic the broker will send you the most recent (if any) retained message. This occurs even if you already had an identical subscription as per the MQTT Spec:
If a Server receives a SUBSCRIBE Packet containing a Topic Filter that is identical to an existing Subscription’s Topic Filter then it MUST completely replace that existing Subscription with a new Subscription. The Topic Filter in the new Subscription will be identical to that in the previous Subscription, although its maximum QoS value could be different. Any existing retained messages matching the Topic Filter MUST be re-sent, but the flow of publications MUST NOT be interrupted [MQTT-3.8.4-3].
In your code you are calling Subscribe in a loop:
while True:
client.subscribe(topic_sub)
time.sleep(1)
To avoid the repeated messages move the subscribe out of the loop (you only need to subscribe once!). Something like the following (simplified!) code:
client = mqtt_connect()
client.subscribe(topic_sub)
while True:
client.wait_msg() // Use client.check_msg() if you have other stuff to do
I am looking to retrieve some Solace queue stats e.g. the current messages spooled count out of the maximum limit for us to set a threshold to stop publishing more messages to the queue.
Also, to subscribe to vpn events to track message discard rates.
By the time we receive errors e.g. MaxMsgUsageExceeded/SpoolOverQuota, it will be too late.
I can't seem to find any of these on SolaceSystems.Solclient.Messaging API
https://docs.solace.com/API-Developer-Online-Ref-Documentation/net/html/7f10bcf6-19f4-beff-0768-ced843e35168.htm
Would be great if someone could help
(using C# for this)
To poll for Solace queue stats from your C# application, you could use legacy SEMP over the message bus to make a SEMP request for the details that you want. Semp (Solace Element Management Protocol) is a request/reply protocol that uses an XML schema to identify all managed objects available in a message broker. Applications can use SEMP to manage and monitor a message broker.
To allow for legacy SEMP to be used over the message bus, as opposed to the management interface, it first needs to be enabled on the Solace PubSub+ message broker at the VPN level.
To publish a SEMP request with the Solace .Net Messaging API, perform the following steps:
Create a Session.
Create the message topic. “#SEMP//SHOW”
ITopic topic = ContextFactory.Instance.CreateTopic( “#SEMP/<router name>/SHOW”);
Create a request message and set its Destination to the topic in Step 2:
IMessage requestMsg = ContextFactory.Instance.CreateMessage();
requestMsg.Destination = topic;
Set the SEMP request string as the binary attachment.
string SOLTR_VERSION = "8_4_0" //change to the message-broker's version
string SEMP_SHOW_QUEUE = "<rpc semp-version=\"soltr/" + SOLTR_VERSION +
"<show><queue><name>queueName</name><detail></detail></queue></show></rpc>";
requestMsg.BinaryAttachment = Encoding.UTF8.GetBytes(SEMP_SHOW_QUEUE);
Call the SendRequest(…) method on Session.
IMessage replyMsg;
ReturnCode rc = session.SendRequest(requestMsg, out replyMsg, timeout);
The SEMP response is returned in replyMsg.
Obtain the binary attachment data from the reply message:
replyMsg.BinaryAttachment
The binary attachment contains the SEMP reply for the command topic in the publish request.
The Solace PubSub+ message broker does raise an event when an egress message is discarded. However, it is only sent out approximately once every 60 seconds for the specified client so it is not possible to get these exact rates.
It is possible for your .NET application to subscribe to VPN-level events over the message-bus. To do this, you must first enable the Solace PubSub+ message broker to publish the events. You can then subscribe to the special topic and receive the events as messages.
The topic to subscribe to is:
#LOG/<level>/VPN/<routerName>/<eventName>/<vpnName>
The different levels can use the * wildcard. For example, if you wish to subscribe to all VPN events of all levels for the VPN apple on router QA-NY1, the topic string would be:
#LOG/*/VPN/QA-NY1/*/apple
SEMP (starting in v2) is a RESTful API for configuring, monitoring, and administering a Solace PubSub+ broker.
1-The swapper page link is SEMP V2 API
2-The Swagger metadata definitions URL is located # http://{solace-sever-url}/SEMP/v2/config/spec
3- From Visual studio, add REST API Client
4-In the configuration dialog pass swagger metadata URL (defined at step 2), for code purpose I choose SolaceSemp as input value parameter for client namespace input.
4 Once you click ok, VS will create the client along with the models under SolaceSemp namespace
5 Start using the client as per following
using SolaceSemp;
using Microsoft.Rest;
var credentials = new BasicAuthenticationCredentials();
credentials.UserName = "place user name";
credentials.Password = "place password";
using (var client = new SolaceSempClient(credentials))
{
var model = client.GetAboutApi();
}
I am trying not to get the offline messages my scenario is if client1 is offline and client2 is sending messages so client1 should not receive any old messages when he reconnects and he should receive messages sent after reconnection. I am using mqtt library(npm) on client side and mosquitto server. I have tried {clean:true} and publish and subscribe using {qos:0} and its not working.This is my code
client2:
this.client = mqtt.connect(url, {
clean: true
}
this.client.publish("mqtt/location", JSON.stringify(data1) ,{qos: 0});
Client1:
this.client = mqtt.connect(url, {
clean: true
}
this.client.subscribe("mqtt/location", {qos: 0});
this.client.on("message", function(topic, payload) {
console.log(payload);
})
Thanks
If you are setting clean session to true, then the only other explanation is that the messages you are receiving were published with the retained bit set. There is nothing you can do to stop your client receiving those messages, but you can detect them. Any published message that you receive from the broker that has the retained bit set is "old".
I want to intercept the offline message of mod_pubsub, if I send a normal message in ejabberd to offline user, I see that message in offline odbc table, if user reconnect the message arrive.
If I publish to a node, and some user are offline, I see nothing in offline message table, but if the user reconnect the item node is sent correctly so the message is saved somewhere.
Can I send the item offline to offline message odbc table? or can I intercept the offline item of mod_pubsub like for message, in fact for message from plugin I can do this:
start(_Host, _Opt) ->
inets:start(),
ejabberd_hooks:add(offline_message_hook, _Host, ?MODULE, create_message, 50).
stop (_Host) ->
ejabberd_hooks:delete(offline_message_hook, _Host, ?MODULE, create_message, 50).
this is my ejabberd.yml config for mod_pubsub:
mod_pubsub:
access_createnode: pubsub_createnode
## reduces resource comsumption, but XEP incompliant
ignore_pep_from_offline: true
## XEP compliant, but increases resource comsumption
## ignore_pep_from_offline: false
last_item_cache: false
db_type: odbc
plugins:
- "flat"
- "hometree"
- "pep" # pep requires mod_caps
As default, pubsub message type are headline. As per XMPP specifications, headline messages should not stored in offline message store.
However, there is a mod_pubsub option to change default notification type. You can for example set mod_pubsub notification_type option to normal. Normal messages are store in offline store.
I am building a simple connector component in go, with these responsibilities:
Open, keep & manage connection to an external service (i.e. run in background).
Parse incoming data into logical messages and pass these messages to business logic component.
Send logical messages from business logic to external service.
I am undecided how to design the interface of the connector in go.
Variant A) Channel for inbound, function call for outbound messages
// Listen for inbound messages.
// Inbound messages are delivered to the provided channel.
func Listen(msg chan *Message) {...}
// Deliver msg to service
func Send(msg *Message) {...}
Variant B) Channel for inbound and outbound messages
// Listen for inbound messages + send outbound messages.
// Inbound messages are delivered to the provided msgIn channel.
// To send a message, put a message into the msgOut channel.
func ListenAndSend(msgIn chan *Message, msgOut chan *Message) {...}
Variant B seems cleaner and more "go-like" to me, but I am looking for answers to:
Is there an "idiomatic" way to do this in go?
alternatively, in which cases should variant A or B be preferred?
any other notable variants for this kind of problem?
Both approaches allow for only one listener (unless you keep track of the amount of listeners, which is a somewhat fragile approach), which is a limitation. It all depends on your programmatic preferences but I'd probably go with callbacks for incoming messages and a send method:
func OnReceive(func(*Message) bool) // If callback returns false, unregister it.
func Send(*Message)
Other than that, both of your proposed models are completely valid. The second seems more "orthogonal". An advantage of using a send method is that you can make sure it never blocks, as opposed to a "bare" channel.