With a MQTT shared subscription, the message on the subscirbed topic would only be sent to one of the subscribing clients. Then, how the other clients of the group receive the message as they also subscribe to the same topic?
With a MQTT shared subscription, the message on the subscribed topic would only be sent to one of the subscribing clients.
Correct. With a normal (non‑shared) subscription the messages are sent to ALL subscribers; with shared subscriptions they are sent to ONE subscriber.
Prior to the introduction of shared subscriptions it was difficult to handle situations where you want multiple servers (for fault tolerance, load balancing etc) but only want to process each message once. Shared subscriptions provide a simple way of accomplishing this.
Then, how the other clients of the group receive the message as they also subscribe to the same topic?
If they are subscribing to the same shared subscription (with the same ShareName) then only one will receive the message; this is by design. If you want them all to receive the message then don't use a shared subscription. Alternatively you can establish multiple subscriptions (so all subscribers receive the message but only one processes it - but note the "could") as per the spec:
If a Client has a Shared Subscription and a Non‑shared Subscription and a message matches both of them, the Client will receive a copy of the message by virtue of it having the Non‑shared Subscription. A second copy of the message will be delivered to one of the subscribers to the Shared Subscription, and this could result in a second copy being sent to this Client.
There is an interesting bug in Java Paho (1.2.5) client that prevents working with shared topics that contains wildcards (#, +) https://github.com/eclipse/paho.mqtt.java/issues/827
Long story short, this will not work:
mqttClient.subscribe("$shared/group/some_topic/#", 1, (topic, message) -> System.out.println(topic));
instead it's required to use callbacks:
mqttClient.subscribe("$shared/group/some_topic/#", 1);
mqttClient.setCallback(new MqttCallback() {
#Override
public void connectionLost(final Throwable cause) {
}
#Override
public void messageArrived(final String topic, final MqttMessage message) throws Exception {
System.out.println(topic);
}
#Override
public void deliveryComplete(final IMqttDeliveryToken token) {
}
});
Related
I am trying to create a subscriber for my durable topic endpoint in solace via .NET APIs.
I have the below method where in I am trying to create a flow for my durable topic endpoint. I don't understand what is the need for ISubscription instance in the Session.CreateFlow method. (https://docs.solace.com/API-Developer-Online-Ref-Documentation/net/html/a548a98a-9134-c167-2517-192a26ceed77.htm)
How do I create an instance of ISubscription and what should it be?
public void Start()
{
//Create a instance of a durable topic endpoint
topic = ContextFactory.Instance.CreateDurableTopicEndpointEx(topicName);
FlowProperties flowProps = new FlowProperties();
flowProps.FlowStartState = false;
if (selector != null)
{
flowProps.Selector = selector;
}
flowProps.AckMode = ackMode == AckMode.ClientAcknowledge ? MessageAckMode.ClientAck : MessageAckMode.AutoAck;
if (windowSize.HasValue)
{
flowProps.WindowSize = windowSize.Value;
}
flow = session.CreateFlow(flowProps, topic, null, HandleFlowMessageEvent, HandleFlowEvent);
flow.Start();
}
I am currently passing it as null and I get the error
subscription must be non-null when endpoint is of type ITopicEndpoint
Secondly, I have a message handler for my flow event and session event, so when a message comes through which handler would it invoke. I would expect that I handle the message in the FlowMessageHandler as I am connecting to a durable topic endpoint. Please can someone shed more light on this?
//session message event handler
session = context.CreateSession(sessionProperties, HandleSessionMessageEvent, HandleSessionEvent);
//Flow message event handler
flow = session.CreateFlow(flowProps, topic, null, HandleFlowMessageEvent, HandleFlowEvent);
For a durable topic endpoint, if you have multiple subscribers then each would have to register with a unique subscription name (you can give any name as you like) so that the broker can maintain a durable connection with the specific subscriber. Even if the subscriber is down for a while and reconnects, it will not lose out on messages in the interim. The broker will push the messages to the subscriber. You can create a subscriber via the Solace Admin client or via code. Solace Admin -> Select the VPN -> Choose Endpoint tab -> select Durable Topic Endpoint -> click the + sign to create a subscriber -> ensure that consume is chosen for 'All other Permissions'.
The messages are sent to the session handler first and then to the flow handler. So you can access the messages from both the handlers.
Device side mqtt publishes data to mosquitto broker installed on Raspberry Pi to a topic.
how to subscribe the data from the mosquitto broker in 2 ways.
Normal google example search gives me code samples. But what i want is 2 ways i thought one is MQTT API to subscribe providing topic which listens every time checking whether data has arrived or not which reduces the CPU performance and speed.
So, Other is if a message has arrived in the topic then it should call back saying message arrived instead of me going and looking into topic everytime checking is cumbersome. The second way increases CPU performance.
MemoryPersistence memoryPersistence = new MemoryPersistence();
MqttConnectOptions conOpt = new MqttConnectOptions();
conOpt.setUserName("mqttuser");
conOpt.setPassword(new String("mqttpassword").toCharArray());
conOpt.setCleanSession(true);
try {
MqttAsyncClient mqttAsyncClient = new MqttAsyncClient("tcp://localhost:1883", "1883", memoryPersistence);
mqttAsyncClient.setCallback(new MqttConnectionCallback());
if (!mqttAsyncClient.isConnected()) {
IMqttToken token = mqttAsyncClient.connect(conOpt);
logger.info();
System.out.println("Connected");
}
} catch (MqttException e) {
e.printStackTrace();
System.out.println("Unable to set up client: "+e.toString());
System.exit(1);
}
this is my listener class overridden method
#Override
public void messageArrived(String topic, MqttMessage msgData) throws Exception {
String time = new Timestamp(System.currentTimeMillis()).toString();
System.out.println("Time:\t" +time +
" Topic:\t" + topic +
" Message:\t" + new String(msgData.getPayload()) +
" QoS:\t" + msgData.getQos());
}
Need some clearance from coding side how this is accompolished in asynchronous ways.
Please correct if my understanding is wrong, specifically on how listener listenes on topicname and how the binding is done.
You need to use the subscribe() method on the mqttAsyncClient object (after connection has completed)
You can call this multiple times to subscribe to multiple topics.
Messages will be delivered to the messageArrived() callback and you can use the topic argument to determine which subscription the message arrived from.
My application only listens to a certain queue, the producer is the 3rd party application. I receive the messages but sometimes based on some logic I need to send fail message to the producer so that the message is resend to my listener again until I decide to consume it and acknowledge it. My current implementation of this process is just throwing some custom exception. But this is not a clean solution, therefore can any one help me to send FAIL to producer without throwing exception.
My JMS Listener Factory settings:
#Bean
public DefaultJmsListenerContainerFactory jmsListenerContainerFactoryForQexpress(SQSErrorHandler errorHandler) {
SQSConnectionFactory connectionFactory = SQSConnectionFactory.builder()
.withRegion(RegionUtils.getRegion(StaticSystemConstants.getQexpressSqsRegion()))
.withAWSCredentialsProvider(new ClasspathPropertiesFileCredentialsProvider(StaticSystemConstants.getQexpressSqsCredentials()))
.build();
DefaultJmsListenerContainerFactory factory = new DefaultJmsListenerContainerFactory();
factory.setConnectionFactory(connectionFactory);
factory.setDestinationResolver(new DynamicDestinationResolver());
factory.setConcurrency("3-10");
factory.setSessionAcknowledgeMode(Session.CLIENT_ACKNOWLEDGE);
factory.setErrorHandler(errorHandler);
return factory;
}
My Listener Settings:
#JmsListener(destination = StaticSystemConstants.QUEXPRESS_ORDER_STATUS_QUEUE, containerFactory = "jmsListenerContainerFactoryForQexpress")
public void receiveQExpressOrderStatusQueue(String text) throws JSONException {
LOG.debug("Consumed QExpress status {}", text);
//here i need to decide either acknowlege or fail
...
if (success) {
updateStatus();
} else {
//todo I need to replace this with explicit FAIL message
throw new CustomException("Not right time to update status");
}
}
Please, share your experience on this. Thank you!
SQS -- internally speaking -- is fully asynchronous and completely decouples the producer from the consumer.
Once the producer successfully hands off a message to SQS and receives the message-id in response, the producer only knows that SQS has received and committed the message to its internal storage and that the message will be delivered to a consumer at least once.¹ There is no further feedback to the producer.
A consumer can "snooze" a message for later retry by simply not deleting it (see setSessionAcknowledgeMode docs) or by actively resetting the visibility timeout on the message instead of deleting it, which triggers SQS to leave the message in the in flight status until the timer expires, at which point it will again deliver the message for the consumer to retry.
Note, too, that a single SQS queue can have multiple producers and/or multiple consumers, as long as all the producers ask for and consumers provide identical services, but there is no intrinsic concept of which consumer or which producer. There is no consumer-to-producer backwards communication channel, and no mechanism for a producer to inquire about the status of an earlier message -- the design assumption is that once SQS has received a message, it will be delivered,² so no such mechanism should be needed.
¹at least once. Unless the queue is a FIFO queue, SQS will typically deliver the message exactly once, but there is not an absolute guarantee that the message will not be delivered more than once. Because SQS is a massive, distributed system that stores redundant copies of messages, it is possible in some edge case conditions for messages to be delivered more than once. FIFO queues avoid this possibility by leveraging stronger internal consistency guarantees, at a cost of reduced throughput of 300 TPS.
²it will be delivered assuming of course that you actually have a consumer running. SQS does not block the producer, and will allow you to enqueue an unbounded number of messages waiting for a consumer to arrive. It accepts messages from producers regardless of whether there are currently any consumers listening. The messages are held until consumed or until the MessageRetentionPeriod (default 4 days, max 14 days) timer expires for each message, whichever comes first.
When didReceiveStatus is called after subscribing to a channel, We are not able to retrieve the channel(s) that was just subscribed.
PNSubscribeStatus.data.subscribedChannel or PNSubscribeStatus.data.actualChannel are always null and PNSubscribeStatus.subscribedChannels gives all currently subscribed channels and not the ones that triggered the didReceiveStatus callback.
What are we doing wrong here ?
In SDK 4.0, didReceiveStatus returns a PNStatus, which according to the class documentation doesn't contain that extra information unless there's an error condition. For our application, we use that handler to monitor connection status to the PubNub server.
PubNub Message Received Channel Name in iOS
You should be able to get the channel that you received the message on but getting it depends on whether you are subscribed to the channel or to a channel group that contains the channel. This is sample code from the PubNub Objective-C for iOS SDK subscribe API Reference:
- (void)client:(PubNub *)client didReceiveMessage:(PNMessageResult *)message {
// Handle new message stored in message.data.message
if (message.data.actualChannel) {
// Message has been received on channel group stored in
// message.data.subscribedChannel
}
else {
// Message has been received on channel stored in
// message.data.subscribedChannel
}
NSLog(#"Received message: %# on channel %# at %#", message.data.message,
message.data.subscribedChannel, message.data.timetoken);
}
If you need other channels that the client is subscribed to, you can call the where-now API.
If you need to be more dynamic about what the reply-to channel should be, then just include that channel name in the message when it is published, assuming the publisher has prior knowledge about which channel this is. Or you can do a just in time lookup on your server as to which channel to reply to.
Here is PubNub support answer on this :
Actually status.data.subscribedChannel and status.data.actualChannel
dedicated to presence events and messages receiving callbacks where
information about sources of event is important.
In -client:didReceiveStatus: client doesn’t give information about
particular channel on which client has been subscribed. If client will
start track this information, there is no guarantee what it will
return expected value (as developer expect some channels to be there).
In previous version (3.x) all this information has been tracked, but
because it can be modified at any moment – result sometimes was
unpredictable.
Subscribe can be done in sequence of methods (one after another) like:
subscribe A1, subscribe C1, subscribe B1 and B2, unsubscribe C1 and B1
– this as result will end up with single call of
-client:didReceiveStatus: with resulting set of channels.
It always best practice just to check whether your channels is in
s_tatus.subscribedChannels_.
My comments:
The point of having asynchronous process is exactly not to think this as sequence of methods... We can not have the guaranty that subscriptions are done in the exact same order as the subscription request unless we block other subscription request until the previous one is done.
I am building a simple connector component in go, with these responsibilities:
Open, keep & manage connection to an external service (i.e. run in background).
Parse incoming data into logical messages and pass these messages to business logic component.
Send logical messages from business logic to external service.
I am undecided how to design the interface of the connector in go.
Variant A) Channel for inbound, function call for outbound messages
// Listen for inbound messages.
// Inbound messages are delivered to the provided channel.
func Listen(msg chan *Message) {...}
// Deliver msg to service
func Send(msg *Message) {...}
Variant B) Channel for inbound and outbound messages
// Listen for inbound messages + send outbound messages.
// Inbound messages are delivered to the provided msgIn channel.
// To send a message, put a message into the msgOut channel.
func ListenAndSend(msgIn chan *Message, msgOut chan *Message) {...}
Variant B seems cleaner and more "go-like" to me, but I am looking for answers to:
Is there an "idiomatic" way to do this in go?
alternatively, in which cases should variant A or B be preferred?
any other notable variants for this kind of problem?
Both approaches allow for only one listener (unless you keep track of the amount of listeners, which is a somewhat fragile approach), which is a limitation. It all depends on your programmatic preferences but I'd probably go with callbacks for incoming messages and a send method:
func OnReceive(func(*Message) bool) // If callback returns false, unregister it.
func Send(*Message)
Other than that, both of your proposed models are completely valid. The second seems more "orthogonal". An advantage of using a send method is that you can make sure it never blocks, as opposed to a "bare" channel.