I have a scenario where mobile app calls rest API hosted by my application. With in this process, I need to send message to downstream system over MQTT and wait until I get the response for that message. And then I have reply back to mobile app.
The challenge here is, messaging over MQTT is asynchronous. So the message which I receive back will be in different thread (some listener class, listening on messageArrived()). How to get back to calling http thread?
Do we have synchronous communication supported by Paho library.? Something like I send a message, open some topic and wait on it till some message is received or timeout?
MQTT by it's very nature is asynchronous, as are all Pub/Sub implementations. There is no concept of a reply to a message at the protocol level, you have no way of knowing if you will EVER get a response (or you may get many) to a published message as you can't know if there is even a subscriber to the topic you publish on.
It is possible to build a system that will work this way, but you need to maintain a state machine of all in flight requests, implement a sensible timeout policy and work out what to do if you get more than one response.
You have not mentioned which of the different Paho libraries you are using, but I'm guessing Java from the method names, but without knowing what HTTP framework you are using and a host of other factors I'm not going to suggest a solution, especially as it will involve a lot of polling and synchronisation.
Is there any reason why the mobile application can't publish and subscribe to MQTT topics directly? This would remove the need for this.
Related
I implemented a MQTT message broker using mosquitto on my network. I have one web app publishing things to the broker and several servers that subscribed the same topic. So i have a redundancy scenario.
My question is, using mosquitto alone, is there any way to configure it to publish data only on the first subscriber? Otherwise, all of them will do the same thing.
I don't think that is possible.
But you can do this.
Have the first subscriber program respond with an ack on the channel as soon as it gets the message, and have the redundancy program look for the ack for a small time after the initial message.
IF the ack is received the redundancy should not do anything.
So if the first subscriber gets and uses the message, the others wont do anything even if they get the message.
No this is not possible with mosquitto at the moment (without communication between the 2 subscribers as described in the other answer).
For the new release of the MQTT spec (v5)* there is a new mode called "Shared Subscriptions". This allow s multiple clients to subscribe to a single topic and messages will be delivered by round robin to each client. This is more for load balancing rather than master/slave fail over.
*There are some brokers (HiveMQ, IBM MessageSight) that already support some version of Shared Subscriptions at MQTT v3.1.1, but they implement it in slightly different ways (different topic prefixes) so they are not cross compatible.
we are implementing (or more reimplementing) a distributed software system. What we have are different processes (possibly running on different computers) that should communicate with each other (let's call these clients). We don't want them to directly communicate with each other, but instead use some kind of message broker.
Since we like to avoid implementing the message broker ourselves we would like to use an existing implementation. But we don't find a protocol or system that fully fulfilles our requirements.
MQTT with its publish-subscribe-mechanism seems nice and could even be used for point-to-point communication (where some specific topics are only subscribed by certain clients).
But it is (like JSM, STOMP, etc.) asynchronous. The sender sends a message into the broker and doesn't know whether it is ever delivered to it's recipient. We want that the sender gets informed about a successful delivery or an elapsed timeout (when no one is receiving the message).
Is there some protocol/implementation available that provides such synchronous messaging functionality?
(It would be nice however if asynchronous delivery would be possible, too)
The messaging by default is ( usually ) asynchronous .
You can considerer RabbitMQ, it contains the following features:
Publisher-confirms (in asynchronous way):
http://www.rabbitmq.com/blog/2011/02/10/introducing-publisher-confirms/
Transaction Commit:
https://www.rabbitmq.com/semantics.html
Messages TTL (to handle time out)
https://www.rabbitmq.com/ttl.html
With this features you can handle the time-out situations and the successful delivery.
If this is not enough you can use the RPC:
https://www.rabbitmq.com/tutorials/tutorial-six-java.html
Let me know if you need more information.
I am using TIdCmdTCPClient and TIdCmdTCPServer. Suddenly I find that I might like to have bi-directional communication.
What would be best? Should I possibly use some other components? If so, which? Or should I kludge and have the 'client' poll the 'server' to ask if it wishes to communciate anything?
This is a very small system. Two clients and ten servers, with a burst of one tarnscation every 30 to 60 seconds for a few minutes once a day, so overhead for polling is inconsequential.
I'm just woder if there is a 'correct' way.
Update: this really is an incredibly simple system. Very little traffic and all of it simple. All transmissions are an indication of even type an an optional single parameter.
<event type> [ <parameter>] e.g. "HERE_IS_SOME_DATA 42"
This can be sent in both directions, hover here is no "reply" as such. Just fire off a message (and hope that it got there)? Receive an Ack with no data? Non-catching of an exception indicates that message was successfully sent?)
Would it be possible (would it be overkill) to use two TIdCmdTCPServer?
Both TIdCmdTCPClient and TIdCmdTCPServer continuously poll their socket endpoints for inbound data during the lifetime of the connection. You do not have to do anything extra for that. So, as soon as a TIdCmdTCPClient connects to the TIdCmdTCPServer, both components will initially be in a reading state until one of them sends a command to the other.
Now, there is a problem with doing that - as soon as either component sends that first command, the receiving component will interpret it as a command and send back a reply, which the other component will interpret as a command and send back a reply, which will be interpretted as a command and send back a reply, and so on, causing an endless cycle of replies back and forth. For that reason, it is not wise to use TIdCmdTCPClient and TIdCmdTCPServer together. You should either use TIdTCPClient with TIdCmdTCPServer, or use TIdCmdTCPClient with TIdTCPServer. Depending on what exactly your protocol looks like, you may have to forgo using TIdCmdTCPClient and TIdCmdTCPServer altogether and just use TIdTCPClient with TIdTCPServer so you have more control over reading and writing on both ends. It is hard to answer with actual code without first knowing what the communication protocol should look like.
A single TCP socket connection can be used in two directions. The server can send data asynchronously to the client at any time. It is up to the client however to read the socket, for asynchronous processing this is done in a listener thread which reads from the socket and synchronizes incoming data operations with the main worker thread.
An example use case in the Indy components is the Telnet client component (TIdTelnet) which has a receive thread listening for server messages.
But you also asked about the 'correct' way - and then the answer depends on other factors such as network stability, guaranteed delivery and how to handle temporary server outages. In enterprise environments, one central messaging hub is preferred in many use cases, so that all parties connect only to this central server which is only responsible for reliable message delivery, and keeps messages until the recipient is available.
You can download the INDY 10 TCP server demo sample code here.
Based on this answer here, I need to put emails in a queue and have a background task run and send them. How do I do this with an architecture that is of ASP.NET-MVC and WCF?
How do I build a queue (sql server)?
How do I build a background task?
You can skin this cat many different ways. The key being that the actual sending of the emails is asynchronous to the queuing of the email.
Queue messages via WCF Service using MSMQ binding via this series of blog posts, which assumes IIS 7: MSMQ, WCF, and IIS: Getting Them to Play Nice.
Queue messages to MSMQ. MSMQ is a nice (sometimes underutilized) queue service built into Windows. You'll write a Windows service to receive messages from this queue. If you have IIS 7, then check out Death to Windows Services, Long Live AppFabric. MSMQ is a breeze, but has some quirky constraints (4MB message size and availability)
Queue messages to a 'sql queue'. Create a table to hold basic queued message information and then stored procedures to wrap the queue semantics (e.g. you don't want multiple consumers to receive the same message). Not difficult, but a little time consuming to get right.
Queue messages to Service Broker (or even MSMQ) and write a Windows service that receives messages from the Service Broker Queue. Service Broker handles the queueing semantics (competing consumers) for you. The downside is that its a pain in the ass to administer.
HTH,
Z
I think your solution is independant of the fact you're using MVC.
The way I've implemented this in the past is to persist the fact you need to sent an e-mail into the database and then process this using a Windows Service.
Another way to do this would be to utilize MSMQ as your storage medium. In general, MSMQ shouldn't be used to "store" data, only as a message transport mechanism, but it's certainly an option in this case.
In terms of developing a "queue", if the e-mails need ordered delivery for some reason, simply having a "RequestedDTTM" column in your database table would allow you to send them in the order they were requested.
Lastly, I would consider implementing a simply multi-threaded e-mail sender to maximize performance. Using the TPL in .NET 4.0 would make this pretty easy. Alternatively, you could use something like the SmartThreadPool library (available at codeplex.com) to manager your e-mail sender threads.
As was mentioned in the other answer you linked to, your UI shouldn't be doing this e-mail sending.
I have a classic client/server (fat client and database) program written in Delphi 2006. When certain conditions are met in the client, I need to notify all the other clients very quickly. Up until now this has been done using UDP broadcasts, but this is no longer viable as clients now connect from outside the LAN and the UDP broadcast is limited to the local network.
I'm aware of the Indy libraries but am not really sure of which components to use and how to structure it. I'm guessing I'll need to have a server that the clients connect to which will receive and distribute the messages...? Any samples out there to get me started?
Are there any other component sets or technologies I should look at instead/as well?
The simple answer is that the standard protocols available in Delphi (and other tools) don't allow for notification in reverse. I looked into this for a project where I wanted to use SOAP. They all assume client asks server, server responds and that's it.
For me, the solution was the RemObjects SDK. This allows you to send notifications to clients, and the notification can have any data you like (just like the client to server). Myself I use the SuperTCP connection, but it works with others too. It can still offer a SOAP interface for clients that must use it, but for where you have control of both client and server it works extremely well.
There are a few really easy ways to do this with Delphi, although I am sure the RemObjects SDK works really well too.
Have a central server that has a * TIdTCPServer listening* on it. Then each client has a TIdTCPClient on it. They connect to the server and block on a read waiting for the server to write. Once the server receives a notification via a listening socket it broadcasts to each of the waiting clients. This is pretty much immediate notification of all the clients.
Have a central server that has a TIdTCPServer listening on it. Then each client has a TIdTCPClient on it. Those clients can "ping" the server to ask for updates at a regular interval (use a session token to maintain state). The frequency of the interval determines how quick the notification will be. When once one of the clients needs to notify the others, it just notifies the server. The server then uses a message queue to make a list of all active client sessions and adds a notification for each. Then the next time each of the clients connects it gives it the notification and remove it from the queue.
Maintain a session table in the database where each client updates regularly that they have an active session, and removes itself when it disconnects. You will need a maintenance process that removes dead sessions. Then you have a message queue table that a client can write an update to with one row for each current active session. Then the other clients can regularly ping that table to see if there are any pending notifications for its session, if there are it can read them, act on them and then remove them.
Some sort of peer to peer approach were the clients are aware of each other through information in the database and then they connect directly to each other and notify or ask for notifications (depending on firewall and NAT configurations). A little more complex, but possible.
Obviously the choice of implementation will depend on your setup and needs. Tunning will be necessary to achieve the best results.
The components you need for this are the TIdTCPServer (listener) and TIdTCPClient (sender). Both of which are in the Indy libraries in Delphi.
ICS components from http://www.overbyte.be are great.
a.) Better compatibility than Indy
b.) PostCard ware
Good examples and support. Use TClientSocket and TServerSocket
FirebirdSQL project use the concept of notifications as being server-client connections that send a string to the client. For this, the db server uses an other port. And require the client to register it's interesting of receiving a certain type of notification through an API call.
You could use the same idea.
RabbitMQ should fit your bill. The server is free and ready to use. You just need a client side to connect, push/send out message and get/pull notified message
Server: http://www.rabbitmq.com/download.html
Do a google for client or implement yourself
Cheers
You should be able to use Multicast UDP for the same purpose. The only difference will be to join the multicast group from every client.
http://en.wikipedia.org/wiki/IP_Multicast
http://en.wikipedia.org/wiki/Internet_Group_Management_Protocol
Edit: Just to clarify, multicast let you join a given "group" associated to a multicast ip address. Any packet sent to that address will reach every client who has join the group
You can watch weonlydo wodVPN component which permit you to create a robust UDP hole punching and gain a port-forwading or a normal VPN (with a fornished network adapter) so you can connect two PC behind a NAT.
I'm using this control for our communication program and works very fine.