RabbitMQ Spring AMQP sync to async conversion
I need to convert synchronous message invocation into asynchronous if the request is running too long. If there a way to achieve this by using Spring AMQP client for RabbitMQ? Thanks.
You can hand off to another thread at any time in your code and the container thread will immediately ack the message. But the logic to do the hand off has to be in your listener.
If you want to control the ack, use mode MANUAL and you'll need a SessionAwareMessageListener so you have access to the Channel to do the basicAck.
Related
We are using spring AMQP to listen to rabbitMQ for messages. I want to be able to report the metrics once we finished processing batch of messages, that means when we exhausts all the message in the queue. I m not sure how to do that in Spring AMQP. browsing spring document, it mentions advice chain to SimpleRabbitListenerContainerFactory, but that's mainly for RetryInterceptor. is there anyway allow me to report?
There is nothing in the framework to notify the listener that there are no new messages available.
You could examine the queue using rabbitadmin to see a message count but that would be expensive to do it on every message delivery.
Some ideas:
You could schedule a task to run after some period when no messages are received (and cancel/reschedule each time a new message arrives).
You could have the sending system add a marker to the "last" message so the receiver knows the batch is complete.
Instead of using the message listener container, use RabbitTemplate.receive() (or receiveAndConvert()) which, by default, returns null when there are no messages in the queue. Call them in a loop until there are no messages. When that happens, issue your report, then go into a polling loop (with a sleep) to poll for the next "batch".
I need to handle a time-consuming and error-prone task (e.g., invoking a SOAP endpoint that will trigger the delivery of an SMS) whenever a given endpoint of my REST API is invoked, but I'd prefer not to make my users wait for that before sending a response back. Spring AMQP is already part of my stack, so I though about leveraging it to establish a "work queue" and have a number of worker processes consuming from the queue and taking care of the "work units". I have, however, the following requirements:
A work unit is guaranteed to be delivered, and delivered to exactly one worker.
Shall a work unit fail to be completed for any reason it must get placed back in the queue so that another worker can pick it up later.
Work units survive server reboots and crashes. This is mandatory because I won't be using a DB of any kind to store them.
I know RabbitMQ and Spring AMQP can be configured in such a way that ensures these three requirements, but I've only ever used it to achieve RPC so I don't know much about anything other than that. Is there any example I might follow? What are some of the pitfalls to watch out for?
While creating queues, rabbitmq gives you two options; transient or durable. Durable messages will be available until you acknowledge them. And messages won't expire if you do not give queue a ttl. For starters you can enable rabbitmq management plugin and play around a little.
But if you really want to guarantee the safety of your messages against hard resets or hardware problems, i guess you need to use a rabbitmq cluster.
Rabbitmq Clustering and you can find high availability subject on the right side of the page.
This guy explaines how to cluster
By the way i like beanstalkd too. You can make it write messages to disk and they will be safe except disk failures.
I would like to know the advantages and disadvantages of using SimpleMessageListenerContainer over receiving a message manually using Spring AMQP. Another question is when we create SimpleMessageListenerContainer setting a queue, does the rabbitmq calls the listeneradaptor or does SimpleMessageListenerContainer keeps polling the queue to check for messages and calls the registered adaptor when their is message.
It depends on your requirements; the listener container gives you an async (message-driven) approach. Otherwise, if you use the RabbitTemplatereceive methods, you are polling the queue. The container does not poll the queue, the broker pushes messages to the container according to the prefetch settings (default 1) - if using ackmode AUTO.
I'm wondering : is it a non-sense to put a timeout on a synchronous web-service call ? I mean, if there is a risk that the server does not respond, I should use an asynchronous call instead, doesn't it ?
(I'm using Jersey)
Thanks !
I'd always advise to set a connection and read timeout on any and all outbound network requests as indefinitely waiting for an answer could eventually consume all your threads and make your app server unresponsive.
In my experience it's no unusual at all to have partners' WS requests not respond withing 60s (which is quite generous).
Handling read timeouts can be tricky for write operations though as you can't tell whether the other system eventually recorded the request or not. In such situation, your partner hopefully provides an idempotent API allowing you to retry at a later time without risk of duplicate execution. Otherwise it may require manual communication with your partner.
I am struggling to work out how I can communicate between rabbitmq and em-websocet.
I want to place a message from a ruby on rails web page on a queue and have the queue handler process the message even if the browser is closed down. If the browser stays open, I want the results of the queue handler to pass json back to the browser.
I did find this but the github page says it is depreceated. Can anyone point me in the right direction?
From what I can gather, you've got a RabbitMQ queue, a way to add items to that queue, something to process items that get added to that queue, and you basically want to notify the browser of progress on that queue.
There are two main ways that you could approach this:
As the final action of the queue processor, publish the item/message via a messaging bus to an instance of em-websocket that's listening on that message bus.
If you can add features to RabbitMQ, then you could do the publish from within RabbitMQ, as a post-processed hook or something like that. (note, I don't know enough about RabbitMQ to say you can definitely do this).
Alternatively with #1, you could use Pusher.com or similar service to offload the handling of the WebSocket connections. Then, from within your queue processor, you would do the publish call to that services' API.
In the case of using Pusher, if you publish to a channel/socket that no longer exists (has any connections), then the message would just get discarded.
Hopefully this helps. Let me know if you want any help in setting up a basic em-websocket server.