Spring JMS timeout expiration during consumption from WebSphere MQ queue - timeout

i have a Spring JMS based client that asynchronously listens for "triggers" on QUEUE1, they indicate that there is a message ready to be consumed on another queue, QUEUE2.
Consumption on QUEUE2 is done with JmsTemplate class, configured as follows:
<bean id="jmsTemplate" class="org.springframework.jms.core.JmsTemplate">
<constructor-arg ref="gpyConnectionFactory" />
<property name="destinationResolver" ref="jndiDestinationResolver" />
<property name="receiveTimeout" value="100" />
</bean>
Notice the little receiveTimeout. This value was already so, before taking charge of this application.
NOW, I noticed that sometimes, specifically when QUEUE2 contains a relative big message,
a call to
jmsTemplate.receiveSelectedAndConvert(destinationName, mqSelector);
retrieves a NULL object, so it is likely that timeout expires!
As far as I know, as JMS spec states (correct me if I'm wrong) timeout would expire only if no message is available on the queue.
The current scenario makes me believe that due to message size and due to the fact that for sure there is a message of that queue, the timeout expires because the consumer doesn't have enough time to read the whole big message.
Is it all that possible?
The provider is WebSphere MQ.
For sure I will set an higher timeout value.

The timeout is not processed by Spring, it is handled in the vendor JMS library...return consumer.receive(timeout).
The broker "pushes" the message to the consumer when it arrives in the queue but, yes, it will take a finite time to transfer a large message and it is certainly possible for the consumer.receive() operation to timeout (maybe repeatedly) until the message is fully transferred.
It's up to the vendor code to actually do the processing, but I would doubt any would block a receive because a message is in the process of being transferred.
So, putting a message in one queue is not a reliable way to notify that a message is available in another queue.
Consider just receiving from the real queue (or use a message-driven approach instead of the JmsTemplate).

Related

Is there a way to receive most messages out of the standard SQS Queue? [NOT FIFO]

I tried using parallel requests but the due to retention by AWS, it does not allow to poll back the same queue unless previously polled messages are deleted.
I however achieved doing the same using the FIFO, but not the standard queue.
Thanks in Advance!
:)
When you say "it does not allow to poll back the same queue unless previously polled messages are deleted", I assume you're talking about the inflight messages per queue limit, which is pretty high at 120,000:
For most standard queues (depending on queue traffic and message backlog), there can be a maximum of approximately 120,000 inflight messages (received from a queue by a consumer, but not yet deleted from the queue). If you reach this limit, Amazon SQS returns the OverLimit error message. To avoid reaching the limit, you should delete messages from the queue after they're processed. You can also increase the number of queues you use to process your messages. To request a limit increase, file a support request.
The expected use case of SQS is to have workers that receive a message, do some work, then delete the message. If you're not following this pattern, I'd strongly recommend reevaluating whether SQS is the right tool for what you're trying to do.
However, if you really have a valid use case for having more than 120K messages inflight at once, you'll need to describe your use case to AWS and get their approval to increase that limit.

Spring AMQP Reporting

We are using spring AMQP to listen to rabbitMQ for messages. I want to be able to report the metrics once we finished processing batch of messages, that means when we exhausts all the message in the queue. I m not sure how to do that in Spring AMQP. browsing spring document, it mentions advice chain to SimpleRabbitListenerContainerFactory, but that's mainly for RetryInterceptor. is there anyway allow me to report?
There is nothing in the framework to notify the listener that there are no new messages available.
You could examine the queue using rabbitadmin to see a message count but that would be expensive to do it on every message delivery.
Some ideas:
You could schedule a task to run after some period when no messages are received (and cancel/reschedule each time a new message arrives).
You could have the sending system add a marker to the "last" message so the receiver knows the batch is complete.
Instead of using the message listener container, use RabbitTemplate.receive() (or receiveAndConvert()) which, by default, returns null when there are no messages in the queue. Call them in a loop until there are no messages. When that happens, issue your report, then go into a polling loop (with a sleep) to poll for the next "batch".

Background Tasks in Spring (AMQP)

I need to handle a time-consuming and error-prone task (e.g., invoking a SOAP endpoint that will trigger the delivery of an SMS) whenever a given endpoint of my REST API is invoked, but I'd prefer not to make my users wait for that before sending a response back. Spring AMQP is already part of my stack, so I though about leveraging it to establish a "work queue" and have a number of worker processes consuming from the queue and taking care of the "work units". I have, however, the following requirements:
A work unit is guaranteed to be delivered, and delivered to exactly one worker.
Shall a work unit fail to be completed for any reason it must get placed back in the queue so that another worker can pick it up later.
Work units survive server reboots and crashes. This is mandatory because I won't be using a DB of any kind to store them.
I know RabbitMQ and Spring AMQP can be configured in such a way that ensures these three requirements, but I've only ever used it to achieve RPC so I don't know much about anything other than that. Is there any example I might follow? What are some of the pitfalls to watch out for?
While creating queues, rabbitmq gives you two options; transient or durable. Durable messages will be available until you acknowledge them. And messages won't expire if you do not give queue a ttl. For starters you can enable rabbitmq management plugin and play around a little.
But if you really want to guarantee the safety of your messages against hard resets or hardware problems, i guess you need to use a rabbitmq cluster.
Rabbitmq Clustering and you can find high availability subject on the right side of the page.
This guy explaines how to cluster
By the way i like beanstalkd too. You can make it write messages to disk and they will be safe except disk failures.

Endpoint Deactivation MQ Adapter BPEL

I am using BPEL process (AIA) to pick message from MQ (Message Queue). MQ Adapter picks message from the queue against a defined schema(nxsd). The nxsd schema has style as terminated. The issue is when an incorrect message is placed onto the queue the process continuously picks message from the queue rejects it and places it back onto the queue. This happens continuously.
Please help as this has caused a lot of server issues
I am not a MQ BPEL adapter guy. I come from MQ background.
MQ Adapter may have ability to route messages that are not understood by it to another queue known as a BACKOUT QUEUE (or BOQNAME in WebSphere MQ). Messages that are not understood by adapter (or for that matter JMS layer) are called poison messages.
There will also be another parameter typically called as a 'Threshold' (BOTHRESH in WebSphere MQ). This attribute tells the adapter when to route the message to a backout queue. For example if the attribute is set to 3, then if the same message is received 3 times (and is sent back to the same queue because the adapter does not understands this message), then when the message arrives for the fourth time, the message is automatically moved to backout queue.
My suggestion would be to look into MQ BPEL adapter documentation to see if it supports Backout queue and threshold attributes. If yes, set these attributes appropriately and try.

How to guarantee that Amazon SQS will receive a message only once?

I'm using an Amazon SQS queue to send notifications to an external system.
If the HTTP request fails when using SQS' SendMessage, I don't know whether the message has been queued or not. My default policy would be to retry posting the message to the queue, but there's a risk to post the message twice, which might not be acceptable depending on the use case.
Is there a way to have SQS refuse the message if there is a duplicate on the message body (or some kind of message metadata, such as a unique ID we could provide) so that we could retry until the message is accepted, and be confident that there won't be a duplicate if the first request had been already queued, but the response had been lost?
No, there's no such mechanism in SQS. Going further, it is also possible that a message will be delivered twice or more (at-least-once delivery semantics). So even if such a mechanism existed, you wouldn't be able to guarantee that the message isn't delivered multiple times.
See: http://docs.aws.amazon.com/AWSSimpleQueueService/latest/SQSDeveloperGuide/DistributedQueues.html
For exactly-once deliveries, you need some form of transactions (and HTTP isn't a transactional protocol) both on the sending and receiving end.
AFAIK, right now SQS does support what was asked!
Please see the "What's new" post entitled Amazon SQS Introduces FIFO Queues with Exactly-Once Processing and Lower Prices for Standard Queues
According to SQS FAQ:
FIFO queues provide exactly-once processing, which means that each message is delivered once and remains available until a consumer processes it and deletes it. Duplicates are not introduced into the queue.
There's also an AWS Blog post with a bit more insight on the subject:
These queues are designed to guarantee that messages are processed exactly once, in the order that they are sent, and without duplicates.
......
Exactly-once processing applies to both single-consumer and multiple-consumer scenarios. If you use FIFO queues in a multiple-consumer environment, you can configure your queue to make messages visible to other consumers only after the current message has been deleted or the visibility timeout expires. In this scenario, at most one consumer will actively process messages; the other consumers will be waiting until the first consumer finishes or fails.
Duplicate messages can sometimes occur when a networking issue outside of SQS prevents the message sender from learning the status of an action and causes the sender to retry the call. FIFO queues use multiple strategies to detect and eliminate duplicate messages. In addition to content-based deduplication, you can include a MessageDeduplicationId when you call SendMessage for a FIFO queue. The ID can be up to 128 characters long, and, if present, takes higher precedence than content-based deduplication.

Resources