I am using BPEL process (AIA) to pick message from MQ (Message Queue). MQ Adapter picks message from the queue against a defined schema(nxsd). The nxsd schema has style as terminated. The issue is when an incorrect message is placed onto the queue the process continuously picks message from the queue rejects it and places it back onto the queue. This happens continuously.
Please help as this has caused a lot of server issues
I am not a MQ BPEL adapter guy. I come from MQ background.
MQ Adapter may have ability to route messages that are not understood by it to another queue known as a BACKOUT QUEUE (or BOQNAME in WebSphere MQ). Messages that are not understood by adapter (or for that matter JMS layer) are called poison messages.
There will also be another parameter typically called as a 'Threshold' (BOTHRESH in WebSphere MQ). This attribute tells the adapter when to route the message to a backout queue. For example if the attribute is set to 3, then if the same message is received 3 times (and is sent back to the same queue because the adapter does not understands this message), then when the message arrives for the fourth time, the message is automatically moved to backout queue.
My suggestion would be to look into MQ BPEL adapter documentation to see if it supports Backout queue and threshold attributes. If yes, set these attributes appropriately and try.
Related
According to this article in TIBCO JMS undelivered message are moved by default to $sys.undelivered (DLQ). The article states:
Generally, a , sometimes referred to as an undelivered-message queue, is a holding queue for messages that cannot be delivered to their destination queues.
In TIBCO EMS, if a message expires or has exceeded the value specified by the maxRedelivery property on a queue, the server checks the message's JMS_TIBCO_PRESERVE_UNDELIVERED property.
If JMS_TIBCO_PRESERVE_UNDELIVERED is set to true, the server moves the message to the undelivered message queue, $sys.undelivered; If JMS_TIBCO_PRESERVE_UNDELIVERED is set to false, the server will delete the message.
This undelivered message queue is a system queue that is always present and cannot be deleted.
Is it possible to put automatically undelivered message of a queue in a dedicated DLQ instead of the $sys.undelivered queue?
Based on this documentation you should be able to create a selector to filter the messages sent to $sys.undelivered and route them to a different queue.
I have a Grails application using the grails RabbitMQ plugin to handle messages asynchronously. The queue is set to durable, the messages are persistent, and there are 20 concurrent consumers. Acknowledgement is turned on and is set to issue an ack/nack based on if the consumer returns normally or throws an exception. These consumers usually handle messages fine, but when the queue fills up very quickly (5,000 or so messages at once) some of the messages get lost.
There is logging in the consumer when the message from Rabbit is received and that logging event never occurs, so the consumer is not receiving the lost messages at all. Further, there are no exceptions that appear in the logs.
I have tried increasing the prefetch value of the consumers to 5 (from 1), but that did not solve the problem. I have checked the RabbitMQ UI and there are no messages stuck in the queue and there are no unacknowledged messages.
I am ruby on rails developer. i am using rabbitMQ in my project to processed some data as soon as the data comes in queue. i am using bunny gem a rabbitMQ client that provide interface to interact with RabbitMq.
My issue is that whenever an exceptions occurred or server stops unexpectedly while processing data from queue my message from the queue is lost.
I want to know how people deal with lost messages from the rabbitMQ queue. is there any way to get those messages back for processing.
There is no way to get the messages back when they're lost. Maybe you could try and track down some entries in RMQ's database cache - but that's just a wild guess/long shot and I don't think that it will help.
What you do need to do for the future is:
in case you are using a single server: make the queues and messages durable, and explicitly acknowledge (so switch off the auto-ACK flag) messages on consumer side only once they're processed.
in case you are using cluster of RMQ nodes (which is of course recommended exactly to avoid these situations): set up queue mirroring
Take a look at RMQ persistance and high availability.
We are using spring AMQP to listen to rabbitMQ for messages. I want to be able to report the metrics once we finished processing batch of messages, that means when we exhausts all the message in the queue. I m not sure how to do that in Spring AMQP. browsing spring document, it mentions advice chain to SimpleRabbitListenerContainerFactory, but that's mainly for RetryInterceptor. is there anyway allow me to report?
There is nothing in the framework to notify the listener that there are no new messages available.
You could examine the queue using rabbitadmin to see a message count but that would be expensive to do it on every message delivery.
Some ideas:
You could schedule a task to run after some period when no messages are received (and cancel/reschedule each time a new message arrives).
You could have the sending system add a marker to the "last" message so the receiver knows the batch is complete.
Instead of using the message listener container, use RabbitTemplate.receive() (or receiveAndConvert()) which, by default, returns null when there are no messages in the queue. Call them in a loop until there are no messages. When that happens, issue your report, then go into a polling loop (with a sleep) to poll for the next "batch".
I'm using an Amazon SQS queue to send notifications to an external system.
If the HTTP request fails when using SQS' SendMessage, I don't know whether the message has been queued or not. My default policy would be to retry posting the message to the queue, but there's a risk to post the message twice, which might not be acceptable depending on the use case.
Is there a way to have SQS refuse the message if there is a duplicate on the message body (or some kind of message metadata, such as a unique ID we could provide) so that we could retry until the message is accepted, and be confident that there won't be a duplicate if the first request had been already queued, but the response had been lost?
No, there's no such mechanism in SQS. Going further, it is also possible that a message will be delivered twice or more (at-least-once delivery semantics). So even if such a mechanism existed, you wouldn't be able to guarantee that the message isn't delivered multiple times.
See: http://docs.aws.amazon.com/AWSSimpleQueueService/latest/SQSDeveloperGuide/DistributedQueues.html
For exactly-once deliveries, you need some form of transactions (and HTTP isn't a transactional protocol) both on the sending and receiving end.
AFAIK, right now SQS does support what was asked!
Please see the "What's new" post entitled Amazon SQS Introduces FIFO Queues with Exactly-Once Processing and Lower Prices for Standard Queues
According to SQS FAQ:
FIFO queues provide exactly-once processing, which means that each message is delivered once and remains available until a consumer processes it and deletes it. Duplicates are not introduced into the queue.
There's also an AWS Blog post with a bit more insight on the subject:
These queues are designed to guarantee that messages are processed exactly once, in the order that they are sent, and without duplicates.
......
Exactly-once processing applies to both single-consumer and multiple-consumer scenarios. If you use FIFO queues in a multiple-consumer environment, you can configure your queue to make messages visible to other consumers only after the current message has been deleted or the visibility timeout expires. In this scenario, at most one consumer will actively process messages; the other consumers will be waiting until the first consumer finishes or fails.
Duplicate messages can sometimes occur when a networking issue outside of SQS prevents the message sender from learning the status of an action and causes the sender to retry the call. FIFO queues use multiple strategies to detect and eliminate duplicate messages. In addition to content-based deduplication, you can include a MessageDeduplicationId when you call SendMessage for a FIFO queue. The ID can be up to 128 characters long, and, if present, takes higher precedence than content-based deduplication.