Spring JMS and AWS SQS FIFO integration - amazon-sqs

Question

I am facing a weird issue while integrating SQS FIFO with Spring JMS. This happens while listening to the message. I have posted only 1 message in the queue and the application keeps on poling the same message again and again. I can see that in the logs. But it never enters the block where the message is read and further operations are done on it.
The visibility time out of the SQS queue is 30 seconds.
#Component
public class SQSListener {
private Logger LOGGER = Logger.getLogger(SQSListener.class);
#Autowired
AmazonSQSAsync amazonSQS;
#JmsListener(destination = "<queue-name>", containerFactory="jmsListenerContainerFactory")
public void onMessage(Message message) {
try {
LOGGER.info("*****Received****" + ((TextMessage)message).getText());
} catch (JMSException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
}
}
Configuration Class:
#Configuration
#ComponentScan("com.ar")
#EnableJms
public class SQSListenerConfig {
private Logger LOGGER = Logger.getLogger(SQSListenerConfig.class);
#Value("${queue.endpoint}")
private String endpoint;
#Value("${queue.name}")
private String queueName;
#Value("${aws.access-key}")
private String accessKey;
#Value("${aws.secret-key}")
private String secretKey;
#Autowired
private SQSListener sqsListener;
private final AWSCredentialsProvider awsCredentialsProvider = new AWSCredentialsProvider() {
#Override
public AWSCredentials getCredentials() {
return new BasicAWSCredentials(accessKey, secretKey);
}
#Override
public void refresh() {
}
};
SQSConnectionFactory connectionFactory = SQSConnectionFactory.builder()
.withAWSCredentialsProvider(awsCredentialsProvider)
.withEndpoint(endpoint)
.withNumberOfMessagesToPrefetch(10).build();
#Bean
public DefaultJmsListenerContainerFactory jmsListenerContainerFactory() {
DefaultJmsListenerContainerFactory factory = new DefaultJmsListenerContainerFactory();
factory.setConnectionFactory(this.connectionFactory);
factory.setDestinationResolver(new DynamicDestinationResolver());
factory.setConcurrency("1-5");
//factory.setMaxMessagesPerTask(10);
//factory.setReceiveTimeout(1000000000L);
factory.setReceiveTimeout(30L);
factory.setSessionAcknowledgeMode(Session.CLIENT_ACKNOWLEDGE);
//factory.setMessageConverter(messageConverter());
return factory;
}
#Bean
public JmsTemplate createJMSTemplate() {
JmsTemplate jmsTemplate = new JmsTemplate(connectionFactory);
jmsTemplate.setDefaultDestinationName(queueName);
jmsTemplate.setDeliveryPersistent(false);
return jmsTemplate;
}
/*#Bean
public DefaultMessageListenerContainer jmsListenerContainer() {
DefaultMessageListenerContainer dmlc = new DefaultMessageListenerContainer();
dmlc.setConnectionFactory(connectionFactory);
dmlc.setDestinationName(queueName);
dmlc.setMessageListener(sqsListener);
return dmlc;
}
*/
#Bean(destroyMethod = "shutdown")
public AmazonSQSAsync amazonSQS() throws Exception {
AmazonSQSAsyncClientBuilder builder = AmazonSQSAsyncClientBuilder.standard();
builder.withCredentials(awsCredentialsProvider);
builder.withRegion(Regions.US_EAST_1);
return builder.build();
}
#Bean
public QueueMessagingTemplate queueMessagingTemplate(AmazonSQSAsync awsSqsClient) throws Exception {
return new QueueMessagingTemplate(awsSqsClient);
}
}
pom.xml
<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-aws-messaging</artifactId>
<version>1.2.2.RELEASE</version>
</dependency>
<dependency>
<groupId>com.amazonaws</groupId>
<artifactId>aws-java-sdk</artifactId>
<version>1.11.292</version>
</dependency>
<dependency>
<groupId>com.amazonaws</groupId>
<artifactId>amazon-sqs-java-messaging-lib</artifactId>
<version>1.0.4</version>
</dependency>
Here is the log. This message is being polled continuously but it never enters the listener method:
amz-sdk-invocation-id;amz-sdk-retry;host;user-agent;x-amz-date
0f481022d919c318ddbb4110196f30d58d35c7fca83b32af5f4db3d9c378e788"
2018-04-18 21:35:35.842 DEBUG 13324 --- [refetchThread-1] com.amazonaws.auth.AWS4Signer : AWS4 String to Sign: '"AWS4-HMAC-SHA256
20180419T013535Z
20180419/us-east-1/sqs/aws4_request
7daaadeb9fe5618925b5a0ab5ab17de7a0fedf77d45fd56984419e530fac3ef7"
2018-04-18 21:35:35.842 DEBUG 13324 --- [refetchThread-1] o.a.h.client.protocol.RequestAddCookies : CookieSpec selected: default
2018-04-18 21:35:35.842 DEBUG 13324 --- [refetchThread-1] o.a.h.client.protocol.RequestAuthCache : Auth cache not set in the context
2018-04-18 21:35:35.842 DEBUG 13324 --- [refetchThread-1] h.i.c.PoolingHttpClientConnectionManager : Connection request: [route: {s}->https://sqs.us-east-1.amazonaws.com:443][total kept alive: 1; route allocated: 1 of 50; total allocated: 1 of 50]
2018-04-18 21:35:35.842 DEBUG 13324 --- [refetchThread-1] h.i.c.PoolingHttpClientConnectionManager : Connection leased: [id: 1][route: {s}->https://sqs.us-east-1.amazonaws.com:443][total kept alive: 0; route allocated: 1 of 50; total allocated: 1 of 50]
2018-04-18 21:35:35.842 DEBUG 13324 --- [refetchThread-1] h.i.c.DefaultManagedHttpClientConnection : http-outgoing-1: set socket timeout to 50000
2018-04-18 21:35:35.842 DEBUG 13324 --- [refetchThread-1] o.a.http.impl.execchain.MainClientExec : Executing request POST /868651433674/aws-nonprod-alm-dev-sqs.fifo HTTP/1.1
2018-04-18 21:35:35.842 DEBUG 13324 --- [refetchThread-1] o.a.http.impl.execchain.MainClientExec : Proxy auth state: UNCHALLENGED
2018-04-18 21:35:35.842 DEBUG 13324 --- [refetchThread-1] org.apache.http.headers : http-outgoing-1 >> POST /868651433674/aws-nonprod-alm-dev-sqs.fifo HTTP/1.1
2018-04-18 21:35:35.842 DEBUG 13324 --- [refetchThread-1] org.apache.http.headers : http-outgoing-1 >> Host: sqs.us-east-1.amazonaws.com
2018-04-18 21:35:35.842 DEBUG 13324 --- [refetchThread-1] org.apache.http.headers : http-outgoing-1 >> Authorization: AWS4-HMAC-SHA256 Credential=AKIAJTHWNVZAZ2T7BW5A/ 20180419/us-east-1/sqs/aws4_request, SignedHeaders=amz-sdk-invocation-id;amz-sdk-retry;host;user-agent;x-amz-date, Signature=74503414f0a291dc4e1fc39c004748fea7565ed8d08d4d9ca810d93d8a550e9e
2018-04-18 21:35:35.842 DEBUG 13324 --- [refetchThread-1] org.apache.http.headers : http-outgoing-1 >> X-Amz-Date: 20180419T013535Z
2018-04-18 21:35:35.842 DEBUG 13324 --- [refetchThread-1] org.apache.http.headers : http-outgoing-1 >> User-Agent: aws-sdk-java/1.11.292 Windows_10/10.0 Java_HotSpot(TM)_64-Bit_Server_VM/25.171-b11 java/1.8.0_171 /SQS Java Messaging Client v1.0
2018-04-18 21:35:35.842 DEBUG 13324 --- [refetchThread-1] org.apache.http.headers : http-outgoing-1 >> amz-sdk-invocation-id: 2dd018d2-da6b-010a-6d41-44ac3b676ed4
2018-04-18 21:35:35.842 DEBUG 13324 --- [refetchThread-1] org.apache.http.headers : http-outgoing-1 >> amz-sdk-retry: 0/0/500
2018-04-18 21:35:35.842 DEBUG 13324 --- [refetchThread-1] org.apache.http.headers : http-outgoing-1 >> Content-Type: application/x-www-form-urlencoded; charset=utf-8
2018-04-18 21:35:35.842 DEBUG 13324 --- [refetchThread-1] org.apache.http.headers : http-outgoing-1 >> Content-Length: 190
2018-04-18 21:35:35.842 DEBUG 13324 --- [refetchThread-1] org.apache.http.headers : http-outgoing-1 >> Connection: Keep-Alive
2018-04-18 21:35:35.842 DEBUG 13324 --- [refetchThread-1] org.apache.http.wire : http-outgoing-1 >> "POST /868651433674/aws-nonprod-alm-dev-sqs.fifo HTTP/1.1[\r][\n]"
2018-04-18 21:35:35.842 DEBUG 13324 --- [refetchThread-1] org.apache.http.wire : http-outgoing-1 >> "Host: sqs.us-east-1.amazonaws.com[\r][\n]"
2018-04-18 21:35:35.842 DEBUG 13324 --- [refetchThread-1] org.apache.http.wire : http-outgoing-1 >> "Authorization: AWS4-HMAC-SHA256 Credential=AKIAJTHWNVZAZ2T7BW5A/ 20180419/us-east-1/sqs/aws4_request, SignedHeaders=amz-sdk-invocation-id;amz-sdk-retry;host;user-agent;x-amz-date, Signature=74503414f0a291dc4e1fc39c004748fea7565ed8d08d4d9ca810d93d8a550e9e[\r][\n]"
2018-04-18 21:35:35.842 DEBUG 13324 --- [refetchThread-1] org.apache.http.wire : http-outgoing-1 >> "X-Amz-Date: 20180419T013535Z[\r][\n]"
2018-04-18 21:35:35.842 DEBUG 13324 --- [refetchThread-1] org.apache.http.wire : http-outgoing-1 >> "User-Agent: aws-sdk-java/1.11.292 Windows_10/10.0 Java_HotSpot(TM)_64-Bit_Server_VM/25.171-b11 java/1.8.0_171 /SQS Java Messaging Client v1.0[\r][\n]"
2018-04-18 21:35:35.842 DEBUG 13324 --- [refetchThread-1] org.apache.http.wire : http-outgoing-1 >> "amz-sdk-invocation-id: 2dd018d2-da6b-010a-6d41-44ac3b676ed4[\r][\n]"
2018-04-18 21:35:35.842 DEBUG 13324 --- [refetchThread-1] org.apache.http.wire : http-outgoing-1 >> "amz-sdk-retry: 0/0/500[\r][\n]"
2018-04-18 21:35:35.843 DEBUG 13324 --- [refetchThread-1] org.apache.http.wire : http-outgoing-1 >> "Content-Type: application/x-www-form-urlencoded; charset=utf-8[\r][\n]"
2018-04-18 21:35:35.843 DEBUG 13324 --- [refetchThread-1] org.apache.http.wire : http-outgoing-1 >> "Content-Length: 190[\r][\n]"
2018-04-18 21:35:35.843 DEBUG 13324 --- [refetchThread-1] org.apache.http.wire : http-outgoing-1 >> "Connection: Keep-Alive[\r][\n]"
2018-04-18 21:35:35.843 DEBUG 13324 --- [refetchThread-1] org.apache.http.wire : http-outgoing-1 >> "[\r][\n]"
2018-04-18 21:35:35.843 DEBUG 13324 --- [refetchThread-1] org.apache.http.wire : http-outgoing-1 >> "Action=ReceiveMessage&Version=2012-11-05&AttributeName.1=All&MessageAttributeName.1=All&MaxNumberOfMessages=10&WaitTimeSeconds=20&ReceiveRequestAttemptId=cd020752-1db2-4385-ba93-3883ea946ab1"
2018-04-18 21:35:35.871 DEBUG 13324 --- [refetchThread-1] org.apache.http.wire : http-outgoing-1 << "HTTP/1.1 200 OK[\r][\n]"
2018-04-18 21:35:35.871 DEBUG 13324 --- [refetchThread-1] org.apache.http.wire : http-outgoing-1 << "Server: Server[\r][\n]"
2018-04-18 21:35:35.871 DEBUG 13324 --- [refetchThread-1] org.apache.http.wire : http-outgoing-1 << "Date: Thu, 19 Apr 2018 01:35:35 GMT[\r][\n]"
2018-04-18 21:35:35.871 DEBUG 13324 --- [refetchThread-1] org.apache.http.wire : http-outgoing-1 << "Content-Type: text/xml[\r][\n]"
2018-04-18 21:35:35.871 DEBUG 13324 --- [refetchThread-1] org.apache.http.wire : http-outgoing-1 << "Content-Length: 3710[\r][\n]"
2018-04-18 21:35:35.871 DEBUG 13324 --- [refetchThread-1] org.apache.http.wire : http-outgoing-1 << "Connection: keep-alive[\r][\n]"
2018-04-18 21:35:35.871 DEBUG 13324 --- [refetchThread-1] org.apache.http.wire : http-outgoing-1 << "x-amzn-RequestId: 7d9a5e2f-0896-533d-a67f-0d7efc3186c3[\r][\n]"
2018-04-18 21:35:35.871 DEBUG 13324 --- [refetchThread-1] org.apache.http.wire : http-outgoing-1 << "[\r][\n]"
2018-04-18 21:35:35.871 DEBUG 13324 --- [refetchThread-1] org.apache.http.wire : http-outgoing-1 << "<?xml version="1.0"?><ReceiveMessageResponse xmlns="http://queue.amazonaws.com/doc/2012-11-05/"><ReceiveMessageResult><Message><MessageId>2cfb5718-16ed-4e2b-ade3-4455889a0016</MessageId><ReceiptHandle>AQEBF5tzKo1hh4/+8molj3+yG20+toxkZq9wZ2RD3TwvwOpfggbs6nTGkgxNsLbyVbh/LJU8/+p6L8oNKkyJTSY7ku47vrMz4RxjgDurFKDUUZwEAAUvp2hSSMrTRVTlWUDpXK7M/YiJ5Mz3EwJqkJfl9zjKJibWHmAdlAtiaEFDapkLa27hSHHozjfmKAVaQU2C+5C1Z3p+cWA06fTQWrT1y80QGADeb0+utjCWQSJEEDOJaXoTC05Ps3Z2R3yq13rz76R79i4X4E0BAyfn2wFZZU1VBMkgaSPWYAO7cQD/5cA=</ReceiptHandle><MD5OfBody>2fbe3ea62f496e8a7b04d351e9305fa4</MD5OfBody><MD5OfMessageAttributes>aec5867692711b413b9dc98b5e238590</MD5OfMessageAttributes><Body><?xml version="1.0" encoding="UTF-8" standalone="yes"?>[\n]"
2018-04-18 21:35:35.871 DEBUG 13324 --- [refetchThread-1] org.apache.http.wire : http-outgoing-1 << "<Fulfillment>[\n]"
2018-04-18 21:35:35.871 DEBUG 13324 --- [refetchThread-1] org.apache.http.wire : http-outgoing-1 << " <Header>[\n]"
2018-04-18 21:35:35.871 DEBUG 13324 --- [refetchThread-1] org.apache.http.wire : http-outgoing-1 << " <GeneratedBy>test</GeneratedBy>[\n]"
2018-04-18 21:35:35.871 DEBUG 13324 --- [refetchThread-1] org.apache.http.wire : http-outgoing-1 << " <GenerationDate> 2018-04-18T21:20:30.095-04:00</GenerationDate>[\n]"
2018-04-18 21:35:35.871 DEBUG 13324 --- [refetchThread-1] org.apache.http.wire : http-outgoing-1 << " </Header>[\n]"
2018-04-18 21:35:35.871 DEBUG 13324 --- [refetchThread-1] org.apache.http.wire : http-outgoing-1 << " <Body>[\n]"
2018-04-18 21:35:35.871 DEBUG 13324 --- [refetchThread-1] org.apache.http.wire : http-outgoing-1 << " <Operations>[\n]"
2018-04-18 21:35:35.871 DEBUG 13324 --- [refetchThread-1] org.apache.http.wire : http-outgoing-1 << " <CreateUser core-id="430ed0fc-9745-4d60-a45d-4fbc7735842c">[\n]"
2018-04-18 21:35:35.871 DEBUG 13324 --- [refetchThread-1] org.apache.http.wire : http-outgoing-1 << " <Active>ACTIVE</Active>[\n]"
2018-04-18 21:35:35.871 DEBUG 13324 --- [refetchThread-1] org.apache.http.wire : http-outgoing-1 << " </CreateUser>[\n]"
2018-04-18 21:35:35.871 DEBUG 13324 --- [refetchThread-1] org.apache.http.wire : http-outgoing-1 << " </Operations>[\n]"
2018-04-18 21:35:35.871 DEBUG 13324 --- [refetchThread-1] org.apache.http.wire : http-outgoing-1 << " </Body>[\n]"
2018-04-18 21:35:35.871 DEBUG 13324 --- [refetchThread-1] org.apache.http.wire : http-outgoing-1 << "</Fulfillment>[\n]"
2018-04-18 21:35:35.871 DEBUG 13324 --- [refetchThread-1] org.apache.http.wire : http-outgoing-1 << "</Body><Attribute><Name>SenderId</Name><Value>AIDAJEEIWD3U2UWQICOBO</Value></Attribute><Attribute><Name>ApproximateFirstReceiveTimestamp</Name><Value>1524100831661</Value></Attribute><Attribute><Name>ApproximateReceiveCount</Name><Value>311</Value></Attribute><Attribute><Name>SentTimestamp</Name><Value>1524100831661</Value></Attribute><Attribute><Name>SequenceNumber</Name><Value>18836913886614767616</Value></Attribute><Attribute><Name>MessageDeduplicationId</Name><Value>2bdaaa01-002b-4daa-9493-761e76101324</Value></Attribute><Attribute><Name>MessageGroupId</Name><Value>user-api-group-id</Value></Attribute><MessageAttribute><Name>MessageGroupId</Name><Value><StringValue>user-api-group-id</StringValue><DataType>String</DataType></Value></MessageAttribute><MessageAttribute><Name>contentType</Name><Value><StringValue>text/plain;charset=UTF-8</StringValue><DataType>String</DataType></Value></MessageAttribute><MessageAttribute><Name>id</Name><Value><StringValue>1672b5dd-3ca5-9128-9ed8-e42d3d9f9714</StringValue><DataType>String</DataType></Value></MessageAttribute><MessageAttribute><Name>timestamp</Name><Value><StringValue>1524100831625</StringValue><DataType>Number.java.lang.Long</DataType></Value></MessageAttribute></Message></ReceiveMessageResult><ResponseMetadata><RequestId>7d9a5e2f-0896-533d-a67f-0d7efc3186c3</RequestId></ResponseMetadata></ReceiveMessageResponse>"
2018-04-18 21:35:35.871 DEBUG 13324 --- [refetchThread-1] org.apache.http.headers : http-outgoing-1 << HTTP/1.1 200 OK
2018-04-18 21:35:35.871 DEBUG 13324 --- [refetchThread-1] org.apache.http.headers : http-outgoing-1 << Server: Server
2018-04-18 21:35:35.871 DEBUG 13324 --- [refetchThread-1] org.apache.http.headers : http-outgoing-1 << Date: Thu, 19 Apr 2018 01:35:35 GMT
2018-04-18 21:35:35.871 DEBUG 13324 --- [refetchThread-1] org.apache.http.headers : http-outgoing-1 << Content-Type: text/xml
2018-04-18 21:35:35.871 DEBUG 13324 --- [refetchThread-1] org.apache.http.headers : http-outgoing-1 << Content-Length: 3710
2018-04-18 21:35:35.871 DEBUG 13324 --- [refetchThread-1] org.apache.http.headers : http-outgoing-1 << Connection: keep-alive
2018-04-18 21:35:35.871 DEBUG 13324 --- [refetchThread-1] org.apache.http.headers : http-outgoing-1 << x-amzn-RequestId: 7d9a5e2f-0896-533d-a67f-0d7efc3186c3
2018-04-18 21:35:35.871 DEBUG 13324 --- [refetchThread-1] o.a.http.impl.execchain.MainClientExec : Connection can be kept alive for 60000 MILLISECONDS
2018-04-18 21:35:35.872 DEBUG 13324 --- [refetchThread-1] h.i.c.PoolingHttpClientConnectionManager : Connection [id: 1][route: {s}->https://sqs.us-east-1.amazonaws.com:443] can be kept alive for 60.0 seconds
2018-04-18 21:35:35.873 DEBUG 13324 --- [refetchThread-1] h.i.c.PoolingHttpClientConnectionManager : Connection released: [id: 1][route: {s}->https://sqs.us-east-1.amazonaws.com:443][total kept alive: 1; route allocated: 1 of 50; total allocated: 1 of 50]
2018-04-18 21:35:35.873 DEBUG 13324 --- [refetchThread-1] com.amazonaws.request : Received successful response: 200, AWS Request ID: 7d9a5e2f-0896-533d-a67f-0d7efc3186c3
2018-04-18 21:35:35.873 DEBUG 13324 --- [refetchThread-1] com.amazonaws.requestId : x-amzn-RequestId: 7d9a5e2f-0896-533d-a67f-0d7efc3186c3
What am I doing wrong here?

Answer 1

Once you receive a message you need to send the acknowledge about the message as you are using Session.CLIENT_ACKNOWLEDGE for acknowledgment of the message.
Just adding message.acknowledge(); after using the message it should delete from the queue.