Spring Cloud Gateway Cors issue - spring-security

I'm having issues setting up CORS configuration in a spring cloud gateway app. I've set the following config to allow everything from everything:
spring:
cloud:
gateway:
globalcors:
corsConfigurations:
'[/**]':
allowedOrigins: "*"
allowedMethods: "*"
allowedHeaders: "authorization"
Yet, when I'm calling an endpoint to the gateway I'm getting:
OPTIONS http://localhost:8080/api/spot/spots 403 (Forbidden)
localhost/:1 Access to XMLHttpRequest at 'http://localhost:8080/api/spot/spots' from origin 'http://localhost:3000' has been blocked by CORS policy: Response to preflight request doesn't pass access control check: No 'Access-Control-Allow-Origin' header is present on the requested resource.
When I call one of the services by-passing the gateway via the UI, the requests are going through fine. However, I don't see any options requests being made.
And as soon as I point the UI to the gateway host, the error occurs.
Also in the Network tab, the request shows up as "failed", no other information. Nothing in any of the application logs either.
These are the response headers:
HTTP/1.1 200 OK
transfer-encoding: chunked
Vary: Origin
Vary: Access-Control-Request-Method
Vary: Access-Control-Request-Headers
Access-Control-Allow-Origin: *
Vary: Origin
Vary: Access-Control-Request-Method
Vary: Access-Control-Request-Headers
Access-Control-Allow-Origin: *
X-Content-Type-Options: nosniff
X-XSS-Protection: 1; mode=block
Cache-Control: no-cache, no-store, max-age=0, must-revalidate
Pragma: no-cache
Expires: 0
X-Frame-Options: DENY
Content-Type: application/json
Date: Tue, 10 Mar 2020 21:24:11 GMT
When restarting the Item service (responsible for this endpoint), I'm getting a 500:
timestamp: "2020-03-10T21:28:33.568+0000"
path: "/api/spot/spots"
status: 500
error: "Internal Server Error"
message: "Connection refused: localhost/127.0.0.1:49508"
requestId: "68570bc1"
Which points to the old random port, meaning the gateway can connect to it somehow.
Any ideas what I'm missing?
Thanks.
The app is open-source and available here.
The SpotController can be found here.
The spring-security configuration for that endpoint can be found here.
The gateway config can be found here.
Here are the logs of the api gateway app, when set on debug (full logs here):
2020-03-10 22:24:10.744 DEBUG 57262 --- [ionManagerTimer] h.i.c.PoolingHttpClientConnectionManager : Closing expired connections
2020-03-10 22:24:10.812 DEBUG 57262 --- [er-item-service] c.netflix.loadbalancer.BaseLoadBalancer : LoadBalancer: PingTask executing [1] servers configured
2020-03-10 22:24:11.464 DEBUG 57262 --- [ctor-http-nio-2] r.n.http.server.HttpServerOperations : [id: 0xd4a47354, L:/0:0:0:0:0:0:0:1:8080 - R:/0:0:0:0:0:0:0:1:59021] Increasing pending responses, now 1
2020-03-10 22:24:11.464 DEBUG 57262 --- [ctor-http-nio-2] reactor.netty.http.server.HttpServer : [id: 0xd4a47354, L:/0:0:0:0:0:0:0:1:8080 - R:/0:0:0:0:0:0:0:1:59021] Handler is being applied: org.springframework.http.server.reactive.ReactorHttpHandlerAdapter#528c0cbe
2020-03-10 22:24:11.464 DEBUG 57262 --- [ctor-http-nio-2] o.s.w.s.adapter.HttpWebHandlerAdapter : [d4a47354] HTTP OPTIONS "/api/spot/spots"
2020-03-10 22:24:11.466 DEBUG 57262 --- [ctor-http-nio-2] o.s.c.g.h.RoutePredicateHandlerMapping : Route matched: item-service
2020-03-10 22:24:11.466 DEBUG 57262 --- [ctor-http-nio-2] o.s.c.g.h.RoutePredicateHandlerMapping : Mapping [Exchange: OPTIONS http://localhost:8080/api/spot/spots] to Route{id='item-service', uri=lb://item-service, order=0, predicate=Paths: [/api/spot/**], match trailing slash: true, gatewayFilters=[], metadata={}}
2020-03-10 22:24:11.466 DEBUG 57262 --- [ctor-http-nio-2] o.s.c.g.h.RoutePredicateHandlerMapping : [d4a47354] Mapped to org.springframework.cloud.gateway.handler.FilteringWebHandler#14780192
2020-03-10 22:24:11.467 DEBUG 57262 --- [ctor-http-nio-2] o.s.w.s.adapter.HttpWebHandlerAdapter : [d4a47354] Completed 200 OK
2020-03-10 22:24:11.468 DEBUG 57262 --- [ctor-http-nio-2] r.n.http.server.HttpServerOperations : [id: 0xd4a47354, L:/0:0:0:0:0:0:0:1:8080 - R:/0:0:0:0:0:0:0:1:59021] Last HTTP response frame
2020-03-10 22:24:11.468 DEBUG 57262 --- [ctor-http-nio-2] r.n.http.server.HttpServerOperations : [id: 0xd4a47354, L:/0:0:0:0:0:0:0:1:8080 - R:/0:0:0:0:0:0:0:1:59021] No sendHeaders() called before complete, sending zero-length header
2020-03-10 22:24:11.468 DEBUG 57262 --- [ctor-http-nio-2] r.n.http.server.HttpServerOperations : [id: 0xd4a47354, L:/0:0:0:0:0:0:0:1:8080 - R:/0:0:0:0:0:0:0:1:59021] Decreasing pending responses, now 0
2020-03-10 22:24:11.468 DEBUG 57262 --- [ctor-http-nio-2] r.n.http.server.HttpServerOperations : [id: 0xd4a47354, L:/0:0:0:0:0:0:0:1:8080 - R:/0:0:0:0:0:0:0:1:59021] Last HTTP packet was sent, terminating the channel
2020-03-10 22:24:11.468 DEBUG 57262 --- [ctor-http-nio-2] r.n.channel.ChannelOperationsHandler : [id: 0xd4a47354, L:/0:0:0:0:0:0:0:1:8080 - R:/0:0:0:0:0:0:0:1:59021] No ChannelOperation attached. Dropping: EmptyLastHttpContent
2020-03-10 22:24:11.486 DEBUG 57262 --- [ctor-http-nio-2] r.n.http.server.HttpServerOperations : [id: 0xd4a47354, L:/0:0:0:0:0:0:0:1:8080 - R:/0:0:0:0:0:0:0:1:59021] Increasing pending responses, now 1
2020-03-10 22:24:11.486 DEBUG 57262 --- [ctor-http-nio-2] reactor.netty.http.server.HttpServer : [id: 0xd4a47354, L:/0:0:0:0:0:0:0:1:8080 - R:/0:0:0:0:0:0:0:1:59021] Handler is being applied: org.springframework.http.server.reactive.ReactorHttpHandlerAdapter#528c0cbe
2020-03-10 22:24:11.486 DEBUG 57262 --- [ctor-http-nio-2] o.s.w.s.adapter.HttpWebHandlerAdapter : [d4a47354] HTTP GET "/api/spot/spots"
2020-03-10 22:24:11.488 DEBUG 57262 --- [ctor-http-nio-2] o.s.c.g.h.RoutePredicateHandlerMapping : Route matched: item-service
2020-03-10 22:24:11.488 DEBUG 57262 --- [ctor-http-nio-2] o.s.c.g.h.RoutePredicateHandlerMapping : Mapping [Exchange: GET http://localhost:8080/api/spot/spots] to Route{id='item-service', uri=lb://item-service, order=0, predicate=Paths: [/api/spot/**], match trailing slash: true, gatewayFilters=[], metadata={}}
2020-03-10 22:24:11.488 DEBUG 57262 --- [ctor-http-nio-2] o.s.c.g.h.RoutePredicateHandlerMapping : [d4a47354] Mapped to org.springframework.cloud.gateway.handler.FilteringWebHandler#14780192
2020-03-10 22:24:11.488 DEBUG 57262 --- [ctor-http-nio-2] o.s.c.g.handler.FilteringWebHandler : Sorted gatewayFilterFactories: [[GatewayFilterAdapter{delegate=org.springframework.cloud.gateway.filter.RemoveCachedBodyFilter#22bf9122}, order = -2147483648], [GatewayFilterAdapter{delegate=org.springframework.cloud.gateway.filter.AdaptCachedBodyGlobalFilter#69aa7d76}, order = -2147482648], [GatewayFilterAdapter{delegate=org.springframework.cloud.gateway.filter.NettyWriteResponseFilter#3d88e6b9}, order = -1], [GatewayFilterAdapter{delegate=org.springframework.cloud.gateway.filter.ForwardPathFilter#9687f55}, order = 0], [GatewayFilterAdapter{delegate=org.springframework.cloud.gateway.filter.GatewayMetricsFilter#671d03bb}, order = 0], [GatewayFilterAdapter{delegate=org.springframework.cloud.gateway.filter.RouteToRequestUrlFilter#208205ed}, order = 10000], [GatewayFilterAdapter{delegate=org.springframework.cloud.gateway.filter.LoadBalancerClientFilter#36068727}, order = 10100], [GatewayFilterAdapter{delegate=org.springframework.cloud.gateway.filter.WebsocketRoutingFilter#5700c9db}, order = 2147483646], [GatewayFilterAdapter{delegate=org.springframework.cloud.gateway.filter.NettyRoutingFilter#72543547}, order = 2147483647], [GatewayFilterAdapter{delegate=org.springframework.cloud.gateway.filter.ForwardRoutingFilter#73afe2b7}, order = 2147483647]]
2020-03-10 22:24:11.489 DEBUG 57262 --- [ctor-http-nio-2] c.n.loadbalancer.ZoneAwareLoadBalancer : Zone aware logic disabled or there is only one zone
2020-03-10 22:24:11.490 DEBUG 57262 --- [ctor-http-nio-2] r.n.resources.PooledConnectionProvider : [id: 0x687fa33b] Created new pooled channel, now 1 active connections and 0 inactive connections
2020-03-10 22:24:11.491 DEBUG 57262 --- [ctor-http-nio-2] reactor.netty.channel.BootstrapHandlers : [id: 0x687fa33b] Initialized pipeline DefaultChannelPipeline{(BootstrapHandlers$BootstrapInitializerHandler#0 = reactor.netty.channel.BootstrapHandlers$BootstrapInitializerHandler), (reactor.left.httpCodec = io.netty.handler.codec.http.HttpClientCodec), (reactor.right.reactiveBridge = reactor.netty.channel.ChannelOperationsHandler)}
2020-03-10 22:24:11.496 DEBUG 57262 --- [ctor-http-nio-2] r.n.resources.PooledConnectionProvider : [id: 0x687fa33b, L:/127.0.0.1:61299 - R:localhost/127.0.0.1:49508] Channel connected, now 1 active connections and 0 inactive connections
2020-03-10 22:24:11.496 DEBUG 57262 --- [ctor-http-nio-2] r.n.resources.PooledConnectionProvider : [id: 0x687fa33b, L:/127.0.0.1:61299 - R:localhost/127.0.0.1:49508] onStateChange(PooledConnection{channel=[id: 0x687fa33b, L:/127.0.0.1:61299 - R:localhost/127.0.0.1:49508]}, [connected])
2020-03-10 22:24:11.496 DEBUG 57262 --- [ctor-http-nio-2] r.n.resources.PooledConnectionProvider : [id: 0x687fa33b, L:/127.0.0.1:61299 - R:localhost/127.0.0.1:49508] onStateChange(GET{uri=/, connection=PooledConnection{channel=[id: 0x687fa33b, L:/127.0.0.1:61299 - R:localhost/127.0.0.1:49508]}}, [configured])
2020-03-10 22:24:11.496 DEBUG 57262 --- [ctor-http-nio-2] r.netty.http.client.HttpClientConnect : [id: 0x687fa33b, L:/127.0.0.1:61299 - R:localhost/127.0.0.1:49508] Handler is being applied: {uri=http://localhost:49508/api/spot/spots, method=GET}
2020-03-10 22:24:11.497 DEBUG 57262 --- [ctor-http-nio-2] reactor.netty.channel.FluxReceive : [id: 0xd4a47354, L:/0:0:0:0:0:0:0:1:8080 - R:/0:0:0:0:0:0:0:1:59021] Subscribing inbound receiver [pending: 0, cancelled:false, inboundDone: true]
2020-03-10 22:24:11.497 DEBUG 57262 --- [ctor-http-nio-2] r.n.resources.PooledConnectionProvider : [id: 0x687fa33b, L:/127.0.0.1:61299 - R:localhost/127.0.0.1:49508] onStateChange(GET{uri=/api/spot/spots, connection=PooledConnection{channel=[id: 0x687fa33b, L:/127.0.0.1:61299 - R:localhost/127.0.0.1:49508]}}, [request_sent])
2020-03-10 22:24:11.579 DEBUG 57262 --- [ctor-http-nio-2] r.n.http.client.HttpClientOperations : [id: 0x687fa33b, L:/127.0.0.1:61299 - R:localhost/127.0.0.1:49508] Received response (auto-read:false) : [Vary=Origin, Vary=Access-Control-Request-Method, Vary=Access-Control-Request-Headers, Access-Control-Allow-Origin=*, X-Content-Type-Options=nosniff, X-XSS-Protection=1; mode=block, Cache-Control=no-cache, no-store, max-age=0, must-revalidate, Pragma=no-cache, Expires=0, X-Frame-Options=DENY, Content-Type=application/json, Transfer-Encoding=chunked, Date=Tue, 10 Mar 2020 21:24:11 GMT]
2020-03-10 22:24:11.579 DEBUG 57262 --- [ctor-http-nio-2] r.n.resources.PooledConnectionProvider : [id: 0x687fa33b, L:/127.0.0.1:61299 - R:localhost/127.0.0.1:49508] onStateChange(GET{uri=/api/spot/spots, connection=PooledConnection{channel=[id: 0x687fa33b, L:/127.0.0.1:61299 - R:localhost/127.0.0.1:49508]}}, [response_received])
2020-03-10 22:24:11.580 DEBUG 57262 --- [ctor-http-nio-2] reactor.netty.channel.FluxReceive : [id: 0x687fa33b, L:/127.0.0.1:61299 - R:localhost/127.0.0.1:49508] Subscribing inbound receiver [pending: 0, cancelled:false, inboundDone: false]
2020-03-10 22:24:11.581 DEBUG 57262 --- [ctor-http-nio-2] r.n.http.client.HttpClientOperations : [id: 0x687fa33b, L:/127.0.0.1:61299 - R:localhost/127.0.0.1:49508] Received last HTTP packet
2020-03-10 22:24:11.581 DEBUG 57262 --- [ctor-http-nio-2] o.s.w.s.adapter.HttpWebHandlerAdapter : [d4a47354] Completed 200 OK
2020-03-10 22:24:11.581 DEBUG 57262 --- [ctor-http-nio-2] r.n.http.server.HttpServerOperations : [id: 0xd4a47354, L:/0:0:0:0:0:0:0:1:8080 - R:/0:0:0:0:0:0:0:1:59021] Last HTTP response frame
2020-03-10 22:24:11.581 DEBUG 57262 --- [ctor-http-nio-2] r.n.http.server.HttpServerOperations : [id: 0xd4a47354, L:/0:0:0:0:0:0:0:1:8080 - R:/0:0:0:0:0:0:0:1:59021] Decreasing pending responses, now 0
2020-03-10 22:24:11.581 DEBUG 57262 --- [ctor-http-nio-2] r.n.http.server.HttpServerOperations : [id: 0xd4a47354, L:/0:0:0:0:0:0:0:1:8080 - R:/0:0:0:0:0:0:0:1:59021] Last HTTP packet was sent, terminating the channel
2020-03-10 22:24:11.581 DEBUG 57262 --- [ctor-http-nio-2] r.n.resources.PooledConnectionProvider : [id: 0x687fa33b, L:/127.0.0.1:61299 - R:localhost/127.0.0.1:49508] onStateChange(GET{uri=/api/spot/spots, connection=PooledConnection{channel=[id: 0x687fa33b, L:/127.0.0.1:61299 - R:localhost/127.0.0.1:49508]}}, [disconnecting])
2020-03-10 22:24:11.581 DEBUG 57262 --- [ctor-http-nio-2] r.n.resources.PooledConnectionProvider : [id: 0x687fa33b, L:/127.0.0.1:61299 - R:localhost/127.0.0.1:49508] Releasing channel
2020-03-10 22:24:11.582 DEBUG 57262 --- [ctor-http-nio-2] r.n.resources.PooledConnectionProvider : [id: 0x687fa33b, L:/127.0.0.1:61299 - R:localhost/127.0.0.1:49508] Channel cleaned, now 0 active connections and 1 inactive connections
2020-03-10 22:24:11.831 DEBUG 57262 --- [erListUpdater-1] c.n.l.DynamicServerListLoadBalancer : List of Servers for item-service obtained from Discovery client: [localhost:49508]
2020-03-10 22:24:11.831 DEBUG 57262 --- [erListUpdater-1] c.n.l.ZoneAffinityServerListFilter : Determining if zone affinity should be enabled with given server list: [localhost:49508]
2020-03-10 22:24:11.831 DEBUG 57262 --- [erListUpdater-1] c.n.l.ZoneAffinityServerListFilter : zoneAffinity is overriden. blackOutServerPercentage: 0.0, activeReqeustsPerServer: 0.0, availableServers: 1
2020-03-10 22:24:11.831 DEBUG 57262 --- [erListUpdater-1] c.n.l.DynamicServerListLoadBalancer : Filtered List of Servers for item-service obtained from Discovery client: [localhost:49508]
2020-03-10 22:24:11.831 DEBUG 57262 --- [erListUpdater-1] c.netflix.loadbalancer.BaseLoadBalancer : LoadBalancer [item-service]: clearing server list (SET op)
2020-03-10 22:24:11.831 DEBUG 57262 --- [erListUpdater-1] c.netflix.loadbalancer.BaseLoadBalancer : LoadBalancer [item-service]: addServer [localhost:49508]
2020-03-10 22:24:11.831 DEBUG 57262 --- [erListUpdater-1] c.n.l.DynamicServerListLoadBalancer : Setting server list for zones: {defaultzone=[localhost:49508]}
2020-03-10 22:24:11.832 DEBUG 57262 --- [erListUpdater-1] c.netflix.loadbalancer.BaseLoadBalancer : LoadBalancer [item-service_defaultzone]: clearing server list (SET op)
2020-03-10 22:24:11.832 DEBUG 57262 --- [erListUpdater-1] c.netflix.loadbalancer.BaseLoadBalancer : LoadBalancer [item-service_defaultzone]: addServer [localhost:49508]
2020-03-10 22:24:11.832 DEBUG 57262 --- [erListUpdater-1] c.netflix.loadbalancer.BaseLoadBalancer : LoadBalancer [item-service]: forceQuickPing invoking
2020-03-10 22:24:11.832 DEBUG 57262 --- [erListUpdater-1] c.netflix.loadbalancer.BaseLoadBalancer : LoadBalancer: PingTask executing [1] servers configured

Did you tried to change the allowedHeaders: "authorization" to allowedHeaders: "*". I had the same issue and was fixed just changing the allowHeaders to the wildcard since I was receiving from the client as header x-requested-with, authorization.

Configuration looks good but it may possible that it is not taking effect. This might happen when both application.properties and application.yml are there under resources, i had same issue.
Keeping only one worked for me. My application.properties file have following cors setting.
spring.cloud.gateway.globalcors.add-to-simple-url-handler-mapping=true
spring.cloud.gateway.globalcors.corsConfigurations.[/**].allowedOrigins=*
spring.cloud.gateway.globalcors.corsConfigurations.[/**].allowedHeaders=*
spring.cloud.gateway.globalcors.corsConfigurations.[/**].allowedMethods=*

Related

How to use an external database from my application in Minikube?

i have a problem, I can't connect to an external database that is in my same network but in another node. My application is built on spring boot and my problem is trying to connect to the database. This is the stacktrace of the error.
22:40:56.393 [main] INFO com.zaxxer.hikari.HikariDataSource - HikariPool-1 - Starting...
22:41:07.480 [main] ERROR com.zaxxer.hikari.pool.HikariPool - HikariPool-1 - Exception during pool initialization.
org.postgresql.util.PSQLException: The connection attempt failed.
at org.postgresql.core.v3.ConnectionFactoryImpl.openConnectionImpl(ConnectionFactoryImpl.java:331)
at org.postgresql.core.ConnectionFactory.openConnection(ConnectionFactory.java:49)
at org.postgresql.jdbc.PgConnection.<init>(PgConnection.java:223)
at org.postgresql.Driver.makeConnection(Driver.java:402)
at org.postgresql.Driver.connect(Driver.java:261)
at com.zaxxer.hikari.util.DriverDataSource.getConnection(DriverDataSource.java:138)
...
...
Caused by: java.net.SocketTimeoutException: connect timed out
at java.base/java.net.PlainSocketImpl.socketConnect(Native Method)
at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source)
at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source)
22:41:07.482 [main] WARN o.h.e.j.e.i.JdbcEnvironmentInitiator - HHH000342: Could not obtain connection to query metadata
org.postgresql.util.PSQLException: The connection attempt failed.
at org.postgresql.core.v3.ConnectionFactoryImpl.openConnectionImpl(ConnectionFactoryImpl.java:331)
at org.postgresql.core.ConnectionFactory.openConnection(ConnectionFactory.java:49)
at org.postgresql.jdbc.PgConnection.<init>(PgConnection.java:223)
at org.postgresql.Driver.makeConnection(Driver.java:402)
I'm using minikube version: v1.26.0
Code for start minikube:
minikube start --driver=hyperkit
My deployment yml file
apiVersion: v1
kind: Service
metadata:
name: web-application
spec:
selector:
role: webapp
ports:
- protocol: TCP
port: 8080
targetPort: 80
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: webapp-gastronomia
spec:
selector:
matchLabels:
role: webapp
replicas: 1
template:
metadata:
labels:
role: webapp
env: production
spec:
containers:
- name: app-gastronomia
image: localhost:5000/app-gastronomia:1.0
imagePullPolicy: IfNotPresent
ports:
- containerPort: 80
My dockerfile:
FROM openjdk:11.0.7-jre-slim
COPY build/libs/register-0.0.1-SNAPSHOT.jar app.jar
EXPOSE 8001
ENTRYPOINT ["java","-jar","app.jar"]
In docker everything works fine, I have no problem.
Ping from ubuntu pod
root#ubuntu:/# ping -c 3 172.16.0.180
PING 172.16.0.180 (172.16.0.180) 56(84) bytes of data.
--- 172.16.0.180 ping statistics ---
3 packets transmitted, 0 received, 100% packet loss, time 2072ms
Ping from my computer's terminal
ping -c 3 172.16.0.180  2 ✘  took 13s   at 05:47:45 PM 
PING 172.16.0.180 (172.16.0.180): 56 data bytes
64 bytes from 172.16.0.180: icmp_seq=0 ttl=63 time=78.136 ms
64 bytes from 172.16.0.180: icmp_seq=1 ttl=63 time=74.404 ms
64 bytes from 172.16.0.180: icmp_seq=2 ttl=63 time=74.904 ms
--- 172.16.0.180 ping statistics ---
3 packets transmitted, 3 packets received, 0.0% packet loss
round-trip min/avg/max/stddev = 74.404/75.815/78.136/1.654 ms
Investigating, create a service and endpoint pointing to the external ip, here my file
apiVersion: v1
kind: Service
metadata:
name: database
spec:
clusterIP: None
ports:
- port: 5432
targetPort: 5432
protocol: TCP
---
kind: Endpoints
apiVersion: v1
metadata:
name: database
subsets:
- addresses:
- ip: 172.16.0.180
ports:
- port: 5432
Doing a ping from the ubuntu pod
root#ubuntu:/# ping -c 3 database
PING database.default.svc.cluster.local (172.16.0.180) 56(84) bytes of data.
--- database.default.svc.cluster.local ping statistics ---
3 packets transmitted, 0 received, 100% packet loss, time 2032ms
Any solution on how to connect from my application (pod) to the minikube database outside?
EDIT
I forgot to mention that I work from home, I access the database from a vpn
This really depends on how you've set up Minikube. Normally, it is running inside a VM which is isolated from the network other local containers are running on.
There are 2 simple ways you can go about to solve your issue -
Run the DB on Minikube as well, which will allow the application to access it using k8s internal dns address
Expose the DB to the public internet using Ngrok or a similar solution. This approach is much less recommended as it needlessly exposes your DB to the internet just so you can access it from Minikube.

Why can't I access my web app from a remote container?

I have looked through past answers since this is a common question but no solution seems to work for me.
I have a Springboot Java app and a postgresql database, each in their own container. Both containers run on a remote headless server on a local network. My remote server's physical IP address is 192.168.1.200. When I enter: 'http://192.168.1.200:8080' in my browser from another machine I get a 'unable to connect' response.
Here is my Docker-compose file:
version: "3.3"
services:
db:
image: postgres
environment:
POSTGRES_PASSWORD: aaa
volumes:
- /var/lib/postgresql/data
ports:
- "5432:5432"
networks:
- nat
web:
image: email-viewer
ports:
- "192.168.1.200:8080:80"
depends_on:
- db
networks:
- nat
networks:
nat:
external:
name: nat
Here is the output when I run docker-compose up:
Recreating email-viewer_db_1 ... done
Recreating email-viewer_web_1 ... done
Attaching to email-viewer_db_1, email-viewer_web_1
db_1 |
db_1 | PostgreSQL Database directory appears to contain a database; Skipping initialization
db_1 |
db_1 | 2022-05-06 16:13:24.300 UTC [1] LOG: starting PostgreSQL 14.2 (Debian 14.2-1.pgdg110+1) on x86_64-pc-linux-gnu, compiled by gcc (Debian 10.2.1-6) 10.2.1 20210110, 64-bit
db_1 | 2022-05-06 16:13:24.300 UTC [1] LOG: listening on IPv4 address "0.0.0.0", port 5432
db_1 | 2022-05-06 16:13:24.300 UTC [1] LOG: listening on IPv6 address "::", port 5432
db_1 | 2022-05-06 16:13:24.305 UTC [1] LOG: listening on Unix socket "/var/run/postgresql/.s.PGSQL.5432"
db_1 | 2022-05-06 16:13:24.311 UTC [27] LOG: database system was shut down at 2022-05-06 13:54:07 UTC
db_1 | 2022-05-06 16:13:24.319 UTC [1] LOG: database system is ready to accept connections
web_1 |
web_1 | . ____ _ __ _ _
web_1 | /\\ / ___'_ __ _ _(_)_ __ __ _ \ \ \ \
web_1 | ( ( )\___ | '_ | '_| | '_ \/ _` | \ \ \ \
web_1 | \\/ ___)| |_)| | | | | || (_| | ) ) ) )
web_1 | ' |____| .__|_| |_|_| |_\__, | / / / /
web_1 | =========|_|==============|___/=/_/_/_/
web_1 | :: Spring Boot :: (v2.6.5)
web_1 |
web_1 | 2022-05-06 16:13:25.487 INFO 1 --- [ main] c.a.emailviewer.EmailViewerApplication : Starting EmailViewerApplication v0.0.1-SNAPSHOT using Java 17.0.2 on 1a13d69d117d with PID 1 (/app/email-viewer-0.0.1-SNAPSHOT.jar started by root in /app)
web_1 | 2022-05-06 16:13:25.490 INFO 1 --- [ main] c.a.emailviewer.EmailViewerApplication : No active profile set, falling back to 1 default profile: "default"
web_1 | 2022-05-06 16:13:26.137 INFO 1 --- [ main] .s.d.r.c.RepositoryConfigurationDelegate : Bootstrapping Spring Data JPA repositories in DEFAULT mode.
web_1 | 2022-05-06 16:13:26.184 INFO 1 --- [ main] .s.d.r.c.RepositoryConfigurationDelegate : Finished Spring Data repository scanning in 38 ms. Found 1 JPA repository interfaces.
web_1 | 2022-05-06 16:13:26.764 INFO 1 --- [ main] o.s.b.w.embedded.tomcat.TomcatWebServer : Tomcat initialized with port(s): 8080 (http)
web_1 | 2022-05-06 16:13:26.774 INFO 1 --- [ main] o.apache.catalina.core.StandardService : Starting service [Tomcat]
web_1 | 2022-05-06 16:13:26.775 INFO 1 --- [ main] org.apache.catalina.core.StandardEngine : Starting Servlet engine: [Apache Tomcat/9.0.60]
web_1 | 2022-05-06 16:13:26.843 INFO 1 --- [ main] o.a.c.c.C.[Tomcat].[localhost].[/] : Initializing Spring embedded WebApplicationContext
web_1 | 2022-05-06 16:13:26.843 INFO 1 --- [ main] w.s.c.ServletWebServerApplicationContext : Root WebApplicationContext: initialization completed in 1297 ms
web_1 | 2022-05-06 16:13:27.031 INFO 1 --- [ main] o.hibernate.jpa.internal.util.LogHelper : HHH000204: Processing PersistenceUnitInfo [name: default]
web_1 | 2022-05-06 16:13:27.077 INFO 1 --- [ main] org.hibernate.Version : HHH000412: Hibernate ORM core version 5.6.7.Final
web_1 | 2022-05-06 16:13:27.222 INFO 1 --- [ main] o.hibernate.annotations.common.Version : HCANN000001: Hibernate Commons Annotations {5.1.2.Final}
web_1 | 2022-05-06 16:13:27.313 INFO 1 --- [ main] com.zaxxer.hikari.HikariDataSource : HikariPool-1 - Starting...
web_1 | 2022-05-06 16:13:27.506 INFO 1 --- [ main] com.zaxxer.hikari.HikariDataSource : HikariPool-1 - Start completed.
web_1 | 2022-05-06 16:13:27.539 INFO 1 --- [ main] org.hibernate.dialect.Dialect : HHH000400: Using dialect: org.hibernate.dialect.PostgreSQL10Dialect
web_1 | 2022-05-06 16:13:28.034 INFO 1 --- [ main] o.h.e.t.j.p.i.JtaPlatformInitiator : HHH000490: Using JtaPlatform implementation: [org.hibernate.engine.transaction.jta.platform.internal.NoJtaPlatform]
web_1 | 2022-05-06 16:13:28.042 INFO 1 --- [ main] j.LocalContainerEntityManagerFactoryBean : Initialized JPA EntityManagerFactory for persistence unit 'default'
web_1 | 2022-05-06 16:13:28.330 WARN 1 --- [ main] JpaBaseConfiguration$JpaWebConfiguration : spring.jpa.open-in-view is enabled by default. Therefore, database queries may be performed during view rendering. Explicitly configure spring.jpa.open-in-view to disable this warning
web_1 | 2022-05-06 16:13:28.663 INFO 1 --- [ main] o.s.b.w.embedded.tomcat.TomcatWebServer : Tomcat started on port(s): 8080 (http) with context path ''
web_1 | 2022-05-06 16:13:28.672 INFO 1 --- [ main] c.a.emailviewer.EmailViewerApplication : Started EmailViewerApplication in 3.615 seconds (JVM running for 4.024)
It turns out that the embedded tomcat server in spring boot uses port 8080 by default and would require 192.168.1.200:8080:8080 instead of 192.168.1.200:8080:80

kubernetes, container crashing on SIGQUIT signal for unknown reasons

I have a container that keeps crashing in my k8s cluster for unknown reasons. The container's process is an nginx server. The container appears to be receiving a SIGQUIT signal.
Dockerfile
# build environment
FROM node:16-alpine as build
WORKDIR /app
ENV PATH /app/node_modules/.bin:$PATH
COPY package.json ./
COPY package-lock.json ./
RUN npm ci --silent
RUN npm install react-scripts#3.4.1 -g --silent
COPY . ./
RUN npm run build
# production environment
FROM nginx:stable-alpine
COPY --from=build /app/build /usr/share/nginx/html
EXPOSE 80
CMD ["nginx", "-g", "daemon off;"]
container logs
/docker-entrypoint.sh: /docker-entrypoint.d/ is not empty, will attempt to perform configuration
/docker-entrypoint.sh: Looking for shell scripts in /docker-entrypoint.d/
/docker-entrypoint.sh: Launching /docker-entrypoint.d/10-listen-on-ipv6-by-default.sh
10-listen-on-ipv6-by-default.sh: info: Getting the checksum of /etc/nginx/conf.d/default.conf
10-listen-on-ipv6-by-default.sh: info: Enabled listen on IPv6 in /etc/nginx/conf.d/default.conf
/docker-entrypoint.sh: Launching /docker-entrypoint.d/20-envsubst-on-templates.sh
/docker-entrypoint.sh: Launching /docker-entrypoint.d/30-tune-worker-processes.sh
/docker-entrypoint.sh: Configuration complete; ready for start up
2021/11/11 06:40:37 [notice[] 1#1: using the "epoll" event method
2021/11/11 06:40:37 [notice[] 1#1: nginx/1.20.1
2021/11/11 06:40:37 [notice[] 1#1: built by gcc 10.2.1 20201203 (Alpine 10.2.1_pre1)
2021/11/11 06:40:37 [notice[] 1#1: OS: Linux 5.4.120+
2021/11/11 06:40:37 [notice[] 1#1: getrlimit(RLIMIT_NOFILE): 1048576:1048576
2021/11/11 06:40:37 [notice[] 1#1: start worker processes
2021/11/11 06:40:37 [notice[] 1#1: start worker process 32
2021/11/11 06:40:37 [notice[] 1#1: start worker process 33
10.15.128.65 - - [11/Nov/2021:06:40:41 +0000] "\x16\x03\x01\x01\x00\x01\x00\x00\xFC\x03\x03>\x85O#\xCC\xB9\xA5j\xAB\x8D\xC1PpZ\x18$\xE5ah\xDF7\xB1\xFF\xAD\x22\x050\xC3.+\xB6+ \x0F}S)\xC9\x1F\x0BY\x15_\x10\xC6\xAAF\xAA\x9F\x9E_#dG\x01\xF5vzt\xB50&;\x1E\x15\x00&\xC0/\xC00\xC0+\xC0,\xCC\xA8\xCC\xA9\xC0\x13\xC0\x09\xC0\x14\xC0" 400 157 "-" "-" "-"
10.15.128.65 - - [11/Nov/2021:06:40:44 +0000] "\x16\x03\x01\x01\x00\x01\x00\x00\xFC\x03\x03\xD8['\xE75x'\xC3}+v\xC9\x83\x84\x96EKn\xC5\xB6}\xEE\xBE\xD9Gp\xE9\x1BX<n\xB2 \xD9n\xD1\xC5\xFC\xF2\x8D\x92\xAC\xC0\xA8mdF\x17B\xA3y9\xDD\x98b\x0E\x996\xB6\xA5\xAB\xEB\xD4\xDA" 400 157 "-" "-" "-"
10.15.128.65 - - [11/Nov/2021:06:40:47 +0000] "\x16\x03\x01\x01\x00\x01\x00\x00\xFC\x03\x03Fy\x03N\x0E\x11\x89k\x7F\xC5\x00\x90w}\xEB{\x7F\xB1=\xF0" 400 157 "-" "-" "-"
2021/11/11 06:40:47 [notice[] 1#1: signal 3 (SIGQUIT) received, shutting down
2021/11/11 06:40:47 [notice[] 32#32: gracefully shutting down
2021/11/11 06:40:47 [notice[] 32#32: exiting
2021/11/11 06:40:47 [notice[] 33#33: gracefully shutting down
2021/11/11 06:40:47 [notice[] 32#32: exit
2021/11/11 06:40:47 [notice[] 33#33: exiting
2021/11/11 06:40:47 [notice[] 33#33: exit
2021/11/11 06:40:47 [notice[] 1#1: signal 17 (SIGCHLD) received from 33
2021/11/11 06:40:47 [notice[] 1#1: worker process 33 exited with code 0
2021/11/11 06:40:47 [notice[] 1#1: signal 29 (SIGIO) received
2021/11/11 06:40:47 [notice[] 1#1: signal 17 (SIGCHLD) received from 32
2021/11/11 06:40:47 [notice[] 1#1: worker process 32 exited with code 0
2021/11/11 06:40:47 [notice[] 1#1: exit
kubectl get pod PODNAME --output=yaml
apiVersion: v1
kind: Pod
metadata:
annotations:
seccomp.security.alpha.kubernetes.io/pod: runtime/default
creationTimestamp: "2021-11-11T06:40:30Z"
generateName: sgb-web-master-fb9f995fb-
labels:
app: sgb-web-master
pod-template-hash: fb9f995fb
name: sgb-web-master-fb9f995fb-zwhgl
namespace: default
ownerReferences:
- apiVersion: apps/v1
blockOwnerDeletion: true
controller: true
kind: ReplicaSet
name: sgb-web-master-fb9f995fb
uid: 96ebf43d-e2e6-4632-a536-764bcab8daeb
resourceVersion: "66168456"
uid: ed80b0d0-6681-4c2a-8edd-16c8ef6bee86
spec:
containers:
- env:
- name: PORT
value: "80"
image: cflynnus/saigonbros-web:master-d70f3001d130bf986da236a08e1fded4b64e8097
imagePullPolicy: Always
livenessProbe:
failureThreshold: 3
httpGet:
path: /
port: 80
scheme: HTTPS
initialDelaySeconds: 3
periodSeconds: 3
successThreshold: 1
timeoutSeconds: 1
name: saigonbros-web
ports:
- containerPort: 80
name: sgb-web-port
protocol: TCP
resources:
limits:
cpu: 500m
ephemeral-storage: 1Gi
memory: 2Gi
requests:
cpu: 500m
ephemeral-storage: 1Gi
memory: 2Gi
securityContext:
capabilities:
drop:
- NET_RAW
terminationMessagePath: /dev/termination-log
terminationMessagePolicy: File
volumeMounts:
- mountPath: /var/run/secrets/kubernetes.io/serviceaccount
name: kube-api-access-rkwb2
readOnly: true
dnsPolicy: ClusterFirst
enableServiceLinks: true
nodeName: gk3-autopilot-cluster-1-default-pool-43dd48b9-tf0n
preemptionPolicy: PreemptLowerPriority
priority: 0
readinessGates:
- conditionType: cloud.google.com/load-balancer-neg-ready
restartPolicy: Always
schedulerName: gke.io/optimize-utilization-scheduler
securityContext:
seccompProfile:
type: RuntimeDefault
serviceAccount: default
serviceAccountName: default
terminationGracePeriodSeconds: 30
tolerations:
- effect: NoExecute
key: node.kubernetes.io/not-ready
operator: Exists
tolerationSeconds: 300
- effect: NoExecute
key: node.kubernetes.io/unreachable
operator: Exists
tolerationSeconds: 300
volumes:
- name: kube-api-access-rkwb2
projected:
defaultMode: 420
sources:
- serviceAccountToken:
expirationSeconds: 3607
path: token
- configMap:
items:
- key: ca.crt
path: ca.crt
name: kube-root-ca.crt
- downwardAPI:
items:
- fieldRef:
apiVersion: v1
fieldPath: metadata.namespace
path: namespace
status:
conditions:
- lastProbeTime: null
lastTransitionTime: null
message: 'Pod is in NEG "Key{\"k8s1-301c19bd-default-sgb-web-master-80-48ae70f6\",
zone: \"asia-southeast1-a\"}". NEG is not attached to any BackendService with
health checking. Marking condition "cloud.google.com/load-balancer-neg-ready"
to True.'
reason: LoadBalancerNegWithoutHealthCheck
status: "True"
type: cloud.google.com/load-balancer-neg-ready
- lastProbeTime: null
lastTransitionTime: "2021-11-11T06:40:33Z"
status: "True"
type: Initialized
- lastProbeTime: null
lastTransitionTime: "2021-11-11T06:44:42Z"
message: 'containers with unready status: [saigonbros-web]'
reason: ContainersNotReady
status: "False"
type: Ready
- lastProbeTime: null
lastTransitionTime: "2021-11-11T06:44:42Z"
message: 'containers with unready status: [saigonbros-web]'
reason: ContainersNotReady
status: "False"
type: ContainersReady
- lastProbeTime: null
lastTransitionTime: "2021-11-11T06:40:33Z"
status: "True"
type: PodScheduled
containerStatuses:
- containerID: containerd://dfc32581c1edda1a221dc00cede918cfb93225e51e505ea7a9f935fc9ab893d5
image: docker.io/cflynnus/saigonbros-web:master-d70f3001d130bf986da236a08e1fded4b64e8097
imageID: docker.io/cflynnus/saigonbros-web#sha256:ff8d6d42511ed6520967007714dfbd46817fca06bb65ae984bc04a8b90346222
lastState:
terminated:
containerID: containerd://dfc32581c1edda1a221dc00cede918cfb93225e51e505ea7a9f935fc9ab893d5
exitCode: 0
finishedAt: "2021-11-11T06:44:41Z"
reason: Completed
startedAt: "2021-11-11T06:44:30Z"
name: saigonbros-web
ready: false
restartCount: 6
started: false
state:
waiting:
message: back-off 2m40s restarting failed container=saigonbros-web pod=sgb-web-master-fb9f995fb-zwhgl_default(ed80b0d0-6681-4c2a-8edd-16c8ef6bee86)
reason: CrashLoopBackOff
hostIP: 10.148.15.200
phase: Running
podIP: 10.15.128.103
podIPs:
- ip: 10.15.128.103
qosClass: Guaranteed
startTime: "2021-11-11T06:40:33Z"
Your liveness probe is configured as HTTPS on port 80. Just change it to HTTP. Look at the key spec.containers.livenessProbe.httpGet.scheme.
Kubernetes thinks that your pod isn't alive (bad liveness probe) and cause the SIGQUIT.
Normally this will help you. When your pod isn't alive, then Kubernetes tries to restart the app for you.
Edit
You can also identify that behavior in the logs of your nginx:
10.15.128.65 - - [11/Nov/2021:06:40:41 +0000] "\x16\x03\x01\x01\x00\x01\x00\x00\xFC\x03\x03>\x85O#\xCC\xB9\xA5j\xAB\x8D\xC1PpZ\x18$\xE5ah\xDF7\xB1\xFF\xAD\x22\x050\xC3.+\xB6+ \x0F}S)\xC9\x1F\x0BY\x15_\x10\xC6\xAAF\xAA\x9F\x9E_#dG\x01\xF5vzt\xB50&;\x1E\x15\x00&\xC0/\xC00\xC0+\xC0,\xCC\xA8\xCC\xA9\xC0\x13\xC0\x09\xC0\x14\xC0" 400 157 "-" "-" "-"
10.15.128.65 - - [11/Nov/2021:06:40:44 +0000] "\x16\x03\x01\x01\x00\x01\x00\x00\xFC\x03\x03\xD8['\xE75x'\xC3}+v\xC9\x83\x84\x96EKn\xC5\xB6}\xEE\xBE\xD9Gp\xE9\x1BX<n\xB2 \xD9n\xD1\xC5\xFC\xF2\x8D\x92\xAC\xC0\xA8mdF\x17B\xA3y9\xDD\x98b\x0E\x996\xB6\xA5\xAB\xEB\xD4\xDA" 400 157 "-" "-" "-"
10.15.128.65 - - [11/Nov/2021:06:40:47 +0000] "\x16\x03\x01\x01\x00\x01\x00\x00\xFC\x03\x03Fy\x03N\x0E\x11\x89k\x7F\xC5\x00\x90w}\xEB{\x7F\xB1=\xF0" 400 157 "-" "-" "-"
2021/11/11 06:40:47 [notice[] 1#1: signal 3 (SIGQUIT) received, shutting down
There are the three configured liveness probes with a period of three seconds. They are unreadable, because kubernetes send TLS packets (which are in a plain-view not human readable).
Immediately after that, there is the shutdown.
The other way is to read the description of your pod. There you can see, that HTTPS and port 80 are configured. HTTPS runs over port 443, so it must be a configuration error.

NGINX Reverse Proxy throws HTTP 503 on JIRA

I'm setting up a internal server for JIRA, Jenkins etc. For simplification i use an Reverse Proxy and docker-compose.
My docker-compose.yml has following configuration:
version: '3.7'
services:
proxy:
container_name: proxy
image: nginx:alpine
init: true
restart: unless-stopped
ports:
- 80:80
- 443:443
networks:
proxynet:
ipv4_address: 172.20.0.2
depends_on:
- jenkins
- nexus
volumes:
- ./reverse-proxy/data/app.conf:/etc/nginx/conf.d
- ./reverse-proxy/logs:/var/logs/nginx/
...
jira:
container_name: jira
image: cptactionhank/atlassian-jira:latest
init: true
restart: unless-stopped
networks:
proxynet:
ipv4_address: 172.20.0.5
volumes:
- ./jira/webapp:/var/atlassian/jira
- ./jira/logs:/opt/atlassian/jira/logs
environment:
- X_PROXY_PORT=8080
- X_PROXY_SCHEME="http"
volumes:
data:
networks:
proxynet:
driver: bridge
ipam:
config:
- subnet: 172.20.0.0/16
reverse-proxy/data/app.conf
server {
listen 80;
server_name 192.123.456.789;
...
location /jira {
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_pass http://172.20.0.5:8080/jira;
proxy_ssl_verify off;
proxy_read_timeout 90;
}
}
If i visit the 192.123.456.789/jira, i get an HTTP ERROR 503.
NGINX access log:
[04/Sep/2019:16:49:33 +0000] "GET /jira HTTP/1.1" 503 36 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/76.0.3809.132 Safari/537.36" "-"
JIRA startup log:
`sMMMMMMMMMMMMMM+
MMMMMMMMMMMMMM
:sdMMMMMMMMMMM
MMMMMM
`sMMMMMMMMMMMMMM+ MMMMMM
MMMMMMMMMMMMMM +MMMMM
:sMMMMMMMMMMM MMMMM
MMMMMM `UOJ
`sMMMMMMMMMMMMM+ MMMMMM
MMMMMMMMMMMMMM +MMMMM
:sdMMMMMMMMMM MMMMM
MMMMMM `UOJ
MMMMMM
+MMMMM
MMMMM
`UOJ
Atlassian Jira
Version : 8.3.3
If you encounter issues starting or stopping JIRA, please see the Troubleshooting guide at https://docs.atlassian.com/jira/jadm-docs-083/Troubleshooting+installation
Using JIRA_HOME: /var/atlassian/jira
Server startup logs are located in /opt/atlassian/jira/logs/catalina.out
04-Sep-2019 17:03:57.136 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Server version: Apache Tomcat/8.5.40
04-Sep-2019 17:03:57.141 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Server built: Apr 10 2019 14:31:19 UTC
04-Sep-2019 17:03:57.142 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Server number: 8.5.40.0
04-Sep-2019 17:03:57.142 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log OS Name: Linux
04-Sep-2019 17:03:57.142 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log OS Version: 4.15.0-54-generic
04-Sep-2019 17:03:57.143 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Architecture: amd64
04-Sep-2019 17:03:57.143 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Java Home: /usr/lib/jvm/java-1.8-openjdk/jre
04-Sep-2019 17:03:57.144 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log JVM Version: 1.8.0_212-b04
04-Sep-2019 17:03:57.144 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log JVM Vendor: IcedTea
04-Sep-2019 17:03:57.144 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log CATALINA_BASE: /opt/atlassian/jira
04-Sep-2019 17:03:57.145 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log CATALINA_HOME: /opt/atlassian/jira
04-Sep-2019 17:03:57.146 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: -Djava.util.logging.config.file=/opt/atlassian/jira/conf/logging.properties
04-Sep-2019 17:03:57.147 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: -Djava.util.logging.manager=org.apache.juli.ClassLoaderLogManager
04-Sep-2019 17:03:57.147 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: -Xms384m
04-Sep-2019 17:03:57.148 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: -Xmx2048m
04-Sep-2019 17:03:57.148 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: -XX:InitialCodeCacheSize=32m
04-Sep-2019 17:03:57.148 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: -XX:ReservedCodeCacheSize=512m
04-Sep-2019 17:03:57.149 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: -Djava.awt.headless=true
04-Sep-2019 17:03:57.149 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: -Datlassian.standalone=JIRA
04-Sep-2019 17:03:57.153 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: -Dorg.apache.jasper.runtime.BodyContentImpl.LIMIT_BUFFER=true
04-Sep-2019 17:03:57.154 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: -Dmail.mime.decodeparameters=true
04-Sep-2019 17:03:57.154 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: -Dorg.dom4j.factory=com.atlassian.core.xml.InterningDocumentFactory
04-Sep-2019 17:03:57.155 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: -XX:-OmitStackTraceInFastThrow
04-Sep-2019 17:03:57.155 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: -Djava.locale.providers=COMPAT
04-Sep-2019 17:03:57.155 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: -Djira.home=/var/atlassian/jira
04-Sep-2019 17:03:57.156 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: -Datlassian.plugins.startup.options=-fg
04-Sep-2019 17:03:57.156 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: -Djdk.tls.ephemeralDHKeySize=2048
04-Sep-2019 17:03:57.156 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: -Djava.protocol.handler.pkgs=org.apache.catalina.webresources
04-Sep-2019 17:03:57.157 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: -Dorg.apache.catalina.security.SecurityListener.UMASK=0027
04-Sep-2019 17:03:57.157 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: -Xloggc:/opt/atlassian/jira/logs/atlassian-jira-gc-%t.log
04-Sep-2019 17:03:57.158 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: -XX:+UseGCLogFileRotation
04-Sep-2019 17:03:57.158 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: -XX:NumberOfGCLogFiles=5
04-Sep-2019 17:03:57.158 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: -XX:GCLogFileSize=20M
04-Sep-2019 17:03:57.159 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: -XX:+PrintGCDetails
04-Sep-2019 17:03:57.159 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: -XX:+PrintGCDateStamps
04-Sep-2019 17:03:57.160 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: -XX:+PrintGCTimeStamps
04-Sep-2019 17:03:57.160 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: -XX:+PrintGCCause
04-Sep-2019 17:03:57.161 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: -Dignore.endorsed.dirs=
04-Sep-2019 17:03:57.161 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: -Dcatalina.base=/opt/atlassian/jira
04-Sep-2019 17:03:57.163 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: -Dcatalina.home=/opt/atlassian/jira
04-Sep-2019 17:03:57.163 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: -Djava.io.tmpdir=/opt/atlassian/jira/temp
04-Sep-2019 17:03:57.164 INFO [main] org.apache.catalina.core.AprLifecycleListener.lifecycleEvent The APR based Apache Tomcat Native library which allows optimal performance in production environments was not found on the java.library.path: [/usr/lib/jvm/java-1.8-openjdk/jre/lib/amd64/server:/usr/lib/jvm/java-1.8-openjdk/jre/lib/amd64:/usr/lib/jvm/java-1.8-openjdk/jre/../lib/amd64:/usr/java/packages/lib/amd64:/usr/lib64:/lib64:/lib:/usr/lib]
04-Sep-2019 17:03:57.505 INFO [main] org.apache.coyote.AbstractProtocol.init Initializing ProtocolHandler ["http-nio-8080"]
04-Sep-2019 17:03:57.555 INFO [main] org.apache.catalina.startup.Catalina.load Initialization processed in 1377 ms
04-Sep-2019 17:03:57.576 INFO [main] org.apache.catalina.core.StandardService.startInternal Starting service [Catalina]
04-Sep-2019 17:03:57.578 INFO [main] org.apache.catalina.core.StandardEngine.startInternal Starting Servlet Engine: Apache Tomcat/8.5.40
2019-09-04 17:04:05,574 localhost-startStop-1 INFO [c.a.jira.startup.JiraHomeStartupCheck] The jira.home directory '/var/atlassian/jira' is validated and locked for exclusive use by this instance.
2019-09-04 17:04:05,783 JIRA-Bootstrap INFO [c.a.jira.startup.JiraStartupLogger]
****************
JIRA starting...
****************
2019-09-04 17:04:05,923 JIRA-Bootstrap INFO [c.a.jira.startup.JiraStartupLogger]
___ Environment _____________________________
JIRA Build : 8.3.3#803004-sha1:4d4040e0714d65b7fffa4801569d014c0b16eaa9
Build Date : Wed Aug 28 00:00:00 GMT 2019
JIRA Installation Type : Standalone
Application Server : Apache Tomcat/8.5.40 - Servlet API 3.1
Java Version : 1.8.0_212 - IcedTea
Current Working Directory : /var/atlassian/jira
Maximum Allowable Memory : 1979MB
Total Memory : 371MB
Free Memory : 330MB
Used Memory : 41MB
Memory Pool: Code Cache : Code Cache: init = 33554432(32768K) used = 8746176(8541K) committed = 33554432(32768K) max = 536870912(524288K)
Memory Pool: Metaspace : Metaspace: init = 0(0K) used = 21664984(21157K) committed = 22331392(21808K) max = -1(-1K)
Memory Pool: Compressed Class Space : Compressed Class Space: init = 0(0K) used = 2323808(2269K) committed = 2490368(2432K) max = 1073741824(1048576K)
Memory Pool: Eden Space : Eden Space: init = 107479040(104960K) used = 22147168(21628K) committed = 107544576(105024K) max = 572653568(559232K)
Memory Pool: Survivor Space : Survivor Space: init = 13369344(13056K) used = 0(0K) committed = 13369344(13056K) max = 71565312(69888K)
Memory Pool: Tenured Gen : Tenured Gen: init = 268435456(262144K) used = 21798312(21287K) committed = 268435456(262144K) max = 1431699456(1398144K)
JVM Input Arguments : -Djava.util.logging.config.file=/opt/atlassian/jira/conf/logging.properties -Djava.util.logging.manager=org.apache.juli.ClassLoaderLogManager -Xms384m -Xmx2048m -XX:InitialCodeCacheSize=32m -XX:ReservedCodeCacheSize=512m -Djava.awt.headless=true -Datlassian.standalone=JIRA -Dorg.apache.jasper.runtime.BodyContentImpl.LIMIT_BUFFER=true -Dmail.mime.decodeparameters=true -Dorg.dom4j.factory=com.atlassian.core.xml.InterningDocumentFactory -XX:-OmitStackTraceInFastThrow -Djava.locale.providers=COMPAT -Djira.home=/var/atlassian/jira -Datlassian.plugins.startup.options=-fg -Djdk.tls.ephemeralDHKeySize=2048 -Djava.protocol.handler.pkgs=org.apache.catalina.webresources -Dorg.apache.catalina.security.SecurityListener.UMASK=0027 -Xloggc:/opt/atlassian/jira/logs/atlassian-jira-gc-%t.log -XX:+UseGCLogFileRotation -XX:NumberOfGCLogFiles=5 -XX:GCLogFileSize=20M -XX:+PrintGCDetails -XX:+PrintGCDateStamps -XX:+PrintGCTimeStamps -XX:+PrintGCCause -Dignore.endorsed.dirs= -Dcatalina.base=/opt/atlassian/jira -Dcatalina.home=/opt/atlassian/jira -Djava.io.tmpdir=/opt/atlassian/jira/temp
Java Compatibility Information : JIRA version = 8.3.3, Java Version = 1.8.0_212
___ Java System Properties _________________
atlassian.plugins.startup.options : -fg
atlassian.standalone : JIRA
awt.toolkit : sun.awt.X11.XToolkit
catalina.base : /opt/atlassian/jira
catalina.home : /opt/atlassian/jira
catalina.useNaming : true
common.loader : "${catalina.base}/lib",
"${catalina.base}/lib/*.jar",
"${catalina.home}/lib",
"${catalina.home}/lib/*.jar"
file.encoding : UTF-8
file.encoding.pkg : sun.io
ignore.endorsed.dirs :
java.awt.graphicsenv : sun.awt.X11GraphicsEnvironment
java.awt.headless : true
java.awt.printerjob : sun.print.PSPrinterJob
java.class.version : 52.0
java.home : /usr/lib/jvm/java-1.8-openjdk/jre
java.io.tmpdir : /opt/atlassian/jira/temp
java.locale.providers : COMPAT
java.naming.factory.initial : org.apache.naming.java.javaURLContextFactory
java.naming.factory.url.pkgs : org.apache.naming
java.protocol.handler.pkgs : org.apache.catalina.webresources
java.runtime.name : OpenJDK Runtime Environment
java.runtime.version : 1.8.0_212-b04
java.specification.name : Java Platform API Specification
java.specification.vendor : Oracle Corporation
java.specification.version : 1.8
java.util.concurrent.ForkJoinPool.common.threadFactory : org.apache.catalina.startup.SafeForkJoinWorkerThreadFactory
java.util.logging.config.file : /opt/atlassian/jira/conf/logging.properties
java.util.logging.manager : org.apache.juli.ClassLoaderLogManager
java.vendor : IcedTea
java.vendor.url : https://icedtea.classpath.org
java.vendor.url.bug : https://icedtea.classpath.org/bugzilla
java.version : 1.8.0_212
java.vm.info : mixed mode
java.vm.name : OpenJDK 64-Bit Server VM
java.vm.specification.name : Java Virtual Machine Specification
java.vm.specification.vendor : Oracle Corporation
java.vm.specification.version : 1.8
java.vm.vendor : IcedTea
java.vm.version : 25.212-b04
jdk.tls.ephemeralDHKeySize : 2048
jira.home : /var/atlassian/jira
mail.mime.decodeparameters : true
org.apache.catalina.security.SecurityListener.UMASK : 0027
org.apache.jasper.runtime.BodyContentImpl.LIMIT_BUFFER : true
org.dom4j.factory : com.atlassian.core.xml.InterningDocumentFactory
os.arch : amd64
os.name : Linux
os.version : 4.15.0-54-generic
package.access : sun.,
org.apache.catalina.,
org.apache.coyote.,
org.apache.jasper.,
org.apache.tomcat.
package.definition : sun.,
java.,
org.apache.catalina.,
org.apache.coyote.,
org.apache.jasper.,
org.apache.naming.,
org.apache.tomcat.
server.loader :
shared.loader :
sun.arch.data.model : 64
sun.boot.library.path : /usr/lib/jvm/java-1.8-openjdk/jre/lib/amd64
sun.cpu.endian : little
sun.cpu.isalist :
sun.io.unicode.encoding : UnicodeLittle
sun.java.command : org.apache.catalina.startup.Bootstrap start
sun.java.launcher : SUN_STANDARD
sun.jnu.encoding : UTF-8
sun.management.compiler : HotSpot 64-Bit Tiered Compilers
sun.os.patch.level : unknown
tomcat.util.buf.StringCache.byte.enabled : true
tomcat.util.scan.StandardJarScanFilter.jarsToScan : log4j-taglib*.jar,
log4j-web*.jar,
log4javascript*.jar,
slf4j-taglib*.jar
tomcat.util.scan.StandardJarScanFilter.jarsToSkip : annotations-api.jar,
ant-junit*.jar,
ant-launcher.jar,
ant.jar,
asm-*.jar,
aspectj*.jar,
bootstrap.jar,
catalina-ant.jar,
catalina-ha.jar,
catalina-jmx-remote.jar,
catalina-storeconfig.jar,
catalina-tribes.jar,
catalina-ws.jar,
catalina.jar,
cglib-*.jar,
cobertura-*.jar,
commons-beanutils*.jar,
commons-codec*.jar,
commons-collections*.jar,
commons-daemon.jar,
commons-dbcp*.jar,
commons-digester*.jar,
commons-fileupload*.jar,
commons-httpclient*.jar,
commons-io*.jar,
commons-lang*.jar,
commons-logging*.jar,
commons-math*.jar,
commons-pool*.jar,
dom4j-*.jar,
easymock-*.jar,
ecj-*.jar,
el-api.jar,
geronimo-spec-jaxrpc*.jar,
h2*.jar,
hamcrest-*.jar,
hibernate*.jar,
httpclient*.jar,
icu4j-*.jar,
jasper-el.jar,
jasper.jar,
jaspic-api.jar,
jaxb-*.jar,
jaxen-*.jar,
jdom-*.jar,
jetty-*.jar,
jmx-tools.jar,
jmx.jar,
jsp-api.jar,
jstl.jar,
jta*.jar,
junit-*.jar,
junit.jar,
log4j*.jar,
mail*.jar,
objenesis-*.jar,
oraclepki.jar,
oro-*.jar,
servlet-api-*.jar,
servlet-api.jar,
slf4j*.jar,
taglibs-standard-spec-*.jar,
tagsoup-*.jar,
tomcat-api.jar,
tomcat-coyote.jar,
tomcat-dbcp.jar,
tomcat-i18n-*.jar,
tomcat-jdbc.jar,
tomcat-jni.jar,
tomcat-juli-adapters.jar,
tomcat-juli.jar,
tomcat-util-scan.jar,
tomcat-util.jar,
tomcat-websocket.jar,
tools.jar,
websocket-api.jar,
wsdl4j*.jar,
xercesImpl.jar,
xml-apis.jar,
xmlParserAPIs-*.jar,
xmlParserAPIs.jar,
xom-*.jar
user.country : US
user.dir : /var/atlassian/jira
user.home : /sbin
user.language : en
user.name : daemon
user.timezone : GMT
2019-09-04 17:04:06,338 JIRA-Bootstrap INFO [c.a.j.config.database.SystemDatabaseConfigurationLoader] Reading database configuration from /var/atlassian/jira/dbconfig.xml
2019-09-04 17:04:06,391 JIRA-Bootstrap INFO [c.a.j.config.database.SystemDatabaseConfigurationLoader] Reading database configuration from /var/atlassian/jira/dbconfig.xml
2019-09-04 17:04:06,392 JIRA-Bootstrap INFO [c.a.j.config.database.SystemDatabaseConfigurationLoader] Reading database configuration from /var/atlassian/jira/dbconfig.xml
2019-09-04 17:04:08,155 JIRA-Bootstrap INFO [c.a.j.config.database.SystemDatabaseConfigurationLoader] Reading database configuration from /var/atlassian/jira/dbconfig.xml
2019-09-04 17:04:08,187 JIRA-Bootstrap INFO [c.a.jira.startup.JiraStartupLogger]
___ Starting the JIRA Plugin System _________________
2019-09-04 17:04:17,775 JIRA-Bootstrap INFO [c.a.j.config.database.SystemDatabaseConfigurationLoader] Reading database configuration from /var/atlassian/jira/dbconfig.xml
2019-09-04 17:04:17,931 JIRA-Bootstrap INFO [c.a.jira.startup.JiraStartupLogger] Running JIRA startup checks.
2019-09-04 17:04:17,933 JIRA-Bootstrap INFO [c.a.jira.startup.JiraStartupLogger] JIRA pre-database startup checks completed successfully.
2019-09-04 17:04:18,023 JIRA-Bootstrap INFO [c.a.j.config.database.DatabaseConfigurationManagerImpl] The database is not yet configured. Enqueuing Database Checklist Launcher on post-database-configured-but-pre-database-activated queue
2019-09-04 17:04:18,024 JIRA-Bootstrap INFO [c.a.j.config.database.DatabaseConfigurationManagerImpl] The database is not yet configured. Enqueuing Post database-configuration launchers on post-database-activated queue
2019-09-04 17:04:18,045 JIRA-Bootstrap INFO [c.a.jira.startup.LauncherContextListener] Memory Usage:
---------------------------------------------------------------------------------
Heap memory : Used: 73 MiB. Committed: 371 MiB. Max: 1980 MiB
Non-heap memory : Used: 68 MiB. Committed: 86 MiB. Max: 1536 MiB
---------------------------------------------------------------------------------
TOTAL : Used: 141 MiB. Committed: 458 MiB. Max: 3516 MiB
---------------------------------------------------------------------------------
04-Sep-2019 17:05:03.590 INFO [main] org.apache.coyote.AbstractProtocol.start Starting ProtocolHandler ["http-nio-8080"]
04-Sep-2019 17:05:03.618 INFO [main] org.apache.tomcat.util.net.NioSelectorPool.getSharedSelector Using a shared selector for servlet write/read
04-Sep-2019 17:05:03.701 INFO [main] org.apache.catalina.startup.Catalina.start Server startup in 66145 ms
I also started JIRA without the X_PROXY_* environment variables, but still no success.
I'm a bit confused because jira has started successfully, but nginx can't access it apparently. What am i missing?
I think i have found your problem. In the nginx config you have this line
location /jira {
...
Change it to
location ^~ /jira {
add_header Channel-id "JIRA";
...

Spring JMS and AWS SQS FIFO integration

I am facing a weird issue while integrating SQS FIFO with Spring JMS. This happens while listening to the message. I have posted only 1 message in the queue and the application keeps on poling the same message again and again. I can see that in the logs. But it never enters the block where the message is read and further operations are done on it.
The visibility time out of the SQS queue is 30 seconds.
#Component
public class SQSListener {
private Logger LOGGER = Logger.getLogger(SQSListener.class);
#Autowired
AmazonSQSAsync amazonSQS;
#JmsListener(destination = "<queue-name>", containerFactory="jmsListenerContainerFactory")
public void onMessage(Message message) {
try {
LOGGER.info("*****Received****" + ((TextMessage)message).getText());
} catch (JMSException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
}
}
Configuration Class:
#Configuration
#ComponentScan("com.ar")
#EnableJms
public class SQSListenerConfig {
private Logger LOGGER = Logger.getLogger(SQSListenerConfig.class);
#Value("${queue.endpoint}")
private String endpoint;
#Value("${queue.name}")
private String queueName;
#Value("${aws.access-key}")
private String accessKey;
#Value("${aws.secret-key}")
private String secretKey;
#Autowired
private SQSListener sqsListener;
private final AWSCredentialsProvider awsCredentialsProvider = new AWSCredentialsProvider() {
#Override
public AWSCredentials getCredentials() {
return new BasicAWSCredentials(accessKey, secretKey);
}
#Override
public void refresh() {
}
};
SQSConnectionFactory connectionFactory = SQSConnectionFactory.builder()
.withAWSCredentialsProvider(awsCredentialsProvider)
.withEndpoint(endpoint)
.withNumberOfMessagesToPrefetch(10).build();
#Bean
public DefaultJmsListenerContainerFactory jmsListenerContainerFactory() {
DefaultJmsListenerContainerFactory factory = new DefaultJmsListenerContainerFactory();
factory.setConnectionFactory(this.connectionFactory);
factory.setDestinationResolver(new DynamicDestinationResolver());
factory.setConcurrency("1-5");
//factory.setMaxMessagesPerTask(10);
//factory.setReceiveTimeout(1000000000L);
factory.setReceiveTimeout(30L);
factory.setSessionAcknowledgeMode(Session.CLIENT_ACKNOWLEDGE);
//factory.setMessageConverter(messageConverter());
return factory;
}
#Bean
public JmsTemplate createJMSTemplate() {
JmsTemplate jmsTemplate = new JmsTemplate(connectionFactory);
jmsTemplate.setDefaultDestinationName(queueName);
jmsTemplate.setDeliveryPersistent(false);
return jmsTemplate;
}
/*#Bean
public DefaultMessageListenerContainer jmsListenerContainer() {
DefaultMessageListenerContainer dmlc = new DefaultMessageListenerContainer();
dmlc.setConnectionFactory(connectionFactory);
dmlc.setDestinationName(queueName);
dmlc.setMessageListener(sqsListener);
return dmlc;
}
*/
#Bean(destroyMethod = "shutdown")
public AmazonSQSAsync amazonSQS() throws Exception {
AmazonSQSAsyncClientBuilder builder = AmazonSQSAsyncClientBuilder.standard();
builder.withCredentials(awsCredentialsProvider);
builder.withRegion(Regions.US_EAST_1);
return builder.build();
}
#Bean
public QueueMessagingTemplate queueMessagingTemplate(AmazonSQSAsync awsSqsClient) throws Exception {
return new QueueMessagingTemplate(awsSqsClient);
}
}
pom.xml
<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-aws-messaging</artifactId>
<version>1.2.2.RELEASE</version>
</dependency>
<dependency>
<groupId>com.amazonaws</groupId>
<artifactId>aws-java-sdk</artifactId>
<version>1.11.292</version>
</dependency>
<dependency>
<groupId>com.amazonaws</groupId>
<artifactId>amazon-sqs-java-messaging-lib</artifactId>
<version>1.0.4</version>
</dependency>
Here is the log. This message is being polled continuously but it never enters the listener method:
amz-sdk-invocation-id;amz-sdk-retry;host;user-agent;x-amz-date
0f481022d919c318ddbb4110196f30d58d35c7fca83b32af5f4db3d9c378e788"
2018-04-18 21:35:35.842 DEBUG 13324 --- [refetchThread-1] com.amazonaws.auth.AWS4Signer : AWS4 String to Sign: '"AWS4-HMAC-SHA256
20180419T013535Z
20180419/us-east-1/sqs/aws4_request
7daaadeb9fe5618925b5a0ab5ab17de7a0fedf77d45fd56984419e530fac3ef7"
2018-04-18 21:35:35.842 DEBUG 13324 --- [refetchThread-1] o.a.h.client.protocol.RequestAddCookies : CookieSpec selected: default
2018-04-18 21:35:35.842 DEBUG 13324 --- [refetchThread-1] o.a.h.client.protocol.RequestAuthCache : Auth cache not set in the context
2018-04-18 21:35:35.842 DEBUG 13324 --- [refetchThread-1] h.i.c.PoolingHttpClientConnectionManager : Connection request: [route: {s}->https://sqs.us-east-1.amazonaws.com:443][total kept alive: 1; route allocated: 1 of 50; total allocated: 1 of 50]
2018-04-18 21:35:35.842 DEBUG 13324 --- [refetchThread-1] h.i.c.PoolingHttpClientConnectionManager : Connection leased: [id: 1][route: {s}->https://sqs.us-east-1.amazonaws.com:443][total kept alive: 0; route allocated: 1 of 50; total allocated: 1 of 50]
2018-04-18 21:35:35.842 DEBUG 13324 --- [refetchThread-1] h.i.c.DefaultManagedHttpClientConnection : http-outgoing-1: set socket timeout to 50000
2018-04-18 21:35:35.842 DEBUG 13324 --- [refetchThread-1] o.a.http.impl.execchain.MainClientExec : Executing request POST /868651433674/aws-nonprod-alm-dev-sqs.fifo HTTP/1.1
2018-04-18 21:35:35.842 DEBUG 13324 --- [refetchThread-1] o.a.http.impl.execchain.MainClientExec : Proxy auth state: UNCHALLENGED
2018-04-18 21:35:35.842 DEBUG 13324 --- [refetchThread-1] org.apache.http.headers : http-outgoing-1 >> POST /868651433674/aws-nonprod-alm-dev-sqs.fifo HTTP/1.1
2018-04-18 21:35:35.842 DEBUG 13324 --- [refetchThread-1] org.apache.http.headers : http-outgoing-1 >> Host: sqs.us-east-1.amazonaws.com
2018-04-18 21:35:35.842 DEBUG 13324 --- [refetchThread-1] org.apache.http.headers : http-outgoing-1 >> Authorization: AWS4-HMAC-SHA256 Credential=AKIAJTHWNVZAZ2T7BW5A/ 20180419/us-east-1/sqs/aws4_request, SignedHeaders=amz-sdk-invocation-id;amz-sdk-retry;host;user-agent;x-amz-date, Signature=74503414f0a291dc4e1fc39c004748fea7565ed8d08d4d9ca810d93d8a550e9e
2018-04-18 21:35:35.842 DEBUG 13324 --- [refetchThread-1] org.apache.http.headers : http-outgoing-1 >> X-Amz-Date: 20180419T013535Z
2018-04-18 21:35:35.842 DEBUG 13324 --- [refetchThread-1] org.apache.http.headers : http-outgoing-1 >> User-Agent: aws-sdk-java/1.11.292 Windows_10/10.0 Java_HotSpot(TM)_64-Bit_Server_VM/25.171-b11 java/1.8.0_171 /SQS Java Messaging Client v1.0
2018-04-18 21:35:35.842 DEBUG 13324 --- [refetchThread-1] org.apache.http.headers : http-outgoing-1 >> amz-sdk-invocation-id: 2dd018d2-da6b-010a-6d41-44ac3b676ed4
2018-04-18 21:35:35.842 DEBUG 13324 --- [refetchThread-1] org.apache.http.headers : http-outgoing-1 >> amz-sdk-retry: 0/0/500
2018-04-18 21:35:35.842 DEBUG 13324 --- [refetchThread-1] org.apache.http.headers : http-outgoing-1 >> Content-Type: application/x-www-form-urlencoded; charset=utf-8
2018-04-18 21:35:35.842 DEBUG 13324 --- [refetchThread-1] org.apache.http.headers : http-outgoing-1 >> Content-Length: 190
2018-04-18 21:35:35.842 DEBUG 13324 --- [refetchThread-1] org.apache.http.headers : http-outgoing-1 >> Connection: Keep-Alive
2018-04-18 21:35:35.842 DEBUG 13324 --- [refetchThread-1] org.apache.http.wire : http-outgoing-1 >> "POST /868651433674/aws-nonprod-alm-dev-sqs.fifo HTTP/1.1[\r][\n]"
2018-04-18 21:35:35.842 DEBUG 13324 --- [refetchThread-1] org.apache.http.wire : http-outgoing-1 >> "Host: sqs.us-east-1.amazonaws.com[\r][\n]"
2018-04-18 21:35:35.842 DEBUG 13324 --- [refetchThread-1] org.apache.http.wire : http-outgoing-1 >> "Authorization: AWS4-HMAC-SHA256 Credential=AKIAJTHWNVZAZ2T7BW5A/ 20180419/us-east-1/sqs/aws4_request, SignedHeaders=amz-sdk-invocation-id;amz-sdk-retry;host;user-agent;x-amz-date, Signature=74503414f0a291dc4e1fc39c004748fea7565ed8d08d4d9ca810d93d8a550e9e[\r][\n]"
2018-04-18 21:35:35.842 DEBUG 13324 --- [refetchThread-1] org.apache.http.wire : http-outgoing-1 >> "X-Amz-Date: 20180419T013535Z[\r][\n]"
2018-04-18 21:35:35.842 DEBUG 13324 --- [refetchThread-1] org.apache.http.wire : http-outgoing-1 >> "User-Agent: aws-sdk-java/1.11.292 Windows_10/10.0 Java_HotSpot(TM)_64-Bit_Server_VM/25.171-b11 java/1.8.0_171 /SQS Java Messaging Client v1.0[\r][\n]"
2018-04-18 21:35:35.842 DEBUG 13324 --- [refetchThread-1] org.apache.http.wire : http-outgoing-1 >> "amz-sdk-invocation-id: 2dd018d2-da6b-010a-6d41-44ac3b676ed4[\r][\n]"
2018-04-18 21:35:35.842 DEBUG 13324 --- [refetchThread-1] org.apache.http.wire : http-outgoing-1 >> "amz-sdk-retry: 0/0/500[\r][\n]"
2018-04-18 21:35:35.843 DEBUG 13324 --- [refetchThread-1] org.apache.http.wire : http-outgoing-1 >> "Content-Type: application/x-www-form-urlencoded; charset=utf-8[\r][\n]"
2018-04-18 21:35:35.843 DEBUG 13324 --- [refetchThread-1] org.apache.http.wire : http-outgoing-1 >> "Content-Length: 190[\r][\n]"
2018-04-18 21:35:35.843 DEBUG 13324 --- [refetchThread-1] org.apache.http.wire : http-outgoing-1 >> "Connection: Keep-Alive[\r][\n]"
2018-04-18 21:35:35.843 DEBUG 13324 --- [refetchThread-1] org.apache.http.wire : http-outgoing-1 >> "[\r][\n]"
2018-04-18 21:35:35.843 DEBUG 13324 --- [refetchThread-1] org.apache.http.wire : http-outgoing-1 >> "Action=ReceiveMessage&Version=2012-11-05&AttributeName.1=All&MessageAttributeName.1=All&MaxNumberOfMessages=10&WaitTimeSeconds=20&ReceiveRequestAttemptId=cd020752-1db2-4385-ba93-3883ea946ab1"
2018-04-18 21:35:35.871 DEBUG 13324 --- [refetchThread-1] org.apache.http.wire : http-outgoing-1 << "HTTP/1.1 200 OK[\r][\n]"
2018-04-18 21:35:35.871 DEBUG 13324 --- [refetchThread-1] org.apache.http.wire : http-outgoing-1 << "Server: Server[\r][\n]"
2018-04-18 21:35:35.871 DEBUG 13324 --- [refetchThread-1] org.apache.http.wire : http-outgoing-1 << "Date: Thu, 19 Apr 2018 01:35:35 GMT[\r][\n]"
2018-04-18 21:35:35.871 DEBUG 13324 --- [refetchThread-1] org.apache.http.wire : http-outgoing-1 << "Content-Type: text/xml[\r][\n]"
2018-04-18 21:35:35.871 DEBUG 13324 --- [refetchThread-1] org.apache.http.wire : http-outgoing-1 << "Content-Length: 3710[\r][\n]"
2018-04-18 21:35:35.871 DEBUG 13324 --- [refetchThread-1] org.apache.http.wire : http-outgoing-1 << "Connection: keep-alive[\r][\n]"
2018-04-18 21:35:35.871 DEBUG 13324 --- [refetchThread-1] org.apache.http.wire : http-outgoing-1 << "x-amzn-RequestId: 7d9a5e2f-0896-533d-a67f-0d7efc3186c3[\r][\n]"
2018-04-18 21:35:35.871 DEBUG 13324 --- [refetchThread-1] org.apache.http.wire : http-outgoing-1 << "[\r][\n]"
2018-04-18 21:35:35.871 DEBUG 13324 --- [refetchThread-1] org.apache.http.wire : http-outgoing-1 << "<?xml version="1.0"?><ReceiveMessageResponse xmlns="http://queue.amazonaws.com/doc/2012-11-05/"><ReceiveMessageResult><Message><MessageId>2cfb5718-16ed-4e2b-ade3-4455889a0016</MessageId><ReceiptHandle>AQEBF5tzKo1hh4/+8molj3+yG20+toxkZq9wZ2RD3TwvwOpfggbs6nTGkgxNsLbyVbh/LJU8/+p6L8oNKkyJTSY7ku47vrMz4RxjgDurFKDUUZwEAAUvp2hSSMrTRVTlWUDpXK7M/YiJ5Mz3EwJqkJfl9zjKJibWHmAdlAtiaEFDapkLa27hSHHozjfmKAVaQU2C+5C1Z3p+cWA06fTQWrT1y80QGADeb0+utjCWQSJEEDOJaXoTC05Ps3Z2R3yq13rz76R79i4X4E0BAyfn2wFZZU1VBMkgaSPWYAO7cQD/5cA=</ReceiptHandle><MD5OfBody>2fbe3ea62f496e8a7b04d351e9305fa4</MD5OfBody><MD5OfMessageAttributes>aec5867692711b413b9dc98b5e238590</MD5OfMessageAttributes><Body><?xml version="1.0" encoding="UTF-8" standalone="yes"?>[\n]"
2018-04-18 21:35:35.871 DEBUG 13324 --- [refetchThread-1] org.apache.http.wire : http-outgoing-1 << "<Fulfillment>[\n]"
2018-04-18 21:35:35.871 DEBUG 13324 --- [refetchThread-1] org.apache.http.wire : http-outgoing-1 << " <Header>[\n]"
2018-04-18 21:35:35.871 DEBUG 13324 --- [refetchThread-1] org.apache.http.wire : http-outgoing-1 << " <GeneratedBy>test</GeneratedBy>[\n]"
2018-04-18 21:35:35.871 DEBUG 13324 --- [refetchThread-1] org.apache.http.wire : http-outgoing-1 << " <GenerationDate> 2018-04-18T21:20:30.095-04:00</GenerationDate>[\n]"
2018-04-18 21:35:35.871 DEBUG 13324 --- [refetchThread-1] org.apache.http.wire : http-outgoing-1 << " </Header>[\n]"
2018-04-18 21:35:35.871 DEBUG 13324 --- [refetchThread-1] org.apache.http.wire : http-outgoing-1 << " <Body>[\n]"
2018-04-18 21:35:35.871 DEBUG 13324 --- [refetchThread-1] org.apache.http.wire : http-outgoing-1 << " <Operations>[\n]"
2018-04-18 21:35:35.871 DEBUG 13324 --- [refetchThread-1] org.apache.http.wire : http-outgoing-1 << " <CreateUser core-id="430ed0fc-9745-4d60-a45d-4fbc7735842c">[\n]"
2018-04-18 21:35:35.871 DEBUG 13324 --- [refetchThread-1] org.apache.http.wire : http-outgoing-1 << " <Active>ACTIVE</Active>[\n]"
2018-04-18 21:35:35.871 DEBUG 13324 --- [refetchThread-1] org.apache.http.wire : http-outgoing-1 << " </CreateUser>[\n]"
2018-04-18 21:35:35.871 DEBUG 13324 --- [refetchThread-1] org.apache.http.wire : http-outgoing-1 << " </Operations>[\n]"
2018-04-18 21:35:35.871 DEBUG 13324 --- [refetchThread-1] org.apache.http.wire : http-outgoing-1 << " </Body>[\n]"
2018-04-18 21:35:35.871 DEBUG 13324 --- [refetchThread-1] org.apache.http.wire : http-outgoing-1 << "</Fulfillment>[\n]"
2018-04-18 21:35:35.871 DEBUG 13324 --- [refetchThread-1] org.apache.http.wire : http-outgoing-1 << "</Body><Attribute><Name>SenderId</Name><Value>AIDAJEEIWD3U2UWQICOBO</Value></Attribute><Attribute><Name>ApproximateFirstReceiveTimestamp</Name><Value>1524100831661</Value></Attribute><Attribute><Name>ApproximateReceiveCount</Name><Value>311</Value></Attribute><Attribute><Name>SentTimestamp</Name><Value>1524100831661</Value></Attribute><Attribute><Name>SequenceNumber</Name><Value>18836913886614767616</Value></Attribute><Attribute><Name>MessageDeduplicationId</Name><Value>2bdaaa01-002b-4daa-9493-761e76101324</Value></Attribute><Attribute><Name>MessageGroupId</Name><Value>user-api-group-id</Value></Attribute><MessageAttribute><Name>MessageGroupId</Name><Value><StringValue>user-api-group-id</StringValue><DataType>String</DataType></Value></MessageAttribute><MessageAttribute><Name>contentType</Name><Value><StringValue>text/plain;charset=UTF-8</StringValue><DataType>String</DataType></Value></MessageAttribute><MessageAttribute><Name>id</Name><Value><StringValue>1672b5dd-3ca5-9128-9ed8-e42d3d9f9714</StringValue><DataType>String</DataType></Value></MessageAttribute><MessageAttribute><Name>timestamp</Name><Value><StringValue>1524100831625</StringValue><DataType>Number.java.lang.Long</DataType></Value></MessageAttribute></Message></ReceiveMessageResult><ResponseMetadata><RequestId>7d9a5e2f-0896-533d-a67f-0d7efc3186c3</RequestId></ResponseMetadata></ReceiveMessageResponse>"
2018-04-18 21:35:35.871 DEBUG 13324 --- [refetchThread-1] org.apache.http.headers : http-outgoing-1 << HTTP/1.1 200 OK
2018-04-18 21:35:35.871 DEBUG 13324 --- [refetchThread-1] org.apache.http.headers : http-outgoing-1 << Server: Server
2018-04-18 21:35:35.871 DEBUG 13324 --- [refetchThread-1] org.apache.http.headers : http-outgoing-1 << Date: Thu, 19 Apr 2018 01:35:35 GMT
2018-04-18 21:35:35.871 DEBUG 13324 --- [refetchThread-1] org.apache.http.headers : http-outgoing-1 << Content-Type: text/xml
2018-04-18 21:35:35.871 DEBUG 13324 --- [refetchThread-1] org.apache.http.headers : http-outgoing-1 << Content-Length: 3710
2018-04-18 21:35:35.871 DEBUG 13324 --- [refetchThread-1] org.apache.http.headers : http-outgoing-1 << Connection: keep-alive
2018-04-18 21:35:35.871 DEBUG 13324 --- [refetchThread-1] org.apache.http.headers : http-outgoing-1 << x-amzn-RequestId: 7d9a5e2f-0896-533d-a67f-0d7efc3186c3
2018-04-18 21:35:35.871 DEBUG 13324 --- [refetchThread-1] o.a.http.impl.execchain.MainClientExec : Connection can be kept alive for 60000 MILLISECONDS
2018-04-18 21:35:35.872 DEBUG 13324 --- [refetchThread-1] h.i.c.PoolingHttpClientConnectionManager : Connection [id: 1][route: {s}->https://sqs.us-east-1.amazonaws.com:443] can be kept alive for 60.0 seconds
2018-04-18 21:35:35.873 DEBUG 13324 --- [refetchThread-1] h.i.c.PoolingHttpClientConnectionManager : Connection released: [id: 1][route: {s}->https://sqs.us-east-1.amazonaws.com:443][total kept alive: 1; route allocated: 1 of 50; total allocated: 1 of 50]
2018-04-18 21:35:35.873 DEBUG 13324 --- [refetchThread-1] com.amazonaws.request : Received successful response: 200, AWS Request ID: 7d9a5e2f-0896-533d-a67f-0d7efc3186c3
2018-04-18 21:35:35.873 DEBUG 13324 --- [refetchThread-1] com.amazonaws.requestId : x-amzn-RequestId: 7d9a5e2f-0896-533d-a67f-0d7efc3186c3
What am I doing wrong here?
Once you receive a message you need to send the acknowledge about the message as you are using Session.CLIENT_ACKNOWLEDGE for acknowledgment of the message.
Just adding message.acknowledge(); after using the message it should delete from the queue.

Resources