Test AWS SQS End Point - amazon-sqs

I have an AWS SQS queue, and for some reason my connections to the the end point are being refused.
If I try:
curl -Is https://sqs.[REDACTED].amazonaws.com/[REDACTED]/[REDACTED]| head -1
I get "HTTP/1.1 404 Not Found" error. I suspect that this is the root cause of the issue I am having with being able to access the queue with an external application. However, I have the conditions set really broadly:
How do I know the issue I am experiencing is not a permissions issue? Is there a way for me to test the queue is actually responsive?
If so, is there a way in CURL for me to get any or all data from a message to verify I am receiving content, and then build my application around my response?

You can use the following curl request:
curl -i https://sqs.{region}.amazonaws.com/{account}/{name}/\?Action=ReceiveMessage
If you want to use long-polling:
while true; do
curl -i https://sqs.{region}.amazonaws.com/{account}/{name}/\?Action=ReceiveMessage\&WaitTimeSeconds\=20
done
See the ReceiveMessage SQS API docs for all supported parameters.
Another alternative is to use the AWS CLI instead, which also works with authentication:
aws sqs receive-message --queue-url https://sqs.{region}.amazonaws.com/{account}/{name}

Related

WSO2 MI Infinite loop on invalid request line

I run a very simple micro integrator service that only has 1 proxy service and a single sequence. In this sequence the incoming XML message is transferred to amazon SQS service.
If I run this in the Integration Studio on the instance that comes built in I have no problems. However, when I package the file into a CAR and feed it to the docker instance it will boot up and instantly gets bombarded with requests? That is to say, the following logs take over and the container can no longer be manually stopped:
[2020-04-15 12:45:44,585] INFO
{org.apache.synapse.transport.passthru.SourceHandler} - Writer null
when calling informWriterError ^[[?62;c^[[?62;c[2020-04-15
12:45:46,589] ERROR
{org.apache.synapse.transport.passthru.SourceHandler} - HttpException
occurred org.apache.http.ProtocolException: Invalid request line:
ÇÃ^ú§ß¡ðO©%åË*29xÙVÀ$À(=À&À*kjÀ at
org.apache.http.impl.nio.codecs.AbstractMessageParser.parse(AbstractMessageParser.java:208)
at
org.apache.synapse.transport.http.conn.LoggingNHttpServerConnection$LoggingNHttpMessageParser.parse(LoggingNHttpServerConnection.java:407)
at
org.apache.synapse.transport.http.conn.LoggingNHttpServerConnection$LoggingNHttpMessageParser.parse(LoggingNHttpServerConnection.java:381)
at
org.apache.http.impl.nio.DefaultNHttpServerConnection.consumeInput(DefaultNHttpServerConnection.java:265)
at
org.apache.synapse.transport.http.conn.LoggingNHttpServerConnection.consumeInput(LoggingNHttpServerConnection.java:114)
at
org.apache.synapse.transport.passthru.ServerIODispatch.onInputReady(ServerIODispatch.java:82)
at
org.apache.synapse.transport.passthru.ServerIODispatch.onInputReady(ServerIODispatch.java:39)
at
org.apache.http.impl.nio.reactor.AbstractIODispatch.inputReady(AbstractIODispatch.java:113)
at
org.apache.http.impl.nio.reactor.BaseIOReactor.readable(BaseIOReactor.java:159)
at
org.apache.http.impl.nio.reactor.AbstractIOReactor.processEvent(AbstractIOReactor.java:338)
at
org.apache.http.impl.nio.reactor.AbstractIOReactor.processEvents(AbstractIOReactor.java:316)
at
org.apache.http.impl.nio.reactor.AbstractIOReactor.execute(AbstractIOReactor.java:277)
at
org.apache.http.impl.nio.reactor.BaseIOReactor.execute(BaseIOReactor.java:105)
at
org.apache.http.impl.nio.reactor.AbstractMultiworkerIOReactor$Worker.run(AbstractMultiworkerIOReactor.java:586)
at java.lang.Thread.run(Thread.java:748) Caused by:
org.apache.http.ParseException: Invalid request line:
ÇÃ^þvHÅFmÉ
(#ë¸'º¯æ¦V
I made sure there were no outside connections possible and I also found the older threads of someone describing this problem, but their solution (changing something in the keystore) did not work.
Also, I made sure to include the SQS certificate in the container as well.
I have no connections setup to connect to the container so that will be out of the equation as well.
What am I missing here?
I have no idea why, but I have identified the culprit to be none other than Portainer. When I shutdown Portainer the stream of requests stops.
According to Wireshark, the requests are all made towards
GET
http://172.17.0.1:9000/api/endpoints/< containerID >/docker/< someId >/logs
It seems that because the WSO2 container I'm trying to run is an ESB that uses endpoints and returns 400 status codes on non-existing endpoints portainer will retry until it succeeds. This is just my observation, so I could be wrong.
I have confirmed my findings by uploading my container to AWS where the problem did not exist.

Error in triggering jenkins job from Slack

I am integrating Jenkins in Slack to facilitate build from Slack.
I configured the Build Trigger section of my job
Then I added Slash Commands to slack and configured it as below.
Now, I went to slack and typed /dev-backend, and it is throwing the error,
/dev-backend failed with the error "invalid_url"
I tried hitting the URL directly from postman which is giving the Authentication required response.
How can I add the authentication so that I can trigger build from slack? I thought the token provided in the build triggers section of Jenkins was enough.
I know it's kind of outdated question but I was also struggling with this error, so maybe it will help someone in the future. If you are getting /command failed with the error "invalid_url" and you are able to hit your application endpoint via eg. Bash shell:
curl -X POST -H 'Authorization: Bearer your-bearer-token' -H 'Content-type: application/json;charset=utf-8' --data '{"channel":"#test","text":"Hello, Slack!"}' http://127.0.0.1:8000/your/application/endpoint
it means that Slack can't access/see the URL that you passed in Slack API Applications Settings.
In my case the problem was that I was passing http://127.0.0.1:8000/some/endpoint in Slash Commands configuration which was visible only by my computer. The easiest way to make it visible for Slack is to use ngrok.
So if you are getting this error the steps are as follows:
Install ngrok
Run your app on localhost, eg. http://127.0.0.1:8000
In Bash shell type ngrok http 8000
Copy url returned by ngrok, in my case http://c609-91-234-49-229.ngrok.io
Paste copied URL to Slack Slash Commands configuration instead of localhost.

Jenkins doesn't update GitHub check status sometimes

I'm using Jenkins 2.15 (GitHub plugin 1.29.3) based CI for my GitHub core repo. It works fine, but sometimes Jenkins build doesn't update GitHub check status.
I see nothing relevant into Jenkins log.
Any idea how to debug and hopefully fix this issue?
As I know, check status update is just an http request to the status api: https://developer.github.com/v3/repos/statuses/
I experienced a similar behavior with a database. The client application and the database had no errors. Each one was on a different host.
What I did was, create a bash script in host A to perform a ping to host B.
ping www.host_B.com | while read pong; do echo "$(date): $pong"; done >> /tmp/ping-test-$(date +%F).log
Then, when the sporadic error related to the connection of the database occurred, the log file helped me to detect that the error was related to:
Network issues
Latency issues
Internet service provider issues
In your case, you could perform a simple curl to the status api and compare to the sporadic behavior detected.

Hyperledger Composer Multiple user mode for REST server throws 401 error

I have been following the tutorials which are provided in Hyperledger Composer docs, but I am not getting the results that they are supposed to I should be getting. Specifically, when I try to enable the multiple user mode for the REST server and I try to call one of the business network REST API operations using the REST API explorer I always get a HTTP 401 Authorization Required. According to the Tutorial, I if get this error is due to I have not authenticated correctly to the REST API, but it does not mention why this error occurs or how I may fix it.
It is very important for application development to be able to authenticate each user who wants to make requests to the API.
What version of Hyperledger Composer are you using?
The tutorial/document you refer to is correct for v0.15.0 and works a little differently for prior versions.
Are you seeing an Access token at the top of the Browser Window - this indicates that you have successfully authenticated and can continue with the steps for the Wallet.
If you are not seeing an Access token displayed, then make sure you hit your REST server with a URL similar to http://localhost:3000/auth/github again and login.
If you are still experiencing problems I would suggest going back to just using authentication without multiuser mode and verify that the authentication works properly from there.
After some research, I found a solution and worked for me.
If you already enable Github authentication then ignore. Otherwise first enable authentication following this tutorial Enaling Authentication.
Before start rest server you will export your admin card from the network by using this command:
composer card export -n admin#sample-network -f admincard.card
Now start rest server with authentication using this command:
composer-rest-server -c admin#sample-network -p 3000 -a true -m true
After some time rest server will start.
Now First, go this link for authentication: http://localhost:3000/auth/github
After successful authentication, you will get an access token and also you will see a Wallet options below.
Now you need to import a card that you already export from your network.
That's it, you can able to add anything to your network.
In my case, I have two missed steps:
enabling authentication for the REST server
https://hyperledger.github.io/composer/v0.19/integrating/enabling-rest-authentication.html
composer-rest-server -c admin#you_project -a true, I just execute composer-rest-server but did not specify the identity "admin#you_project" before.

Faye: Can't send GET messages in browser even though socket is open, working and responding with data via CURL

So my websocket is opening properly to faye, I'm using the nginx_tcp_proxy module. When I run a curl it looks good:
$ curl http://now.2u.fm:9200/faye\?message\=%5B%7B%22channel%22%3A%22%2Fmeta%2Fhandshake%22%2C%22version%22%3A%221.0%22%2C%22supportedConnectionTypes%22%3A%5B%22callback-polling%22%5D%2C%22id%22%3A%221%22%7D%5D\&jsonp\=__jsonp6__
__jsonp6__([{"id":"1","channel":"/meta/handshake","successful":true,"version":"1.0","supportedConnectionTypes":["long-polling","cross-origin-long-polling","callback-polling","websocket","eventsource","in-process"],"clientId":"jls0srprht51xb368yrojft3h4drgu0","advice":{"reconnect":"retry","interval":0,"timeout":45000}}]);#
And curl with the -I flag
HTTP/1.1 200 OK
...
Connection: close
But I'm getting failed GET requests when my website tries to hit this as GET (no error code):
edit: I noticed it says "switching protocols" just now!
Same as when I try and hit the url directly in my browser.
My gut says, "hey, thats because you have a tcp connection open not an http one!", but then is private_pub using GET? But for all I know a GET request is just fine over TCP and I'm doing something wrong.
I found out the issue was that Faye needs to be able to open a socket and use http. Because private_pub assumes you do both on the same domain/port this was impossible using the tcp module without having to modify private_pub and the faye extension it was loading to use the right ports.
In the end I used HAProxy instead as it was much simpler to set up and required no modification of private pub.

Resources