Http POST request taking lot of time to complete - net/http - Go - post

I am using net/http in Go, and one of my HTTPS POST request is taking lot of time like approx 16 seconds to complete . Though the backend server processing is only taking 2.2 seconds but over all process (when the response is received from the REST API after full processing ) is like 16 seconds .
My call looks like :
https://xxxxx.com/abcded
And the body has a JSON which contains a GIF file base64 of size approx 1.68 MB
{
"username": "test",
"update_username": "0",
"name": "test name",
"isprivate": "0",
"avatar_url": "",
"description": "I am Test , updated using api ",
"location": "test world",
"phone": "12345678",
"age": "23",
"gender": "m",
"update_profilepic": "1",
"profilepic_lat": "28.593",
"profilepic_lon": "77.1981",
"profilepic_height": "480",
"profilepic_width": "480",
"profilepic_type": "photo",
"profilepic": "Dummy String due to Size Constraint in StackOverFlow"
}
Details :
The POST request was invoked from the client (Postman) at 16:59:00.100 But the server recieves the call at 16:59:13.240 (which is shown in the Server logs below) . And finally the server does all the processing and responds back at 16:59:15.073 . And finally I recieved the response code at the client (Postman) at 16:59:15.773
Server Logs Start *****************************************
16:59:13.240
Just Entered into AUTHENTICATION method
16:59:13.246
--------------In Profile Related--------------
16:59:13.246
--------------In Edit Setting--------------
16:59:13.363
---------- In Edit Setting --------
16:59:13.363
Username direct rcvd :%!(EXTRA string=morninglad)
16:59:13.363
Username rcvd :%!(EXTRA string=morninglad)
16:59:13.363
Name rcvd :%!(EXTRA string=morning)
16:59:13.363
Update NON USER NAME UPDATE FLAG RCVD
16:59:13.363
About to call Update Query
16:59:13.374
About to Update the Profile Pic as found the flag
16:59:13.374
Updating the Profile Pic------------
16:59:14.906
About to Insert Media into DB
16:59:14.906
Got the GeoHash :%!(EXTRA string=ttnc9j3tgujw)
16:59:14.952
Found location :%!(EXTRA string=Gurugram)
16:59:14.955
Media Play value is :%!(EXTRA string=)
16:59:14.955
Username :%!(EXTRA string=morninglad)
16:59:14.983
Updating the Profile Pic Done ------------
16:59:14.983
Profile Pic Updated !
16:59:15.041
-------------------Fetch Avatars-----------------
16:59:15.073
The Avatar URL got :Avatar1
16:59:15.073
The Avatar URL got : Avatar2
16:59:15.073
---------- In Edit Setting DONE - END --------
16:59:15.073
--------------Response SENT--------------
Server Logs End *****************************************
So if I go by the above mentioned timestamps ,then time to reach to server was almost 13 seconds , and the total processing time of server was about 2 seconds . And finally recieved the response code in within 1 second.So total time is like 16 seconds for this call.
So I was trying to figure out why the call to reach to server is taking 13 seconds ?
When I am invoking the same call with the smaller body i.e instead of GIF of 1.68 MB to JPEG of 1.4 KB the total time is like 2.2 seconds . So shall I assume that this body size is the reason for increased time ?
But with the kind of internet I am working on (Download speed - 72.33 Mbps and Upload Speed - 6.07 Mbps)this sounds weird .
Any pointer if I am doing something wrong over here ?

Related

Rails - multiple theads to avoid the slack 3 second API response rule

I am working with the slack API. My script does a bunch of external processing and in some cases it can take around 3-6 seconds. What is happening is the Slack API expects a 200 response within 3 seconds and because my function is not finished within 3 seconds, it retries again and then it ends up posting the same automated responses 2-3 times.
I confirmed this by commenting out all the functions and I had no issue, it posted the responses to slack fine. I then added sleep 10 and it done the same responses 3 times so the ohly thing different was it took longer.
From what I read, I need to have threaded responses. I then need to first respond to the slack API in thread 1 and then go about processing my functions.
Here is what I tried:
def events
Thread.new do
json = {
"text": "Here is your 200 response immediately slack",
}
render(json: json)
end
puts "--------------------------------Json response started----------------------"
sleep 30
puts "--------------------------------Json response completed----------------------"
puts "this is a successful response"
end
When I tested it the same issue happened so I tried using an online API tester and it hits the page, waits 30 seconds and then returns the 200 response but I need it to respond immediately with the 200, THEN process the rest otherwise I will get duplicates.
Am I using threads properly or is there another way to get around this Slack API 3 second response limit? I am new to both rails and slack API so a bit lost here.
Appreciate the eyes :)
I would recommend using ActionJob to run the code in the background if you don't need to use the result of the code in the response. First, create an ActiveJob job by running:
bin/rails generate job do_stuff
And then open up the file created in app/jobs/do_stuff_job.rb and edit the #perform function to include your code (so the puts statements and sleep 30 in your example). Finally, from the controller action you can call DoStuff.perform_later and your job will run in the background! Your final controller action will look something like this:
def events
DoStuff.perform_later # this schedules DoStuff to be done later, in
# the background, so it will return immediately
# and continue to the next line.
json = {
"text": "Here is your 200 response immediately slack",
}
render(json: json)
end
As an aside, I'd highly recommend never using Thread.new in rails. It can create some really confusing behavior especially in test scripts for a number of reasons, but usually because of how it interacts with open connections and specifically ActiveRecord.

My rails app is mixing up api requests and responses from different threads when called at the same time

I am working on a Rails app that has a form that makes 1st party API requests to Odoo v12, where I have all of my data stored. When a user submits 2 forms simultaneously, if there's any delay in the API response (as few as .2 seconds), the server begins work on the other request, which is expected.
However, when the responses come back in the order they were made, rails assign the response to the request with an open connection. (For e.g. response 1 is being assigned to request 2, and response 2 is being assigned to request 1.)
Inspecting the request_ids confirms that this is what is happening.
Example of created record IDs returning to the wrong request
res.partner create
Starting api call for res.partner on request
63f02dd4-8242-4043-9926-1e34133be2e1 at 1576616788523
[{:odoo_po=>"7643153000",
:quickbooks_invoice_url=>"", :deal_name=>"Delete ME PLEASE",
:deal_id=>"427759097", :reorder=>0, :company=>333, :sale_contact=>337,
:submitted_by=>98, :approver=>119, :hubspot_id=>"427759097",
:shipping_deadline=>"2020-01-07", :delivery_deadline=>"2020-01-13",
:knitting_deadline=>"2020-01-03", :internal_notes=>"",
"distributor_po"=>"14215",
"kit_arrays"=>"1,85996,15578;2,15578,85996;"}]
sale_form.sale_form create
Starting api call for sale_form.sale_form on request
673fa655-907a-416f-8ed9-7c97d220fc0b at 1576616788650
18805 _
673fa655-907a-416f-8ed9-7c97d220fc0b
51 ms api call for
sale_form.sale_form on request 673fa655-907a-416f-8ed9-7c97d220fc0b
9067 _ 63f02dd4-8242-4043-9926-1e34133be2e1
680 ms api call for res.partner
on request 63f02dd4-8242-4043-9926-1e34133be2e1
In the code above, the '9067' should be record_id associated with the sale_form.sale_form request(request_id 673fa655-907a-416f-8ed9-7c97d220fc0b), while the '18805' should be the record_id associated with the res.partner request(request_id 63f02dd4-8242-4043-9926-1e34133be2e1)
The code runs fine when forms are submitted more than a few seconds apart so I don't think it is an issue with the code I've written. Is this an issue with Rails/Puma/some other Gem? Or should I shift my focus towards how Odoo is handling the requests/responses?

Can Firebase Realtime Database be an WebHook Endpoint?

My Firebase database is structured to successfully save data from HTTP POST requests, as confirmed by sending test requests directly from my Angular development app:
CreatePost(){
let body = {
"host_id" : 1,
"uuid" : 1,
"id" : 1,
"status" : "ENDED"
}
this.http.post( this.url, body )
Firebase database update result:
-zoom
-L1dZbBWMTIr7ojjodzg
host_id: 1
id: 1
status: "ENDED"
uuid: 1
However, with the same database URL used as WebHook endpoint for a 3rd party service (Zoom conferencing), the database is not updating in response to WebHook triggering events, such as starting a meeting.
The Zoom WebHook POST is being sent correctly, as confirmed by a test to RequestBin:
FORM/POST PARAMETERS
host_id: w_1a9RDvTKqiG_BBdV7kuw
status: STARTED
id: 3544245503
uuid: oJ+nrTm7Rwq1NYlpML7W/Q==
Raw Body:
id=3544245503&uuid=X%2F1R2AC1QS%2Btjuhxc0Kt%2Bw%3D%3D&
host_id=w_1a9RDvTKqiG_BBdV7kuw&status=STARTED
Has anyone had experience with using Firebase for their WebHook endpoint? Does the WebHook POST need to be configured in a particular way for Firebase?
To POST (or PUT) data to the Firebase Database in a REST request, the body of your request must contain the JSON object to be written. From the Firebase documentation:
curl -X POST -d '{
"author": "alanisawesome",
"title": "The Turing Machine"
}' 'https://docs-examples.firebaseio.com/rest/saving-data/fireblog/posts.json'
Your code is posting the form fields in the body as url-encoded, which is simply a different format. See this answer for a comparison of the same data in both formats: https://stackoverflow.com/a/9880122.
If Zoom doesn't have support for posting the form fields as JSON, you could consider creating a Cloud Function to do the conversion.

Only I can see my output from custom slash commands

I created a custom slash command in Slack. The backend code, not that it's important, is a Lambda function in Python in AWS.
My problem is that when I enter the slash command, I am the only one who can see the message. Otherwise, it works perfectly. Is there a way to get others to see the output from my custom slash command?
See "'In Channel' vs. 'Ephemeral' responses" here: https://api.slack.com/slash-commands#responding_to_a_command.
By default, the response messages sent to commands will only be
visible to the user that issued the command (we call these "ephemeral"
messages). However, if you would like the response to be visible to
all members of the channel in which the user typed the command, you
can add a response_type of in_channel to the JSON response, like this:
{
"response_type": "in_channel",
"text": "It's 80 degrees right now.",
"attachments": [
{
"text":"Partly cloudy today and tomorrow"
}
]
}
When the response_type is in_channel, both the response message and the initial message typed by the user will be shared in
the channel.
If you have a block in your JSON payload to slack (you used slacks block-kit) e.g
`"blocks": []`
you'll have to put the "response_type": "in_channel" above blocks for it to work :) e.g
{
"response_type": "in_channel",
"blocks": [....]
}

I'm getting troubles while uploading to Youtube with google java client: "Invalid value for: listed is not a valid value"

I'm trying to upload videos using https://code.google.com/p/youtube-api-samples/source/browse/samples/java/youtube-cmdline-uploadvideo-sample/src/main/java/com/google/api/services/samples/youtube/cmdline/youtube_cmdline_uploadvideo_sample/UploadVideo.java example
I've successfully registered on the youtube console and have obtained the client_secrets.json at https://code.google.com/apis/console/?api=youtube#project:340964720118:access
I have installed the secrets so my app can access them; everything is compiled well, but when the program is executed, I'me getting that strange errors:
GoogleJsonResponseException code: 400 : Invalid value for: listed is not a valid value
com.google.api.client.googleapis.json.GoogleJsonResponseException: 400 Bad Request
{
"code" : 400,
"errors" : [ {
"domain" : "global",
"message" : "Invalid value for: listed is not a valid value",
"reason" : "invalid"
} ],
"message" : "Invalid value for: listed is not a valid value"
}
at com.google.api.client.googleapis.json.GoogleJsonResponseException.from(GoogleJsonResponseException.java:145)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:113)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:40)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:423)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:343)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.execute(AbstractGoogleClientRequest.java:460)
I can't find out what is wrong; since the request itself is deeply encapsulated, I can't understand what is wrong with the request...
UPD.
Thanks to
System.out.println(videoInsert.getHttpContent().toString());
call inserted before the
Video returnedVideo = videoInsert.execute();
in the sample code, I got that the JSOn formed and sent to the google is
{snippet={description=Video uploaded via YouTube Data API V3 using the Java library on Sun Sep 29 20:29:30 MSK 2013, tags=[test], title=Test Upload via Java on Sun Sep 29 20:29:30 MSK 2013}, status={privacyStatus=listed}}
But actually there is still a question what is wong with this json and why it causes the error on the the server side.
UPD 2:
And no I no longer get the message above, but the upload is still impossible. I'm getting 401 Error while trying to upload the file in that lines of the example above:
System.out.println(videoInsert.getJsonContent().toString());
System.out.println(videoInsert.getHttpContent().getType());
Video returnedVideo = videoInsert.execute();
System.out.println(returnedVideo.getId());
I supposed that I haven't authorized the request to my personal data, but I actually DID it (the request was loaded to the default browser where I have accepted it and specified the account I would like to work with). However, there is no way to check that authoriation; my google account data holds only Apps and Activity log for the google+ apps; I haven't fount my app in the list. So where I can check where I'm not logged in?
IOException: 401 Unauthorized
com.google.api.client.googleapis.json.GoogleJsonResponseException: 401 Unauthorized
at com.google.api.client.googleapis.json.GoogleJsonResponseException.from(GoogleJsonResponseException.java:145)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:113)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:40)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:423)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:343)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.execute(AbstractGoogleClientRequest.java:460)
at ScreenRecordingExample.UploadToYoutube(ScreenRecordingExample.java:356)
Ok, I have found out that the problem appeared due to the incorrect privacyStatus value:
I had
status.setPrivacyStatus("listed");
in my code, while it seems that only public, unlisted and private are possible values
And the UPD-2 problem was resolved unfortunately in the way i don't know exactly how:
i have noticed, that in my code example there was a line
return new AuthorizationCodeInstalledApp(flow, localReceiver).authorize("user");
I have replaced 'user' with my google user name (but actually I'm not sure that it was a thing that was really needed). After that replacement the authorization attempt started again (default browser loaded the accept/decline page and there I should select one of two accounts). Since I have two accounts, the original youtube one and the common google one, and during the last attempt I have selected the google's one and got 401 error, I have selected the youtube's one this time.
so, after that the problem was resolved. I still don't know do it due to the correct username instead of 'user' or due to the correct account selection. If someoune kows the truth, please comment!

Resources