I am working with the slack API. My script does a bunch of external processing and in some cases it can take around 3-6 seconds. What is happening is the Slack API expects a 200 response within 3 seconds and because my function is not finished within 3 seconds, it retries again and then it ends up posting the same automated responses 2-3 times.
I confirmed this by commenting out all the functions and I had no issue, it posted the responses to slack fine. I then added sleep 10 and it done the same responses 3 times so the ohly thing different was it took longer.
From what I read, I need to have threaded responses. I then need to first respond to the slack API in thread 1 and then go about processing my functions.
Here is what I tried:
def events
Thread.new do
json = {
"text": "Here is your 200 response immediately slack",
}
render(json: json)
end
puts "--------------------------------Json response started----------------------"
sleep 30
puts "--------------------------------Json response completed----------------------"
puts "this is a successful response"
end
When I tested it the same issue happened so I tried using an online API tester and it hits the page, waits 30 seconds and then returns the 200 response but I need it to respond immediately with the 200, THEN process the rest otherwise I will get duplicates.
Am I using threads properly or is there another way to get around this Slack API 3 second response limit? I am new to both rails and slack API so a bit lost here.
Appreciate the eyes :)
I would recommend using ActionJob to run the code in the background if you don't need to use the result of the code in the response. First, create an ActiveJob job by running:
bin/rails generate job do_stuff
And then open up the file created in app/jobs/do_stuff_job.rb and edit the #perform function to include your code (so the puts statements and sleep 30 in your example). Finally, from the controller action you can call DoStuff.perform_later and your job will run in the background! Your final controller action will look something like this:
def events
DoStuff.perform_later # this schedules DoStuff to be done later, in
# the background, so it will return immediately
# and continue to the next line.
json = {
"text": "Here is your 200 response immediately slack",
}
render(json: json)
end
As an aside, I'd highly recommend never using Thread.new in rails. It can create some really confusing behavior especially in test scripts for a number of reasons, but usually because of how it interacts with open connections and specifically ActiveRecord.
class myTest extends Simulation {
val headers = Map("Authorization" -> "longAuthHeader")
val httpProtocol = http
.baseUrl("http://baseurl.com:8000")
.headers(headers)
val scn = scenario("Scenario Name")
.exec(http("request")
.get("/data/url/"))
setUp(scn.inject(constantUsersPerSec(40) during (2 minutes)))
.protocols(httpProtocol)
.throttle(jumpToRps(40), holdFor(2 minutes))
}
Using the above, I am creating a test using gatling which performs 40 RPS to baseurl.com:8000/data/url and maintains this for 2 minutes.
The problem with above approach is that only one user (as identified by the auth header) is performing the test.
What steps do I need to take to change this test to perform a request with, say, 40 users (40 different auth headers) that perform 1RPS each? So I will have 40 RPS distributed over 40 users rather than 40 RPS from one user.
This is important because my application performs slightly different behaviour based on the user context (different auth headers = different behaviour).
the gatling advanced tutorial describes how to do exactly this
gatling.io/docs/current/advanced_tutorial#advanced-tutorial You're interested the sections on configuring virtual users and feeders –
I want to POST data from Postman Google Chrome extension.
I want to make 10 requests with different data and it should be at the same time.
Is it possible to do such in Postman?
If yes, can anyone explain to me how can this be achieved?
I guess there's no such feature in postman as to run concurrent tests.
If I were you, I would consider Apache jMeter, which is used exactly for such scenarios.
Regarding Postman, the only thing that could more or less meet your needs is - Postman Runner.
There you can specify the details:
number of iterations,
upload CSV file with data for different test runs, etc.
The runs won't be concurrent, only consecutive.
Do consider jMeter (you might like it).
Postman doesn't do that but you can run multiple curl requests asynchronously in Bash:
curl url1 & curl url2 & curl url3 & ...
Remember to add an & after each request which means that request should run as an async job.
Postman however can generate curl snippet for your request: https://learning.getpostman.com/docs/postman/sending_api_requests/generate_code_snippets/
I don't know if this question is still relevant, but there is such possibility in Postman now. They added it a few months ago.
All you need is create simple .js file and run it via node.js. It looks like this:
var path = require('path'),
async = require('async'), //https://www.npmjs.com/package/async
newman = require('newman'),
parametersForTestRun = {
collection: path.join(__dirname, 'postman_collection.json'), // your collection
environment: path.join(__dirname, 'postman_environment.json'), //your env
};
parallelCollectionRun = function(done) {
newman.run(parametersForTestRun, done);
};
// Runs the Postman sample collection thrice, in parallel.
async.parallel([
parallelCollectionRun,
parallelCollectionRun,
parallelCollectionRun
],
function(err, results) {
err && console.error(err);
results.forEach(function(result) {
var failures = result.run.failures;
console.info(failures.length ? JSON.stringify(failures.failures, null, 2) :
`${result.collection.name} ran successfully.`);
});
});
Then just run this .js file ('node fileName.js' in cmd).
More details here
Not sure if people are still looking for simple solutions to this, but you are able to run multiple instances of the "Collection Runner" in Postman. Just create a runner with some requests and click the "Run" button multiple times to bring up multiple instances.
Run all Collection in a folder in parallel:
'use strict';
global.Promise = require('bluebird');
const path = require('path');
const newman = Promise.promisifyAll(require('newman'));
const fs = Promise.promisifyAll(require('fs'));
const environment = 'postman_environment.json';
const FOLDER = path.join(__dirname, 'Collections_Folder');
let files = fs.readdirSync(FOLDER);
files = files.map(file=> path.join(FOLDER, file))
console.log(files);
Promise.map(files, file => {
return newman.runAsync({
collection: file, // your collection
environment: path.join(__dirname, environment), //your env
reporters: ['cli']
});
}, {
concurrency: 2
});
In postman's collection runner you can't make simultaneous asynchronous requests, so instead use Apache JMeter instead. It allows you to add multiple threads and add synchronizing timer to it
If you are only doing GET requests and you need another simple solution from within your Chrome browser, just install the "Open Multiple URLs" extension:
https://chrome.google.com/webstore/detail/open-multiple-urls/oifijhaokejakekmnjmphonojcfkpbbh?hl=en
I've just ran 1500 url's at once, did lag google a bit but it works.
The Runner option is now on the lower right side of the panel
If you need to generate more consecutive requests (instead of quick clicking SEND button). You can use Runner. Please note it is not true "parallel request" generator.
File->New Runner Tab
Now you can "drag and drop" your requests from Collection and than keep checked only request you would like to generate by a Runner setting 10 iterations (to generate 10 requests ) and delay for example to 0 (to make it as fast as possible).
Easiest way is to get => Google Chrome "TALEND API TESTER"
Go to help + type in Create Scenario
...or just go to this link => https://help.talend.com/r/en-US/Cloud/api-tester-user-guide/creating-scenario
I was able to send several POST API calls simultaneously.
You can use Fiddler with started traffic capture to record manual queries from Postman, then select them in Fiddler's sessions list as much as you want and replay (press R key) - they would run in parallel.
https://docs.telerik.com/fiddler/generate-traffic/tasks/resendrequest
You can run multiple instances of postman Runner and run the same collection with different data files in each instance.
Open multiple postman. It replicates it and run concurrently.
I'm using rspec to test my application and I'm having a hard time figuring out how to test this. The Slack::Notifier's job is to send a post request to a webhook. Once I call this method in Rspec, I don't know how to see the response. Also, is it possible to match the format of this text to an expected text somewhere? My method is below. Thanks.
def notify
offset = 14400 #UTC to EST
notifier = Slack::Notifier.new Rails.application.secrets.slack_organization_name, Rails.application.secrets.slack_token, channel: "##{Rails.application.secrets.slack_channel}", username: Rails.application.secrets.slack_user_name
notifier.ping(":white_check_mark: *USAGE SUMMARY for #{(Time.now - offset).to_formatted_s(:long) }*")
count = 0
current_time = Time.now.to_i
live_response.each do |r|
if r["properties"]["time"] > ((current_time - offset) - 60) #&& r["properties"]["$initial_referring_domain"] == "capture.com"
notifier.ping("
*Name:* #{r["properties"]["$name"]}
*Event:* #{r["event"]}
*Keywords:* #{r["properties"]["keywords"]}
*Organization:* #{r["properties"]["organizationName"]}
*Email:* #{r["properties"]["$email"]}
*Time:* #{Time.at(r["properties"]["time"] + offset).utc.to_datetime.in_time_zone("Eastern Time (US & Canada)").to_formatted_s(:long_ordinal)}
*More Data:* #{ANALYTICS_URL}#{r["properties"]["distinct_id"]}
__________________________________________________
")
count +=1
end
end
notifier.ping("*There were #{count} events in this report.*")
end
Testing network communications (like API calls) is a tricky thing. Personally I would rely on programming by contract and testing in isolation - i.e. assume the external service is working fine and it responds positively for valid request.
Then you test your client code by checking that you are actually sending a valid request. For this stub the method where control exits your code into a library/system code. For example if you are making a HTTP GET request using a gem like HTTParty, then stub HTTParty.get i.e. HTTParty.stub(:get) and in that stub verify that correct parameters were sent.
On the other side of the spectrum you should also simulated both positive and negative responses from the web service and make sure your client code handles it in expected manner.
If you are making a real then you are introducing a lot of dependencies on your test : a test setup of external service, risk of network issues (timeout, n/w breakdown, etc) problems with external service and may be more.
If you yourself are writing that webservice too then test that one also in isolation, i.e by simulating valid and invalid inputs and making sure they are handled properly. This part is pretty much your controller specs or request specs.
Once again, this is my opinion. Suggestions to do this in a better way and constructive criticism on the shortcomings of this approach are definitely welcome.
I'm getting messages of a RabbitMQ queue and each message is a URL that I want to make a request to. Now I'm using the AMQP gem to subscribe to the queue and that uses EventMachine, so I'm using the the em-http-request library to make the http requests. According to the documentation here: https://github.com/igrigorik/em-http-request/wiki/Parallel-Requests
The following will issue asynchronous http-requests:
EventMachine.run {
http1 = EventMachine::HttpRequest.new('http://google.com/').get
http2 = EventMachine::HttpRequest.new('http://yahoo.com/').get
http1.callback { }
http2.callback { }
end
So when I subscribe to the RabbitMQ queue I have the following code:
x = 0
EventMachine.run do
connection = AMQP.connect(:host => '127.0.0.1')
channel = AMQP::Channel.new(connection)
channel.prefetch(50)
queue = channel.queue("http.requests")
exchange = channel.direct("")
queue.subscribe do |metadata, payload|
url = payload.inspect
eval "
#http#{x} = EventMachine::HttpRequest.new(url).get
#http#{x}.callback do
puts \"got a response\"
puts #http#{x}.response
end
x = x+1
"
end
end
This dynamically creates new variables and creates new http requests, similar to the way described in the em-http-request documentation. But is there a way to test whether the requests are actually being made asynchronously? Is it possible to write to the console every time a get request is fired off so I can see they are fired off one after the other without waiting for a response?
You can try running tcpdump and analysing the output. If you see the TCP three-way handshakes for the two connections being interleaved then the connections are happening in parallel.
This can't really be part of an automated test though, if that's what you're trying to aim for. I would be happy just to verify that the library does what it says it does once and not make it part of a test suite.
A very simple example, demonstrating exactly what you want:
require 'em-http-request'
EM.run do
# http://catnap.herokuapp.com/3 delays the HTTP response by 3 seconds.
http1 = EventMachine::HttpRequest.new('http://catnap.herokuapp.com/3').get
http1.callback { puts 'callback 1' }
http1
puts 'fired 1'
http2 = EventMachine::HttpRequest.new('https://www.google.com/').get
http2.callback { puts 'callback 2' }
puts 'fired 2'
end
Output (for me):
fired 1
fired 2
callback 2
callback 1
Depending on your internet connection, Heroku and Google, the response to the second HTTP request will likely come in first and you can be sure, the requests are indeed done in parallel.