I am trying to model a server-to-server REST API interaction in Gatling 2.2.0. There are several interactions of the type "request a list and then request all items on the list at in parallel", but I can't seem to model this in Gatling. Code so far:
def groupBy(dimensions: Seq[String], metric: String) = {
http("group by")
.post(endpoint)
.body(...).asJSON
.check(
...
.saveAs("events")
)
}
scenario("Dashboard scenario")
.exec(groupBy(dimensions, metric)
.resources(
// a http() for each item in session("events"), plz
)
)
I have gotten as far as figuring out that parallel requests are performed by .resources(), but I don't understand how to generate a list of requests to feed it. Any input is appreciated.
Below approach is working for me. Seq of HttpRequestBuilder will be executed concurrently:
val numberOfParallelReq = 1000
val scn = scenario("Some scenario")
.exec(
http("first request")
.post(url)
.resources(parallelRequests: _*)
.body(StringBody(firstReqBody))
.check(status.is(200))
)
def parallelRequests: Seq[HttpRequestBuilder] =
(0 until numberOfParallelReq).map(i => generatePageRequest(i))
def generatePageRequest(id: Int): HttpRequestBuilder = {
val body = "Your request body here...."
http("page")
.post(url)
.body(StringBody(body))
.check(status.is(200))
}
Not very sure of your query but seems like you need to send parallel request which can be done by
setUp(scenorio.inject(atOnceUsers(NO_OF_USERS)));
Refer this http://gatling.io/docs/2.0.0-RC2/general/simulation_setup.html
Related
def get_nlg(graph_query):
driver = Graph("neo4j://localhost:7687", auth=("neo4j","password"))
graph_response = graph.evaluate(graph_query)
For the above code, I replaced with the driver code as below, but its not working, what is the function in neo4j driver equivalent to evaluate() function in py2neo?
def get_nlg(graph_query):
driver = GraphDatabase.driver("neo4j://localhost:7687", auth=("neo4j","password"))
with driver.session() as session:
graph_response = session.run(graph_query)
return graph_response
When the result from graph_response of 2nd code is passed to the below code, I am getting an error
TypeError: <neo4j.work.result.Result object at 0x7f94cf7f31d0> is not JSON serializable
class GetBiggestComponent(Action):
def name(self):
return "action_get_biggest_component"
def run(self, dispatcher, tracker, domain):
query = None
intent = tracker.latest_message['intent']
child_comp = tracker.get_slot('component_type_child')
parent_comp = tracker.get_slot('component_type_parent')
error = None
graph_response = GenerateQuery.get_biggest_component(child_comp, parent_comp)
graph_response['intent_name'] = intent['name']
dispatcher.utter_message(json.dumps(graph_response))
return []
the error is coming when it is passed in the line
dispatcher.utter_message(json.dumps(graph_response))
The output of session.run is an object that lets you navigate the result, not the result itself. I strongly recommend reading the manual and API docs for the driver, as this is all described in there.
https://neo4j.com/docs/driver-manual/current/session-api/simple/#driver-simple-result-consume
https://neo4j.com/docs/api/python-driver/current/api.html#result
As per my answer to your other question, to simulate evaluate, you will simply need to navigate to the first record of the result and then return the first value of that record.
From Limo anywhere trigger, I am getting the values in my node.js code action. Suppose we are getting following value from the trigger:
null,the four seasons hotel;
Zapier is sending only the four seasons hotel in the code. Is there a way in Zapier to get the rawJSON and parse it in the code?
My code :
let rows = "";
const totalRecords = firstNames.length;
const toLocations = inputData.to_location.split(",");
const firstNames = inputData.first_name.split(",");
const lastNames = inputData.last_name.split(",");
const pickupDates = inputData.pickup_date.split(",");
const fromLocations = inputData.from_location.split(",");
for(let i = 0; i < totalRecords; i++){
rows += `${firstNames[i]} ${lastNames[i]} ${fromLocations[i]} ${toLocations[i]} ${pickupDates[i]}`;
if(i !== (totalRecords - 1)){
rows += ",";
}
}
return {
rows: rows
};
Following is the complex solution of the Above problem :
Make a separate "Catch Raw Webhook" in Zapier.
Use the Webhook URL from #1 above and add it as a webhook action on existing Limo anywhere trigger. This webhook action will send json as post under "raw" parameter. Choose custom request then method will be POST, data pass through will be yes.
Add a code action after webhook trigger to parse the Python Dictionary string in Zapier :
import json, ast
output = {'res': json.dumps(ast.literal_eval(input_data["raw"]))}
This will now give you proper JSON compatible with Javascript and from here on you can deal with the data.
I had to use python in #3 because raw JSON was only compatible with Python and it looked like following :
{u'due_date': u'2019-03-22T00:00:00', u'terms': u'due_upon_receipt'}
I am trying to call a REST API through LUA. However, I am not able to capture full raw response returned by the API. Below is the code sample:
local http_socket = require("socket.http")
local pretty_print = require("pl.pretty")
local header = {
["x-device-type"] = "M",
["authorization"] = "ashdjkashd",
["x-app-secret"] = "asdasda",
["x-user-id"] = "asdasdasd"
}
r, c, h = http_socket.request {
method = "GET", -- Validation API Method
url = "http://google.com", -- Validation API URL
headers = header
}
print(r .. c)
pretty_print.dump(h)
I'm using lua 5.3, and luarocks version=2.4.1.
In variable c i am getting code, and in h there are a few headers. I need to capture full response returned by the API.
As you may know, luasocket's http.request supports two forms of usage. I'm assuming you need the second form to customize the resty request for that particular API.
In this case to capture the response body you'll need to use the sink field with ltn12.sink module. For example
local ltn12 = require 'ltn12'
-- ...
local res = {}
r, c, h, s = http_socket.request
{
method = "GET", -- Validation API Method
url = "http://google.com", -- Validation API URL
headers = header,
sink = ltn12.sink.table(res)
}
res = table.concat(res)
print(res)
The table.concat is needed since the response could be comprised of multiple chunk sizes(appended to res as it's received).
You can also write it out to file by replacing above with ltn12.sink.file, eg. using ltn12.sink.file(io.stdout) will dump the response to standard output.
I am trying to export JIRA tasks via API and I hit a wall on excel due to JIRA only allowing a 1000 limit. I can do an export manually to CSV and get over 1000 results and was wondering if anyone had any luck with large JIRA exports via REST API and can help point me in the right direction on this.
Guessing an export to CSV then pull into excel for reporting might work?
Thanks!
JIRA's REST API supports pagination to prevent that clients of the API can put too much load on the application. This means you cannot just pull in all issue data with 1 REST call.
You can only retrieve "pages" of max 1000 issues using the paging query parameters startAt and maxResults. See the Pagination section here.
If you run a JIRA standalone server then you can tweak the maximum number of results that JIRA returns, but for a cloud instance this is not possible. See this KB article for more info.
using jira-python (according to your tag)
# search_issues can only return 1000 issues, so if there are more we have to search again, thus startAt=count
issues = []
count = 0
while True:
tmp_issues = jira_connection.search_issues('', startAt=count, maxResults=count + 999)
if len(tmp_issues) == 0:
# Since Python does not offer do-while, we have to break here.
break
issues.extend(tmp_issues)
count += 999
The code below will fetch results 200 records at a time, till all records are exported.
you can export max 1000 records at a go by updating the page size, it will recursively fetch 1000 records until everything is exported
var windowSlider = 200
const request = require('request')
const fs = require('fs')
const chalk = require('chalk')
var windowSlider = 200
var totlExtractedRecords = 0;
fs.writeFileSync('output.txt', '')
const option = {
url: 'https://jira.yourdomain.com/rest/api/2/search',
json: true,
qs: {
jql: "project in (xyz)",
maxResults: 200,
startAt: 0,
}
}
const callback = (error, response) => {
const body = response.body
console.log(response.body)
const dataArray = body.issues
const total = body.total
totlExtractedRecords = dataArray.length + totlExtractedRecords
if (totlExtractedRecords > 0) {
option.qs.startAt = windowSlider + option.qs.startAt
}
dataArray.forEach(element => {
fs.appendFileSync('output.txt', element.key + '\n')
})
console.log(chalk.red.inverse('Total extracted data : ' + totlExtractedRecords))
console.log(chalk.red.inverse('Total data: ' + total))
if (totlExtractedRecords < total) {
console.log('Re - Running with start as ' + option.qs.startAt)
console.log('Re - Running with maxResult as ' + option.qs.maxResults)
request(option, callback).auth('api-reader', 'APITOKEN', true)
}
}
request(option, callback).auth('api-reader', 'APITOKEN', true)
I tired to port the request-oauth library (based on python-request) to Python 3 (with help of 2to3) but I have problems to validate a signature with StatusNet (same API as Twitter).
When I do a request to oauth/request_token, I have no problem but to oauth/access_token I have an error 401 Invalid signature. I don't understand why because it seems to me that what I sign is correct.
For example, with the python 2 code, cf hook.py and auth.py (original from the git repo), I get :
signing_key = '0de1456373dfc9349dd38a48e61fc844&136d6b9a597ee57d4338254812681acd',
signing_raw = 'POST&http%3A%2F%2Fstatus2.dotzero.me%2Fapi%2Foauth%2Faccess_token&oauth_consumer_key%3Dec3ad931b294b51a5ff595c732acb7a5%26oauth_nonce%3D33448267%26oauth_signature_method%3DHMAC-SHA1%26oauth_timestamp%3D1332279747%26oauth_token%3D2131043f3516bcb730d391ed2033a880%26oauth_verifier%3D8816492%26oauth_version%3D1.0'
oauth_hook.token.key = '2131043f3516bcb730d391ed2033a880'
oauth_hook.token.secret = '136d6b9a597ee57d4338254812681acd'
request.data_and_params = {'oauth_version': '1.0', 'oauth_signature': 'xyjxH5QcfZXnG111L7qANZ+ahRI=',
'oauth_token': '2131043f3516bcb730d391ed2033a880', 'oauth_nonce': '33448267',
'oauth_timestamp': '1332279747', 'oauth_verifier': '8816492',
'oauth_consumer_key': 'ec3ad931b294b51a5ff595c732acb7a5',
'oauth_signature_method': 'HMAC-SHA1'}
and with my python 3 port, cf hook.py and auth.py, I get :
signing_key = '0de1456373dfc9349dd38a48e61fc844&136d6b9a597ee57d4338254812681acd',
signing_raw = 'POST&http%3A%2F%2Fstatus2.dotzero.me%2Fapi%2Foauth%2Faccess_token&oauth_consumer_key%3Dec3ad931b294b51a5ff595c732acb7a5%26oauth_nonce%3D52360702%26oauth_signature_method%3DHMAC-SHA1%26oauth_timestamp%3D1332278837%26oauth_token%3D2131043f3516bcb730d391ed2033a880%26oauth_verifier%3D8816492%26oauth_verifier%3D8816492%26oauth_version%3D1.0'
oauth_hook.token.key = '2131043f3516bcb730d391ed2033a880'
oauth_hook.token.secret = '136d6b9a597ee57d4338254812681acd'
request.data_and_params = {'oauth_nonce': '52360702', 'oauth_timestamp': '1332278837',
'oauth_verifier': '8816492', 'oauth_consumer_key': 'ec3ad931b294b51a5ff595c732acb7a5',
'oauth_signature_method': 'HMAC-SHA1', 'oauth_version': '1.0',
'oauth_token': '2131043f3516bcb730d391ed2033a880',
'oauth_signature': 'BRsb11dk++405uaq5pRS+CMUzbo='}
Both looks good to me but the first one succeed and the second returns a 401 error, invalid signature.
In both cases, I get the token.key and token.secret as the result of :
OAuthHook.consumer_key = self.ckey
OAuthHook.consumer_secret = self.csecret
oauth_hook = OAuthHook()
client = requests.session(hooks={'pre_request': oauth_hook})
response = client.post('%soauth/request_token' % (self.url), {'oauth_callback': 'oob'})
# new oauth_hook with the request token
oauth_hook = OAuthHook(response[b'oauth_token'][0],response[b'oauth_token_secret'][0])
Them, I go to oauth/authorize?oauth_token=%s" % oauth_hook.token.key to get authorize the app and get a pincode. After that I can do the problematic request
...
response = client.post('%soauth/request_token' % (self.url), {'oauth_callback': 'oob'})
oauth_hook = OAuthHook(response[b'oauth_token'][0],response[b'oauth_token_secret'][0])
# get the pincode from %soauth/authorize?oauth_token=%s" % (self.url, oauth_hook.token.key)
oauth_hook.token.set_verifier(pincode)
client = requests.session(hooks={'pre_request': oauth_hook})
response = client.post("%soauth/access_token" % (self.url),
{'oauth_verifier': pincode})
The signature code from the auth.py file is
def sign(self, request, consumer, token):
"""Builds the base signature string."""
key, raw = self.signing_base(request, consumer, token)
hashed = hmac.new(key.encode(), raw.encode(), sha1)
# Calculate the digest base 64.
return binascii.b2a_base64(hashed.digest())[:-1]
Any idea why it doesn't work with the py3k code ?
Thank you
Found the answer ! There were two oauth_verifier in the POST request, leading to a wrong signature...
You may need to verify the Authorization header string in your request. Normally it would be of the format:
'Authorization' => 'OAuth
realm="",oauth_timestamp="1243392158",oauth_nonce="VsaPHb",oauth_consumer_key="xxxxxxxxxxxxxxxxxx",oauth_token="xxxxxx-xxxx-xxxxxxxxxxxxxx",oauth_version="1.0",oauth_signature_method="HMAC-SHA1",oauth_signature="xxxxxxxxxxxxxxxxxxxx"'
In the above header value, check that the "oauth_signature" is decoded properly. That is, it should not contain values like: %3D. You can use this tool to decode the string.
This has worked for me. Hope it helps someone.