I have an issue with JSON fields not extracted properly.
First of all I have an JSON Extractor on my input that extracts the message field, this will result in a new MESSAGE field, with the following contents:
{"kind":"Event","apiVersion":"audit.k8s.io/v1","level":"Metadata","auditID":"e03a5c76-2122-4c1d-8265-be6e9cdc5e11","stage":"ResponseComplete","requestURI":"/api/v1/namespaces/ingress-nginx/configmaps/ingress-controller-leader-nginx","verb":"update","user":{"username":"system:serviceaccount:ingress-nginx:ingress-nginx","uid":"42491580-cda5-4856-9f1b-ae133faadba7","groups":["system:serviceaccounts","system:serviceaccounts:ingress-nginx","system:authenticated"]},"sourceIPs":["10.81.100.2"],"userAgent":"nginx-ingress-controller/v0.40.2 (linux/amd64) ingress-nginx/fc4ccc5eb0e41be2436a978b01477fc354f31643","objectRef":{"resource":"configmaps","namespace":"ingress-nginx","name":"ingress-controller-leader-nginx","uid":"7bd1f348-9586-45ea-b41c-897c8ba83985","apiVersion":"v1","resourceVersion":"178373"},"responseStatus":{"metadata":{},"code":200},"requestReceivedTimestamp":"2020-11-04T09:21:18.756400Z","stageTimestamp":"2020-11-04T09:21:18.759305Z","annotations":{"authentication.k8s.io/legacy-token":"system:serviceaccount:ingress-nginx:ingress-nginx","authorization.k8s.io/decision":"allow","authorization.k8s.io/reason":"RBAC: allowed by RoleBinding "ingress-nginx/ingress-nginx" of Role "ingress-nginx" to ServiceAccount "ingress-nginx/ingress-nginx""}}
But then the issue is that the JSON path annotations (in bold) is not extracted into fields. Is there any way to accomplish this?
Thanks!
This is most probably because annotations and more specifically authorization.k8s.io/reason's property value breaks the JSON format.
Related
Inline with this thread
How do we get the document file url using the Watson Discovery Service?,
I tried adding url field inside HTML document under various tags like p, directly as
<"url">...<"/url"> etc inside <"body"> tag.
Whatever I do, this url field that I am trying to add is not getting reflected either under extracted_metadata or under html or under text in the json when I see the resulting json after conversion / enrichment in the discovery tooling. Can someone throw some hint on what I can do to get url field just like title under extracted_metadata section in the json output?
I am performing load testing on my server using jmeter.
In one of my post requests, I receive a unique id in the response.
I need to send this id as a parameter in the following post requests.
I am new to jmeter and don't have any idea how to do this.
Help would be really appreciated.
If you need to store the whole response into a variable - take the following steps:
Add Beanshell PostProcessor as a child of the request which returns response you're looking for
Put the following line into the PostProcessor's "Script" area:
vars.put("response", new String(data));
Refer extracted value as ${response} where required
See How to Use BeanShell: JMeter's Favorite Built-in Component guide to lean more about Beanshell scripting in JMeter
Alternatively you can do the same with the Regular Expression Extractor, in that case relevant configuration will be:
Reference Name: response
Regular Expression: (?s)(^.*)
Template: $1$
If you need a part of response, not the whole response you can amend Regular Expression according to your needs as per Regular Expressions chapter of JMeter's User Manual
If you really need to store the whole response into a variable, do the following:
Add JSR223 PostProcessor as a child of the request which returns response you're looking for
Put the following line into the "Script" area:
vars.put("response", prev.getResponseDataAsString());
Use then this response as ${response} where you need it
But you rarely need to use the whole response and you should avoid it for big , in this case it is much better to use the Extractor that suits your response format:
JSON Extractor for JSON
CSS/JQuery Extractor for HTML extraction
XPath Extractor for XML
Regular Expression Extractor for all of them or any textual format
You can use JMeter's Post-Processor Regular Expression Extractor to extract the required value from response. Just Add this under the sampler whose response will contain the required value.
In Reg expression extractor, you will define the variable name (referenceName), RegularExpression, template etc. Later you can use the value in this variable. To learn how to use Reg expression extractor you can refer to following tutorial.
https://docs.blazemeter.com/customer/portal/articles/1743642-using-regex-regular-expression-extractor-with-jmeter
I know this question is old but I agree with #UBIK you should probably use a JSON extractor. I am working with a load test that is sending over 100 requests per second and I need to reuse the value in a specific JSON key, so I'm using a JSON extractor and saving the values in a .csv file to be used by the next request.
extract the JSON
This is the Groovy script to write it to a .csv file
def myRandomIds = new File("randomIds.csv")
myRandomIds << vars.get("id") + ","
myRandomIds << System.getProperty("line.separator")
log.info(vars.get("id") + " saved to randomIds.csv...")
This is the CSV data config I have set up in my other request that is reading from the csv file. (In my case these .jmx files are automated and parametized using jenkins)
CSV data set config
With a request to Twitter for example:
https://api.twitter.com/1/users/show.json?screen_name=stephenfry&include_entities=true
I can extract an element like followers_count using result["followers_count"]
I've tried a similar request to LastFM but their JSON is structured differently as it's a translation of their default XML.
With their demo request:
http://ws.audioscrobbler.com/2.0/?method=artist.getinfo&artist=Cher&api_key=b25b959554ed76058ac220b7b2e0a026&format=json
How would I get the listeners value?
I've tried
result["listeners"]
result["artist.listeners"]
result["artist.stats.listeners"]
I understand I need to access a node, but have no idea how to go about it.
Can anyone help?
It's a nested hash, so you can reach it with:
result["artist"]["stats"]["listeners"]
Example:
require('open-uri')
require('json')
result = JSON.parse(open('http://ws.audioscrobbler.com/2.0/?method=artist.getinfo&artist=Cher&api_key=b25b959554ed76058ac220b7b2e0a026&format=json').read)
result["artist"]["stats"]["listeners"].to_i
The Last.fm data is a series of nested hashes, so you need to access them as such. Give the following a try, it should do the trick:
result["artist"]["listeners"]
Hi I am using biztalk's FlatFile parser (using XML schema) to part a CSV file. The CSV File sometimes contains invalid date - 1/1/1900. Currently the schema validation for the flat file fails because of invalid date. Is there any setting that I can use to allow the date to be used?
I dont want to read the date as string. I might be forced to if there is no other way.
You could change it to a valid XML date time (e.g., 1900-01-00:00:00Z) using a custom pipeline component (see examples here). Or you can just treat it as a string in your schema and deal with converting it later in a map, in an orchestration, or in a downstream system.
Here is a a C# snippet that you could put into a scripting functoid inside a BizTalk map to convert the string to an xs:dateTime, though you'll need to do some more work if you want to handle the potential for bad input data:
public string ConvertStringDateToDateTime(string param1)
{
return DateTime.Parse(inputDate).ToString("s",System.Globalization.DateTimeFormatInfo.InvariantInfo);
}
Also see this blog post if you're looking to do that in multiple places in a single map.
If I use permalink_fu on a text field I get this for the resulting permalink:
http://localhost:3000/243webb4ee4ff2227230b4232438591810c35d3f
instead of
http://localhost:3000/where-is-my-lost-cat
I assume this is because the text field becomes a blob in the database and permalink_fu only knows how to operate on string fields.
How could I convert this to a readable string version?
(without changing the database type from text to string)
Try commenting out these lines in your local version. See what the exception is, and trace that to your issue.