Looping in a C-style For loop instead of using each - jenkins

how can i loop through a json file using a FOR loop in groovy? I am able to do it with .each but i am in a situation/bug where i cannot use .each loops.
The json file is being read and parsed into an object.
The json looks like this:
{
"workflows1": {
"name": "/wf_multifolder",
"file": "release1/wf_multifolder.XML",
"folderNames": {
"multifolder": "{{multifolder}}",
"agent1": "{{agentx}}"
}
},
"workflows2": {
"name": "/wf_multifolder",
"file": "release1/wf_multifolder.XML",
"folderNames": {
"multifolder": "{{multifolder}}",
"agent1": "{{agentx}}"
}
}
}
Note: i can modify the json file, if need to make the process simpler.. All i am try to do is to loop throgh and extract the values for the keys.

So given the json in a String like so:
def jsonText = '''{
"workflows1": {
"name": "/wf_multifolder",
"file": "release1/wf_multifolder.XML",
"folderNames": {
"multifolder": "{{multifolder}}",
"agent1": "{{agentx}}"
}
},
"workflows2": {
"name": "/wf_multifolder",
"file": "release1/wf_multifolder.XML",
"folderNames": {
"multifolder": "{{multifolder}}",
"agent1": "{{agentx}}"
}
}
}'''
You can just do:
import groovy.json.*
def json = new JsonSlurper().parseText(jsonText)
for(entry in json) {
println "$entry.key has file $entry.value.file"
}
to print:
workflows1 has file release1/wf_multifolder.XML
workflows2 has file release1/wf_multifolder.XML

Related

Parsing boto3 invoke_endpoint response from AWS SageMaker

I have a Sagemaker endpoint that I can infer to from boto3 client and get response.
Per boto3 doc, the Body of the response result is a Byte object StreamingBody type. I convert it to a dictionary
response = client.invoke_endpoint(EndpointName=endpoint_name, Body=json.dumps(data))
response_body = response['Body']
dict_response = response_body.read().decode('utf-8')
print(dict_response)
The above code gives me a response like below (stripped down for this post)
I need to retrieve the array from the "floatVal" key. How do I do that?
{
"outputs": {
"score": {
"dtype": "DT_FLOAT",
"floatVal": [
0.00012408883776515722,
...........
-0.8316119909286499,
-0.24423488974571228
],
"tensorShape": {
"dim": [
{
"size": "1"
},
{
"size": "1024"
}
]
}
}
},
"modelSpec": {
"version": "1",
"name": "generic_model",
"signatureName": "serving_default"
}
}
Actually the dict_response is not really a dictionary here, rather a string type. So I had to convert the dict_response to an actual dictionary and then I could retrieve the floatVal key.
Updated code
response = client.invoke_endpoint(EndpointName=endpoint_name, Body=json.dumps(data))
response_body = response['Body']
response_str = response_body.read().decode('utf-8')
response_dict = eval(response_str)
print(response_dict['outputs']['score']['floatVal'])

How to get a sub-field of a struct type map, in the search response of YQL query in Vespa?

Sample Data:
"fields": {
"key1":0,
"key2":"no",
"Lang": {
"en": {
"firstName": "Vikrant",
"lastName":"Thakur"
},
"ch": {
"firstName": "维克兰特",
"lastName":"塔库尔"
}
}
}
Expected Response:
"fields": {
"Lang": {
"en": {
"firstName": "Vikrant",
"lastName":"Thakur"
}
}
}
I have added the following in my search-definition demo.sd:
struct lang {
field firstName type string {}
field lastName type string {}
}
field Lang type map <string, lang> {
indexing: summary
struct-field key {
indexing: summary | index | attribute
}
}
I want to write a yql query something like this (This doesn't work):
http://localhost:8080/search/?yql=select Lang.en from sources demo where key2 contains 'no';
My temporary workaround approach
I have implemented a custom searcher in MySearcher.java, through which I am able to extract the required sub-field and set a new field 'defaultLang', and remove the 'Lang' field. The response generated by the searcher:
"fields": {
"defaultLang": {
"firstName": "Vikrant",
"lastName":"Thakur"
}
}
I have written the following in MySearcher.java:
for (Hit hit: result.hits()) {
String language = "en"; //temporarily hard-coded
StructuredData Lang = (StructuredData) hit.getField("Lang");
Inspector o = Lang.inspect();
for (int j=0;j<o.entryCount();j++){
if (o.entry(j).field("key").asString("").equals(language)){
SlimeAdapter value = (SlimeAdapter) o.entry(j).field("value");
hit.setField("defaultLang",value);
break;
}
}
hit.removeField("Lang");
}
Edit-1: A more efficient way instead is to make use of the Inspectable interface and Inspector, like above (Thanks to #Jo Kristian Bergum)
But, in the above code, I am having to loop through all the languages to filter out the required one. I want to avoid this O(n) time-complexity and make use of the map structure to access it in O(1). (Because the languages may increase to 1000, and this would be done for each hit.)
All this is due to the StructuredData data type I am getting in the results. StructureData doesn't keep the Map Structure and rather gives an array of JSON like:
[{
"key": "en",
"value": {
"firstName": "Vikrant",
"lastName": "Thakur"
}
}, {
"key": "ch",
"value": {
"firstName": "维克兰特",
"lastName": "塔库尔"
}
}]
Please, suggest a better approach altogether, or any help with my current one. Both are appreciated.
The YQL sample query I guess is to illustrate what you want as that syntax is not valid. Picking a given key from the field Lang of type map can be done as you do in your searcher but deserializing into JSON and parsing the JSON is probably inefficient as StructuredData implements the Inspectable interface and you can inspect it directly without the need to go through JSON format. See https://docs.vespa.ai/documentation/reference/inspecting-structured-data.html

Accessing a specific item in a JSON return in Rails

I have this JSON from the NHL API, and I tried to access the value gamesPlayed:
"teams"=>[
{
"id"=>5,
"name"=>"Pittsburgh Penguins",
"link"=>"/api/v1/teams/5",
"venue"=>{
"id"=>5034,
"name"=>"PPG Paints Arena",
"link"=>"/api/v1/venues/5034",
"city"=>"Pittsburgh",
"timeZone"=>{
"id"=>"America/New_York",
"offset"=>-5,
"tz"=>"EST"
}
},
"abbreviation"=>"PIT",
"teamName"=>"Penguins",
"locationName"=>"Pittsburgh",
"division"=>{
"id"=>18,
"name"=>"Metropolitan",
"nameShort"=>"Metro",
"link"=>"/api/v1/divisions/18",
"abbreviation"=>"M"
},
"conference"=>{
"id"=>6,
"name"=>"Eastern",
"link"=>"/api/v1/conferences/6"
},
"franchise"=>{
"franchiseId"=>17,
"teamName"=>"Penguins",
"link"=>"/api/v1/franchises/17"
},
"teamStats"=>[
{
"type"=>{
"displayName"=>"statsSingleSeason"
},
"splits"=>[
{
"stat"=>{
"gamesPlayed"=>16,
"wins"=>7,
"losses"=>6,
"ot"=>3,
"pts"=>17,
"ptPctg"=>"53.1",
"goalsPerGame"=>3.313,
"goalsAgainstPerGame"=>3.063,
"evGGARatio"=>1.0833,
"powerPlayPercentage"=>"23.4",
"powerPlayGoals"=>11.0,
"powerPlayGoalsAgainst"=>8.0,
"powerPlayOpportunities"=>47.0,
"penaltyKillPercentage"=>"84.0",
"shotsPerGame"=>32.625,
"shotsAllowed"=>33.6875,
"winScoreFirst"=>0.6,
"winOppScoreFirst"=>0.167,
"winLeadFirstPer"=>0.5,
"winLeadSecondPer"=>1.0,
"winOutshootOpp"=>0.333,
"winOutshotByOpp"=>0.444,
"faceOffsTaken"=>1035.0,
"faceOffsWon"=>534.0,
"faceOffsLost"=>501.0,
"faceOffWinPercentage"=>"51.6",
"shootingPctg"=>10.2,
"savePctg"=>0.909
},
"team"=>{
"id"=>5,
"name"=>"Pittsburgh Penguins",
"link"=>"/api/v1/teams/5"
}
},
{
"stat"=>{
"wins"=>"24th",
"losses"=>"15th",
"ot"=>"9th",
"pts"=>"24th",
"ptPctg"=>"19th",
"goalsPerGame"=>"8th",
"goalsAgainstPerGame"=>"19th",
"evGGARatio"=>"11th",
"powerPlayPercentage"=>"10th",
"powerPlayGoals"=>"22nd",
"powerPlayGoalsAgainst"=>"4th",
"powerPlayOpportunities"=>"31st",
"penaltyKillOpportunities"=>"1st",
"penaltyKillPercentage"=>"6th",
"shotsPerGame"=>"12th",
"shotsAllowed"=>"27th",
"winScoreFirst"=>"15th",
"winOppScoreFirst"=>"27th",
"winLeadFirstPer"=>"27th",
"winLeadSecondPer"=>"7th",
"winOutshootOpp"=>"25th",
"winOutshotByOpp"=>"25th",
"faceOffsTaken"=>"25th",
"faceOffsWon"=>"19th",
"faceOffsLost"=>"6th",
"faceOffWinPercentage"=>"8th",
"savePctRank"=>"13th",
"shootingPctRank"=>"12th"
},
"team"=>{
"id"=>5,
"name"=>"Pittsburgh Penguins",
"link"=>"/api/v1/teams/5"
}
}
]
}
],
"shortName"=>"Pittsburgh",
"officialSiteUrl"=>"http://pittsburghpenguins.com/",
"franchiseId"=>17,
"active"=>true
}
}
I am working in ruby on rails and would like to access the gamesPlayed value.
So far I have:
url = 'https://statsapi.web.nhl.com/api/v1/teams/5?expand=team.stats'
uri = URI(url)
response = Net::HTTP.get(uri)
response = JSON.parse(response)
#awayteamgamesplayed = response["teams"][0]["teamStats"]["stat"]["gamesPlayed"]
I can get to the team name using: response["teams"][away_team]["name"] but cant't work out gamesPlayed.
But it doesn't seem to work for gamesPlayed.
Value of teamStats is an Array. You need to access it via an index.
Same for splits
response["teams"][0]["teamStats"][0]["splits"][0]["stat"]["gamesPlayed"]
# => 16
teamStats is an array try this
response["teams"][0]["teamStats"][0]["stat"]["gamesPlayed"]

Extracting value of a node in Java using contains with JsonPath in RestAssured

I have to extract value of book title using JsonPath in RestAssured in Java from following json response
{
"spec": {
"groups": [
{
"name": "book",
"title": "classic-books:1.0.2"
},.......
]
}
}
I am looking to use contains to get the book with a specific title.Please help.
Assume you have response with JSON in it:
response.body().jsonPath().get("spec.groups[i].title");

Grails - ElasticSearch - QueryParsingException[[index] No query registered for [query]]; with elasticSearchHelper; JSON via curl works fine though

I have been working on a Grails project, clubbed with ElasticSearch ( v 20.6 ), with a custom build of elasticsearch-grails-plugin(to support geo_point indexing : v.20.6)
have been trying to do a filtered Search, while using script_fields (to calculate distance). Following is Closure & the generated JSON from the GXContentBuilder :
Closure
records = Domain.search(searchType:'dfs_query_and_fetch'){
query {
filtered = {
query = {
if(queryTxt){
query_string(query: queryTxt)
}else{
match_all {}
}
}
filter = {
geo_distance = {
distance = "${userDistance}km"
"location"{
lat = latlon[0]?:0.00
lon = latlon[1]?:0.00
}
}
}
}
}
script_fields = {
distance = {
script = "doc['location'].arcDistanceInKm($latlon)"
}
}
fields = ["_source"]
}
GXContentBuilder generated query JSON :
{
"query": {
"filtered": {
"query": {
"match_all": {}
},
"filter": {
"geo_distance": {
"distance": "5km",
"location": {
"lat": "37.752258",
"lon": "-121.949886"
}
}
}
}
},
"script_fields": {
"distance": {
"script": "doc['location'].arcDistanceInKm(37.752258, -121.949886)"
}
},
"fields": ["_source"]
}
The JSON query, using curl-way, works perfectly fine. But when I try to execute it from Groovy Code, I mean with this (taken from ElasticSearchService.groovy) where request is SearchRequest instance :
elasticSearchHelper.withElasticSearch { Client client ->
def response = client.search(request).actionGet()
}
It throws following error :
Failed to execute phase [dfs], total failure; shardFailures {[1][index][3]: SearchParseException[[index][3]: from[0],size[60]: Parse Failure [Failed to parse source [{"from":0,"size":60,"query_binary":"eyJxdWVyeSI6eyJmaWx0ZXJlZCI6eyJxdWVyeSI6eyJtYXRjaF9hbGwiOnt9fSwiZmlsdGVyIjp7Imdlb19kaXN0YW5jZSI6eyJkaXN0YW5jZSI6IjVrbSIsImNvbXBhbnkuYWRkcmVzcy5sb2NhdGlvbiI6eyJsYXQiOiIzNy43NTIyNTgiLCJsb24iOiItMTIxLjk0OTg4NiJ9fX19fSwic2NyaXB0X2ZpZWxkcyI6eyJkaXN0YW5jZSI6eyJzY3JpcHQiOiJkb2NbJ2NvbXBhbnkuYWRkcmVzcy5sb2NhdGlvbiddLmFyY0Rpc3RhbmNlSW5LbSgzNy43NTIyNTgsIC0xMjEuOTQ5ODg2KSJ9fSwiZmllbGRzIjpbIl9zb3VyY2UiXX0=","explain":true}]]]; nested: QueryParsingException[[index] No query registered for [query]]; }
The above Closure works if I only use filtered = { ... } script_fields = { ... } but it doesn't return the calculated distance.
Anyone had any similar problem ?
Thanks in advance :)
It's possible that I might have been dim to point out the obvious here :P

Resources