How to limit Jenkins API response to last n build IDs - jenkins

http://xxx/api/xml?&tree=builds[number,description,result,id,actions[parameters[name,value]]]
Above API returns all the build IDs. Is there a way to limit results to get last 5 build IDS?

The tree query parameter allows you to explicitly specify and retrieve only the information you are looking for, by using an XPath-ish path expression. The value should be a list of property names to include, with sub-properties inside square braces. Try tree=jobs[name],views[name,jobs[name]] to see just a list of jobs (only giving the name) and views (giving the name and jobs they contain). Note: for array-type properties (such as jobs in this example), the name must be given in the original plural, not in the singular as the element would appear in XML (). This will be more natural for e.g. json?tree=jobs[name] anyway: the JSON writer does not do plural-to-singular mangling because arrays are represented explicitly.
For array-type properties, a range specifier is supported. For example, tree=jobs[name]{0,10} would retrieve the name of the first 10 jobs. The range specifier has the following variants:
{M,N}: From the M-th element (inclusive) to the N-th element (exclusive).
{M,}: From the M-th element (inclusive) to the end.
{,N}: From the first element (inclusive) to the N-th element (exclusive). The same as {0,N}.
{N}: Just retrieve the N-th element. The same as {N,N+1}.
Another way to retrieve more data is to use the depth=N query parameter . This retrieves all the data up to the specified depth. Compare depth=0 and depth=1 and see what the difference is for yourself. Also note that data created by a smaller depth value is always a subset of the data created by a bigger depth value.
Because of the size of the data, the depth parameter should really be only used to explore what data Jenkins can return. Once you identify the data you want to retrieve, you can then come up with the tree parameter to exactly specify the data you need.
I'm on version 1.509.4. which doesn't support range specifier.
Source: http://ci.citizensnpcs.co/api/

You can create an xml object with the build numbers via xpath and parse it yourself with via different means.
http://xxx/api/xml?xpath=//build/number&wrapper=meep
Creates an xml that looks like:
<meep>
<number>n</number>
<number>n+1</number>
...
<number>m</number>
</meep>
And will be populated with the build numbers n through m that are currently in jenkins for the specified job in the url. You can substitute anything for the word "meep", that will become the wrapper object for the newly created xml object.
How are you collecting/manipulating the api xml output once you get it? Because there is a solution here for How do I select the last N elements with XPath?. I tried using some of these xpath manipulations but I couldn't get it to work when playing with the url in my browser; it might work if you are doing something else.
When I get the xml object, I happen to manipulate it via shell scripts.
#!/bin/sh
# NOTE: To get the url to work with curl, you need a valid jenkins user and api token
# Put all build numbers in a variable called build_ids
build_ids="$(curl -sL --user ${_jenkins_api_user}:${_jenkins_api_token} \
"${_jenkins_url}/job/${_job_name}/api/xml?xpath=//build/number&wrapper=meep" \
| sed -e 's/<[^>]*>/ /g' | sed -e 's/ / /g')"
# Print the last 5 items with awk
echo "${build_ids}" | awk '{n = 5; for (--n; n >= 0; n--){ printf "%s\t",$(NF-n)} print ""}';
Once you have your xml object you can essentially parse it however you want.
NOTE: I am running Jenkins ver. 2.46.1

Looking at the doco at the raw .../api/ endpoint (on Jenkins 2.60.3) it says
For array-type properties, a range specifier is supported. For
example, tree=jobs[name]{0,10} would retrieve the name of the first 10
jobs. The range specifier has the following variants:
{M,N}: From the M-th element (inclusive) to the N-th element (exclusive).
{M,}: From the M-th element (inclusive) to the end.
{,N}: From the first element (inclusive) to the N-th element (exclusive). The same as {0,N}.
{N}: Just retrieve the N-th element. The same as {N,N+1}.
For the OP's case, you'd append {,5} to the end of the URL to get the first 5 results:
http://xxx/api/xml?&tree=builds[number,description,result,id,actions[parameters[name,value]]]{,5}

Related

Importing bulk json data into neo4j

I am trying to load json file of size about 700k. But it is showing me the heap memory out of space error.
My query is as below:
WITH "file:///Users//arundhathi.d//Documents//Neo4j//default.graphdb//import//tjson.json" as url
call apoc.load.json(url) yield value as article return article
Like in csv I tried to use USING PERIODIC COMMIT 1000 with json. But I am not allowed to use with loading json.
How to load bulk json data?.
You can also convert JSON into CSV files using jq - a uber fast json converter. https://stedolan.github.io/jq/tutorial/
This is the recommended way according to: https://neo4j.com/blog/bulk-data-import-neo4j-3-0/
If you have many files, write a python program or similar that iterates through the length of files calling:
os.system("cat file{}.json | jq '. [.entity1, .entity2, .entity3] | #csv' >> concatenatedCSV.csv".format(num))
or in Go:
exec.Command("cat file"+num+".json | jq '. [.entity1, .entity2, .entity3] | #csv' >> concatenatedCSV.csv")
I recently did this for about 700GB of JSON files. It takes some thought to get the csv files in the right format, but if you follow the tutorial on jq you'll pickup how to do it. Additionally, check out how the headers need to be and what not here: https://neo4j.com/docs/operations-manual/current/tools/import/
It took about a day to convert it all, but given the transaction overhead of using apoc, and the ability to reimport at anytime once the files are in the format it is worth it in the long run.
apoc.load.json now supports a json-path as a second parameter.
To get the first 1000 JSON objects from the array in the file, try this:
WITH "file:///path_to_file.json" as url
CALL apoc.load.json(url, '[0:1000]') YIELD value AS article
RETURN article;
The [0:1000] syntax specifies a range of array indices, and the second number is exclusive (so, in this example, the last index in the range is 999).
The above should at least work in neo4j 3.1.3 (with apoc release 3.1.3.6). Note also that the Desktop versions of neo4j (installed via the Windows and OSX installers) have a new requirement concerning where to put plugins like apoc in order to import local files.

Query Influxdb based on tags?

I have started playing around with Influxdb v0.13 and I have some dummy values in my test db where id is a tag and value is a field:
> SELECT * FROM dummy
name: dummy
--------------
time id value
1468276508161069051 1234 12345
1468276539152853428 613 352
1468276543470535110 613 4899
1468276553853436191 1234 12
I get no results returned when I run this query:
> SELECT * FROM dummy WHERE id=1234
but I do get the following when querying with the field instead:
> SELECT * FROM dummy WHERE value=12
name: dummy
--------------
time id value
1468276553853436191 1234 12
Am I doing something wrong here? I thought the point of tags were to be queried (since they are indexed and fields are not), but they seem to break my queries.
It appears that Influx will treat every tag key and value we insert as string and this is evidently shown in their official documentation.
See: https://docs.influxdata.com/influxdb/v0.13/guides/writing_data/
When writing points, you must specify an existing database in the db
query parameter. See the HTTP section on the Write Syntax page for a
complete list of the available query parameters. The body of the POST
- we call this the Line Protocol - contains the time-series data that you wish to store. They consist of a measurement, tags, fields, and a
timestamp. InfluxDB requires a measurement name. Strictly speaking,
tags are optional but most series include tags to differentiate data
sources and to make querying both easy and efficient. Both tag keys
and tag values are strings.
Note: the text in bold.
Hence to filter by tag key value - the query must be enquoted.
Example:
SELECT * FROM dummy WHERE id='1234'

Java 8- forEach method iterator behaviour

I recently started checking new Java 8 features.
I've come across this forEach iterator-which iterates over the Collection.
Let's take I've one ArrayList of type <Integer> having values= {1,2,3,4,5}
list.forEach(i -> System.out.println(i));
This statement iteates over a list and prints the values inside it.
I'd like to know How am I going to specify that I want it to iterate over some specific values only.
Like, I want it to start from 2nd value and iterate it till 2nd last value. or something like that- or on alternate elements.
How am I going to do that?
To iterate on a section of the original list, use the subList method:
list.subList(1, list.length()-1)
.stream() // This line is optional since List already has a foreach method taking a Consumer as parameter
.forEach(...);
This is the concept of streams. After one operation, the results of that operation become the input for the next.
So for your specific example, you can follow #Joni's command. But if you're asking in general, then you can create a filter to only get the values you want to loop over.
For example, if you only wanted to print the even numbers, you could create a filter on the streams before you forEached them. Like this:
List<Integer> intList = Arrays.asList(1,2,3,4,5);
intList.stream()
.filter(e -> (e & 1) == 0)
.forEach(System.out::println);
You can similarly pick out the stuff you want to loop over before reaching your terminal operation (in your case the forEach) on the stream. I suggest you read this stream tutorial to get a better idea of how they work: http://winterbe.com/posts/2014/07/31/java8-stream-tutorial-examples/

Listing two or more variables alongside each other

I want an alternative to running frequency for string variables because I also want to get a case number for each of the string value (I have a separate variable for case ID).
After reviewing the string values I will need to find them to recode which is the reason I need to know the case number.
I know that PRINT command should do what I want but I get an error - is there any alternative?
PRINT / id var2 .
EXECUTE.
>Error # 4743. Command name: PRINT
>The line width specified exceeds the output page width or the record length or
>the maximum record length of 2147483647. Reduce the number of variables or
>split the output line into several records.
>Execution of this command stops.
Try the LIST command.
I often use the TEMPORARY commond prior to the LIST command, as often there is only a small select of record of interest I may want to "list"/investigate.
For example, in the below, only to list the records where VAR2 is not a blank string.
TEMP.
SELECT IF (len(VAR2)>0).
LIST ID VAR2.
Alternatively, you could also (but dependent on having CUSTOM TABLES add-on module), do something like below which would get the results into a tabular format also (which may be preferable if then exporting to Excel, for example.
CTABLES /TABLE CTABLES /VLABELS VARIABLES=ALL DISPLAY=NONE
/TABLE A[C]>B[C]
/CATEGORIES VARIABLES=ALL EMPTY=EXCLUDE.

Is there a way to receive builds list from Jenkins from a specific number?

I'm using the following API to receive a list of builds per a job:
http://example.jenkins.com/job/someJob/api/json/parameters?pretty=true&tree=builds[number,result,building,actions[parameters[*]]]
The response is a JSON string with the list of all of the builds of someJob.
is there a way to get the list starting from a specific build_number?
For array-type properties, a range specifier is supported. For example, tree=jobs[name]{0,10} would retrieve the name of the first 10 jobs. The range specifier has the following variants:
{M,N}: From the M-th element (inclusive) to the N-th element (exclusive).
{M,}: From the M-th element (inclusive) to the end.
{,N}: From the first element (inclusive) to the N-th element (exclusive). The same as {0,N}.
{N}: Just retrieve the N-th element. The same as {N,N+1}.
[above is gracefully stolen borrowed from built-in Jenkins documentation at ${JENKINS_URL}/api ]
What you want to achieve can be done this way:
for M in {10,20,30}:
Open browser link( http://example.jenkins.com/job/someJob/api/json/parameters?pretty=true&tree=builds[number,result,building,actions[parameters[*]]]{M,} )
Note that build_number has little to do with this query. It'll count the existing build logs, so if you have some kind of log rotation mechanism there - you can't count on build_number.

Resources