Jenkins Pipeline - how to get logs from parallel builds - jenkins

Question

Is it possible, and if yes: how?, to get the log output for each parallel step separately?
I.e.:
def projectBranches = [:]
for (int i = 0; i < projects.size(); i++) {
def _i = i
projectBranches[_i] = {
someFunction(_i)
}
}
parallel projectBranches
Is it now possible to get the log for each of projectBranches[_i]?

Answer 1

I needed to access logs from within the pipeline code
so I implemented the algorithm proposed by キキジキ (really helpful) with a few adjustments (to add branchName prefix on each line to be able to get the whole log and still figure out which branch corresponds to each line ; and to support nested branches, which I needed) in https://github.com/gdemengin/pipeline-logparser :
to get logs programmatically
to get the full logs with branch prefix (similar to what currentBuild.rawBuild.log returned before version 2.2.5 of workflow-job plugin. but in version 2.26 they got rid of the branch information and I could not find any built-in function with the same information)
String logs = logparser.getLogsWithBranchInfo()
[Pipeline] Start of Pipeline
[Pipeline] parallel
[Pipeline] { (Branch: branch1)
[Pipeline] { (Branch: branch2)
[Pipeline] }
[Pipeline] echo
[branch1] in branch1
[Pipeline] sleep
[branch1] Sleeping for 1 sec
[Pipeline] echo
[branch2] in branch2
[Pipeline] sleep
[branch2] Sleeping for 1 sec
get the logs from 'branch2' only
String logsBranch2 = logparser.getLogsWithBranchInfo(filter: ['branch2'])
[branch2] in branch2
[branch2] Sleeping for 1 sec
to archive logs (in as $JOB_URL/<runId>/artifacts) to have them available as a link for later use
to archive the full logs (with branch prefix)
logparser.archiveLogsWithBranchInfo('consoleText.txt')
to archive the logs from branch2 only
logparser.archiveLogsWithBranchInfo('logsBranch2.txt', [filter: ['branch2']])

Answer 2

I found a way to achieve that, but you need to access the build folder directly (for example using currentBuild.rawBuild.getLogFile().getParent()).
Parse the xml files (or the single flowNodeStore.xml file) inside the workflow directory:
Build a hierarchy of nodes using the <id> and <parentIds> values.
If <branchName> is defined associate it to the current node and recursively to all nodes that have this node as parent. If a node has multiple parents assign no branch value to it.
Read the log file as byte[].
Read each line of log-index to find log ranges to assign to each node. The format of a line can be one of the following:
offset nodeId -> start of new node range, end of the previous (if present).
offset: end of current node range.
Convert the byte range back to a utf8 string (new String(range, "UTF-8")).
You might want to strip away all embedded codes with something like replaceAll("\u001B.*?\u001B\\[0m", "")

Answer 3

You could get your nodes by use of Jenkins REST API: job/test/1/api/json?depth=2
Result should contain something like:
{"_class":"org.jenkinsci.plugins.workflow.cps.nodes.StepStartNode","actions":[{},{},{}],"displayName":"Branch: 0","iconColor":"blue","id":"13","parents":["3"],"running":false,"url":"job/test/1/execution/node/13/"},
{"_class":"org.jenkinsci.plugins.workflow.cps.nodes.StepStartNode","actions":[{},{},{}],"displayName":"Allocate node : Start","iconColor":"blue","id":"23","parents":["13"],"running":false,"url":"job/test/1/execution/node/23/"},
{"_class":"org.jenkinsci.plugins.workflow.cps.nodes.StepStartNode","actions":[{},{}],"displayName":"Allocate node : Body : Start","iconColor":"blue","id":"33","parents":["23"],"running":false,"url":"job/test/1/execution/node/33/"},
{"_class":"org.jenkinsci.plugins.workflow.cps.nodes.StepAtomNode","actions":[{},{}],"displayName":"Print Message","iconColor":"blue","id":"37","parents":["33"],"running":false,"url":"job/test/1/execution/node/37/"}
So for your case you are interested in the child with type StepAtomNode of your Branch with name given (0-9 for this case). From this you could obtain console output address by simply adding log to the address (like: job/test/1/execution/node/37/log).
Now this is where it gets a bit ugly, you need to parse the html to get the actual log from the
<pre class="console-output">log here
</pre>