Reading a HTML file through Groovy Script - jenkins

I need to write a Jenkins pipeline script using Groovy where the below HTML is the input.
<table style="width:30%">
<TR>
<TD>Failed A Count</TD>
<TD>2869</TD>
</TR>
<TR>
<TD>Failed B Count</TD>
<TD>9948</TD>
</TR>
<TR>
<TD>Failed C Count</TD>
<TD>3456</TD>
</TR></table>
I am getting it from a RestAPI, and if any of the value is more than 100 I need to trigger an email.
def response = httpRequest 'REST_API_URI'
println("Status: "+response.status)
def responseBody = response.content
String[] TDcollection;
String[] splitData = responseBody.split("\n");
for (String eachSplit : splitData) {
if (eachSplit.contains("Failed")) {
print(eachSplit);
}
}
I have tried this, But not able to pick up the value and validate it.
This might seem very easy, but as I am very
new to Groovy, I am kind of stuck on it. Thanks In Advance.

No-brainer groovy:
String input = '''\
<table style="width:30%">
 <TR>
 <TD>Failed A Count</TD>
 <TD>2869</TD>
 </TR>
 <TR>
 <TD>Failed B Count</TD>
 <TD>9948</TD>
 </TR>
 <TR>
 <TD>Failed B Count</TD>
 <TD>10000</TD>
 </TR>
 <TR>
 <TD>Failed C Count</TD>
 <TD>3456</TD>
 </TR></table>'''
Map<String,Integer> failedValues = [:].withDefault{ 0 }
input.eachMatch( /<TD>Failed (\w+) Count<\/TD>\s*<TD>(\d+)<\/TD>/ ){ _, name, count -> failedValues[ name ] += count.toInteger() }
assert failedValues == [A:2869, B:19948, C:3456]
boolean errorOccured = failedValues.any{ 100 <= it.value }
assert errorOccured
Note also the summing up of counts for the same "name".

Related

How to display iteration index in Thymeleaf? [duplicate]

This question already has answers here:
Thymeleaf - How to loop a list by index
(2 answers)
Closed 1 year ago.
I'm attempting to display each row from an array using Thymeleaf - following their documentation I am unable to use any of the following attributes from th:each:
The current iteration index, starting with 0. This is the index
property.
The current iteration index, starting with 1. This is the count
property.
userinput.html:
<tr th:each="year : ${years}">
<th scope="row" th:text="${years}"></th>
<th scope="row" th:text="${bdcInterest}"></th>
<td th:text="${bdAmount}"></td>
</tr>
CalculatorController.java:
#RequestMapping(value = "/submit", method = RequestMethod.GET)
public String userInput(Model model, BigDecimal lumpsum, BigDecimal interestrate, BigDecimal monthlywithdrawal) {
BigDecimal initialinvestment = lumpsum;
BigDecimal[] bdAmount = new BigDecimal[11];
BigDecimal[] bdcInterest = new BigDecimal[11];
BigDecimal[] initialInvestment = new BigDecimal[11];
int[] years = new int[11];
bdcInterest[0] = new BigDecimal(0);
initialInvestment[0] = initialinvestment;
int increment = 1;
while(increment < 10) {
BigDecimal amount = lumpsum
.multiply(BigDecimal
.valueOf(1)
.add(interestrate
.divide(BigDecimal
.valueOf(100)))
.subtract(monthlywithdrawal
.multiply(BigDecimal
.valueOf(12)))); // Calculate the total yearly amount
BigDecimal cInterest = amount.subtract(initialinvestment); // Calculate only the interest earned
bdAmount[increment] = amount;
bdcInterest[increment] = cInterest;
initialInvestment[increment] = initialinvestment;
years[increment] = increment;
lumpsum = amount;
increment++;
}
model.addAttribute("years", years);
model.addAttribute("initialInvestment", initialInvestment);
model.addAttribute("bdAmount", bdAmount);
model.addAttribute("bdcInterest", bdcInterest);
return "userinput";
}
The necessary data is submitted correctly in each respective array, however I believe I've misunderstood the documentation:
Thymeleaf maintains the iteration status of the th:each tag in a special variable. Please, see the relevant documentation.
Among the different information provided in that variable, you can find an index property, which corresponds to the actual iteration index.
In you example, you probably could iterate your results like this:
<tr th:each="year, iterStat : ${years}">
<th scope="row" th:text="${year}"></th>
<th scope="row" th:text="${bdcInterest[iterStat.index]}"></th>
<td th:text="${bdAmount[iterStat.index]}"></td>
</tr>
To avoid this kind of problems, please, consider define in your code a simple java object that agglutinates the four properties you are iterating:
public class MuCustomObject {
private BigDecimal bdAmount;
private BigDecimal bdcInterest;
private BigDecimal initialInvestment;
private int year;
// getters and setters omitted for brevity
}
Then, use the object in your controller:
#RequestMapping(value = "/submit", method = RequestMethod.GET)
public String userInput(Model model, BigDecimal lumpsum, BigDecimal interestrate, BigDecimal monthlywithdrawal) {
BigDecimal initialinvestment = lumpsum;
List<MyCustomObject> myCustomObjectList = new ArrayList<MyCustomObject>();
MyCustomObject myCustomObject = new MyCustomObject();
myCustomObject.setBdcInterest(new BigDecimal(0));
myCustomObject.setInitialInvestment(initialinvestment);
myCustomObjectList.add(myCustomObject);
int increment = 1;
while(increment < 10) {
BigDecimal amount = lumpsum
.multiply(BigDecimal
.valueOf(1)
.add(interestrate
.divide(BigDecimal
.valueOf(100)))
.subtract(monthlywithdrawal
.multiply(BigDecimal
.valueOf(12)))); // Calculate the total yearly amount
BigDecimal cInterest = amount.subtract(initialinvestment); // Calculate only the interest earned
myCustomObject = new MyCustomObject();
myCustomObject.setBdAmount(amount);
myCustomObject.setBdcInterest(cInterest);
myCustomObject.setInitialInvestment(initialinvestment);
myCustomObject.setYear(increment);
myCustomObjectList.add(myCustomObject);
lumpsum = amount;
increment++;
}
model.addAttribute("myCustomObjects", myCustomObjectList);
return "userinput";
}
With that information you could directly iterate the collection:
<tr th:each="myCustomObject, iterStat : ${myCustomObjects}">
<th scope="row" th:text="${myCustomObject.year}"></th>
<th scope="row" th:text="${myCustomObject.bdcInterest}"></th>
<td th:text="${myCustomObject.bdAmount}"></td>
</tr>

Extract visual text from Google Classic Site page using Apps Script in Google Sheets

I have about 5,000 Classic Google Sites pages that I need to have a Google Apps script under Google Sheets examine one by one, extract the data, and enter that data into the Google Sheet row by row.
I wrote an apps script to use one of the sheets called "Pages" that contains the exactly URL of each page row by row, to run down while doing the extraction.
That in return would get the HTML contents and I would then use regex to extract the data I want which is the values to the right of each of the following...
Job name
Domain owner
Urgency/Impact
ISOC instructions
Which would then write that date under the proper columns in the Google Sheet.
This worked except for one big problem. The HTML is not consistent. Also, ID's and tags were not used so really it makes trying to do this through SitesApp.getPageByUrl not possible.
Here is the code I came up with for that attempt.
function startCollection () {
var masterList = SpreadsheetApp.getActiveSpreadsheet().getSheetByName("Pages");
var startRow = 1;
var lastRow = masterList.getLastRow();
for(var i = startRow; i <= lastRow; i++) {
var target = masterList.getRange("A"+i).getValue();
sniff(target)
};
}
function sniff (target) {
var pageURL = target;
var pageContent = SitesApp.getPageByUrl(pageURL).getHtmlContent();
Logger.log("Scraping: ", target);
// Extract the job name
var JobNameRegExp = new RegExp(/(Job name:<\/b><\/td><td style='text-align:left;width:738px'>)(.*?)(\<\/td>)/m);
var JobNameValue = JobNameRegExp.exec(pageContent);
var JobMatch = JobNameValue[2];
if (JobMatch == null){
JobMatch = "NOTE FOUND: " + pageURL;
}
// Extract domain owner
var DomainRegExp = new RegExp(/(Domain owner:<\/b><\/td><td style='text-align:left;width:738px'><span style='font-family:arial,sans,sans-serif;font-size:13px'>)(.*?)(<\/span>)/m);
var DomainValue = DomainRegExp.exec(pageContent);
Logger.log("DUMP1:",SitesApp.getPageByUrl(pageURL).getHtmlContent());
var DomainMatch = DomainValue[2];
if (JobMatch == null){
DomainMatch = "N/A";
}
// Extract Urgency & Impact
var UrgRegExp = new RegExp(/(Urgency\/Impact:<\/b><\/td><td style='text-align:left;width:738px'>)(.*?)(<\/td>)/m);
var UrgValue = UrgRegExp.exec(pageContent);
var UrgMatch = UrgValue[2];
if (JobMatch == null){
UrgMatch = "N/A";
}
// Extract ISOC Instructions
var ISOCRegExp = new RegExp(/(ISOC instructions:<\/b><\/td><td style='text-align:left;width:738px'>)(.*?)(<\/td>)/m);
var ISOCValue = ISOCRegExp.exec(pageContent);
var ISOCMatch = ISOCValue[2];
if (JobMatch == null){
ISOCMatch = "N/A";
}
// Add record to sheet
var row_data = {
Job_Name:JobMatch,
Domain_Owner:DomainMatch,
Urgency_Impact:UrgMatch,
ISOC_Instructions:ISOCMatch,
};
insertRowInTracker(row_data)
}
function insertRowInTracker(rowData) {
var sheet = SpreadsheetApp.getActiveSpreadsheet().getSheetByName("Jobs");
var rowValues = [];
var columnHeaders = sheet.getDataRange().offset(0, 0, 1).getValues()[0];
Logger.log("Writing to the sheet: ", sheet.getName());
Logger.log("Writing Row Data: ", rowData);
columnHeaders.forEach((header) => {
rowValues.push(rowData[header]);
});
sheet.appendRow(rowValues);
}
So for my next idea, I have thought about using UrlFetchApp.fetch. The one problem I have though is that these pages on that Classics Google Site sit behind a non-shared with the public domain. While using SitesApp.getPageByUrl has the script ask for authorization and works, SitesApp.getPageByUrl does not meaning when it tries to call the direct page, it just gets the Google login page.
I might be able to work around this and turn them public, but I am still working on that.
I am running out of ideas fast on this one and hoping there is another way I have not thought of or seen. What I would really like to do is not even mess with the HTML content. I would like to use apps script under the Google Sheet to just look at the actual data presented on the page and then match a text and capture the value to the right of it.
For example have it go down the list of URLS on sheet called "Pages" and do the following for each page:
Find the following values:
Find the text "Job name:", capture the text to the right of it.
Find the text "Domain owner:", capture the text to the right of it.
Find the text "Urgency/Impact:", capture the text to the right of it.
Find the text "ISOC instructions:", capture the text to the right of it.
Write those values to a new row in sheet called "Jobs" as seen below.
Then move on the the next URL in the sheet called "Pages" and repeat until all rows in the sheet "Pages" have been completed.
Example of the data I want to capture
I have created an exact copy of one of the pages for testing and is public.
https://sites.google.com/site/2020dump/test
An inspect example
The raw HTML of the table which contains all the data I am after.
<tr>
<td style="width:190px"><b>Domain owner:</b></td>
<td style="text-align:left;width:738px">IT.FinanceHRCore </td>
</tr>
<tr>
<td style="width:190px"> <b>Urgency/Impact:</b></td>
<td style="text-align:left;width:738px">Medium (3 - Urgency, 3 - Impact) </td>
</tr>
<tr>
<td style="width:190px"><b>ISOC instructions:</b></td>
<td style="text-align:left;width:738px">None </td>
</tr>
<tr>
<td style="width:190px"></td>
<td style="text-align:left;width:738px"> </td>
</tr>
</tbody>
</table>
Any examples of how I can accomplish this? I am not sure how from an apps script perspective to go about not looking at HTML and only looking at the actual data displayed on the page. For example looking for the text "Job name:" and then grabbing the text to the right of it.
The goal at the end of the day is to transfer the data from each page into one big Google Sheet so we can kill off the Google Classic Site.
I have been scraping data with apps script using regular expressions for a while, but I will say that the formatting of this page does make it difficult.
A lot of the pages that I scrape have tables in them so I made a helper script that will go through and clean them up and turn them into arrays. Copy and paste the script below into a new google script:
function scrapetables(html,startingtable,extractlinksTF) {
var totaltables = /<table.*?>/g
var total = html.match(totaltables)
var tableregex = /<table[\s\S]*?<\/table>/g;
var tables = html.match(tableregex);
var arrays = []
var i = startingtable || 0;
while (tables[i]) {
var thistable = []
var rows = tables[i].match(/<tr[\s\S]*?<\/tr>/g);
if(rows) {
var j = 0;
while (rows[j]) {
var thisrow = tablerow(rows[j])
if(thisrow.length > 2) {
thistable.push(tablerow(rows[j]))
} else {thistable.push(thisrow)}
j++
}
arrays.push(thistable);
}
i++
}
return arrays;
}
function removespaces(string) {
var newstring = string.trim().replace(/[\r\n\t]/g,'').replace(/ /g,' ');
return newstring
}
function tablerow(row,extractlinksTF) {
var cells = row.match(/<t[dh][\s\S]*?<\/t[dh]>/g);
var i = 0;
var thisrow = [];
while (cells[i]) {
thisrow.push(removehtmlmarkup(cells[i],extractlinksTF))
i++
}
return thisrow
}
function removehtmlmarkup(string,extractlinksTF) {
var string2 = removespaces(string.replace(/<\/?[A-Za-z].*?>/g,''))
var obj = {string: string2}
//check for link
if(/<a href=.*?<\/a>/.test(string)) {
obj['link'] = /<a href="(.*?)"/.exec(string)[1]
}
if(extractlinksTF) {
return obj;
} else {return string2}
}
Running this got close, but at the moment, this doesn't handle nested tables well so I cleaned up the input by sending only the table that we want by isolating it with a regular expression:
var tablehtml = /(<table[\s\S]{200,1000}Job Name[\s\S]*?<\/table>)/im.exec(html)[1]
Your parent function will then look like this:
function sniff(pageURL) {
var html= SitesApp.getPageByUrl(pageURL).getHtmlContent();
var tablehtml = /(<table[\s\S]{200,1000}Job Name[\s\S]*?<\/table>)/im.exec(html)[1]
var table = scrapetables(tablehtml);
var row_data =
{
Job_Name: na(table[0][3][1]), //indicates the 1st table in the html, row 4, cell 2
Domain_Owner: na(table[0][4][1]), // indicates 1st table in the html, row 5, cell 2 etc...
Urgency_Impact: na(table[0][5][1]),
ISOC_Instructions: na(table[0][6][1])
}
insertRowInTracker(row_data)
}
function na(string) {
if(string) {
return string
} else { return 'N/A'}
}

Query ViewData, then change value based on result

I have this code snippet in my controller
outputmodel4.Add(new SP_data.student()
{
student_id = (decimal)SPOutput4["student_id"],
student_no = (string)SPOutput4["student_no"],
course_id = (decimal)SPOutput4["course_id"],
floor_no = (int)SPOutput4["floor_no"],
tutor_id = (decimal)SPOutput4["tutor_id"],
capacity_id = (decimal)SPOutput4["capacity_id"],
});
}
ViewData["Output"] = outputmodel4;
what I am trying to do it query Capacity_id, and if the number is (say) 2, then display "All Most Full" on my view.
I think the say to accomplish this is
#foreach (var item in ViewData["Output"] as IEnumerable<app.Models.SP_data.student>)
{
<TD>
#if (item.capacity_id = 2)
{
All Most Full
}
else if (item.capacity_id = 5)
{
Full
}
else if etc..
</TD>
}
.. but I getting errors in the view
Cannot implicitly convert type decimal to bool
can someone please advise, and help ?
thanks

How would I use button in the following situation in grails?

I am returning a map value that contain multiple items. The map is as follows:
def myList = []
//Some code to obtain the List
List << ["Id": Id,"Name": Name,"Code": sCode, "RunType": x ]
return [items: List]
In my GSP page I am using the following code to print them out in my web page.
<%
def counter = 0
for (i in items) {
counter = counter + 1
println("<td>" + i + "</td>" + "\n")
if (counter == 1) {
println("</tr><tr>")
counter = 0
}
}
%>
And the output looks as follows:
[Id:i-d0f5, Name:es_test_1b_110.test.com, Code:16, RunType:On Demand]
[Id:i-7890, Name:namc-qc.test.com, Code:16, RunType:On Demand]
[Id:i-ee56, Name:abcdef.test.com, Code:16, RunType:On Demand]
[Id:i-c41e, Name:backup.grails.test.com, Code:80, RunType:On Demand]
What I need now is to add a button for each of the values printed (so if 6 servers are printed 6 buttons will show up and if 10 servers are printed 10 buttons will show up) i will be using the button to pass the Id to start and stop the server.
Grails has some powerful built-in GSP tags which come in quite handy here. For example:
<g:each in="${items}" var="i">
<tr>
<td>${i.Id}</td>
<td>${i.Name}</td>
<td>${i.Code}</td>
<td>${i.RunType}</td>
<td><a class="button" href="${createLink(controller: '<controllerName>', action: 'start', id: i.Id)}">Start</a></td>
</tr>
</g:each>
There are, of course, many other ways to approach this, but I hope this is a start. For more, see these sections in the Grails ref doc: each, createLink

ActionLink only if a certain condition is met

If s = "N/A" then I don't want to use the ActionLink. In other words, if the inventory item is not currently being used on a project, then don't provide the link (just show N/A instead). Also, how do I send the link to Projects/Details? Right now, it will go to "Nails/Projects/Details" instead, because I'm using the NailsController class.
<td class="table-normal-data">
<% Dim l As Integer = InStr(item.CurrentProject, " [")
Dim s As String = item.CurrentProject
Dim projectID As String = ""
If l > 0 Then
s = Mid(item.CurrentProject, 1, l - 1)
projectID = Mid(item.CurrentProject, l + 2, Len(item.CurrentProject) - l - 2)
Else
s = ""
End If
%>
<%: Html.ActionLink(s, "Projects/Details", New With {.id = projectID}) %>
</td>
I'm much more familiar with MVC3/Razor and C#, but I often do something like this in my views:
#if( Model.Flag )
{
<span>n/a</span>
}
else
{
#Html.ActionLink(....
}
Basically, you output different stuff through the view depending on the state of the model.

Resources