Jenkins groovy - how to separate url path and filename - jenkins

$url = "http://bus00cyb.ind.testserver.com:8080/job/IOTF-7.4.x-BITBUCKET-REPO/51/artifact/output.txt"
I have a Jenkins job parameter which has takes 'url' value as an user input. I want to separate the param as below:
$url_path = "http://bus00cyb.ind.testserver.com:8080/job/IOTF-7.4.x-BITBUCKET-REPO/51/artifact"
$filename = "output.txt"
How to do this using jenkins pipeline groovy script? Pls suggest.

Don't know if I'm understanding your question correctly but in groovy you will have something like this:
url = "http://bus00cyb.ind.testserver.com:8080/job/IOTF-7.4.x-BITBUCKET-REPO/51/artifact/output.txt"
urlsplitted = url.split('/')
filename = urlsplitted[urlsplitted.length-1]
shorturl = url-filename
println(shorturl)
println(filename)

U can use regexp (.*)\/(\w*\..*)$
Pattern p = Pattern.compile("(.*)\/(\w*\..*)$");
Matcher m = p.matcher("http://bus00cyb.ind.testserver.com:8080/job/IOTF-7.4.x-BITBUCKET-REPO/51/artifact/output.txt");
If (m.find()){
filename= m.group(2)
url=m.group(1)
}

Related

how to pass additional parametes to spider parse function in scrapy during web scraping

I am trying to pass additional information to the parse function but it is giving a type error.
TypeError: parse() got an unexpected keyword argument 'body'
i am unable to resolve this issue.
"""
return [scrapy.Request(url=website.search_url.format(prod), callback=self.parse,
cb_kwargs = {"body":website.body_xpath,"product_list":website.products_list_xpath,
"names":website.products_name_xpath,"selling_price":website.selling_price_xpath,
"market_price":website.market_price_xpath}) for website in websites for prod in modified_products]
def parse(self, response):
body = response.cb_kwargs.get("body")
product_list = response.cb_kwargs.get("product_list")
name = response.cb_kwargs.get("names")
selling_price = response.cb_kwargs.get("selling_price")
market_price = response.cb_kwargs.get("market_price")
"""
I forgot to write those names in parse function definition, after adding them i am getting the correct result. Thanks for having a look at it.
"""
return [scrapy.Request(url=website.search_url.format(prod), callback=self.parse,
cb_kwargs = dict(body = website.body_xpath, product_list = website.products_list_xpath,
name = website.products_name_xpath, selling_price = website.selling_price_xpath,
market_price = website.market_price_xpath)) for website in websites for prod in modified_products]
def parse(self, response, body, product_list, name, selling_price, market_price):
body = response.cb_kwargs["body"]
product_list = response.cb_kwargs["product_list"]
name_ = response.cb_kwargs["name"]
selling_price_ = response.cb_kwargs["selling_price"]
market_price_ = response.cb_kwargs["market_price"]
"""

hmac authentication in vcl

I am trying to authenticate URL using hamc. I can do the following to verify.My question is how do I parse the URL to extract only part of the URL excluding the hmac parameter. I tried using local variables in vcl but it threw an error.
Any suggestions on how to extract the hmac value and URL query parameters as shown below.
http://localhost/zzz/?q1=xxx&q2=yyy&hmac=hash
if (digest.hmac_md5("key", "q1=xxx&q2=yyy") != "value")
{
return (synth(401, digest.hmac_md5("key", "http://localhost/zzz/?q1=xxx&q2=yyy")));
}
Thanks
You'll want to use the [querystring][1] vmod. As far as I know it does not come pre-packaged, so you will need to build it but it should do exactly what you need.
With that you can define regexes/static values to match querystring arguments, and then filter those out or in.
there is no need for a external plugin in that case you can just strip out the hmac=XXX query string parameter, from req.url and store the result in a new variable req.http.url_without_hmac and req.http.hmac to the digest.hmac_md5
see a sample test case:
varnishtest "Strip query parameter"
server s1 {
rxreq
txresp
rxreq
txresp
} -start
varnish v1 -vcl+backend {
import std;
sub vcl_recv {
# Strip out HMAC parameter
# get only the query string, ignore uri
set req.http.qs = regsuball(req.url, ".*\?(.*?)$", "?\1");
# strip hmac= from the qs
set req.http.url_without_hmac = regsuball(req.http.qs,"\?hmac=[^&]+$",""); # strips when QS = "?hmac=AAA"
set req.http.url_without_hmac = regsuball(req.http.url_without_hmac,"\?hmac=[^&]+&","?"); # strips when QS = "?hmac=AAA&foo=bar"
set req.http.url_without_hmac = regsuball(req.http.url_without_hmac,"&hmac=[^&]+",""); # strips when QS = "?foo=bar&hmac=AAA" or QS = "?foo=bar&hmac=AAA&bar=baz"
# remove the leading ? from the url_without_hmac
set req.http.url_without_hmac = regsuball(req.http.url_without_hmac,"^\?(.*)$", "\1");
# extract the hmac= value from the req.http.qs
set req.http.hmac = regsuball(req.http.qs, ".*[?&]hmac=([^&]*).*", "\1");
# NOW USE req.http.url_without_hmac for your digest validation and req.http.hmac as the value
}
sub vcl_deliver {
set resp.http.url_without_hmac = req.http.url_without_hmac;
set resp.http.hmac = req.http.hmac;
}
} -start
client c1 {
txreq -url "/1?a=1&hmac=2&b=1"
rxresp
expect resp.http.url_without_hmac == "a=1&b=1"
expect resp.http.hmac == "2"
} -run
client c2 {
txreq -url "/1?hmac=hello&a=1&b=1"
rxresp
expect resp.http.url_without_hmac == "a=1&b=1"
expect resp.http.hmac == "hello"
} -run

For Loop to call multiple variables from properties file

I have a properties file which I call inside my Jenkins Pipeline Script to get multiple variables.
BuildCounter = n
BuildName1 = Name 1
BuildName2 = Name 2
...
Buildnamen = Name n
I call my properties file with: def props = readProperties file: Path
Now I want to create a loop to print all my BuildNames
for (i = 0; i < BuildJobCounterInt; i++){
tmp = 'BuildName' + i+1
println props.tmp
}
But of course this is not working. ne last println call is searching for a variable called tmp in my properties file. Is there a way to perform this or am I completely wrong?
EDIT:
This is my .properties file:
BuildJobCounter = 1
BuildName1 = 'Win32'
BuildPath1 = '_Build/MBE3_Win32'
BuildName2 = 'empty'
BuildPath2 = 'empty'
TestJobCounter = '0'
TestName1 = 'empty'
TestPath1 = 'empty'
TestName2 = 'empty'
TestPath2 = 'empty'
In my Jenkins pipeline I want to have the possibility to check the ammount of Build/TestJobs and automatically calle the Jobs (each BuildName and BuildPath is a Freestyle Job) To call all these Job I thought of calling the variables inside a for loop. So for every istep I have the Name/Path pair.
Try the below:
Change from:
println props.tmp
To:
println props[tmp]
or
println props."$tmp"
EDIT : based on OP comment
change from:
tmp = 'BuildName' + i+1
To:
def tmp = "BuildName${(i+1).toString()}"

Select All Choices in Jenkins Groovy

I want to select all the choices in my Groovy script so it defaults to all. I am using the Active Choices Reactive Parameter because I am reading in the previous option. How do I make my "output" variable so it is all selected without having the user select them all?
def output = []
def line
def release = RELEASE_NUMBER
releaseNumber = release.replaceAll("\\n", "");
String[] number = releaseNumber.split("\\.");
def list = "cmd /c e:\\tools\\wget --no-check-certificate --http-user=username--http-password=password-qO- \"https://1.1.1.1:443/svn/Instructions/trunk/${number[0]}.${number[1]}.${number[2]}/ICAN/EI_${releaseNumber}.txt\"".execute().text
list.eachLine {
if (it.contains("- LH")) {
String newName = it.replaceAll("\\s", "");
String newName2 = newName.replaceAll("-", "");
output.add(newName2)
}
}
return output
I don't know anything about Jenkins, but reading the documentation for the plugin you mention you should be able to simply use output.add("${newName2}:selected").

Is is possible to have strings evaluate as variables in groovy?

I am playing with grails and groovy. I wondered if its possible to do something like this.
def inbuiltReqAttributes = ['actionName','actionUri','controllerName','controllerUri']
inbuiltReqAttributes.each() { print " ${it} = ? " };
what would i put in the ? to get groovy to evaluate the current iterator value as a variable e.g. to do it the long way
print " actionName = $actionName "
Thanks
I believe off the top of my head, this should work:
print " ${it} = ${this[ it ]}"
Or:
print " ${it} = ${getProperty( it )}"
But i'm not at a computer to 100% verify this atm...
Try this:
inbuiltReqAttributes.each() {
evaluate("value = ${it}")
print "$it = $value"
}

Resources