How to specify branch name for shared library as environment variable - docker

I want to refer/specify (syntax) branch which is set as an environment variable for Jenkins shared library which will be provide during docker container.
For Example:
#Library(['my-shared-library', BRANCH_NAME])
Tried using ${BRANCH_NAME} ${env.BRANCH_NAME}
I will provide BRANCH_NAME as environment variable in docker-compose.yml
Also i want to get the env variable effected in org.jenkinsci.plugins.workflow.libs.GlobalLibraries.xml
like if i set PIPELINE_VERSION as env variable
<?xml version='1.1' encoding='UTF-8'?>
<org.jenkinsci.plugins.workflow.libs.GlobalLibraries plugin="workflow-cps-global-lib#2.15">
<libraries>
<org.jenkinsci.plugins.workflow.libs.LibraryConfiguration>
<name>XXXXXXXXXXXX</name>
<retriever class="org.jenkinsci.plugins.workflow.libs.SCMSourceRetriever">
<scm class="jenkins.plugins.git.GitSCMSource" plugin="git#3.12.0">
<id>XXXXXXXXXXXXXXXXXXXXXXXX</id>
<remote>XXXXXXXXXXXXXXXXXXX</remote>
<credentialsId>jXXXXXXXXXXXXXXXXXXXX</credentialsId>
<traits>
<jenkins.plugins.git.traits.BranchDiscoveryTrait/>
</traits>
</scm>
</retriever>
<defaultVersion>${PIPELINE_RELEASE_VERSION}</defaultVersion>
<implicit>true</implicit>
<allowVersionOverride>true</allowVersionOverride>
<includeInChangesets>false</includeInChangesets>
</org.jenkinsci.plugins.workflow.libs.LibraryConfiguration>
</libraries>
</org.jenkinsci.plugins.workflow.libs.GlobalLibraries>
Thanks,
Kusuma

I don't think anyway it possible to make the environment variable available for org.jenkinsci.plugins.workflow.libs.GlobalLibraries.xml. But If you use Jenkins Code As Configuration plugins, you can pass the variable from docker-compose and make that available to the config file, and when Jenkins load the config file to prepare config for Jenkins would work.
An example can be found here

Related

Log4j2 environment lookup does not find variable in AWS Linux EC2 instance

I have a Tomcat WAR project running in AWS Elastic Beanstalk EC2 instances. I have configured the instances to ensure that they have an environment variable CLUSTER_NAME. I can verify that the variable is available in the EC2 instance.
[ec2-user#ip-10* ~]$ cat /etc/environment
export CLUSTER_NAME=sandbox
ec2-user#ip-10* ~]$ echo $CLUSTER_NAME
sandbox
This variable is looked up in a Log4j2 XML file like this:
<properties>
<property name="env-name">${env:CLUSTER_NAME}</property>
</properties>
The env-name property is used in a Coralogix appender like this:
<Coralogix name="Coralogix" companyId="--" privateKey="--"
applicationName="--" subSystemName="${env-name}">
<PatternLayout>
<pattern>%d{yyyy-MM-dd HH:mm:ss.SSS}{GMT+0}\t%p\t%c\t%m%n</pattern>
</PatternLayout>
</Coralogix>
I see that this lookup is not working, as the env-name is just shown as ${env:CLUSTER_NAME} in Coralogix dashboard. The value works if I hardcode it.
What can be done to fix this lookup? There are several related questions for this, but they seem to refer to log4j1.x. https://stackoverflow.com/a/22296362. I have ensured that this project uses log4j2.
The solution was to add the CLUSTER_NAME variable in the /etc/profile.d/env.sh. This variable is available in the log4j2.xml with the following lookup.
<property name="env-name">
${env:CORALOGIX_CLUSTER_NAME}
</property>
I am still not clear of the difference between adding a variable to /etc/environment vs /etc/profile.d/env.sh.

The term 'System.DefaultWorkingDirectory' is not recognized in YAML pipeline

I had a pipeline with PowerShell task that run some python script. It worked without any problems.
After I convert my pipeline into YAML format to store it as code and got something like this (a part of whole yaml pipeline):
variables:
Build.SyncSources: false
REPO_PATH_DA: '/asdfg/qwerty'
REPO_PATH_DS: '/zxcvbn/tyuio'
PIP_REPO_HOST: 'bbbb.nnnn.yyyy.com'
PIP_REPO_URL: 'https://$(PIP_REPO_HOST)/api/pypi/pypi/simple'
PIP_VENV_NAME: 'my_test_venv'
SelectedBranch: ''
WorkingDirectory: $(System.DefaultWorkingDirectory)
…………………………………………….
- task: PowerShell#1
displayName: 'Install package'
inputs:
scriptType: inlineScript
inlineScript: |
.\$(PIP_VENV_NAME)\Scripts\activate
python.exe -m pip install --index-url=$(PIP_REPO_URL) --trusted-host=$(PIP_REPO_HOST) mypackage
python.exe -m pip install --index-url=$(PIP_REPO_URL) --trusted-host=$(PIP_REPO_HOST) $(WorkingDirectory)$(REPO_PATH_DA)\qwerty
And after I run this pipeline I get an error:
##[error]System.DefaultWorkingDirectory : The term 'System.DefaultWorkingDirectory' is not recognized as the name of a cmdlet, function, script file, or operable program. Check the spelling of the name, or if a path was included, verify that the path is correct and try again.
I tried to change the name of variable in format: $(env:System_DefaultWorkingDirectory) , but no success. I suppose that predefined variables are not passed into yaml pipeline. Do you have any ideas how to resolve it?
Based on my test, the same script could work fine in my yaml pipeline. The $(WorkingDirectory) will be converted to paths xxx/xx/s.
To check if the predefined variable: $(System.DefaultWorkingDirectory) has been passed to Yaml Pipeline.
You could add a task to list all Environment variables:
steps:
- script: SET | more
In the task log, you could search the SYSTEM_DEFAULTWORKINGDIRECTORY and check if the variable exists.
For example:
If this environment variable exists, you could try to use the following format: $env:SYSTEM_DEFAULTWORKINGDIRECTORY
Here is the example:
variables:
Build.SyncSources: false
.....
WorkingDirectory: '$env:SYSTEM_DEFAULTWORKINGDIRECTORY'
If you couldn't find this variable, you can also check if there are equivalent variables.
For example:
BUILD_SOURCESDIRECTORY , BUILD_REPOSITORY_LOCALPATH
Is this a build or a release pipeline? If it's a release, you may need to use $(Pipeline.Workspace) instead of $(System.DefaultWorkingDirectory).
Also, have you tried using $(System.WorkingDirectory) directly in your Powershell task, instead of declaring a variable that references it?

Jenkins Windows Slave setup using Winsw not working

Using this info https://hayato-iriumi.net/2019/05/23/how-to-install-jenkins-slave-as-windows-service/ we are setting up the Jenkins Slave on Windows server. Jenkins agent start from command line. but when we start from Windows service, its giving below error message? How to resolve this error message?
Service cannot be started. System.IO.InvalidDataException: Attribute <className> is missing in configuration XML
at winsw.Util.XmlHelper.SingleAttribute[TAttributeType](XmlElement node, String attributeName)
at winsw.Extensions.WinSWExtensionDescriptor.FromXml(XmlElement node)
at winsw.Extensions.WinSWExtensionManager.LoadExtension(String id)
at winsw.Extensions.WinSWExtensionManager.LoadExtensions()
at winsw.WrapperService.OnStart(String[] args)
at System.ServiceProcess.ServiceBase.ServiceQueuedMainCallback(Object state)
xml file we have
<service>
<id>JenkinsSlave</id>
<name>Jenkins agent</name>
<description>This service runs an agent for Jenkins automation server.</description>
<executable>c:\java\jdk-11\bin\java.exe</executable>
<arguments>-Xrs -jar "c:\jenkins\slave.jar" -jnlpUrl https://jenkinsmaster/jenkins/computer/slave01/slave-agent.jnlp -secret a4b5b4ddfd34a016cd3a8eb94cbe8f908613e33a66db5fa6f5f43a080aea3116 -workDir=c:\jenkins</arguments>
<workingdirectory>c:\jenkins</workingdirectory>
<logmode>rotate</logmode>
<onfailure action="restart">
<download from="https://jenkinsmaster/jenkins/jnlpJars/slave.jar" to="c:\jenkins\slave.jar">
<extensions>
<extension enabled="false" classname="winsw.Plugins.RunawayProcessKiller.RunawayProcessKillerExtension" id="killOnStartup">
<pidfile>c:\jenkins\jenkins_agent.pid</pidfile>
<stoptimeout>5000</stoptimeout>
<stopparentfirst>false</stopparentfirst>
</extension>
</extensions>
</download>
</onfailure>
</service>
Thanks
There are some errors in the sample "Jenkins-Slave.xml" that is presented at the linked web page (which yours is apparently based on). It has all elements and attributes named in lowercase, but actually some of them should be mixed case (as can be seen by the error message that it doesn't find attribute className).
Try this one instead:
<service>
<id>YourJenkinsSlaveServiceId</id>
<name>Your Jenkins Slave Service Name</name>
<description>This service runs an agent for Jenkins automation server.</description>
<executable>C:\Program Files\Java\JRE8\bin\java.exe</executable>
<arguments>-Xrs -jar "%BASE%\slave.jar" -jnlpUrl http://YourJenkinsServer:8080/computer/YourNodeName/slave-agent.jnlp -secret YourSecretStringConsistingOfHexadecimalCharacters -workDir=C:\YourNodeWorkDir</arguments>
<logmode>rotate</logmode>
<onfailure action="restart" />
<download from="http://YourJenkinsServer:8080/jnlpJars/agent.jar" to="%BASE%\slave.jar"/>
<extensions>
<extension enabled="true" className="winsw.Plugins.RunawayProcessKiller.RunawayProcessKillerExtension" id="killOnStartup">
<pidfile>%BASE%\jenkins_agent.pid</pidfile>
<stopTimeout>5000</stopTimeout>
<stopParentFirst>false</stopParentFirst>
</extension>
</extensions>
</service>
This is from a more detailed explanation of how to install an agent as a Windows service which I have given in this answer.
I too has the same issue similarly..
System.IO.FileNotFoundException: Unable to locate jenkins.xml file within executable directory or any parents
at winsw.ServiceDescriptor..ctor()
at winsw.WrapperService.Run(String[] _args, ServiceDescriptor descriptor)
at winsw.WrapperService.Main(String[] args)
Solution:
Save your jenkins-agent file as xml format. not just jenkins-agent.xml
we can use notepad++ to save the xml type.
similarly remove .exe at end of jenkins-agent.exe
Hope its useful.... cheers !!
The className, stopTimeout and stopParentFirst should not be in lower case.

Create Job Dynamically in Jenkins

I have a job in jenkins with a configuration, then, with the jenkins API in /cli i can get-job (API method) with an xml structure of my job and then i can create-job (API method) in jenkins with the followed xml.
?xml version='1.0' encoding='UTF-8'?>
<project>
<actions/>
<description></description>
<keepDependencies>false</keepDependencies>
<properties/>
<scm class="hudson.plugins.git.GitSCM" plugin="git#2.2.7">
<configVersion>2</configVersion>
<userRemoteConfigs>
<hudson.plugins.git.UserRemoteConfig>
<url>https://username:password#bitbucket.org/repoowner/project.git</url>
<credentialsId>550e8400-e29b-41d4-a716-446655440000</credentialsId>
</hudson.plugins.git.UserRemoteConfig>
</userRemoteConfigs>
....
Even if i give this url tag "https://username:password#bitbucket.org/repoowner/project.git" jenkins needs authentication to work, so in credentialsId tag jenkins give an UUID.
I want to be able to create a job dynamically by an external application with a given URL in this format "https://username:password#bitbucket.org/repoowner/project.git".
How can it be done?
Thanks.
You can get the credentialsId via the API and the credentials-store plugin.
e.g. for credentials in global Domain
${ your-jenkins-domain }/credential-store/domain/_/api/xml
<domainWrapper>
<credentials>
<_XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXX/>
</credentials>
<description></description>
<displayName></displayName>
<fullDisplayName></fullDisplayName>
<fullName>credential-store/_</fullName>
<global>true</global>
<urlName>_</urlName>
</domainWrapper>
But on some point it is a bit tricky:
when accessing the xml api for global domain the id already has a '_' as prefix. When using other domains the prefix is missing (but in a job a prefix is added... couldn't figure out where the prefix can be found)
e.g. I stored github access data in a separated domain, the credentialsId tag was:
<XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXX/>
but used in a job id was:
0XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXX
Actually i solve the problem just creating an generic user in bitbucket, then the UUID its always the same and i can just copy and paste that UUID to the others project.xml files.

Ant scp task failure

I have one requirement: copy local files to remote system.
I have done the following:
downloaded jsch-0.1.44.jar and copied to lib folder of Ant
set the path and every thing
My buildfile is:
<project name="ImportedBuild" default="all">
<target name="copyFileToRemote">
<echo>2222222222 copyFileToRemote Examples:::::::::::::</echo>
<scp file="sample.txt" todir="${username}:${password}#${hostname}:/shared"/>
</target>
</project>
When I run Ant, I get this error:
BUILD FAILED com.jcraft.jsch.JSchException: reject HostKey: 10.184.74.168
at com.jcraft.jsch.Session.checkHost(Session.java:712)
at com.jcraft.jsch.Session.connect(Session.java:313)
at com.jcraft.jsch.Session.connect(Session.java:154)
at org.apache.tools.ant.taskdefs.optional.ssh.SSHBase.openSession(SSHBase.java:212)
at org.apache.tools.ant.taskdefs.optional.ssh.Scp.upload(Scp.java:291)
at org.apache.tools.ant.taskdefs.optional.ssh.Scp.execute(Scp.java:203)
at org.apache.tools.ant.UnknownElement.execute(UnknownElement.java:288)
... etc ...
Any ideas how to resolve this?
According to the Ant scp task docs, trust attribute:
This trusts all unknown hosts if set
to yes/true. Note If you set this to
false (the default), the host you
connect to must be listed in your
knownhosts file, this also implies
that the file exists.
The trust attribute is not used in your task call, so it appears that the host (10.184.74.168) is not in your knownhosts file. Suggest you add trust="true", or add the host to the knownhosts file.
Be sure your ~/.ssh/known_hosts file is using un-hashed hostnames; if the lines start |1|base64data..., JSch appears unable to parse them. Create lines of the format hostname[,hostname|ip]* ssh-keytype base64data....
See man 8 sshd on the precise format of known_hosts, and tips on where to find the host's public key.

Resources