TFS linux agent ignores Build.Repository.Clean variable - tfs

Situation:
TFS 2018
build process consists of two phases -- executed on Windows and Linux agents respectively
Get sources (TFVC) has Clean flag set to false
Problem:
Linux agent (unlike it's counterpart) ignores Clean flag -- it always executes tf workspace -delete, ... -new, etc.
I've added "print variables" step and could clearly see that Build.Repository.Clean is false in every phase.
Is there any way to fix this?
More info:
Setting system.debug to true produces interesting log:
Querying workspace information.
##[debug]tf workspaces -format:xml "-collection:<tfs-server-url>" -jwt:******** -noprompt
##[debug]No workspace matching *;Project Collection Build Service (<collection-name>) on computer <computer-name> found in Team Foundation Server <tfs-server-url>.
##[debug]<?xml version="1.0" encoding="utf-8"?><workspaces/>
##[debug]Sources directory does not exist or is empty.
##[debug]tf workspaces ws_1_14 -computer:* -format:xml "-collection:<tfs-server-url>" -jwt:******** -noprompt
##[debug]No workspace matching ws_1_14;Project Collection Build Service (<collection-name>) on computer * found in Team Foundation Server <tfs-server-url>.
##[debug]Deleting: '/home/<user>/agent/_work/1/s'.
##[command]tf workspace -new -location:local -permission:Public ws_1_14 "-collection:<tfs-server-url>" -jwt:******** -noprompt
Workspace 'ws_1_14' created.
But when I login to that machine and run these commands myself (without -jwt:... -noprompt) -- I get correct information about ws_1_14 workspace. Agent is being executed as daemon/service created via sudo ./svc.sh install; sudo ./svc.sh start. Which means problem is somewhere in that daemon's environment.

This happens because my hostname was FQDN (instead of shortname). It is an existing bug.
Solution is to change hostname to be short and without dots(e.g. sudo hostnamectl set-hostname shortname) and run build again (it will recreate workspace with short hostname in Computer column) and all subsequent builds should work fine.

Related

Jenkins Terminal does not support executing hg purge

I am using jenkins, when executing Hg purge --all -R D:\path in jenkins pipeline it will return hg: unknown command 'purge'.
But when executing the same command in Windows Terminal it will execute correctly
Note : Jenkins in the same machine the Mercurial exist and purge plugin exist in the big repo and and sub repo
How to solve that issue ?
On the rights on delirium:
You may have more than one Mercurial on your host and Jenkins configured to use another instance (without extension), than command-prompt
You run Jenkins under different user and different mercurial.ini used, without extension (per-repository config can't be read)
You can (select any method):
enable extension on system-wide basis (and disable it in repositories, there you don't want it)
replace extension by "old-style" mercurial alias
forcibly enable extension for single command hg purge --config extensions.purge=
From my POV, system-wide extension in mercurial.ini will be most easy and logical solution

TFS Release CopyFiles task gives code EPERM

I'm running latest version of TFS 2015 release manager. I have a simple release that includes a "CopyFiles" task. I need to have the "Clean Target Folder" option of the task enabled.
I have three target environments configured. Both are newly created identical Windows servers. The CopyFiles task works fine against the first two servers but for the next environment in the pipeline, when copying to the second server I get the following error:
2016-11-11T21:41:07.4324714Z found 2 files
2016-11-11T21:41:07.4324714Z ##[debug]file:D:/Build/_work/df9f2c4cf/(master) Orchard BTDF/drop/20161111.5.zip will be copied.
2016-11-11T21:41:07.4324714Z ##[debug]file:D:/Build/_work/df9f2c4cf/(master) Orchard BTDF/drop/Unzip.ps1 will be copied.
2016-11-11T21:41:07.4324714Z Cleaning target folder: \\Steatbt02\BizTalkDeployments2\Orchard
2016-11-11T21:41:07.4324714Z ##[debug]rm -rf \\Steatbt02\BizTalkDeployments2\Orchard
2016-11-11T21:41:07.4480915Z rm: could not remove directory (code EPERM): \\Steatbt02\BizTalkDeployments2\Orchard
2016-11-11T21:41:07.4480915Z ##[debug]task result: Failed
There's no sign of errors in the event log.
I created the share "\Steatbt02\BizTalkDeployments2" and granted "full-access" permissions to the account running the build agent. I even tried logging onto the build server and run as for the command prompt using the credentials of the build agent. From this command window I was able to remove the target folder
"\Steatbt02\BizTalkDeployments2\Orchard" - so I can't think why the build agent would have trouble doing the same!
Any ideas what could be wrong, or what extra steps I could take to track down the root cause of the problem?
The problem was the agent queue selected for the problem environment.

Running a Windows batch file in Jenkins

I trying to run my Tests cases for a project in Jenkins through DOS Command line on Windows server . I am using Jenkins ver. 1.559.
I am trying to run a Windows batch file through a Jenkins job that has the java command.
Under the "Build" section >> "Execute Windows Batch command" >> Command: call E:\Jenkins\App\UnitTests\App_UnitTests.bat
Save and Build the job.
I am getting an error and here is my Console Output
C:\Program Files (x86)\Jenkins\jobs\App Test Cases\workspace>java.exe -cp "E:\Jenkins\App\PPS\App\bin\;E:\Jenkins\App\PPS\App\lib\junit-4.10.jar;E:\Jenkins\App\PPS\App\lib\*.jar;" org.junit.runner.JUnitCore com.omnitracs.fra.junit.EventTests
'java.exe' is not recognized as an internal or external command, operable program or batch file.
C:\Program Files (x86)\Jenkins\jobs\App Test Cases\workspace>PAUSE
Press any key to continue . . .
C:\Program Files (x86)\Jenkins\jobs\App Test Cases\workspace>EXIT
Build step 'Execute Windows batch command' marked build as failure
Finished: FAILURE
The java command runs fine outside Jenkins.
I have set the JAVA_HOME for the local user and also the PATH variable to point to where java.exe is.
What am I doing wrong? Please help.
For a windows environment, there is some extra configuration needed for more advanced builds. Ultimately you have two options
From Jenkins, The Definitive Guide:
This basic installation will work fine in a simple context, but you
will often need to fine-tune your service. For example, by default,
the Jenkins service will be running under the local System account.
However, if you are using Maven, Jenkins will need an .m2 directory
and a settings.xml file in the home directory. Similarly, if you
are using Groovy, you might need a .groovy/lib directory. And so on.
To allow this, and to make testing your Jenkins install easier, make
sure you run this service under a real user account with the correct
development environment set up. Alternatively, run the application as the
system user, but use the System Information page in Jenkins to check
the /Users/johnsmart/Projects/Books/jenkins-thedefinitive- guide
directory, and place any files that must be placed in the user home
directory here....
To configure your Jenkins server to run under a service account (The suggested option), Install Jenkins as a service, and under the services properties set the log on user info.

Jenkins time-out while fetching from Github in 10 mins.

I've stuck with this issue while configuring Jenkins for Nightly build. Please note that the repository project "project1" is large and is about 900MB. Please let me know how should I go around this problem.
Started by user anonymous
Building in workspace C:\Users\user1\.jenkins\jobs\Nightly Build\workspace
Fetching changes from the remote Git repository
Fetching upstream changes from git#github.com:MyOrg/projectgroup/project1
ERROR: Timeout after 10 minutes
FATAL: Failed to fetch from git#github.com:MyOrg/projectgroup/project1
hudson.plugins.git.GitException: Failed to fetch from git#github.com:MyOrg/projectgroup/project1
at hudson.plugins.git.GitSCM.fetchFrom(GitSCM.java:612)
at hudson.plugins.git.GitSCM.retrieveChanges(GitSCM.java:836)
at hudson.plugins.git.GitSCM.checkout(GitSCM.java:861)
at hudson.model.AbstractProject.checkout(AbstractProject.java:1412)
at hudson.model.AbstractBuild$AbstractBuildExecution.defaultCheckout(AbstractBuild.java:652)
at jenkins.scm.SCMCheckoutStrategy.checkout(SCMCheckoutStrategy.java:88)
at hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:557)
at hudson.model.Run.execute(Run.java:1679)
at hudson.maven.MavenModuleSetBuild.run(MavenModuleSetBuild.java:509)
at hudson.model.ResourceController.execute(ResourceController.java:88)
at hudson.model.Executor.run(Executor.java:230)
Caused by: hudson.plugins.git.GitException: Command "fetch -t git#github.com:MyOrg/projectgroup/project1 +refs/heads/*:refs/remotes/origin/*" returned status code -1:
stdout:
stderr:
at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandIn(CliGitAPIImpl.java:981)
at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandWithCredentials(CliGitAPIImpl.java:920)
at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.fetch(CliGitAPIImpl.java:187)
at hudson.plugins.git.GitAPI.fetch(GitAPI.java:229)
at hudson.plugins.git.GitSCM.fetchFrom(GitSCM.java:610)
... 10 more
I was about to use Workaround #2 but it seems as of git plugin 2.0.3 this can now be configured in the UI, although it is a bit hidden away and is per project.
Go to the configure screen for a project, Source Code Management section, Git, Additional Behaviors, Add, Advanced clone behaviors, Timeout (in minutes) for clone and fetch operation.
I would use workaround #2 as suggested by DevHopeful_2012 if you want a global setting.
It looks like this is a known issue. See JENKINS-20445, JENKINS-20387, and several other issues, which seem to be popping up.
According to these bug reports, this is only happening in the newer version of the Git plugin, so you could downgrade, or try some workarounds:
Workaround #1 - create a local bare reference clone to reduce the time, as stated here.
If you have only 30 kb/second throughput from Jenkins to your git repository, you should probably consider cloning a bare copy of the
repository to central location on the Jenkins server, then perform the
clone with the "Advanced clone behaviour" to use a reference
repository. That will significantly reduce the amount of data
transferred from the git server to Jenkins.
On my Debian Jenkins machine, I do that with:
$ sudo mkdir -p /var/cache/git/mwaite
$ sudo chown mwaite.mwaite /var/cache/git/mwaite
$ cd /var/cache/git/mwaite
$ git clone --bare https://github.com/jenkinsci/jenkins.git
After that bare clone is available on the Jenkins machine, add that
advanced behavior to the job and it should perform much better.
Workaround #2 - increase the timeout to allow for the long initial clone process to complete, using the Git.timeOut property:
java -Dorg.jenkinsci.plugins.gitclient.Git.timeOut=60 -jar jenkins.war
This SO question is a good example of how to set properties for the Jenkins service. Also, note that this value is in minutes and not seconds. This workaround is courtesy of David.
The latter workaround worked for me. Although, I must admit that I'd prefer to use the reference repo, but I don't believe that it works with a multiconfiguration job, such as mine. And, be forewarned that it was surprisingly painful to set a property for every slave node and then internally document that we have to set this on all nodes, using large Git repos, going forward.
WINDOWS ONLY
There is a possibility that your ssh connection is not configured properly. If the ssh connection to your git repository is not estabilished, Jenkins build may appear as if it hung and eventually timeout.
By default, the Jenkins installer sets up Jenkins to run as a service,
which runs as the “Local System account”, NOT your user account. Since
the “Local System account” does not have SSH keys or known_hosts set
up, “git clone” will fail.
You can just copy C:\Program Files (x86)\Git\.ssh to
C:\Windows\SysWOW64\config\systemprofile\.ssh (the “Local System
account” home)
Also,
IMPORTANT: make sure your ssh keys do NOT have a password! Jenkins
will appear to hang when cloning the repository, but really it’s ssh
blocking in the background waiting for you to input your password.
Detailed steps in http://computercamp-cdwilson-us.tumblr.com/post/48589650930/jenkins-git-clone-via-ssh-on-windows-7-x64
I just found the solution for jenkins timout #10 min errors. This error was coming because of low Internet connection or may be the size of you project in github is large, so jenkins not able load that project from git server.For resolve the problem we have to edit "Additional Behaviours" in Git in jenkins project configuration and increase "Timeout (in minutes) for clone and fetch operations" to "60" or "120" minutes. So now whenever jenkins clone the git project from git server, its enough time to load/clone a project from git server.
Workaround #2 worked for me, I had to change jenkins.xml file.
On Windows:
Go to C:\Program Files (x86)\Jenkins
open jenkins.xml file
add -Dorg.jenkinsci.plugins.gitclient.Git.timeOut=60 inside <arguments> tag
This is how my configuration looks like
Before:
<arguments>-Xrs -Xmx256m -Dhudson.lifecycle=hudson.lifecycle.WindowsServiceLifecycle -jar "%BASE%\jenkins.war" --httpPort=8080 --webroot="%BASE%\war"</arguments>
After:
<arguments>-Xrs -Xmx256m -Dhudson.lifecycle=hudson.lifecycle.WindowsServiceLifecycle -Dorg.jenkinsci.plugins.gitclient.Git.timeOut=60 -jar "%BASE%\jenkins.war" --httpPort=8080 --webroot="%BASE%\war"</arguments>
In Ubuntu 14.04 lts
Goto /etc/default/jenkins edit with vi or gedit
update java orgs like below
Before update, it will be like
JAVA_ARGS="-Djava.awt.headless=true"
Update like below
JAVA_ARGS="-Djava.awt.headless=true -Dorg.jenkinsci.plugins.gitclient.Git.timeOut=30"
and restart jenkins like
sudo /etc/init.d/jenkins start
On CentOS/RedHat 7.4 Linux with Jenkins-2.138
Edit /etc/sysconfig/jenkins, modify the JENKINS_ARGS (last line) with the timeout setting mentioned by others in their answers:
# Pass arbitrary arguments to Jenkins.
# Full option list: java -jar jenkins.war --help
#
JENKINS_ARGS="-Dorg.jenkinsci.plugins.gitclient.Git.timeOut=25"

Jenkins: How To Build multiple projects from a TFS repository?

I have set my workspace directory to C:\jenkins_builds\workspace and I want to build ProjA and ProjB, each having a local workfolder (same as project name).
When fetching the source code from my repository, the first two things the TFS plugin does are:
tf workspace -new %workspace-name-A%;%user-name% -server:%my-server%
tf workfold -map $%branch% ProjA -workspace:%workspace-name-A% -server:%my-server%
Which goes fine when building ProjA. The problem is, the first command maps the root directory from the repository directly to my C:\jenkins_builds\workspace directory. The second command does what I actually want, i.e. mapping %branch% to the ProjA subfolder. Later on, when building ProjB, the first command fails (and consequently the build) with the following error message:
The path C:\jenkins_builds\workspace is already mapped in workspace %workspace-name-A%;%user-name%.
OK, it seems like a bad idea to map the root directory to the work directory. But why does this automatically happen when the TFS plugin runs the workspace new line? Currently I have to clean things up between building ProjA and ProjB by running the -unmap command.
My team is using Team Foundation 3.0.
We have the same situation and there are 2 ways to solve this:
use different workspace-root-directories for the two builds
This results in the need for two checkouts => double the space and slower, but better isolation between the two builds
"hardcode" the workspace name to the same for both builds
By default jenkins creates a workspace containing the build name, which can be changed in the "advanced" section of the TFS config, and then you can use the same workspace-/workfolder-mapping for several builds - in our case we called them ProjectName_${NODE_NAME} so it even works on several nodes

Resources