Earlier I used my local workspace to keep all the project contents. Now I moved that to a remote server. I just created empty folder (remote_ws) in my local PC and mount that with remote workspace (ws). Before I open eclipse giving following command in a terminal and then it will mount local workspace folder(remote_ws) and remote workspace folder(ws). By giving that command all the remote contents are available in local PC.
Command is,
sshfs -o nonempty hera#192.168.1.83:/projects/project_hera/ws /external/remote_ws/
I am using ant build to build this project. My problem is earlier it took around 2 minutes to build this project. But now it is taking very long time (around 15 minutes) to build the same project. To copy content to a mount local folder (remote_ws) is taking very long time.
How can I speedup this build process? Please help.
Build (compilation, creation jar, war) etc involves a lot IO. You cannot make it fast without improving over-all IO performance - faster disk, faster network etc.
Build will be faster on the machine where source code exist. I would advice to use a CI (say jenkins and do a build on the remote server. They can checkout, build , test and deploy without any manual intervention.
Related
Currently my TFS build is creating the workspace in build machine and all my remaining task are performed in a network share in UNIX machine. So i copy my workspace to that network share before proceeding with the build. Is there any way to directly create the workspace in NetworkShare, so that i can stop copying process.
If you mean the UNIX project have dependencies which generated on windows build, I am afraid you have to add a copy task to copy them first, unless you can build the windows and UNIX projects on the same agent, but generally it's not possible.
However if you want to build them on a network shared path, then you just need to deploy a new build agent and set the network path as the working folder. Thus you can directly reference the dependencies on the shared path.
I have set up a CI Build that is also executing some tests.
In the GetSources step Clean is set to true.
I use a git repo.
When I run the build in a hosted agent, "getSources" takes about 20 seconds. When I run the build on a on premise agent, "getSources" takes about 20 minutes!
I can see, that the on premise agent is mostly idle in terms of cpu and memory. I also verified that network speed is at around 50 MBit/s.
Why does getSources take so long?
The Clean option has no effect for Hosted agent.
No matter what you set for the clean option (false or true for clean sources/all build directories/output directory etc), when you queue build with Hosted agent, it always download the sources only.
Assume if you set Clean as true and clean all build directories. For private agent, it will delete the entire working folder that contains the sources folder, binaries folder, artifact folder, and so on. But for Hosted agent, it only download the sources each time.
So the execute time for Hosted agent is different from private agent for most time.
To speed up private agent to build, you can follow any of below aspect:
set Clean as false (more efficient).
Since you are queuing CI build, if you set Clean as false, the private agent will only update the files which has been modified/created/deleted to your local source folder.
Only clean sources if you still need to set Clean as true.
It will only clean up the files and subfolders source folder (s/). But if your project is large enough, you'd better use the way to set Clean as false.
To answer the "why?", it's the fact that you're copying all those files between your system and Microsoft's data center. I've experienced the same and bought some additional pipelines to accommodate.
I have found many examples of Linux to Windows or OSX to Windows, but battling to find how to copy build files from a Windows server installation of Jenkins, to the
Windows server which will host the website.
I tried Copy Artifact Plugin - but it seems this is used to copy other builds into the current build - is this correct?
Or can it be used to copy the build that just execute to a remote Windows folder?
What should I use to copy to another Windows server?
I would like to not use batch files. Can/should powershell be used?
The idea would be to copy the files to a timestamped folder on the hosting Windows server
e.g.
xcopy *.* \\MyHostingServer\Temp
Then only after everything has copied successfully - to copy the files from MyHostingServer\Temp into the correct web folder. This is to prevent the deployment to the web folder if e.g. network connection was lost halfway.
I am having a really hard time getting Windows specific information.
I suggest to use two jobs:
build job is running on the first Windows server. Archive your build files as an artifacts to the job
deployment job should run on second Windows server. You can use "Copy artifacts from another project" to get your build files locally (on second server)
We have Jenkins Docker Slave template that successfully builds a piece of software for example a Gradle project. This is based on the https://hub.docker.com/r/evarga/jenkins-slave/).
When we fire up the docker slave the dependencies are downloaded everytime we do a build. We would like to speed up the build so dependencies that are downloaded can be reused by the same build or even by other builds.
Is there a way to specify an external folder so that cache is used? Or another solution that reuses the same cache?
I think, the described answers work only for exclusive caches for every build-job. If I have different jenkins-jobs running on docker-slaves, I will get some trouble with this scenario. If the jobs run on the same time and write to the same mounted cache in the host-filesystem, it can become corrupted. Or you must mount a folder with the job-name as part of the filesystem-path (one jenkins-job run only once at a time).
Here's an example for maven dependencies, it's exactly what Opal suggested. You create a Volume, wich refers to cache folder of the host.
I'm trying to reduce build times and right now Source Indexing and Symbol Publishing with TFS 2015 takes (~1hr). Maybe indexing sources and publishing symbols is just heavy on disk I/O and bottle-necked there -- I'm unsure. I want sources to continue to be indexed and symbols to continue to be published for this particular build as it makes debugging exponentially easier.
Are there any ways to make source indexing and symbol publishing with TFS 2015 faster?
It's hard to just reduce the time of this task "Source Indexing/Symbol Publishing "
However, there are other ways to reduce the build timeļ¼Such as setting clean workspace to none. Changing the workspace setting from recreate a fresh workspace every time to incremental by which it will incrementally download the source to the build workspace only.
During the build process, the build agent compiles and does other work with your source files. Before the build agent can do this work, it downloads the files from folders on your version control server into a local working directory. To facilitate downloading these files, the build agent creates a version control workspace, which maps the folders on the server to the local folders in the working directory for the build agent. If you set clean workspace , it will delete the old files and get down the sources during every triggered build. So set clean workspace to none can reduce the time of the build.
And it's also related to the hardware of your server. Improve the performance of the server will also reduce your build times.