What is standard for building large projects in jenkins (eg. 80Gig of dependent jobs building libs and binaries from one repo)?
if you split this large build into jobs that build libs and others that build downstream apps, should you use jenkins user to build the project, or setup a separate build user to build the project?
What if you farm out to a remote agent using SSH?
jenkins user is not allowed to remote by default into another host because of this /bin/false here:
$ grep jenkins /etc/passwd
jenkins:x:996:992:Jenkins Automation Server:/var/lib/jenkins:/bin/false
so does this mean all farm jobs run as a build user or should you change this setting to /bin/sh to complete the ssh session as jenkins user? Or use only setup JNLP agents?
If you build the libs on the jenkins master should the .o and binaries be owned as jenkins or not?
If you build on a dedicated jenkins master should the executors be agents setup to use build user rather than the jenkins user?
If using a NFS type mount for sharing build artifacts how does that look?
Related to the 2 previous questions - all local jobs would have build ownership under jenkins user. So does that mean you use a build user and use remote node to localhost as the build user just to make the artifacts all have the same user for all local and remote agents to use?
These may sound stupid questions but I can't find any guidelines on who should be the build owner or what is best practice in making a very large single git repo project build in a sane way (repo owners do not want to split the code up into different repos because of static linking).
In our experience (very large monorepo, 250+ slaves):
We united several jobs into one big job, with parallel stages where applicable, so independent things can be built at the same time on different slaves (to cut time). Thus, it is easier to follow what failed and why, and you have all the artifacts in one place, and there's one Jenkinsfile to follow.
All our slaves are set up as JLNP, and when they reboot they start jenkins-agent. There's no jenkins user on our slaves.
As you are supposed to pick all the artifacts and archive them in the end, preferably cleaning the slave into zero state, it does not matter who owns that, and you can always change it with chown.
NFS would not be a great idea for this in our place, as it would be severely constrained by network and disk usage. We use Docker registry for docker images, but Artifactory might work if you're not using Docker. minio would be another option.
Related
I have a Windows VM that hosts a VSTS build agent. Due to the number and length of builds that are running I would like to know whether multiple build agents can be hosted on one computer? That would allow a dedicated agent for slow builds, and a dedicated agent for quick builds.
https://www.visualstudio.com/en-us/docs/build/admin/agents/v2-windows
Yes you can run multiple agents in a single VM.
Make two directories say Agent1 and Agent2, extract the agent in each one of them and configure them with different names against your VSTS/TFS account.
It should work out of the box.
We run 4 agent jobs per machine concurrently with no issues. As mentioned above, should work out of the box. Just make sure you clean up directories. We have a script to do it every night
Yes, this works, I did the following:
Created a PAT for agent installation needs
Downloaded agent binaries from the agent creation page
Unpacked the archive contents into 2 different directories ("c:\ado-build-agents\agent1" and "c:\ado-build-agents\agent2")
Ran "config.cmd" and followed configuration instructions, provided by it.
Updated pipelines to build the agent pool, which those agents reside in ("Default" in my case)
To test the setup - triggered all 15 pipelines, that I had. As the result I was able to see two pipelines running at the same time, while others were in the "Queued" state (according to my expectations).
I will be also testing out how resources are consumed by the agents to try to understand if I should deploy more agents on the build machine.
There is a job controlled by Development team which built in a different node. I am on Testing team who want to take the artifacts and deploy on test device.
I can see those Artifacts from dev are stored in some path in dev's node. Does it means it must first archived in Jenkins master before I can copy it to my job?
I am using Copy Artifact plugin and constantly getting the error
Failed to copy artifacts from <dev-job> with filter: <path-in-dev-node>
*Some newbie question since i just moved from TeamCity
You probably want to use: Copy Artifact plugin.
Adds a build step to copy artifacts from another project.
Consider also, the Jenkins post-buid step "Archive the artifacts".
If you copy from the other job's workspace, what happens if another job is in progress or the workspace is wiped? That step copies them from the node to the master and stores a copy along with the build logs, etc. That makes them available via the UI as long as the build logs remain. It can take up space tho.
If you do use archive artifacts, consider using the system property jenkins.model.Jenkins.buildsDir to store all the build logs (and artifacts) outside of the jobs config directory. Some downtime and work required to separate the two (config / logs) .
You may also want to consider using a proper repository manager (Nexus / artifactory)
Finally, you may want to learn about using a Jenkins pipeline rather the relying on chained jobs, triggers or users and so forth. Why? 'cos it's much more controlled and easier to maintain.
ps: I'm not a huge fan of artifactDeployer, but it may work for you.
pps: you may want to review this in depth answer: Jenkis downstream job fails to find upstream artifacts
Let's say you have a gitlab instance and it already uses Jenkins for all its CI builds via the gitlab Jenkins plugin, etc. The Jenkins setup has a modest collection of build slaves providing a variety of platforms, etc. and each slave is set up to run just one job at a time (i.e. a Jenkins job gets exclusive access to the build slave, which is important for reasons I won't go into here).
Now let's say you want to consider using gitlab's own native CI support, moving one or more projects over to gitlab instead of Jenkins. The gitlab CI would need to use the same set of build slaves, but it needs to play nice with Jenkins and the two need to cooperate so that if one runs a job on a particular slave, the other won't submit a job to that same slave until the first finishes. In effect, while Jenkins is running a job on a slave, gitlab should see that slave as unavailable and vice versa.
Anyone have working methods for getting gitlab to tell Jenkins it is using a slave while it runs a CI job on there and vice versa? The method doesn't have to be 100% bullet proof, it would potentially be okay if both gitlab and Jenkins run a job on the same slave at the same time if it is a rare event (i.e. race conditions could potentially be tolerated if the frequency of occurrence is likely to be low).
Additional info:
Build slaves include Linux, Windows and Apple.
Docker is not used and would not be permitted at this time.
We have full admin access to everything, but changing code in gitlab or Jenkins themselves would be rejected. Adding scripts or plugins would be okay.
I would like to set up jenkins server that would run test scripts based on successful build deployments on other Jenkins servers. for example, if the QA jenkins server is named JQA1OnMachine1 and i have three others that are named
J2OnMachine2, J3OnMachine3, J4OnMachine4 (different jenkins server on different boxes) can the JQA1OnMachine1 (QA jenkis) poll the others at regular interval to see if a build was deployed successfully? if so can anyone tell me how?
Jenkins master slave along with Jenkins Pipeline Plugin would be one of the better ways to implement this however, since you don't want to use that approach you can explore PSTools to remotely capture processes or files on different server.
Your builds may update a file on the build server post completion of the build and your QA machine can run script with PSTools to monitor and trigger the QA testing based on the file content
I have recently started to mess about with Jenkins and am unsure how to deploy my web app to a basic server. I've gotten into the Pipeline (https://jenkins.io/doc/book/pipeline/) and it seems like a fantastic way to work.
Where I'm a bit stuck is in two spots:
Once my repo is in my workspace within Jenkins, how do I prep it so I am only deploying the files necessary for the application? For example, I don't need my src/ directory or my Vagrantfile when I'm deploying things.
How do I deploy my app to the server? I see examples all over the place, but I am getting a bit lost since there seems to be so many ways to do this. I'm assuming scp or something like that...?
To build off of #2, is there a way to deploy web apps as transactions (in one shot) rather than file-by-file?
Please let me know if I can provide any information for potential answers!
I can't speak to your specific use case but a common way to do this is the build-and-deploy model, where you will have 2 Jenkins jobs. The "build" job will check out from source, run build commands such as maven or make, and lastly will "archive" the build artifacts. The latter is an option under the 'post-build actions' tab at the bottom.
In the "deploy" job, you will grab the artifacts of your choice. You can fetch a single file, all of them, and everything in between. This requires use of the 'Copy Artifact' plug-in and it allows you to copy files generated by other jobs. Now you can run your usual deploy script in the 'Execute Command' box. Most command line paradigms are supported out of the box such as setting environment variables.
The instructions above assume that you want to run your application off of a host that you've provisioned as a Jenkins slave.
Use artifacts as mentioned by Paul Back, or a 3rd party artifactory server as in video
This is always tricky and error-prone. Why not spin up a fresh server with new release (humanly verified once)
Jenkins & Ansible is the answer here. This is how I deploy to production, since I am in no need to use anything like Docker (too many issues with particular app) so have to run the app natively. Quick example would be
You monitor a specific branch in gitlab / github or whatever else and then call a webhook on push / merge etc on that branch, at this point you deal with anything you need to do by running a playbook on the jenkins job that monitors that branch (jenkins).
in my case jenkins and ansible run on the same server. Jenkins runs the ansible playbook that does whatever I need to do.
for example with ansible, I copy certain files that need to be there, run configs / change filenames etc. setup nginx, run composer,
you get the point.