I'm trying to create a CircleCI job in which after a specific step, I want to save the output of that step somewhere that other jobs/builds can read it
The pseudo flow would look something like:
A git branch triggers a build that has multiple jobs
The output of one of the steps gets uploaded somewhere in circle (maybe artifacts?)
Build completes
Then:
A different branch runs a build
When it gets to a job, it reads data that was uploaded by the previous build
Is there a good place to start for reading how to do this? I've been looking in circle's docs, but started getting a bit confused. Happy to answer any clarifying questions
Related
I setup my jmeter to run on jenkins daily. Everything works fine except the past jenkins build automatically flip to fail status the next day when a new schedule run kicks in and it also removed the past build results. I attached a screenshot of the build history. For sep 12, they were all green.
Anyone experience this issue or know which area I can look into?
My goal is to try to compare with past build result and send an email if the performance didn't pass criteria.
I am happy to share more details.
Unfortunately we're not telepathic enough to guess the reason by looking at a single screenshot.
Double check your job configuration, especially location of the JMeter's jtl result files as it might be the case you're storing them in current job folder instead of workspace
Double check your Performance Plugin configuration, especially "thresholds" section
Inspect Console Output for the previous/latest builds
Check jenkins.log file for any suspicious entries
I have a stage which checks whether any hardcoded credentials aren't going into the build.
This is done using a custom jenkins library.
Now, there are many pipelines in which I need to add, so I figured out two solutions :
Make a python script and manually figure out the text pattern in each/most of these Jenkinsfiles where I can add my stage.
Let Jenkins do the work of checking whether the stage exists and if not fail the build, so that the developer could herself/himself add the stage.
Now, 2nd is the one I would like to go with as it's quite scalable ( so to say ) and I don't have to go with the unreliableness of pattern searching to add the stage using python and moreover I already have tried out the 1st one.
This question is similar to this : jenkins-making-a-build-fail-if-javadoc-is-missing
In the above question, the solution seeker wishes to fail the build based on javadoc string.
The solution as suggested is a plugin, but I don't want to increase the complexity to this solution because for this I will have to learn plugin development for Jenkins and seeing that it's in JAVA, it will take even more time ( I am fluent in Python ).
I have worked, rather struggled with groovy to make a Jenkin libs, but I am ready to walk that path, if need be.
Thanks.
Just thinking if you could introduce some kind of global variable in your flow which will store all the env.STAGE_NAME at the beginning of each stage (need just one-liner on every stage). At the end of the flow, you validate all the stage names from the list and see if you are missing anything.
I'm slowly replacing traditional jobs with Jenkins pipelines. We've got some jobs which I've previously optimised by only deleting some key files from the workspace of a previous build - thus we end up with incremental builds rather than full ones. FTR this makes our basic builds 3/4 times faster, and I'm keen to preserve it.
I need to delete those files (to simplify real scenario) that contain "cache". I currently use "**/cache" as an include parameter to the Delete Workspace build step. Question: is there something similar already in pipeline steps? I could probably do it using find or similar, but this has to work on Windows too and that has portability implications.
You could use the cleanWS step to clean up certain parts of the workspace. However, it is a plugin you can find here: Workspace Cleanup Plugin.
You can find syntax about a snippet generator for this step at your-jenkins-url/pipeline-syntax/
I've switched away from using cleanWS having used it. Rather I am using the file operations to explicitly delete the files concerned.
The file operations act there and then. The cleanWs acts at the end of a run and can't be relied upon if that run went wrong and did not finish - e.g. syntax error - or that was running a different script.
I've used jenkins for quite a few years but have never set it up myself which I did at my new job. There are a couple questions and issues that I ran into.
Default workspace location - It seems like the latest Jenkins has the default workspace in Jenkins\jobs[projectName]\workspace and is overwritten (or wiped if selected) for every build. I thought that it should instead be in Jenkins\jobs[projectName]\builds[build_id]\ so that it would be able to store the workspace state for every build for future reference?
Displaying workspace on the project>Build_ID page - This goes along with the previous as I expected each 'workspace' for previous builds to show here. Currently in my setup this individual page gives you nothing except the Git revision and what repo changes triggered the build. As well as console output. Where are the artifacts? Where is the link to this build's workspace that was used?
Archiving Artifacts in builds - When choosing artifacts, the filter doesn't seem to work. My build creates a filestructure with the artifacts in it inside workspace. I want to store this and the artifacts filter says it starts at workspace. So I put in 'artifacts' and nothing gets stores (also where would this get stored?). I have also tried '/artifacts' and 'artifacts/*'.
Any help would be great! Thanks!
It does seem like you are confused about several aspects of Jenkins.. I think your question basically boils down to the following.
What is a difference between a workspace and a build?
So, here are some thoughts on this topic:
Builds are historical data. They (usually) don't change like a workspace does during building/checkout.
Builds contain information about a run (e.g. its status, build number, change log, etc) and any artifacts that you tell it to archive (logs, test results, etc). They (usually) don't contain source code like a workspace.
Builds are stored in the Jenkins\jobs\[projectName]\builds\[build_id]\ directory. This is a directory managed by Jenkins and you (usually) do not need to modify anything in this directory. However, workspaces are directories meant for the build and you can do pretty much anything with them and place them anywhere (it does not need to be in the default Jenkins\jobs\[projectName]\workspace directory.
Workspaces should be able to be wiped at any given time. To restore it, just rebuild the job with the same parameters/revision. If you need to keep something after a build, tell Jenkins to archive it before the build is done.
In regard to saving the entire state, I don't think you need to do that. As mentioned in #4, you should be able to reproduce the same build by kicking off the same revision/parameters as the build in question. If you cannot get back to the original state from the same revision/parameters, then that might be something to strive for as debugging is going to be a nightmare. :)
A workspace is an aspect of the project and not a build and that is why there is no link to the workspace from that page. Again, a build is just saved data from a previous run. A project uses the workspace to build stuff and that is why you can get to the workspace from that page.
In regard to how to save artifacts, you must specify the names of the files you want to save. Unless you are trying to save a file called "artifacts", then you should probably use something else. How about **/*.log for all log files? or **/*.xml for all xml files?
Hope this helps.
We've got a Jenkins setup where we do incremental builds on SCM change, validate and then if this works do a full build (from scratch). This basically works but we waste time doing full builds during the day that we don't normally use.
I know we could trigger full builds every night, but many of our branches won't change for a few days - and then we might get a rush of changes. Thus building every branch every night is wasteful too.
What I really want is some mechanism where we only do the full builds once (say at night) if there has been an SCM change and the incremental build and validate worked - there is no point auto-triggering full builds where the incremental build and validate failed. Actually just "the incremental build and validate worked" should suffice - as these normally just run on SCM change.
Any suggestions? Is there some Jenkins extension that would help with this?
To achieve what you've asked for you can create a new job that is the same as your existing one, but have it only poll the SCM once a day, for a nightly build.
Set the schedule to something like this: H H(0-5) * * *.
In your original job, remove the post-build triggering of a full build.
That will give you pretty much what you've asked for, except the nightly build will do an incremental build and then a full build if the incremental one succeeded, rather than just checking the result of the last incremental build.
BUT...
What is the cost of the 'waste' you are trying to avoid? How much does running a full build every night actually cost you? And wouldn't you be better off finding out when the full build is broken as soon as possible, i.e. during the day when it was broken rather than only the following morning?