I have inherited a system that runs helm through jenkins. I have editted the helm config-maps file to the point that it creates a file and that file should then be run from another file.
I cannot run any logs outside of Jenkins, so currently am unable to work out what the issue is.
Does helm log what it is doing anywhere?
What could I do in Jenkins to force extra logging to happen within Helm?
I have been googling for a few days now without any luck, so am hoping someone reads this will knows something about helm.
Related
I'm trying to build our code through jenkins. I set it up 4 days ago and everything worked back then, but now every build fails. At first I thought it was because of the Jest testing I added (jest kept hanging instead of stopping) but I fixed that with --forceExit and the problem still exists.
With every build, it seems like Jenkins is crashing silently. I couldn't find any useful logs or anything. What is weird though, is that I tried to execute the commands through ssh in the jenkins folder where the project was placed by jenkins. When I did that, the npm install was really slow and the whole server was hanging. There are two other projects running from that server with nodejs, one with keystonejs that crashes every time this happens, the other one is plain express and had no problems. Any idea what can be the problem here?
EDIT: It's also happening more randomly and I couldn't connect to my server for a while through ssh or any other possibility.
If is simple what you need to do, then don't use jenkins, is just to heavy.
Instead build an sh file that builds what you need, the down side of this is that you have to execute it by hand when you need to build.
I ran a build yesterday, hoping I would read some logs today.
I came today, and got an error 404 when trying to access the build. Strange.
Running another build, shows my build actually did run, but it is unreachable.
Is there a way to get my hands on the logs?
Notice build #10 is missing, even though it did start.
Probably a windows update is to blame for this.
The broken link is http://192.168.80.10:8080/job/Dev_git/10
More information on a run can usually be found using the context menu under Console Output. This is only accessible if you have the correct permissions set in Jenkins.
This of course does not work, if a build is missing. One reason could be that your Jenkins is configured in a way that only a certain number of historic builds are kept, see Build History Missing in Jenkins for an explanation how to deal with that.
However, your case seems to be different, because a build in the middle of the history is missing. For this, I suggest to look around in the jobs directory of your Jenkins installation where it stores all the configuration and run data.
References
https://wiki.jenkins.io/display/JENKINS/Administering+Jenkins
Where does Jenkins store configuration files for the jobs it runs?
Jenkins (on a Kubernetes node) is complaining it requires a newer version of Jenkins to run some of my plug-ins.
SEVERE: Failed Loading plugin Matrix Authorization Strategy Plugin
v2.4.2 (matrix-auth) java.io.IOException: Matrix Authorization
Strategy Plugin v2.4.2 failed to load.
- You must update Jenkins from v2.121.2 to v2.138.3 or later to run this plugin.
The same log file also complains farther down that it can't read my config file... I'm hoping this is just because of the version issue above, but I'm including it here in case it is a sign of deeper issues:
SEVERE: Failed Loading global config
java.io.IOException: Unable to read /var/jenkins_home/config.xml
I'd either like to disable the plug-ins that are causing the issue so I can see the Jenkins UI and manage the plug-ins from there, or I'd like to update Jenkins in a way that DOES NOT DELETE MY USER DATA AND JOB CONFIG DATA.
So far, I tried disabling ALL the plug-ins by adding .disabled files to the Jenkins plug-ins folder. That got rid of most of the errors, but it still complained about the plug-in above. So I removed the .disabled file for that, and now it's complaining about Jenkins not being a new enough version again (the error above).
Note: this installation of Jenkins is using a persistent storage volume, mounted with EFS. So that will probably help alleviate some of the restrictions around upgrading Jenkins, if that's what we need to do.
Finally, whatever we do with the plug-ins and Jenkins version, I need to make sure the change is going to persist if Kubernetes re-starts the node in the future. Unfortunately, I am pretty unfamiliar with Kubernetes, and I haven't discovered yet where these changes need to be made. I'm guessing the file that controls the Kubernetes deployment configuration?
This project is using Helm, in case that matters. But again, I hardly know anything about Helm, so I don't know what files you might need to see to make this question solvable. Please comment so I know what to include here to help provide the needed information.
We faced the same problem with our cluster, and we have a basic explanation about that, but not sure about it (The following fix works)
That error come with the fact that you have installed Jenkins via Helm, and their plugins through the Jenkins UI. It works if you decide to never reboot the pod, but if one day, jenkins have to make his initialization again, you will face that error.
Jenkins try to load plugins from the JENKINS_PLUGINS_DIR, which is empty, so the pod die.
To fix the current error, you should specify your plugin in the master.installPLugins parameter.
If you followed a normal install, just go on your cluster and
helm get values jenkins_release_name
So you may have something like that:
master:
enableRawHtmlMarkupFormatter: true
installPlugins:
- kubernetes:1.16.0
- workflow-job:2.32
By default, some values are "embedded" by helm to be sure that jenkins works, see here for more details: Github Helm Charts Jenkins
So, just copy it in a file with the same syntax and add your plugins with their versions. After, you have just to use the helm upgrade command with your file on your release:
helm upgrade [RELEASE] [CHART] -f your_file.yaml
Good luck !
I realize that this question has been asked before but I feel like my case is different.
About five hours after I launch jenkins I sometimes see this message appear when loading the jenkins administrator panel:
Jenkins detected that you appear to be running more than one instance of Jenkins that share the same home directory '’. This greatly confuses Jenkins and you will likely experience strange behaviors, so please correct the situation."
This is strange, it happens every time I launch it, eventually making Jenkins unusable. I installed Jenkins via homebrew and launch is via the jenkins CLI. I manually do this so I don't see how it's possible that Jenkins launches twice.
I reinstalled Jenkins after completely removing it. Still no luck.
Anyone have any issues like this before?
I also had a similar problem and solved it with following checks:
Check if another java process is running which runs jenkins e.g. in unix using top
If you're using tomcat, search for a second instance of tomcat having the jenkins.war exploded somewhere
Could be helpful to just have one version of java installed
I am on a redhat linux box. I recently updated Jenkins to version 1.509 only to find that after doing so it has "forgotten" two of my jobs/projects. The jobs can still be found on my Jenkins machine under /var/lib/jenkins/jobs, but they no longer show up in the Jenkins GUI. I attempted to re-create them based off the configuration file I have, but I am not confident I have totally re-created the functionality they had.
I also tried to copy the job and or rename it hoping that would get jenkins to see it, but no luck. I had tried cp -r /var/lib/jenkins/jobs/JOB1 /var/lib/jenkins/jobs/JOB2. I also restarted the service a number of times. Finally I updated all of my plugins on the off chance that was somehow related.
So my question is "How can I get Jenkins to notice these jobs?" or failing that "can I run these jobs from in the terminal?"
NOTE: I am not discouraging others from upgrading Jenkins. After I upgraded Jenkins did complain about a number of things which I didn't pay enough attention to which I believe got me into this mess in the first place.
If I were you, I would try the Jenkins CLI (from $JENKINS_URL/cli) and use the create-job command and feed the job configuration file to the cli's stdin.
If that does not help, I would inspect Jenkins log files (you are saving stdout and stderr of Jenkins somewhere, right?) for any errors or clues. If the job failed to load because of some tag that you can guess is provided by a plugin, try to remove that part from the config file.
If that does not help, I would upgrade Jenkins. I think there might be some fixes related to this in the LTS version changelog since 1.509.
And above all... if I were you, I would start making backups of the job configuration files.
I regularly back up the global config.xml, all the job config.xml files and all the plugins. Using these I can set up my Jenkins from scratch. And I do that to set up a test instance where I try any plugin or Jenkins core upgrade. If I see no problems after running a few of the trickiest builds, I know I can upgrade the production instance with much more confidence.