My jenkins setup is not overly complicated, there are just a bit over 200 jobs; the problem I'm having is as follows:
The jobs folder is mounted on NFS drive;
Some of the jobs are creating log file fine, but then it is loosing permissions completely (it becomes 000), resulting in an error on the console regarding log file permissions:
I've checked and rechecked permissions on the folder and all the jobs, but nothing is there stands out that could explain what's the cause of the problem. It's not an issue on its own, but some of the jobs are quite important, and without manually fixing permissions, they can't be debugged.
Any hints would be welcome.
I had the same issue, it's more than likely that your version control (e.g Perforce/SVN) sets log files to read only permissions when synced.
An easy work around to this problem is to add an "Execute Windows Batch Command" build step where you cd into the directory containing your log file and change it's permissions.
Use the commands:
cd
attrib -r
This will alter the permissions of your log file and allow your jobs to write to the file. I'm sure there's other ways of dealing with this issue but this is a pretty quick and easy way.
Related
I do something like this
-javaagent:/usr/local/lib/perfino/perfino.jar=server=ybperfino,name=${HSTNAMESHORT}-${APPNAME},group=${YBENV}/${HSTNAMESHORT},logMBean=10,logFile=${LOG_DIR}/perfinologs/${HSTNAMESHORT}-${APPNAME}.log
basically I want the log files to be created in the log directory for the app not the home directory for the userid
but it seems like the log file isn't being created either with logfile argument or with out !
using java11 if that makes any difference.
Found the answer - I had a competing java agent that was loading before it.
After I changed the order both java agents worked.
I have changed our Jenkins setup from everything running on one machine to a master-agent/slave setup. Before that everything worked fine, now I am facing issues that some programs I am calling that access files can't find them.
Case 1:
(Pls don't ask why it is so complicated, but the file structure is given and I can't change it)
I am calling a python script, that iself calls a batch file
filepath= os.path.abspath(os.path.join(pamFolder, "run.bat"))
p = subprocess.Popen(filepath, cwd=pamFolder, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
... and the batch file again a jar file with the actual program
java -XX:-UseGCOverheadLimit -cp "../..;../../libs/*" -jar ..\..\myjarfile.jar
Within the jar file there is an access to a file on the disk failing with an error message that the file can't be found:
ERR : The file was not found in the specified path 'U:\somefile.txt'. Please check this path for access and your configuration!
Case 2:
I am calling a batch file from Jenkins that is calling some other exe and in the end trying to open a file in Excel via the COM interface. Here I am getting the following exception (Excel can't access the file):
Unhandled Exception: System.Runtime.InteropServices.COMException: Microsoft Excel kann auf die Datei 'D:\Jenkins\workspace\myJob\someDir\someFile.xlsm' nicht zugreifen.
Question
As previously mentioned, both jobs were working in the previous setup. Both files DO exist.
I suspect that Jenkins / the programs are trying to find the files on the master where they are not available.
Is there any way to tell Jenkins that the called tools are fully executed on the slave node or in some other way tell them where to find these files?
EDIT
The job is already running on the slave. The console shows Running on [slave name] in D:/Jenkins/workspace/xxxxx.
The master is configured in a way that only jobs assigned to it run on the master. So pretty much all jobs should run on the slave.
EDIT2 / SOLUTION
It turned out that the 2 issues are caused by different things.
Case1: Solved this by using the UNC path
Case2: Solved by a mixture of giving the necessary permissions as described here and starting the slave service with a user with admin rights.
From my experience with this issue, usually it has to do with your SCM setup.
But as you stated that the files DO exist I think there might be the possibility that U:\ is a network share? Then consider changing your path to use a UNC path.
If that's not the case check if your jenkins slave as sufficient user rights to access said file.
You can tell Jenkins to run the job on the designated slave as follows:
Under Nodes > [SLAVE] > Configure, specify a label for the slave.
Under [Job] > Configure > Restrict where this project can be run, enter the label.
Now when you build, the console output of the job should read correctly along the lines Running on [SLAVE] (build_agent_01) in C:/jenkins and the files must be accessible.
There's a problem with TFS everywhere plugin for Eclipse if I try to revert or restore a file under a source control and if the folder/file is mapped to a different volume than the actual project.
When I try to restore or revert it, I get an error:
java.io.IOException: Failed to rename /Users/*/*/*/*/.tf1/8/d54f18aa-bdce-4ab7-958a-01eaaf0c36c1.tmp to /Volumes/macOSData/*/*/*/*/*/some_file.cs. Check the file and directory permissions.
Log has additional line:
2018-07-31 12:44:39,814 WARN [ModalContext] (com.microsoft.tfs.util.FileHelpers) Main rename failed (source permissions problem?), trying to rename temp file back
There's no problem while I get specific version of the project, even with overriding existing files, so there permissions are fine, at least for getting files. Also, this does not happen when the file is on the same volume as the local TFS mapping.
Setting all permissions to 777 does fix the problem, but this marks all files with +x making them all "changed", making this solution unacceptable.
I tried mapping the base folder to a second drive (i.e. force creating a .tf folder on a second drive), but this doesn't help. The error will appear when I try to revert stuff on the main volume.
I tried using symlinks so the mapping stays within the same volume, but still no luck.
Is there any way to solve this? Or everything should be on the same volume?
After a whole day of digging around and poking with jshell, this appears to be a bug with File.renameTo() in Java on macOS. renameTo function silently fails without any exception even though there's no permission issue to write to the destination. This happens only when writing to a different volume.
I have submitted a pull request into TFS Everywhere repo on github with a workaround for this issue.
Anyone interested can compile plugin with this changes to get things working:
https://github.com/Microsoft/team-explorer-everywhere/pull/276
I have a test which tries to start an embedded mysql.
It tries to write a socket file to Files.createTempFile(null, ".sock") which returns with the following error:
mysql start failed with error: [ERROR] The socket file path is too long (> 103): /private/var/tmp/_bazel_user/1c8ed8d84f6cb79483aa3cc4da758c86/bazel-sandbox/2478112867584790357/execroot/some_workspace/_tmp/dfebe48cda4dfdc8739653efedfa4933/394798020705754292.sock.
I worked around it by re-pointing java temp dir to /tmp using jvm_flags but this doesn't work when I try to use sandboxing since I guess the test isn't allowed to write there.
I've tried setting a symbolic link from the java code (like so Files.createSymbolicLink(Paths.get("/tmp/foo"),Paths.get(System.getProperty("java.io.tmpdir"))) but this doesn't seem to help.
I've also tried setting the output_base but that didn't help either.
Would really appreciate pointers and tips since I currently can't run my tests under sandbox and so can't parallelize them.
I think your approach of re-pointing Java to /tmp should work. The macOS sandbox always allows writing to a number of directories and /tmp and /private/tmp are included in this set. I tried to reproduce the failure with a genrule: genrule(name = "write_to_tmp", outs = ["out.txt"], cmd = "touch /tmp/something.sock && touch $#"), which works fine and creates the file /tmp/something.sock.
I think in general using /tmp should work fine, although it does seem to be a bit unusual on macOS. $TMPDIR is set to a user-specific folder with a randomized name underneath /var/folders by the OS and it seems to be generally encouraged to use that instead of /tmp. But if you know what you're doing, I don't see a real problem.
Note that we don't have tmpfs or similar mechanisms available on macOS, so we can't automatically guarantee that your usage of writable folders like /tmp by actions is hermetic, won't leak state between runs or that file names won't conflict. Make sure to generate file names in a secure way via mkstemp or similar.
In a bazel test, you can use the TEST_TMPDIR environment variable for a test-private writeable area.
See https://docs.bazel.build/versions/master/test-encyclopedia.html
I have restarted Jenkins using the following:
service jenkins stop
service jenkins start
Followed to that I can see some jobs are missing from the GUI.
I have also tried to go the job URL using http://<jenkins_url>/job/<JOBNAME>/
Unfortunately it is also giving:
HTTP ERROR 404
Problem accessing /job/<JOBNAME>/. Reason:
Not Found
Powered by Jetty://
Also performed Doing a Reload Configuration from Disk with no luck.
I checked the config.xml file and I can see it is corrupted. The size of config.xml file is around 110 MB. Why this file got corrupted? How to trace it.
Can anyone give me any pointer how to troubleshoot this problem?
I had the same symptoms, but I'm using a homebrew installation of Jenkins.
The Jenkins machine was shut down improperly, likely from a power outage, so when it came back up it was basically a clean instance. No jobs and no system configurations.
The following solution isn't for your exact use case, but it does solve the problem for some users who return to Jenkins to find it without any jobs.
The solution basically involves you checking to see if you have started the Jenkins service incorrectly or from the wrong place.
...
On to the specific homebrew issue:
For whatever reason, the homebrew.mxcl.jenkins.plist file was found in ~/Library/LaunchDaemons/
It belongs in ~/Library/LaunchAgents/ only.
If this happens, it can be solved as follows
Stop the service:
sudo launchctl unload ~/Library/LaunchDaemons/homebrew.mxcl.jenkins.plist
Reload the correct file, located in ~/Library/LaunchAgents/ by trying the following line in case it's running:
launchctl unload ~/Library/LaunchAgents/homebrew.mxcl.jenkins.plist
Note: the above line may yell at you if it's not running, which is ok.
Start it up again:
launchctl load ~/Library/LaunchAgents/homebrew.mxcl.jenkins.plist
If all looks good when Jenkins loads again, you can and should
delete homebrew.mxcl.jenkins.plist in ~/Library/LaunchDaemons/:
sudo rm ~/Library/LaunchDaemons/homebrew.mxcl.jenkins.plist