I am trying to get my Jenkins publish files over ssh. I use pipeline scripts, that is because I have to use the ssh agent. But I am not able to configure it correctly, it is always throwing an exception. This is what I did so far for debugging:
I configured ssh credentials on the "Publish over SSH" part in global settings and as credentials (SSH Username with private key).
I use a file on the master without passphrase.
I created a freestyle job and configured the "Publish SSH"-Plugin to make an "ls" on the remote machine. This works.
I checked the box to use "ssh agent", but this leads to an exception:
[ssh-agent] Looking for ssh-agent implementation...
[ssh-agent] Java/JNR ssh-agent
[ssh-agent] Skipped registering BouncyCastle, not running on a remote agent
[ssh-agent] Started.
[ssh-agent] Unable to read key: Unable to create OpenSSL PBDKF: Could not generate secret key
org.bouncycastle.openssl.PEMException: Unable to create OpenSSL PBDKF: Could not generate secret key
at org.bouncycastle.openssl.jcajce.PEMUtilities.getKey(Unknown Source)
at org.bouncycastle.openssl.jcajce.PEMUtilities.getKey(Unknown Source)
at org.bouncycastle.openssl.jcajce.PEMUtilities.crypt(Unknown Source)
at org.bouncycastle.openssl.jcajce.JcePEMDecryptorProviderBuilder$1$1.decrypt(Unknown Source)
at org.bouncycastle.openssl.PEMEncryptedKeyPair.decryptKeyPair(Unknown Source)
at jenkins.bouncycastle.api.PEMEncodable.decode(PEMEncodable.java:162)
at com.cloudbees.jenkins.plugins.sshagent.jna.JNRRemoteAgent.addIdentity(JNRRemoteAgent.java:79)
at com.cloudbees.jenkins.plugins.sshagent.SSHAgentBuildWrapper$SSHAgentEnvironment.add(SSHAgentBuildWrapper.java:395)
at com.cloudbees.jenkins.plugins.sshagent.SSHAgentBuildWrapper.preCheckout(SSHAgentBuildWrapper.java:229)
at jenkins.scm.SCMCheckoutStrategy.preCheckout(SCMCheckoutStrategy.java:76)
at hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:528)
at hudson.model.Run.execute(Run.java:1741)
at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
at hudson.model.ResourceController.execute(ResourceController.java:98)
at hudson.model.Executor.run(Executor.java:410)
Caused by: java.security.spec.InvalidKeySpecException: Could not generate secret key
at javax.crypto.SecretKeyFactory.generateSecret(SecretKeyFactory.java:347)
... 15 more
Caused by: java.lang.IllegalArgumentException: password empty
at org.bouncycastle.jcajce.provider.symmetric.OpenSSLPBKDF$PBKDF.engineGenerateSecret(Unknown Source)
at javax.crypto.SecretKeyFactory.generateSecret(SecretKeyFactory.java:336)
... 15 more
I tried to type some dummy passphrase, this leads to a different exception (something with "check your credentials").
I tried to set the path for the key file to something different, this leads to a third exception (Like FileNotFound)
There is no difference between running on master or slave.
The error message "password empty" looks as if the empty passphrase makes problems - but the "publish over ssh" plugin has no problems with it, so I think it should be ok like this.
I have no idea what else I could check. I am near to create a freestyle job with "publish over ssh" which is triggered by my pipeline job... but seriously this is not the way it should be done?...
Related
I am searching for a best practice for switching svn repo paths from many jenkins jobs at once.
We have many different jobs which all define an svn path and poll our svn repo every 5 minutes.
Normally we build all projects from their trunks (subversion plugin 2.7.1).
But sometimes for bugfix reasons we want to be able to build all or certain projects from predefined tags or branches.
This switch should be made on a global job level not in the actual jobs themselves.
I have defined a job which lists all svn paths for our projects (using List Subversion tags plugin).
When the job gets fired every selected path gets written into a properties file with a key-value pair like "SVN_Projectname=tags/TAGNAME".
When no subversion tag is selected for a project the default vaule "SVN_Projectname=trunk" is written to the file for that project.
This is running without any issues...
I created a test job with a textparameter called "SVN_Projectname" and used the envInject plugin option "Prepare an environment for the run" with "Override Build Parameters" to read the already existent properties file.
Also the subversion plugin is provided with the parameter "${SVN_Projectname}" for the read parameter value from file.
Additional credentials (realm and credentials) are also provided and polling is activated for that job too.
Running that job manually is no problem, but the actual value from the file is generating polling results of the following error (scm-polling.log):
Started on 11.10.2017 14:53:00
Received SCM poll call on master for XXX_TAG on 11.10.2017 14:53:00
ERROR: Failed to check repository revision for http://XXX/svn/repository_XXX/Project/tags/tagname
[8mha:AAAAWB+LCAAAAAAAAP9b85aBtbiIQSmjNKU4P08vOT+vOD8nVc8DzHWhSE4tKMnjz/PLL0ldFVf2c+b/lb8MDAwVRQxSaBqxXTRIIQMEMIIUFgAAckCEiWAAAAA=[0morg.tmatesoft.svn.core.SVNCancelException: svn: E200015: E200015: ISVNAuthentication provider did not provide credentials; HTTP authorization cancelled.
svn: E200015: ISVNAuthentication provider did not provide credentials; HTTP authorization cancelled.
at org.tmatesoft.svn.core.internal.wc.SVNErrorManager.error(SVNErrorManager.java:66)
at org.tmatesoft.svn.core.internal.wc.SVNErrorManager.error(SVNErrorManager.java:57)
at org.tmatesoft.svn.core.internal.io.dav.http.HTTPConnection.request(HTTPConnection.java:798)
at org.tmatesoft.svn.core.internal.io.dav.http.HTTPConnection.request(HTTPConnection.java:391)
at org.tmatesoft.svn.core.internal.io.dav.http.HTTPConnection.request(HTTPConnection.java:379)
at org.tmatesoft.svn.core.internal.io.dav.DAVConnection.performHttpRequest(DAVConnection.java:862)
at org.tmatesoft.svn.core.internal.io.dav.DAVConnection.exchangeCapabilities(DAVConnection.java:698)
at org.tmatesoft.svn.core.internal.io.dav.DAVConnection.open(DAVConnection.java:118)
at org.tmatesoft.svn.core.internal.io.dav.DAVRepository.openConnection(DAVRepository.java:1049)
at org.tmatesoft.svn.core.internal.io.dav.DAVRepository.getLatestRevision(DAVRepository.java:189)
at org.tmatesoft.svn.core.internal.wc2.ng.SvnNgRepositoryAccess.getRevisionNumber(SvnNgRepositoryAccess.java:119)
at org.tmatesoft.svn.core.internal.wc2.SvnRepositoryAccess.getLocations(SvnRepositoryAccess.java:195)
at org.tmatesoft.svn.core.internal.wc2.ng.SvnNgRepositoryAccess.createRepositoryFor(SvnNgRepositoryAccess.java:46)
at org.tmatesoft.svn.core.internal.wc2.remote.SvnRemoteGetInfo.run(SvnRemoteGetInfo.java:46)
at org.tmatesoft.svn.core.internal.wc2.remote.SvnRemoteGetInfo.run(SvnRemoteGetInfo.java:31)
at org.tmatesoft.svn.core.internal.wc2.SvnOperationRunner.run(SvnOperationRunner.java:21)
at org.tmatesoft.svn.core.wc2.SvnOperationFactory.run(SvnOperationFactory.java:1235)
at org.tmatesoft.svn.core.wc2.SvnOperation.run(SvnOperation.java:294)
at org.tmatesoft.svn.core.wc.SVNWCClient.doInfo(SVNWCClient.java:2535)
at hudson.scm.SubversionSCM.parseSvnInfo(SubversionSCM.java:1228)
at hudson.scm.CompareAgainstBaselineCallable.call(CompareAgainstBaselineCallable.java:79)
at hudson.scm.CompareAgainstBaselineCallable.call(CompareAgainstBaselineCallable.java:27)
at hudson.remoting.LocalChannel.call(LocalChannel.java:45)
at hudson.scm.SubversionSCM.compareRemoteRevisionWith(SubversionSCM.java:1425)
at hudson.scm.SCM.compareRemoteRevisionWith(SCM.java:391)
at hudson.scm.SCM.poll(SCM.java:408)
at hudson.model.AbstractProject._poll(AbstractProject.java:1460)
at hudson.model.AbstractProject.poll(AbstractProject.java:1363)
at jenkins.triggers.SCMTriggerItem$SCMTriggerItems$Bridge.poll(SCMTriggerItem.java:128)
at hudson.triggers.SCMTrigger$Runner.runPolling(SCMTrigger.java:557)
at hudson.triggers.SCMTrigger$Runner.run(SCMTrigger.java:603)
at hudson.util.SequentialExecutionQueue$QueueEntry.run(SequentialExecutionQueue.java:119)
at java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)
at java.util.concurrent.FutureTask.run(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
at java.lang.Thread.run(Unknown Source)
Caused by: org.tmatesoft.svn.core.SVNCancelException: svn: E200015: ISVNAuthentication provider did not provide credentials; HTTP authorization cancelled.
at org.tmatesoft.svn.core.internal.io.dav.http.HTTPConnection.request(HTTPConnection.java:728)
... 34 more
Done. Took 0,17 sec
No changes
While i have provided all credentials to the job and starting the job manually, there is no problem. I dont understand what the actual problem is...
Or is there a better way of automating such a behavior?
Maybe manipulating the projects config.xmls with a changed svn path is a possibility?
But then jenkins service would have to be restarted? and i would prefer to make the change on the fly ^^
Whenever a build of my Jenkins job finishes, I'm sending an email with the results using the Editable Email Notification plugin (a.k.a. Email-ext plugin).
Also, if a certain file is present in the system, I'm modifying the default content of the message to notify of the existence of that file.
For that end, I'm using the Pre-send Script field of the plugin.
Test job configuration:
Prepare an environment for the run
Properties Content
LOG="log.txt"
Build stage
Execute shell
#!/bin/bash
touch ${WORKSPACE}/${LOG}
echo "this is just a log" >> ${WORKSPACE}/${LOG}
Post-build Actions
Editable Email Notification
Pre-send Script
File log = new File("${WORKSPACE}/${LOG}");
logger.println(log.text);
When executing the build, Jenkins will create the file in the ${WORKSPACE} and fill it (I'm able to print the contents from the terminal in the Jenkins slave).
But when trying to access it from the Email plugin, the exception java.io.FileNotFoundException is raised:
java.io.FileNotFoundException: /home/jenkins/workspace/testJob/log.txt (No such file or directory)
at java.io.FileInputStream.open(Native Method)
at java.io.FileInputStream.<init>(FileInputStream.java:146)
at groovy.util.CharsetToolkit.<init>(CharsetToolkit.java:69)
at org.codehaus.groovy.runtime.DefaultGroovyMethods.newReader(DefaultGroovyMethods.java:16958)
at org.codehaus.groovy.runtime.DefaultGroovyMethods.getText(DefaultGroovyMethods.java:16006)
at org.codehaus.groovy.runtime.dgm$381.doMethodInvoke(Unknown Source)
at org.codehaus.groovy.reflection.GeneratedMetaMethod$Proxy.doMethodInvoke(GeneratedMetaMethod.java:70)
at groovy.lang.MetaClassImpl$GetBeanMethodMetaProperty.getProperty(MetaClassImpl.java:3500)
at org.codehaus.groovy.runtime.callsite.GetEffectivePojoPropertySite.getProperty(GetEffectivePojoPropertySite.java:61)
at org.codehaus.groovy.runtime.callsite.AbstractCallSite.callGetProperty(AbstractCallSite.java:227)
at Script1.run(Script1.groovy:59)
at groovy.lang.GroovyShell.evaluate(GroovyShell.java:580)
at groovy.lang.GroovyShell.evaluate(GroovyShell.java:618)
at groovy.lang.GroovyShell.evaluate(GroovyShell.java:589)
at hudson.plugins.emailext.ExtendedEmailPublisher.executePresendScript(ExtendedEmailPublisher.java:450)
at hudson.plugins.emailext.ExtendedEmailPublisher.sendMail(ExtendedEmailPublisher.java:311)
at hudson.plugins.emailext.ExtendedEmailPublisher._perform(ExtendedEmailPublisher.java:297)
at hudson.plugins.emailext.ExtendedEmailPublisher.perform(ExtendedEmailPublisher.java:244)
at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:20)
at hudson.model.AbstractBuild$AbstractBuildExecution.perform(AbstractBuild.java:782)
at hudson.model.AbstractBuild$AbstractBuildExecution.performAllBuildSteps(AbstractBuild.java:723)
at hudson.model.Build$BuildExecution.cleanUp(Build.java:195)
at hudson.model.Run.execute(Run.java:1785)
at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
at hudson.model.ResourceController.execute(ResourceController.java:98)
at hudson.model.Executor.run(Executor.java:410)
I'm at a loss as why Groovy is complaining about the missing file, whereas if I instead print the path with
logger.println(log.getPath());
it will print it successfully.
NOTE:
The logger variable in the Pre-send Script is provided by the notification plugin to access directly the build log.
java.io.File methods will refer to files on the master where Jenkins is running, and not in the current workspace on the slave machine (in case this job indeed runs on a slave machine).
To refer to files on the slave machine, you should use the readFile method
def log = readFile("${WORKSPACE}/${LOG}");
Within a Jenkins pipeline, File log = new File("${WORKSPACE}/${LOG}"); doesn't work.
You can use readFile instead, e.g.:
def log = readFile "${WORKSPACE}/${LOG}"
I had a jenkins Job with the same problem. readFilemethod didn't work for me because I used a Job DSL (had an error like No signature of method readFile()), so I used the readFileFromWorkspace method and it worked for me.
Your code should be modified like:
def log = readFileFromWorkspace("${WORKSPACE}/${LOG}");
I'm trying to use the Jenkins Ansible plugin to deploy a project. I figured this should be pretty easy as i'm pretty familiar with both Jenkins and Ansible. However, i'm having issues getting the Ansible command to successfully execute. I believe the problem lies in the use of the credentials. We have a jenkins user that owns the Jenkins execution process and we have a jenkins user setup on the machine being deployed to. We are using SSH keys and I have added the jenkins users private key to the Jenkins credential store and selected those in the Ansible configuration. However, when i run the job, this is the snippet of my output (replaced some information).
[my-job] $ ansible-playbook my-job.yml -i inventories/dev -l 1_2_3_4 -f 1 --private-key /tmp/ssh7229752594712048879.key -u jenkins --diff --vault-password-file ~/.vault-pass
PLAY [tag_ansible_groups_my_job] ***********************
TASK [setup] *******************************************************************
fatal: [1_2_3_4]: UNREACHABLE! => {"changed": false, "msg": "Failed to connect to the host via ssh: Permission denied (publickey,password).\r\n", "unreachable": true}
to retry, use: --limit #/data/jenkinsdata/workspace/my-job/my-job.retry
PLAY RECAP *********************************************************************
1_2_3_4 : ok=0 changed=0 unreachable=1 failed=0
FATAL: command execution failed
hudson.AbortException: Ansible playbook execution failed
at org.jenkinsci.plugins.ansible.AnsiblePlaybookBuilder.perform(AnsiblePlaybookBuilder.java:227)
at org.jenkinsci.plugins.ansible.AnsiblePlaybookBuilder.perform(AnsiblePlaybookBuilder.java:200)
at hudson.tasks.BuildStepCompatibilityLayer.perform(BuildStepCompatibilityLayer.java:78)
at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:20)
at hudson.model.AbstractBuild$AbstractBuildExecution.perform(AbstractBuild.java:779)
at hudson.model.Build$BuildExecution.build(Build.java:205)
at hudson.model.Build$BuildExecution.doRun(Build.java:162)
at hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:534)
at hudson.model.Run.execute(Run.java:1729)
at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
at hudson.model.ResourceController.execute(ResourceController.java:98)
at hudson.model.Executor.run(Executor.java:404)
ERROR: Ansible playbook execution failed
Finished: FAILURE
I've run the same command successfully from the Jenkins machine and it works just fine. The only difference was that I referenced the original SSH private key instead of the copied temporary key. To see if this was the issue, I decided to check the contents of the copied temporary SSH private key and the file was empty. I checked the file before the job finished executing as I am assuming it stays there for the life of the job and removed only after the job has finished executing. To verify this, I removed the use of credentials from the configuration and added in the private-key and user parameters pointing directly to the appropriate key and the job runs successfully.
[my-job] $ ansible-playbook my-job.yml -i inventories/dev -l 1_2_3_4 -f 1 --private-key ~/.ssh/id_rsa -u jenkins --diff --vault-password-file ~/.vault-pass
PLAY [tag_ansible_groups_my_job] ***********************
TASK [setup] *******************************************************************
ok: [1_2_3_4]
So, to me, it seems that Jenkins is failing to copy the SSH private key to the temporary file(s). I've tried with other SSH keys but still the same problem each time. I've tried specifying the private key in Jenkins credential store itself rather than specifying the location and that also did not work. I've checked the Jenkins logs and there is nothing that would indicate why the private SSH key would fail to copy to the /tmp directory.
I'd prefer not to reference the key directly in the additional parameters as this limits me to only keys located on that server. Any suggestions?
I found this solution at Edureka. It worked for me.
set "host_key_checking = False" in /etc/ansible/ansible.cfg
https://www.edureka.co/community/42595/not-able-connect-remote-host-via-jenkins-run-ansible-playbook
In the ssh-agent credentials section of my jenkins job, I have two options:
Specific credentials
Parameter expression
My job works fine when I select Specific credentials, but when I choose Parameter expression, and fill in the value as ${CREDENTIAL_ID}, and run my job with parameter CREDENTIAL_ID=<hex id of credential>, the job dies with this trace:
[WS-CLEANUP] Deleting project workspace...
[WS-CLEANUP] Done
FATAL:
java.io.IOException: [ssh-agent] Could not find specified credentials
at com.cloudbees.jenkins.plugins.sshagent.SSHAgentBuildWrapper.preCheckout(SSHAgentBuildWrapper.java:204)
at jenkins.scm.SCMCheckoutStrategy.preCheckout(SCMCheckoutStrategy.java:76)
at hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:528)
at hudson.model.Run.execute(Run.java:1738)
at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
at hudson.model.ResourceController.execute(ResourceController.java:98)
at hudson.model.Executor.run(Executor.java:410)
FATAL: [ssh-agent] Could not find specified credentials
java.io.IOException: [ssh-agent] Could not find specified credentials
at com.cloudbees.jenkins.plugins.sshagent.SSHAgentBuildWrapper.preCheckout(SSHAgentBuildWrapper.java:204)
at jenkins.scm.SCMCheckoutStrategy.preCheckout(SCMCheckoutStrategy.java:76)
at hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:528)
at hudson.model.Run.execute(Run.java:1738)
at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
at hudson.model.ResourceController.execute(ResourceController.java:98)
at hudson.model.Executor.run(Executor.java:410)
Finished: FAILURE
Any idea what might be going on? I'm running Jenkins ver. 1.654, Credentials Binding Plugin 1.6, SSH credentials plugin 1.11, and SSH agent plugin 1.9
I was trying to achieve the same thing, using a String Parameter with the credential ID (the hex value, just like you posted in your problem's description).
The issue is that it seems that the Parameter Expression expects you to use a Credential Parameter instead of a String Parameter.
Although it's not exactly how you wanted to use it (you'll have to choose the key from a dropdown list when you trigger a build), changing the parameter type may be a suitable solution for you.
I am trying to launch new slave agent but I am seeing the below error. Can you please help in resolving this.
- I have copied the keys in both the machines in .ssh/authorized_keys and able to connect manually but failing from jenkins.
- All permissions also looks gud.
[02/19/16 13:12:34] [SSH] Opening SSH connection to sdc-caoneops- app1.qa.xxxxx.com:22.
ERROR: Server rejected the 1 private key(s) for App (credentialId:6aced962- 26cd-4c8f-97d3-305c2e2a6540/method:publickey)
[02/19/16 13:12:34] [SSH] Authentication failed.
hudson.AbortException: Authentication failed.
at hudson.plugins.sshslaves.SSHLauncher.openConnection(SSHLauncher.java:1178)
at hudson.plugins.sshslaves.SSHLauncher$2.call(SSHLauncher.java:701)
at hudson.plugins.sshslaves.SSHLauncher$2.call(SSHLauncher.java:696)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
[02/19/16 13:12:34] Launch failed - cleaning up connection
[02/19/16 13:12:34] [SSH] Connection closed.
When you ssh manually from master to slave, you are doing so as the Jenkins master user, right? You only need to copy the public key from the master into ~/.ssh/authorized_keys of the slaves.
Add Jenkins Credentials for the jenkins private key and you use that pair in slave configuration "Credentials".
In slave configuration add in "Remote root directory" the home directory of jenkins user on the slave (which contains .ssh/authorized_keys)