I have got my code files in bitbucket and have configured a jenkins build job to run where there is a change in the bitbucket repository. At the end of which it has to copy the files from the repo to a directory located at a production server from where the application is running.
Is there a away to copy the files from repo to a server using a script inputed to jenkins?
I asume you have the files in the workspace of the job. How about copying the files via the command line? If you want to do so, insert a batch block for windows nodes or a shell block for linux nodes and use
cp original_file new_file
You have 2 possibilites:
Run a slave on the production server
In this case you run a slave on the production server, which connects to your master jenkins. The slave has to be run under a user, which is able to write the directory, where you want to copy the files too.
2 Variations of this possibility:
You can execute the clone (checkout) of the bitbucket repository on the master and then use stash to make the files accessible on the slave, running on the production server (https://jenkins.io/doc/pipeline/steps/workflow-basic-steps/#stash-stash-some-files-to-be-used-later-in-the-build).
You run the whole pipeline on the slave, which is running on the production server, which means the production server needs access to bitbucket.
There are several possibilities to connect a slave to a master: https://wiki.jenkins.io/display/JENKINS/Distributed+builds#Distributedbuilds-Differentwaysofstartingagents
Use remote copy possibilites
You copy the files with eg. scp in linux.
This has some security implications:
You have to add the password of the production to the jenkins credential store and pass it to the copy command
if using keys (recommended). You have to add the private key to the jenkins credential store and pass it to the command.
Related
I've successfully connected to a Bitbucket repository within my Jenkins job. My issue is that I haven't been able to find any information about how to access/use the files that the repository contains. I added an "Execute Shell" step after connecting the repo, but don't know where the files the repo contains are. I tried cd'ing into /NameOfRepo/NameOfSubfolder but it says the file/directory does not exist in the console output when I run the job. Where does Jenkins store files it has gained access to that live in a remote repository? Do I need to use shell commands to clone my repo to a specific location?
The scenario is - I have a job A which runs my ant script and packages the artifact's for me.
I am also using parametrized Triggered plug in to Trigger my "Job B" which will deploy my artifact on remote machine.
The job A is working fine and also Job B.
The tasks that i have to perform with Job B are
GIT checkout (which contains my deployment scripts) (successfully doning).
Copying artifacts from previous build to Remote machine. (successfully doing)
Run shell script on remote machine(script present in workspace folder )- Facing issues.
I browsed various plug ins for the same but no one is allowing me to run shell script after , "SCP to remote machine" which is present in Post build action.
I would like to execute the same sequence, however if you guys have any other suggestions please share.
Thanks in Advance.!
As part of Publish Over SSH Plugin, you can execute a script after the files had been copied over.
Under Post-build Actions
Add Send build artifacts over SSH
Select a preconfigured server (done in global configuration)
Select files to copy from workspace
Enter Exec command
If one of the files you copy is your shell script, you can enter it here as an "exec command"
To solve my query i used Jenkins SSH Plugin. This provides a configuration tab where i can add multiple hosts and after that used them in my job level configuration.
Link to Plugin
you get privilege to execute shell script on remote host as pre-build step or post build step.
updated the path of publish over ssh it worked for me
I have a Jenkins server setup with two jobs
first job polls the develop branch and builds the project on the jenkins server. i then have another job that polls the production branch this builds this branch on another jenkins slave which is the staging server. This job is configured so that on a successful build it should publish the artefacts over ssh to the production server.
All the SSh keys are setup and the staging server connects to production server but 0 files are transferred
using GIT_SSH to set credentials Bitbucket Repo
using .gitcredentials to set credentials
Checking out Revision 89874cc01a9f669df69817b1049b1ab98ecb19d3 (origin/Production)
SSH: Connecting from host [nginx-php-fastcgi]
SSH: Connecting with configuration [AmazonAWS] ...
SSH: Disconnecting configuration [AmazonAWS] ...
SSH: Transferred 0 file(s)
Finished: SUCCESS
I checked the staging workspace and files are being built there, just not sent to the prod server. Any suggestions??
i have also tried a different remove prefix as suggested bellow and here Jenkins transferring 0 files using publish over SSH plugin
You should remove /* from the Remove prefix line
Edit:
Your Source files cannot be outside of the job's workspace. If your files are in the root of workspace, just set it to * to transfer all workspace files, or **/* to include subdirectories. Else specify a pattern relative to ${WORKSPACE}.
Even adding a leading / will not escape that, as all it does is append that to workspace, in your case it becomes ${WORKSPACE}/var/www/workspace/opms-staging-server. Even using parent directory ../ will not work. This is for security concerns, else a job configurer could transfer private files off the Jenkins server.
If you need to get files from another job, you need to use Copy Artifacts build step. Tell me if that's your case, and I will explain further.
I am using jenkins for build process. I am running some scripts on remote server using jenkins server. It's fine but remote host generates some html file. I want to copy that file back to jenkins server. Is it possible to do it from jenkins server?
If you want to archive it permanently, you can use the Archive Artifacts option in the Post-build step in Jenkins. In the case of builds that happen on slaves, Archive Artifacts copies the artifacts back to the Jenkins server for archiving and reuse.
If you want to then use this in a subsequent build, you can use the Copy Artifacts step to introduce an artifact from another build into your subsequent process on any Jenkins slave.
We use this to move production builds into our test environment after packaging and it works great.
Copy to slave plugin also copies files back from slaves to the master. Might be worth the look.
I want to deploy with jenkins to the test environment and to the production environment. To do so I need to connect to the server of the wanted environment, something like ssh/scp.
I would like to know what the best way is.
I found some plugins to do this, like the Jenkins-Deploy-Plug-in or Jenkins Publish over SSH Plugin. The first has lots of issues, which is not really trustworthy to deploy to production and for the second you need to change the global configuration, which is manual work for every deploy.
Any ideas how to solve this? Maybe with some scripts or plugins?
The only current idea I have is: to connect with jenkins to a server (maybe with the SSH Plugin) and to execute there a script that connects to the wished environment. But that are two connections. Is that really neccessary? I hope for a more straightforward way for this.
thanks for any hint.
I suggest the following procedure:
one single shell script (stored somewhere on the jenkins server) does everything.
Basically, the script does scp of the build artifact and then connects to the server (ssh) and does all the necessary tasks to deploy (setup maintenance page, backup the current app, deploy the new app, ...).
On the jenkins server, there are at least 2 jobs:
the first one simply does the build (using maven, or any other build script)
the second job does the deploy : so this job only runs the shell script.
(I suggest one deploy job for each target environment : testing, production, ...)
It does not require any "special" jenkins plugin to achieve this "one click deployment".
It only requires that the jenkins user has ssh access to the target server.
EDIT
Here is a sample shell script to illustrate my post
#This script will copy the last artifact build by the job "MyApp" to test.myserver.com
#and remotely execute the deployment script.
#copy the war to the server
#(the job "MyApp" is using maven, that's why the war can be found at this location)
scp -i <HOME_DIR>/.ssh/id_dsa $HUDSON_HOME/jobs/MyApp_Build/workspace/myapp/target/myapp.war deployeruser#test.myserver.com:/tmp/
#connect to the server and execute the deployment script
ssh -i <HOME_DIR>/.ssh/id_dsa deployeruser#test.myserver.com
#The following is just an example of what a deployment script can be.
#of course you must adapt it to your needs and environment
"cd <TOMCAT_DIR>;
#first copy the current war to a backup directory (additionaly, I have a cron task deleting old undeployed apps)
cp -rf myapp-apps/myapp* undeployed/myapp-apps/;
#execute a script (stored on the server) to properly stop the app
sh bin/myapp.sh stop;
#delete current app
rm -rf myapp-apps/myapp;
rm -rf myapp-apps/myapp.war;
#copy the uploaded war in tomcat app directory
cp /tmp/myapp.war myapp-apps/;
#execute a script (stored on the server) to start the app
sh bin/myapp.sh start"
Using SSH compromises security on your environment
and is quite hard to troubleshoot.
It is better to install a Jenkins-Slave on the remote machine
and run the tests there by executing a Job on the Slave.
The Slave is monitored by the Server, which saves you a lot of trouble
managing the connection.
You can trigger the remote Job at the end of a successful build
and pass it the artifact of that build.
(can also have the first Job store the artifacts on a shared drive
and pass the location of those artifacts to the next Job).
See here:
Jenkins - Distributed builds
Jenkins - Parameterized Trigger Plugin
Jenkins - Copy Artifact Plugin
Ideally, you should be using something like Fabric or Capistrano for deployments, and call those scripts from Jenkins. I've used Capistrano extensively for both Ruby On Rails and Non-Ruby applications too. The biggest advantage I see are:
The intelligence built in to rollback the deployment in case there are errors while deployments.
Hooks it provides to run a set of scripts such as DB migration, service restarts etc.
Manual rollback in case you need to.