I tried many times to execute a Continuous Integration via BitBucket Pipeline (beta). For the moment I need a very simple task, just update my remote server when a push is made on repository (In the past I used for this purpose CodeShip with a very similar syntax).
In Pipelines is necessary to set up a file called bitbucket-pipelines.yml which contains several rows to differentiate between branches, etc. but the main instruction is:
- lftp -c "open -u $FTP_USER,$FTP_PASSWORD ftp.mydomain.com; set ssl:verify-certificate no; mirror -Rne /opt/atlassian/bitbucketci/agent/build /clone/ /public_html/dev"
Unfortunately it does not run correctly because it failed (apparently with infinite loop and new attempts).
I tried to discuss this topic with Support but I did not recieve any useful help and in the final message, they simply suggested me other resources.
Maybe, is there anybody that set up succesfully a similar things?
Thanks
If its just a git push you want, you could try this.
image: samueldebruyn/debian-git
pipelines:
default:
- step:
script:
- echo "Pipeline Init"
- apt-get update
- apt-get -qq install git-ftp
- echo "'_$(git status -uno --porcelain | wc -l)_'"
- git status -uno --porcelain
- echo "Initiating Push site:Source."
- git config git-ftp.syncroot Source/
- git ftp init --user $Username --passwd $Pwd ftp://domain.com/public_html/
Once you have done the first push (init), change the code git ftp init to git ftp push
Related
I have a CI pipeline in Bitbucket which is building, testing and deploying an application.
The thing is that after the deploy I want to run selenium tests.
Selenium tests are in an another repository in Bitbucket and they have their own pipeline.
Is there a trigger step in the Bitbucket pipeline to trigger a pipeline when a previous one has finished?
I do not want to do a fake push to the test repository to trigger those tests.
The most "correct" way I can think of doing this is to use the Bitbucket REST API to manually trigger a pipeline on the other repository, after your deployment completes.
There are several examples of how to create a pipeline here: https://developer.atlassian.com/bitbucket/api/2/reference/resource/repositories/%7Bworkspace%7D/%7Brepo_slug%7D/pipelines/#post
Copy + pasting the first example. How to trigger a pipeline for the latest commit on master:
$ curl -X POST -is -u username:password \
-H 'Content-Type: application/json' \
https://api.bitbucket.org/2.0/repositories/jeroendr/meat-demo2/pipelines/ \
-d '
{
"target": {
"ref_type": "branch",
"type": "pipeline_ref_target",
"ref_name": "master"
}
}'
according to their official documentation there is no "easy way" to do that, cause the job are isolated in scope of one repository, yet you can achieve your task in following way:
create docker image with minimum required setup for execution of your tests inside
upload to docker hub (or some other repo if you have such)
use docker image in last step of you pipeline after deploy to execute tests
Try out official component Bitbucket pipeline trigger: https://bitbucket.org/product/features/pipelines/integrations?p=atlassian/trigger-pipeline
You can run in after deploy step
script:
- pipe: atlassian/trigger-pipeline:4.1.7
variables:
BITBUCKET_USERNAME: $BITBUCKET_USERNAME
BITBUCKET_APP_PASSWORD: $BITBUCKET_APP_PASSWORD
REPOSITORY: 'your-awesome-repo'
ACCOUNT: 'teams-in-space'
#BigGinDaHouse I did something more or less like you say.
My step is built on top of docker image with headless chrome, npm and git.
I did follow the steps below:
I have set private key for the remote repo in the original repo. Encoded base 64. documentation. The public key is being set into the remote repo in SSH Access option in bitbucket menu.
In the pipeline step I am decoding it and setting it to a file. I am also changing its permission to be 400.
I am adding this Key inside the docker image. ssh-add
Then I am able to do a git clone followed by npm install and npm test
NOTE: The entry.sh is because I am starting the headless browser.
- step:
image: kimy82/headless-selenium-npm-git
script:
- echo $key_in_env_variable_in_bitbucket | base64 --decode > priv_key
- chmod 400 ./priv_key
- eval `ssh-agent -s`
- ssh-agent $(ssh-add priv_key; git clone git#bitbucket.org:project.git)
- cd project
- nohup bash /usr/bin/entry.sh >> out.log &
- npm install
- npm test
Top answers (this and this) are correct, they work.
Just adding that we found out (after a LOT of trial and error) that the user executing the pipeline must have WRITE permissions on the repo where the pipeline is invoked (even though his app password permissions were set to "WRITE" for repos and pipelines...)
Also, this works for executing pipelines in Bitbucket's cloud or on-premise, through local runners.
(Answering as I am lacking reputation for commenting)
The following code is an "Execute Shell" build step in Jenkins. The job pulls from a repo which contains a file ranger-policies/policies.json. What I'd like to do is update that file (with a curl command, in this case) and then commit the change to source control and update the remote repo. The job successfully pulls from the remote repo in the "Source Code Management" section of the job configuration page over SSH using SSH keys. However, when the job gets to the "git push origin master" line in the "Execute Shell" step, I get a Permission denied (publickey) error, as if those same SSH keys which allowed me to successfully pull the repo are not available in the "Execute Shell" step when I want to push.
curl -X GET --header "text/json" -H "Content-Type: text/json" -u user:pass "http://my-url.com/exportJson" > ranger-policies/policies.json
git add ranger-policies/policies.json
git commit -m "udpate policies.json with latest ranger policies `echo "$(date +'%Y-%m-%d')"`"
git push origin master
I ended up figuring out how to make it work. The solution involves using the SSH Agent plugin. Here's a step-by-step that describes how I did it, hopefully it helps someone else:
First, create a new pipeline job.
Then, as hinted at in this post from Jenkins' documentation, go to the home screen for your new pipeline job, and click on "Pipeline Syntax." Choose "git: Git" as the "Sample Step, and enter the git repo you want to push to in the "Repository URL" field. Then choose the corresponding valid SSH keys for that repo from the "Credentials dropdown." Everything should look like this:
Grab the value of "credentialsId", highlighted with red in the above screenshot. You'll need it later.
Install the "Workspace Cleanup Plugin" (https://wiki.jenkins.io/display/JENKINS/Workspace+Cleanup+Plugin, optional) and the "SSH Agent Plugin" (https://jenkins.io/doc/pipeline/steps/ssh-agent/, not optional, required for this process to work).
Now go back to your new pipeline job and hit "Configure," which will take you to the screen where you define the job. Drop the following code into the "Pipeline" section ("Definition" should be set to "Pipeline script"): https://gist.github.com/ScottNeaves/5cdce294296437043b24f0f3f0a8f1d8
Drop your "credentialsId" into the appropriate places in the above Jenkinsfile, and fix up the repo names to target the repo you want, and you should be good to go.
Relevant documentation:
https://jenkins.io/doc/pipeline/examples/#push-git-repo
https://gist.github.com/blaisep/eb8aa720b06eff4f095e4b64326961b5#file-jenkins-pipeline-git-cred-md
https://issues.jenkins-ci.org/browse/JENKINS-28335?focusedCommentId=269000&page=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-269000
As per this gist, you need to set the remote origin url as per:
git remote set-url origin git#github.com:username/your-repository.git
I'm new to this continuous integration thing. I want to use Jenkins as my CI system, but I can't get it to pull the build everytime there's a new one.
Using mercurial's plugin I'm able to connect to my repository and pull my builds normally, but I don't want Jenkins to keep polling, I want it to update the build only when there's a new one instead. On the plugin's wiki I found this:
As of version 1.38 it's possible to trigger builds using push
notifications instead of polling. In your repository's .hg/hgrc file
add:
[hooks]
commit.jenkins = wget -q -O /dev/null <jenkins root>/mercurial/notifyCommit?url=<repository remote url>
incoming.jenkins = wget -q -O /dev/null <jenkins root>/mercurial/notifyCommit?url=<repository remote url>
For now I'm keeping Jenkis local, so I used this o my hgrc file:
commit.jenkins = wget -q -O /dev/null http://localhost:8080/mercurial/notifyCommit?url=<my repository remote url>
incoming.jenkins = wget -q -O /dev/null http://localhost:8080/mercurial/notifyCommit?url=<my repository remote url>
But builds aren't being triggered. Could someone help me?
[UPDATE]
I didn't pay attention to the wget command, which doesn't exist on windows. Installed it and it's still the same. Jenkins is not pulling the builds.
You must to have wget on PATH (I'll recommend native port of GOW, not Cygwin - or Bash in Win10)
Your hooks must be in working state
wget ... must produce the expected result
you have threenow two possible points of failure and have to test all independently
Does my hooks work?
Replace your current content of hooks with dumb billet like
commit.jenkins = echo Commit hook here
incoming.jenkins = echo Incoming hook here
and test hooks (in console for better visibility) by executing commit into repo with added hook and pull|push to it|unbundle anything. If you'll see hook output - they are usable
Does Jenkins integration work?
After commit to repo you can perform task of hook by hand: run wget -q -O /dev/null ... and check results in Jenkins
I've managed to create a CircleCI build that triggers a subsequent build using their API using curl. I've added this to my circle.yml:
test:
override:
- mvn test -s settings.xml
- mvn deploy -Prun-its -s settings.xml
- curl -v -X POST https://circleci.com/api/v1/project/alexec/docker-maven-plugin/tree/master?circle-token=$CIRCLE_TOKEN
How do I trigger only if all of the previous steps are green?
I think you should do this in the deployment section: Since this is - by definition - only run if everything is fine, this should do the trick.
See their documentation on deployment for details. There it says:
These commands are triggered only after a successful (green) build.
You should have a requires variable in your job that you want to run only if the previous job has run. So you give the requires variable a value of the job name that you want to succeed first before the jobs resume running.
See this example: https://circleci.com/docs/2.0/configuration-reference/
Is there a way to specify a hook in the single repository?
Now we have specified the hook in the "/etc/mercurial/hgrc" file, but every time it builds twice, and it builds for each commit in each repository.
So we want to specify a build per repository.
This is how we implemented the hook:
[hooks]
changegroup = curl --silent http://jenkins:8080/job/ourProject/build
It's on a Ubuntu server.
Select the Poll SCM option under Build Triggers.
Make sure that schedule form is empty.
You should be creating in the .hg directory, /home/user/mercurial/.hg/hgrc and add hooks as:
[hooks]
commit.jenkins = wget -q http://localhost:8080/mercurial/notifyCommit?url=file:///home/user/mercurial > /dev/null
incoming.jenkins = wget -q http://localhost:8080/mercurial/notifyCommit?url=file:///home/user/mercurial > /dev/null
You should make sure that
Your Jenkins project doesn't poll
You use the proper notifyCommit URLs for your Mercurial hooks: https://wiki.jenkins-ci.org/display/JENKINS/Mercurial+Plugin
Ok, I found what I looked for (I'm the bounty; my case is Mercurial with a specific branch).
In the main/origin repository, place a hook with your desired build script. Pregroupchange is to maintain the incoming changes. I have a rhodecode installed on the main repository and itself has its own hooks.
In this way, I still trigger Jenkins and still have the changes afther the trigger for rhodecode push notifications and others.
[hooks]
pregroupchange = /path/to/script.extention
In the script, place your desired actions, also a trigger for Jenkins. Don't forget to enable in Jenkins:Job:Configure:Build Triggers:checkbox Trigger builds remotely + put here your desired_token (for my case: Mercurial).
Because you can't trigger only to a specific branch in Mercurial, I found the branch name in this way. Also, to trigger from a remote script, you need to give in Jenkins read permission for anonymous overall, or create a specific user with credentials and put them into the trigger URL.
Bash example:
#!/bin/bash
BRANCH_NAME=`hg tip --template "{branch}"`
if [ $BRANCH_NAME = "branch_name" ]; then
curl --silent http://jenkins_domain:port/path/to/job?token=desired_token
fi
For the original question:
In this way you only execute one build, for a desired branch. Hooks are meant only for the main repository in case you work with multiple clones and multiple developers. You may have your local hooks, but don't trigger Jenkins from you local, for every developer. Trigger Jenkins only from the main repository when a push came (commit, incoming, and groupchange). Your local hooks are for other things, like email, logs, configuration, etc.