Wtth help of below article i've setup SSH keys for bitbucket so i can use it in pipelines
https://support.atlassian.com/bitbucket-cloud/docs/set-up-an-ssh-key/
When tested on terminal window by entering following command it works fine:
$ ssh -T git#bitbucket.org
but when i run my pipelines it fails
Added public key under my bitbucket profile
My Pipeline:
image:
name: abhisheksaxena7/salesforcedockerimg
pipelines:
branches:
feature/**:
- step:
script:
- ant -buildfile build/build.xml deployEmptyCheckOnly -Dsfdc.username=$SFDC_USERNAME -Dsfdc.password=$SFDC_PASS$SFDC_TOKEN -Dsfdc.serverurl=https://$SFDC_SERVERURL
# master:
# - step:
# script:
# - ant -buildfile build/build.xml deployCode -Dsfdc.username=$SFDC_USERNAME -Dsfdc.password=$SFDC_PASS$SFDC_TOKEN -Dsfdc.serverurl=https://$SFDC_SERVERURL
Admin-Changes:
- step:
script:
- echo my_known_hosts
# Set up SSH key; follow instructions at https://confluence.atlassian.com/display/BITBUCKET/Set+up+SSH+for+Bitbucket+Pipelines
- (mkdir -p ~/.ssh ; cat my_known_hosts >> ~/.ssh/known_hosts; umask 077 ; echo $SSH_KEY | base64 --decode -i > ~/.ssh/id_rsa)
# Read update_to_trigger_pipelines.txt into commitmsg variable
- commitmsg="$(<update_to_trigger_pipelines.txt)"
# Set up repo and checkout master
- echo git#bitbucket.org:$BITBUCKET_REPO_OWNER/$BITBUCKET_REPO_SLUG.git
- git remote set-url origin git#bitbucket.org:$BITBUCKET_REPO_OWNER/$BITBUCKET_REPO_SLUG.git
- git config --add remote.origin.fetch +refs/heads/*:refs/remotes/origin/*
- git fetch
- git checkout master
# Get metadata from server
- ant -buildfile build/build.xml getCode -Dsfdc.username=$SFDC_USERNAME -Dsfdc.password=$SFDC_PASS$SFDC_TOKEN -Dsfdc.serverurl=https://$SFDC_SERVERURL
# Commit any changes to master
- git add force-app/main/default/*
- git config user.name "$GIT_USERNAME"
- git config user.email "$GIT_EMAIL"
- if [[ -n $(git status -s) ]] ; then filelist=`git status -s` ; git commit -a -m "$commitmsg" -m "$filelist" ; git push origin master:master ; else echo "No changes detected"; fi
I was adding my local server ssh key to my profile instead repository SSH KEY, so i had to get repository pipelines SSH Keys and add it to my profile.
Related
I would like to edit the gitlab cicd variables thru my pipeline script.
Flow:
Submit merge request
CI pipeline retrieve the version number (variable stored in the CICD project setting (refer to pic although the variable version is not included)
version+1 and set back into the CICD project setting
git tag the file based on version number
the file i want to git tag is a bash script file and I am using Linuxx docker image. please advise
current script
variables:
PROFILE_NAME: default
default:
image: docker-image
stages:
- tagging
Tag:
stage: tag
script:
- yum install git -y
- git --version
- git remote set-url --push origin ${CI_SERVER_PROTOCOL}://${GITLAB_PERSONAL_ACCESS_TOKEN_NAME}:${GITLAB_PERSONAL_ACCESS_TOKEN}#${CI_SERVER_HOST}/${CI_PROJECT_PATH}.git
- export "VERSION=$(($VERSION +1 ))" > $INC_VERSION
- echo 'after version:' $VERSION
- echo 'after increment version:' $INC_VERSION
- git push origin --tags
only:
refs:
- merge_requests
variables:
- ($CI_MERGE_REQUEST_TARGET_BRANCH_NAME =~ /^dev/ || $CI_COMMIT_BRANCH =~ /^dev/ || $CI_MERGE_REQUEST_TARGET_BRANCH_NAME =~ /^release-/ || $CI_COMMIT_BRANCH =~ /^release-/)
except:
variables:
- ($CI_COMMIT_BEFORE_SHA == '0000000000000000000000000000000000000000' && $CI_MERGE_REQUEST_TARGET_BRANCH_NAME !~ /^./)
I have the following pipelines file:
node('git') {
stage('Set Git Config') {
sh 'git config --global user.email "jenkins#test.com"'
sh 'git config --global user.name "jenkins"'
sh 'git config --global credential.helper cache'
sh "git config --global credential.helper 'cache --timeout=3600'"
}
stage('Set Git Credentials') {
git credentialsId: 'gitlab1', url: '${GITLAB1_REPO}'
git credentialsId: 'gitlab2', url: '${GITLAB2_REPO}'
}
stage('Synchronize with Gitlab2'){
sh 'git clone --bare ${GITLAB1_REPO} tfs'
dir("tfs") {
//add a remote repository
sh 'git remote add --mirror=fetch second ${GITLAB2_REPO}'
// update the local copy from the first repository
sh 'git fetch origin --tags'
// update the local copy with the second repository
sh 'git fetch second --tags'
// sync back the second repository
sh 'git push second --all'
sh 'git push second --tags'
}
}
}
Stage 1 and Stage 2 work perfectly. Stage 3 fails with permission denied.
I find this strange because on Stage 2, I can already see what the last commit was so it indicates that the credentials do work. Why aren't they working on stage 3?
This is the error I am seeing:
git clone --bare git#bitbucket.test/test.git tfs Cloning
into bare repository 'tfs'... Permission denied (publickey). fatal:
Could not read from remote repository.
While in stage 2, I see:
git config core.sparsecheckout # timeout=10
git checkout -f 30f1a7d1b77ef64e1cd44eab11a6ef4541c23b43
git branch -a -v --no-abbrev # timeout=10
git branch -D master # timeout=10
git checkout -b master 30f1a7d1b77ef64e1cd44eab11a6ef4541c23b43 Commit message: "test commit"
Stage 1 - you add some settings in shell to local git
Stage 2 - you point to actual credentials to be used and use a Jenkins plugin - which would just work
Satge 3 - back to shell, no credentials provided from jenkins, so the context is slave/local jenkins user.
Solution would be to use withCredentials for username and password or sshagent(credentials...) for private key
// credentialsId here is the credentials you have set up in Jenkins for pushing
// to that repository using username and password.
withCredentials([usernamePassword(credentialsId: 'git-pass-credentials-ID', passwordVariable: 'GIT_PASSWORD', usernameVariable: 'GIT_USERNAME')]) {
sh("git tag -a some_tag -m 'Jenkins'")
sh('git push https://${GIT_USERNAME}:${GIT_PASSWORD}#<REPO> --tags')
}
// For SSH private key authentication, try the sshagent step from the SSH Agent plugin.
sshagent (credentials: ['git-ssh-credentials-ID']) {
sh("git tag -a some_tag -m 'Jenkins'")
sh('git push <REPO> --tags')
}
I am trying to deploy the build to gh pages from travis ci. But, I am not able to access the variables from within the git commands, otherwise the variables are accessible when simply trying to echo them.
jobs:
include:
- stage: "lint"
name: "Check for code smell"
script: yarn lint
- stage: "deploy"
name: "Deploy to GH Pages"
script:
- git config --global user.name ${Name}
- git config --global user.email ${Email}
- git remote rm origin
- git remote add origin https://linux-nerd:${GITHUB_TOKEN}#${GH_REF}
- yarn run deploy
- echo ${Email}
- echo $Email
- echo https://linux-nerd:${GITHUB_TOKEN}#${GH_REF}
The last three echos are printing correctly, but the git commands do not take the correct values.
What am I missing?
this is how I configure git user and email:
git config --global user.name "username"
git config --global user.email "email"
Please not the use of "" above
thats what u are missing in your file
I am a CircleCI user, and I am setting up an integration with Heroku.
I want to do the following, and setup security with integrations with dockerHub and also to Heroku from the CircleCI portal page, using this config.yml file.
The problem is that CircleCI doesn't seem to know what these variables should be set to, and instead just echos.
${HEROKU_API_KEY} ${HEROKU_APP}
config.yml
version: 2
jobs:
build:
working_directory: ~/springboot_swagger_example-master-cassandra
docker:
- image: circleci/openjdk:8-jdk-browsers
steps:
- checkout
- restore_cache:
key: springboot_swagger_example-master-cassandra-{{ checksum "pom.xml" }}
- run: mvn dependency:go-offline
- save_cache:
paths:
- ~/.m2
key: springboot_swagger_example-master-cassandra-{{ checksum "pom.xml" }}
- type: add-ssh-keys
- type: deploy
name: "Deploy to Heroku"
command: |
if [ "${CIRCLE_BRANCH}" == "master" ]; then
# Install Heroku fingerprint (this is heroku's own key, NOT any of your private or public keys)
echo 'heroku.com ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAu8erSx6jh+8ztsfHwkNeFr/SZaSOcvoa8AyMpaerGIPZDB2TKNgNkMSYTLYGDK2ivsqXopo2W7dpQRBIVF80q9mNXy5tbt1WE04gbOBB26Wn2hF4bk3Tu+BNMFbvMjPbkVlC2hcFuQJdH4T2i/dtauyTpJbD/6ExHR9XYVhdhdMs0JsjP/Q5FNoWh2ff9YbZVpDQSTPvusUp4liLjPfa/i0t+2LpNCeWy8Y+V9gUlDWiyYwrfMVI0UwNCZZKHs1Unpc11/4HLitQRtvuk0Ot5qwwBxbmtvCDKZvj1aFBid71/mYdGRPYZMIxq1zgP1acePC1zfTG/lvuQ7d0Pe0kaw==' >> ~/.ssh/known_hosts
# git push git#heroku.com:yourproject.git $CIRCLE_SHA1:refs/heads/master
# Optional post-deploy commands
# heroku run python manage.py migrate --app=my-heroku-project
fi
- run: mvn package
- run:
name: Install Docker client
command: |
set -x
VER="17.03.0-ce"
curl -L -o /tmp/docker-$VER.tgz https://get.docker.com/builds/Linux/x86_64/docker-$VER.tgz
tar -xz -C /tmp -f /tmp/docker-$VER.tgz
mv /tmp/docker/* /usr/bin
- run:
name: Build Docker image
command: docker build -t joethecoder2/spring-boot-web:$CIRCLE_SHA1 .
- run:
name: Push to DockerHub
command: |
docker login -u$DOCKERHUB_LOGIN -p$DOCKERHUB_PASSWORD
docker push joethecoder2/spring-boot-web:$CIRCLE_SHA1
- run:
name: Setup Heroku
command: |
curl https://cli-assets.heroku.com/install-ubuntu.sh | sh
chmod +x .circleci/setup-heroku.sh
.circleci/setup-heroku.sh
- run:
name: Deploy to Heroku
command: |
mkdir app
cd app/
heroku create
# git push https://heroku:$HEROKU_API_KEY#git.heroku.com/$HEROKU_APP.git master
echo ${HEROKU_API_KEY}
echo ${HEROKU_APP}
git push https://heroku:${HEROKU_API_KEY}#git.heroku.com/${HEROKU_APP}.git master
- store_test_results:
path: target/surefire-reports
- store_artifacts:
path: target/spring-boot-web-0.0.1-SNAPSHOT.jar
The problem is that CircleCI doesn't seem to know what these variables should be set to, and instead just echos.
${HEROKU_API_KEY}
${HEROKU_APP}
The question, and problem is why aren't these settings being detected automatically?
You need to set the value for the variables: https://circleci.com/docs/2.0/env-vars/
They are being echo'd because you're echoing them.
echo ${HEROKU_API_KEY}
echo ${HEROKU_APP}
I have a repository with two branches: master and Dev and I want to configure that pipline in such a way that when I push code to Dev branch and code build was successfull, the Dev was merged to master. Unfortunatly I can't find any information about merge in bitbucket piplines docs.
That's my yml file:
pipelines:
branches:
Dev:
- step:
script:
- ant deployCodeCheckOnly -Dsf.username=$SF_USERNAME -Dsf.password=$SF_PASSWORD
Could somebody help me with that case? If it possible?
--Edit
I try to change script as sugest:
pipelines:
branches:
Dev:
- step:
script:
- ant deployCodeCheckOnly -Dsf.username=$SF_USERNAME -Dsf.password=$SF_PASSWORD
- git remote -v
- git fetch
- git checkout master
- git merge Dev
- git push -v --tags origin master:master
Result:
git remote -v
+ git remote -v
origin git#bitbucket.org:repository/project.git (fetch)
origin git#bitbucket.org:repository/project.git (push)
git fetch origin
+ git fetch origin
Warning: Permanently added the RSA host key for IP address ..... to the list of known hosts.
And error:
+ git checkout master
error: pathspec 'master' did not match any file(s) known to git.
--Solution
Dev:
- step:
script:
- ant deployCodeCheckOnly -Dsf.username=$SF_USERNAME Dsf.password=$SF_PASSWORD
- git fetch
- git checkout -b master
- git merge Dev
- git push -v --tags origin master:master
I was facing the same issue, but wanted to use pull requests instead of simple git merge. So I ended up utilising bitbucket API for the job:
Create "App password"
--
Create "App password" so you don't have to push your own credentials to pipelines
(bitbucket settings -> app passwords)
Set environment variables for pipelines
--
BB_USER = your username
BB_PASSWORD = app password
Create bash script
--
I have a bash script that creates pull request from $BITBUCKET_BRANCH and merge it immediately
#!/usr/bin/env bash
# Exit immediately if a any command exits with a non-zero status
# e.g. pull-request merge fails because of conflict
set -e
# Set destination branch
DEST_BRANCH=$1
# Create new pull request and get its ID
echo "Creating PR: $BITBUCKET_BRANCH -> $DEST_BRANCH"
PR_ID=`curl -X POST https://api.bitbucket.org/2.0/repositories/$BITBUCKET_REPO_OWNER/$BITBUCKET_REPO_SLUG/pullrequests \
--fail --show-error --silent \
--user $BB_USER:$BB_PASSWORD \
-H 'content-type: application/json' \
-d '{
"title": "Automerger: '$BITBUCKET_BRANCH' -> '$DEST_BRANCH'",
"description": "Automatic PR from pipelines",
"state": "OPEN",
"destination": {
"branch": {
"name": "'$DEST_BRANCH'"
}
},
"source": {
"branch": {
"name": "'$BITBUCKET_BRANCH'"
}
}
}' \
| sed -E "s/.*\"id\": ([0-9]+).*/\1/g"`
# Merge PR
echo "Merging PR: $PR_ID"
curl -X POST https://api.bitbucket.org/2.0/repositories/$BITBUCKET_REPO_OWNER/$BITBUCKET_REPO_SLUG/pullrequests/$PR_ID/merge \
--fail --show-error --silent \
--user $BB_USER:$BB_PASSWORD \
-H 'content-type: application/json' \
-d '{
"close_source_branch": false,
"merge_strategy": "merge_commit"
}'
usage: ./merge.sh DESTINATION_BRANCH
see pipelines environment variables documentation to understand better used variables
see bitbucket API docs for more info about used API
Finally in pipelines
--
just call the script:
Dev:
- step:
script:
- chmod +x ./merge.sh
- ./merge.sh master
Benefits:
Pipeline will fail if there is conflict (if you want it to fail)
better control of what's happening
In the “script” section of the YAML configuration, you can do more or less anything you can do at the shell, so (although I’ve never tried it) don’t see a reason why this shouldn’t be possible.
In other words, you’d have to:
Switch the branch to master
Merge dev (optionally, using the predefined BITBUCKET_COMMIT environment variable, which identifies your dev commit)
Commit to master (and probably also push)
As git is available in script, you can use normal git commands and do not need anything specific to Bb Pipelines, like so:
script:
- git fetch
- git checkout -b master
- git merge Dev
- git push -v --tags origin master:master
To make sure this is only done when your Ant job is successful, you should make sure that in case of an error you’ll get a non-zero exit status (which, I assume, is Ant’s default behaviour).