I am setting up a continuous integration of a node project and I am unable to share a folder between two different jobs.
workflows:
version: 2
build-and-deploy:
jobs:
- build:
filters:
branches:
only: circleci
- deploy:
filters:
branches:
only: circleci
requires:
- build
Is there a way to share the /dist folder from 'build' job to the 'deploy'
job?
Related
ALM used Bitbucket Cloud
CI system used Bitbucket Cloud
Languages of the repository: Angular (Other (for JS, TS, Go, Python, PHP, …))
Error observed
ERROR: Error during SonarScanner execution
ERROR: Not authorized. Please check the property sonar.login or SONAR_TOKEN env variable
Steps to reproduce
SONAR_TOKEN already generated and added to my ENV_VAR
Bitbucket.yaml
image: ‘node:12.22’
clone:
depth: full # SonarCloud scanner needs the full history to assign issues properly
definitions:
caches:
sonar: ~/.sonar/cache # Caching SonarCloud artifacts will speed up your build
steps:
step: &build-test-sonarcloud
name: Build, test and analyze on SonarCloud
caches:
- sonar
script:
- pipe: sonarsource/sonarcloud-scan:1.2.1
variables:
EXTRA_ARGS: ‘-Dsonar.host.url=https://sonarcloud.io -Dsonar.login=${SONAR_TOKEN}’
step: &check-quality-gate-sonarcloud
name: Check the Quality Gate on SonarCloud
script:
- pipe: sonarsource/sonarcloud-quality-gate:0.1.4
pipelines:
branches
Potential workaround
No idea.
if you already install the sonar cloud app to your workspace environment, there is no need to give the sonar url again. The integration process is handling the URL part. Also, you should add your Sonar token variable to Workspace or repo environment. After that, you should login to Sonar Cloud organization account and bind your repo to SonarCloud to be able to evaluate it by Sonar Cloud. Here is my Sonar Cloud setup;
bitbucket-pipelines.yml file,
image:
name: <base image>
clone:
# SonarCloud scanner needs the full history to assign issues properly
depth: full
definitions:
caches:
# Caching SonarCloud artifacts will speed up your build
sonar: ~/.sonar/cache
pipelines:
pull-requests:
'**':
- step:
name: "Code Quality and Security on PR"
script:
- pipe: sonarsource/sonarcloud-scan:1.2.1
variables:
SONAR_TOKEN: '$SONAR_CLOUD_TOKEN'
SONAR_SCANNER_OPTS: -Xmx512m
DEBUG: "true"
branches:
master:
- step:
name: "Code Quality and Security on master"
script:
- pipe: sonarsource/sonarcloud-scan:1.2.1
variables:
SONAR_TOKEN: '$SONAR_CLOUD_TOKEN'
SONAR_SCANNER_OPTS: -Xmx512m
DEBUG: "true"
tags:
'*.*.*-beta*':
- step:
name: "Image Build & Push"
services:
- docker
caches:
- docker
clone:
depth: 1
script:
- <build script>
- step:
name: "Deploy"
deployment: beta
clone:
enabled: false
script:
- <deploy script>
'*.*.*-prod':
- step:
name: "Image Build & Push"
services:
- docker
caches:
- docker
clone:
depth: 1
script:
- <build script>
- step:
name: "Deploy"
deployment: prod
clone:
enabled: false
script:
- <deploy script>
sonar-project.properties file,
sonar.organization=<sonar cloud organization name>
sonar.projectKey=<project key>
sonar.projectName=<project name>
sonar.sources=<sonar evaluation path>
sonar.language=<repo language>
sonar.sourceEncoding=UTF-8
i wrote a pipeline in bitbucket environment but i would like the pipeline to be triggered only when the user run it and not automatically on push or commit.
here is the code:
pipelines:
branches:
new_ui_apk:
- step:
name: Build apk
size: 2x
script:
- JAVA_OPTS="-Xmx2048m -XX:MaxPermSize=2048m -XX:+HeapDumpOnOutOfMemoryError -Dfile.encoding=UTF-8"
- docker build -t app-release:1.0.0 .
services:
- docker
definitions:
services:
docker:
memory: 7128
actually i use the skip ci tip to avoid it but if another team member push or commit any change, the pipeline will run, how else can i avoid it please?
if you mention the definition under "custom" property it stops listening branches and only acts when a user triggers it.
use this.
pipelines:
custom:
new_ui_apk:
- step:
name: Build apk
size: 2x
script:
- JAVA_OPTS="-Xmx2048m -XX:MaxPermSize=2048m -XX:+HeapDumpOnOutOfMemoryError -Dfile.encoding=UTF-8"
- docker build -t app-release:1.0.0 .
services:
- docker
definitions:
services:
docker:
memory: 7128
The Answer is not good you only need to add trigger: manual
-step
image: XXX
name: XXXX
deployment: XXXX
trigger: manual
script:
- whatever....
And it will be shown a option to be run inside the pipeline options.
I have this simple config:
version: 2
jobs:
build:
machine: true
steps:
- checkout
- run: |
docker-compose -f docker-compose.test.yml build
test:
machine: true
working_directory: ~/app
steps:
- checkout
- run:
command: make test
name: Test
workflows:
version: 2
build_and_test:
jobs:
- build
- test:
requires:
- build
My test job, the second one, fails for some reasons. But whenever I check the logs on CircleCI, I can see that the image is always built in this job. I was expecting that the job would use the container that was built in the build job. So question is, why are containers not shared across jobs?
Struggling for a few last days to migrate from CircleCI 1.0 to 2.0 and while the build process is done, deployment is still a big issue. CircleCI documentation is not really of a big help.
Here is a similar config.yml to what I have:
version 2
jobs:
build:
docker:
- image: circleci/node:8.9.1
steps:
- checkout
- setup_remote_docker
- run
name: Install required stuff
command: [...]
- run:
name: Build
command: docker build -t project .
deploy:
docker:
- image: circleci/node:8.9.1
steps:
- checkout
- run:
name: Deploy
command: |
bash scripts/deploy/deploy.sh
docker tag project [...]
docker push [...]
workflows:
version: 2
build-deploy:
jobs:
- build
- deploy:
requires:
- build
filters:
branches:
only: develop
The issue is in deploy job. I have to specify the docker: -image point but I want to reuse the environment from build job where all the required stuff is installed already. Surely, I could just install them in deploy job, but having multiple deploy jobs leads to code duplication which is something I do not want.
you probably want to persist to workspace and attach it in your deploy job.
you wont need to use '- checkout' after that
https://circleci.com/docs/2.0/configuration-reference/#persist_to_workspace
jobs:
build:
docker:
- image: circleci/node:8.9.1
steps:
- checkout
- setup_remote_docker
- run
name: Install required stuff
command: [...]
- run:
name: Build
command: docker build -t project .
- persist_to_workspace:
root: ./
paths:
- ./
deploy:
docker:
- image: circleci/node:8.9.1
steps:
- attach_workspace:
at: ./
- run:
name: Deploy
command: |
bash scripts/deploy/deploy.sh
docker tag project [...]
docker push [...]
workflows:
version: 2
build-deploy:
jobs:
- build
- deploy:
requires:
- build
filters:
branches:
only: develop
If you label the image built by the build stage, you can then reference it in the deploy stage: https://docs.docker.com/compose/compose-file/#labels
Is it possible to share steps between branches and still run branch specific steps? For example, the develop and release branch has the same build process, but uploaded to separate S3 buckets.
pipelines:
default:
- step:
script:
- cd source
- npm install
- npm build
develop:
- step:
script:
- s3cmd put --config s3cmd.cfg ./build s3://develop
staging:
- step:
script:
- s3cmd put --config s3cmd.cfg ./build s3://staging
I saw this post (Bitbucket Pipelines - multiple branches with same steps) but it's for the same steps.
Use YAML anchors:
definitions:
steps:
- step: &Test-step
name: Run tests
script:
- npm install
- npm run test
- step: &Deploy-step
name: Deploy to staging
deployment: staging
script:
- npm install
- npm run build
- fab deploy
pipelines:
default:
- step: *Test-step
- step: *Deploy-step
branches:
master:
- step: *Test-step
- step:
<<: *Deploy-step
name: Deploy to production
deployment: production
trigger: manual
Docs: https://confluence.atlassian.com/bitbucket/yaml-anchors-960154027.html
Although it's not officially supported yet, you can pre-define steps now.
You can use yaml anchors.
I got this tip from bitbucket staff when I had an issue running the same steps across a subset of branches.
definitions:
step: &Build
name: Build
script:
- npm install
- npm build
pipelines:
default:
- step: *Build
branches:
master:
- step: *Build
- step:
name: deploy
# do some deploy from master only
I think Bitbucket can't do it. You can use one pipeline and check the branch name:
pipelines:
default:
- step:
script:
- cd source
- npm install
- npm build
- if [[ $BITBUCKET_BRANCH = develop ]]; then s3cmd put --config s3cmd.cfg ./build s3://develop; fi
- if [[ $BITBUCKET_BRANCH = staging ]]; then s3cmd put --config s3cmd.cfg ./build s3://staging; fi
The two last lines will be executed only on the specified branches.
You can define and re-use steps with YAML Anchors.
anchor & to define a chunk of configuration
alias * to refer to that chunk elsewhere
And the source branch is saved in a default variable called BITBUCKET_BRANCH
You'd also need to pass the build results (in this case the build/ folder) from one step to the next, which is done with artifacts.
Combining all three will give you the following config:
definitions:
steps:
- step: &build
name: Build
script:
- cd source
- npm install
- npm build
artifacts: # defining the artifacts to be passed to each future step.
- ./build
- step: &s3-transfer
name: Transfer to S3
script:
- s3cmd put --config s3cmd.cfg ./build s3://${BITBUCKET_BRANCH}
pipelines:
default:
- step: *build
develop:
- step: *build
- step: *s3-transfer
staging:
- step: *build
- step: *s3-transfer
You can now also use glob patterns as mentioned in the referenced post and steps for both develop and staging branches in one go:
"{develop,staging}":
- step: *build
- step: *s3-transfer
Apparently it's in the works. Hopefully available soon.
https://bitbucket.org/site/master/issues/12750/allow-multiple-steps?_ga=2.262592203.639241276.1502122373-95544429.1500927287