We use Bitbucket Cloud
I have the following bitbucket-pipelines.yml file. I removed the actual script steps to make it more concise.
image: node:9.8.0
definitions:
caches:
node-custom-staging: my_project/node_modules
steps:
- step: &npm_install
name: NPM install
caches:
- node
- node-custom-staging
script:
- ...
- step: &build_and_test
name: Build and Test
script:
- parallel:
- step:
caches:
- node
- node-custom-staging
name: Check coding and scss standards
script:
- ...
- step:
caches:
- node
- node-custom-staging
size: 2x
name: Check Build
script:
- ...
- step:
caches:
- node
- node-custom-staging
size: 2x
name: Execute unit test cases
script:
- ...
- step: &manage_version
name: Manage Version
size: 2x
script:
- ...
pipelines:
pull-requests:
'{feature/*,hotfix/*,bugfix/*,patch/*,breaking/*,release/*}':
- step: *manage_version
- step: *npm_install
- step: *build_and_test
branches:
master:
- step: *npm_install
- step: *build_and_test
Bitbucket gives me this error
There is an error in your bitbucket-pipelines.yml at [pipelines > pull-requests > {feature/*,hotfix/*,bugfix/*,patch/*,breaking/*,release/*} > 2 > step > script > 0]. To be precise: Missing or empty command string. Each item in this list should either be a single command string or a map defining a pipe invocation.
But yet if I cut/paste in Validator for bitbucket-pipelines.yml, it passes validation. What am I missing? TIA
Your parallel build and test pipeline is all wrong. The parallel definition must be at the same level as any other step, not inside a step's script!
image: node:9.8.0
definitions:
caches:
node-custom-staging: my_project/node_modules
steps:
- &npm_install_step
name: NPM install
caches:
- node
- node-custom-staging
script:
- ...
- &build_and_test_workflow
parallel:
- step:
caches:
- node
- node-custom-staging
name: Check coding and scss standards
script:
- ...
- step:
caches:
- node
- node-custom-staging
size: 2x
name: Check Build
script:
- ...
- step:
caches:
- node
- node-custom-staging
size: 2x
name: Execute unit test cases
script:
- ...
- &manage_version_step
name: Manage Version
size: 2x
script:
- ...
pipelines:
pull-requests:
'{feature/*,hotfix/*,bugfix/*,patch/*,breaking/*,release/*}':
- step: *manage_version_step
- step: *npm_install_step
- *build_and_test_workflow
branches:
master:
- step: *npm_install_step
- *build_and_test_workflow
I guess your code is syntactically valid YAML but the script elements aren't tested until an agent attempts to run them.
Related
Does anyone know why this script isn't working?
version: 2.1
orbs:
android: circleci/android#1.0.3
gcp-cli: circleci/gcp-cli#2.2.0
jobs:
build:
working_directory: ~/code
docker:
- image: cimg/android:2022.04
auth:
username: mydockerhub-user
password: $DOCKERHUB_PASSWORD
environment:
JVM_OPTS: -Xmx3200m
steps:
- checkout
- run:
name: Chmod permissions
command: sudo chmod +x ./gradlew
- run:
name: Download Dependencies
command: ./gradlew androidDependencies
- run:
name: Run Tests
command: ./gradlew lint test
- store_artifacts:
path: app/build/reports
destination: reports
- store_test_results:
path: app/build/test-results
nightly-android-test:
parameters:
system-image:
type: string
default: system-images;android-30;google_apis;x86
executor:
name: android/android-machine
resource-class: xlarge
steps:
- checkout
- android/start-emulator-and-run-tests:
test-command: ./gradlew connectedDebugAndroidTest
system-image: << parameters.system-image >>
- run:
name: Save test results
command: |
mkdir -p ~/test-results/junit/
find . -type f -regex ".*/build/outputs/androidTest-results/.*xml" -exec cp {} ~/test-results/junit/ \;
when: always
- store_test_results:
path: ~/test-results
- store_artifacts:
path: ~/test-results/junit
workflows:
unit-test-workflow:
jobs:
- build
nightly-test-workflow:
triggers:
- schedule:
cron: "0 0 * * *"
filters:
branches:
only:
- develop
jobs:
- nightly-android-test:
matrix:
alias: nightly
parameters:
system-image:
- system-images;android-30;google_apis;x86
- system-images;android-29;google_apis;x86
- system-images;android-28;google_apis;x86
- system-images;android-27;google_apis;x86
name: nightly-android-test-<<matrix.system-image>>
I keep getting the following build error:
Config does not conform to schema: {:workflows {:nightly-test-workflow {:jobs
[{:nightly-android-test {:matrix disallowed-key, :name disallowed-key}}]}}}
The second workflow seems to fail due to the matrix and name parameters but I can't see anything wrong in the script that would make them fail. I've tried looking at a yaml parser and couldn't see any null vaules and I tried the circle ci discussion forum with not a lot of luck.
I don't think that's the correct syntax. See the CircleCI documentation:
https://circleci.com/docs/2.0/configuration-reference/#matrix-requires-version-21
https://circleci.com/docs/2.0/using-matrix-jobs/
According to the above references, I believe it should be:
- nightly-android-test:
matrix:
alias: nightly
parameters:
system-image: ["system-images;android-30;google_apis;x86", "system-images;android-29;google_apis;x86", "system-images;android-28;google_apis;x86", "system-images;android-27;google_apis;x86"]
name: nightly-android-test-<<matrix.system-image>>
I'm new in circleci. I want to install my infrastructure via terraform after that I also want to trigger my build, deploy and push command for aws side. But workflow does not allow me to use plan_approve_apply and build-and-deploy together in understand one workflow. I also try to create multiple workflows (like below example) for each one but also it didn't work. How can I call both in single circli config file
My Circleci config yml file:
version: 2.1
orbs:
aws-ecr: circleci/aws-ecr#8.1.0
aws-ecs: circleci/aws-ecs#2.2.1
jobs:
init-plan:
working_directory: /tmp/project
docker:
- image: docker.mirror.hashicorp.services/hashicorp/terraform:light
steps:
- checkout
- run:
name: terraform init & plan
command: |
terraform init
terraform plan
- persist_to_workspace:
root: .
paths:
- .
apply:
docker:
- image: docker.mirror.hashicorp.services/hashicorp/terraform:light
steps:
- attach_workspace:
at: .
- run:
name: terraform
command: |
terraform apply
- persist_to_workspace:
root: .
paths:
- .
destroy:
docker:
- image: docker.mirror.hashicorp.services/hashicorp/terraform:light
steps:
- attach_workspace:
at: .
- run:
name: destroy
command: |
terraform destroy
- persist_to_workspace:
root: .
paths:
- .
workflows:
version: 2
plan_approve_apply:
jobs:
- init-plan
- apply:
requires:
- init-plan
- hold-destroy:
type: approval
requires:
- apply
- destroy:
requires:
- hold-destroy
workflows: # didn't work
build-and-deploy:
jobs:
- aws-ecr/build_and_push_image:
account-url: "${AWS_ACCOUNT_ID}.dkr.ecr.${AWS_DEFAULT_REGION}.amazonaws.com"
repo: "${AWS_RESOURCE_NAME_PREFIX}"
region: ${AWS_DEFAULT_REGION}
tag: "${CIRCLE_SHA1}"
- aws-ecs/deploy-service-update:
requires:
- aws-ecr/build_and_push_image
aws-region: ${AWS_DEFAULT_REGION}
family: "${AWS_RESOURCE_NAME_PREFIX}-service"
cluster-name: "${AWS_RESOURCE_NAME_PREFIX}-cluster"
container-image-name-updates: "container=${AWS_RESOURCE_NAME_PREFIX}-service,image-and-tag=${AWS_ACCOUNT_ID}.dkr.ecr.${AWS_DEFAULT_REGION}.amazonaws.com/${AWS_RESOURCE_NAME_PREFIX}:${CIRCLE_SHA1}"
ALM used Bitbucket Cloud
CI system used Bitbucket Cloud
Languages of the repository: Angular (Other (for JS, TS, Go, Python, PHP, …))
Error observed
ERROR: Error during SonarScanner execution
ERROR: Not authorized. Please check the property sonar.login or SONAR_TOKEN env variable
Steps to reproduce
SONAR_TOKEN already generated and added to my ENV_VAR
Bitbucket.yaml
image: ‘node:12.22’
clone:
depth: full # SonarCloud scanner needs the full history to assign issues properly
definitions:
caches:
sonar: ~/.sonar/cache # Caching SonarCloud artifacts will speed up your build
steps:
step: &build-test-sonarcloud
name: Build, test and analyze on SonarCloud
caches:
- sonar
script:
- pipe: sonarsource/sonarcloud-scan:1.2.1
variables:
EXTRA_ARGS: ‘-Dsonar.host.url=https://sonarcloud.io -Dsonar.login=${SONAR_TOKEN}’
step: &check-quality-gate-sonarcloud
name: Check the Quality Gate on SonarCloud
script:
- pipe: sonarsource/sonarcloud-quality-gate:0.1.4
pipelines:
branches
Potential workaround
No idea.
if you already install the sonar cloud app to your workspace environment, there is no need to give the sonar url again. The integration process is handling the URL part. Also, you should add your Sonar token variable to Workspace or repo environment. After that, you should login to Sonar Cloud organization account and bind your repo to SonarCloud to be able to evaluate it by Sonar Cloud. Here is my Sonar Cloud setup;
bitbucket-pipelines.yml file,
image:
name: <base image>
clone:
# SonarCloud scanner needs the full history to assign issues properly
depth: full
definitions:
caches:
# Caching SonarCloud artifacts will speed up your build
sonar: ~/.sonar/cache
pipelines:
pull-requests:
'**':
- step:
name: "Code Quality and Security on PR"
script:
- pipe: sonarsource/sonarcloud-scan:1.2.1
variables:
SONAR_TOKEN: '$SONAR_CLOUD_TOKEN'
SONAR_SCANNER_OPTS: -Xmx512m
DEBUG: "true"
branches:
master:
- step:
name: "Code Quality and Security on master"
script:
- pipe: sonarsource/sonarcloud-scan:1.2.1
variables:
SONAR_TOKEN: '$SONAR_CLOUD_TOKEN'
SONAR_SCANNER_OPTS: -Xmx512m
DEBUG: "true"
tags:
'*.*.*-beta*':
- step:
name: "Image Build & Push"
services:
- docker
caches:
- docker
clone:
depth: 1
script:
- <build script>
- step:
name: "Deploy"
deployment: beta
clone:
enabled: false
script:
- <deploy script>
'*.*.*-prod':
- step:
name: "Image Build & Push"
services:
- docker
caches:
- docker
clone:
depth: 1
script:
- <build script>
- step:
name: "Deploy"
deployment: prod
clone:
enabled: false
script:
- <deploy script>
sonar-project.properties file,
sonar.organization=<sonar cloud organization name>
sonar.projectKey=<project key>
sonar.projectName=<project name>
sonar.sources=<sonar evaluation path>
sonar.language=<repo language>
sonar.sourceEncoding=UTF-8
I'm trying to configure a Bitbucket pipeline to execute the SonarQube pipe, but Bitbucket complains that the pipeline step is empty, null or missing.
I've got SONAR_TOKEN defined as a project variable with the correct token.
Here's my current bitbucket-pipeline.yml file:
image: atlassian/default-image:2
clone:
depth: full
definitions:
caches:
sonar: ~/.sonar/cache
steps:
- step: &sonarcloud
name: Analyze on SonarCloud
caches:
- sonar
script:
- pipe: sonarsource/sonarcloud-scan:0.1.5
variables:
SONAR_TOKEN: ${SONAR_TOKEN}
pipelines:
branches:
'*':
- step: *sonarcloud
Any ideas?
Found the issue.
The problem is that the step details in the definition area is incorrectly indented and is missing one extra indentation level.
Instead of:
...
- steps: &sonarcloud
name: ...
...
It's
...
- steps: &sonarcloud
name: ... // Notice the extra level of indentation
...
The correct YAML is:
image: atlassian/default-image:2
clone:
depth: full
definitions:
caches:
sonar: ~/.sonar/cache
steps:
- step: &sonarcloud
name: Analyze on SonarCloud
caches:
- sonar
script:
- pipe: sonarsource/sonarcloud-scan:0.1.5
variables:
SONAR_TOKEN: ${SONAR_TOKEN}
pipelines:
branches:
'*':
- step: *sonarcloud
Is it possible to share steps between branches and still run branch specific steps? For example, the develop and release branch has the same build process, but uploaded to separate S3 buckets.
pipelines:
default:
- step:
script:
- cd source
- npm install
- npm build
develop:
- step:
script:
- s3cmd put --config s3cmd.cfg ./build s3://develop
staging:
- step:
script:
- s3cmd put --config s3cmd.cfg ./build s3://staging
I saw this post (Bitbucket Pipelines - multiple branches with same steps) but it's for the same steps.
Use YAML anchors:
definitions:
steps:
- step: &Test-step
name: Run tests
script:
- npm install
- npm run test
- step: &Deploy-step
name: Deploy to staging
deployment: staging
script:
- npm install
- npm run build
- fab deploy
pipelines:
default:
- step: *Test-step
- step: *Deploy-step
branches:
master:
- step: *Test-step
- step:
<<: *Deploy-step
name: Deploy to production
deployment: production
trigger: manual
Docs: https://confluence.atlassian.com/bitbucket/yaml-anchors-960154027.html
Although it's not officially supported yet, you can pre-define steps now.
You can use yaml anchors.
I got this tip from bitbucket staff when I had an issue running the same steps across a subset of branches.
definitions:
step: &Build
name: Build
script:
- npm install
- npm build
pipelines:
default:
- step: *Build
branches:
master:
- step: *Build
- step:
name: deploy
# do some deploy from master only
I think Bitbucket can't do it. You can use one pipeline and check the branch name:
pipelines:
default:
- step:
script:
- cd source
- npm install
- npm build
- if [[ $BITBUCKET_BRANCH = develop ]]; then s3cmd put --config s3cmd.cfg ./build s3://develop; fi
- if [[ $BITBUCKET_BRANCH = staging ]]; then s3cmd put --config s3cmd.cfg ./build s3://staging; fi
The two last lines will be executed only on the specified branches.
You can define and re-use steps with YAML Anchors.
anchor & to define a chunk of configuration
alias * to refer to that chunk elsewhere
And the source branch is saved in a default variable called BITBUCKET_BRANCH
You'd also need to pass the build results (in this case the build/ folder) from one step to the next, which is done with artifacts.
Combining all three will give you the following config:
definitions:
steps:
- step: &build
name: Build
script:
- cd source
- npm install
- npm build
artifacts: # defining the artifacts to be passed to each future step.
- ./build
- step: &s3-transfer
name: Transfer to S3
script:
- s3cmd put --config s3cmd.cfg ./build s3://${BITBUCKET_BRANCH}
pipelines:
default:
- step: *build
develop:
- step: *build
- step: *s3-transfer
staging:
- step: *build
- step: *s3-transfer
You can now also use glob patterns as mentioned in the referenced post and steps for both develop and staging branches in one go:
"{develop,staging}":
- step: *build
- step: *s3-transfer
Apparently it's in the works. Hopefully available soon.
https://bitbucket.org/site/master/issues/12750/allow-multiple-steps?_ga=2.262592203.639241276.1502122373-95544429.1500927287