Circleci Can we use multiple workflows for multiple type? - circleci

I'm new in circleci. I want to install my infrastructure via terraform after that I also want to trigger my build, deploy and push command for aws side. But workflow does not allow me to use plan_approve_apply and build-and-deploy together in understand one workflow. I also try to create multiple workflows (like below example) for each one but also it didn't work. How can I call both in single circli config file
My Circleci config yml file:
version: 2.1
orbs:
aws-ecr: circleci/aws-ecr#8.1.0
aws-ecs: circleci/aws-ecs#2.2.1
jobs:
init-plan:
working_directory: /tmp/project
docker:
- image: docker.mirror.hashicorp.services/hashicorp/terraform:light
steps:
- checkout
- run:
name: terraform init & plan
command: |
terraform init
terraform plan
- persist_to_workspace:
root: .
paths:
- .
apply:
docker:
- image: docker.mirror.hashicorp.services/hashicorp/terraform:light
steps:
- attach_workspace:
at: .
- run:
name: terraform
command: |
terraform apply
- persist_to_workspace:
root: .
paths:
- .
destroy:
docker:
- image: docker.mirror.hashicorp.services/hashicorp/terraform:light
steps:
- attach_workspace:
at: .
- run:
name: destroy
command: |
terraform destroy
- persist_to_workspace:
root: .
paths:
- .
workflows:
version: 2
plan_approve_apply:
jobs:
- init-plan
- apply:
requires:
- init-plan
- hold-destroy:
type: approval
requires:
- apply
- destroy:
requires:
- hold-destroy
workflows: # didn't work
build-and-deploy:
jobs:
- aws-ecr/build_and_push_image:
account-url: "${AWS_ACCOUNT_ID}.dkr.ecr.${AWS_DEFAULT_REGION}.amazonaws.com"
repo: "${AWS_RESOURCE_NAME_PREFIX}"
region: ${AWS_DEFAULT_REGION}
tag: "${CIRCLE_SHA1}"
- aws-ecs/deploy-service-update:
requires:
- aws-ecr/build_and_push_image
aws-region: ${AWS_DEFAULT_REGION}
family: "${AWS_RESOURCE_NAME_PREFIX}-service"
cluster-name: "${AWS_RESOURCE_NAME_PREFIX}-cluster"
container-image-name-updates: "container=${AWS_RESOURCE_NAME_PREFIX}-service,image-and-tag=${AWS_ACCOUNT_ID}.dkr.ecr.${AWS_DEFAULT_REGION}.amazonaws.com/${AWS_RESOURCE_NAME_PREFIX}:${CIRCLE_SHA1}"

Related

CircleCI terraform destroy fails?

I am new to CircleCI and trying to implement a simple workflow to apply and delete my terraform infrastructure using the terraform orb. This is my .circleci/config.yml file :-
version: 2.1
orbs:
terraform: circleci/terraform#3.1
jobs:
deploy_infrastructure:
executor: terraform/default
steps:
- checkout
- terraform/init
- terraform/validate
- terraform/plan
- terraform/apply
- terraform/destroy
workflows:
deploy_infrastructure:
jobs:
- deploy_infrastructure
I am able to create the infrastructure but as soon as it gets to destroy, it fails with the following error :-
Error: Module not installed
│
│ on main.tf line 1:
│ 1: module "vpc" {
│
│ This module is not yet installed. Run "terraform init" to install all modules required by this configuration
I even added terraform/init step before the terraform/destroy step and it still fails with the same error. Any help will be appreciated.
version: '2.1'
jobs:
init:
docker:
- image: hashicorp/terraform
steps:
- checkout
- run: terraform init
- run: terraform plan
- run: terraform apply -auto-approve
- persist_to_workspace:
root: .
paths:
- .
hold:
docker:
- image: hashicorp/terraform
steps:
- run: echo "Placeholder"
destroy:
docker:
- image: hashicorp/terraform
steps:
- attach_workspace:
at: .
- run: terraform destroy -auto-approve
workflows:
test:
jobs:
- init
- hold:
type: approval
requires:
- init
- destroy:
requires:
- hold
This worked!!!

Circle ci Workflow Build error. Matrix and Name parameters not working

Does anyone know why this script isn't working?
version: 2.1
orbs:
android: circleci/android#1.0.3
gcp-cli: circleci/gcp-cli#2.2.0
jobs:
build:
working_directory: ~/code
docker:
- image: cimg/android:2022.04
auth:
username: mydockerhub-user
password: $DOCKERHUB_PASSWORD
environment:
JVM_OPTS: -Xmx3200m
steps:
- checkout
- run:
name: Chmod permissions
command: sudo chmod +x ./gradlew
- run:
name: Download Dependencies
command: ./gradlew androidDependencies
- run:
name: Run Tests
command: ./gradlew lint test
- store_artifacts:
path: app/build/reports
destination: reports
- store_test_results:
path: app/build/test-results
nightly-android-test:
parameters:
system-image:
type: string
default: system-images;android-30;google_apis;x86
executor:
name: android/android-machine
resource-class: xlarge
steps:
- checkout
- android/start-emulator-and-run-tests:
test-command: ./gradlew connectedDebugAndroidTest
system-image: << parameters.system-image >>
- run:
name: Save test results
command: |
mkdir -p ~/test-results/junit/
find . -type f -regex ".*/build/outputs/androidTest-results/.*xml" -exec cp {} ~/test-results/junit/ \;
when: always
- store_test_results:
path: ~/test-results
- store_artifacts:
path: ~/test-results/junit
workflows:
unit-test-workflow:
jobs:
- build
nightly-test-workflow:
triggers:
- schedule:
cron: "0 0 * * *"
filters:
branches:
only:
- develop
jobs:
- nightly-android-test:
matrix:
alias: nightly
parameters:
system-image:
- system-images;android-30;google_apis;x86
- system-images;android-29;google_apis;x86
- system-images;android-28;google_apis;x86
- system-images;android-27;google_apis;x86
name: nightly-android-test-<<matrix.system-image>>
I keep getting the following build error:
Config does not conform to schema: {:workflows {:nightly-test-workflow {:jobs
[{:nightly-android-test {:matrix disallowed-key, :name disallowed-key}}]}}}
The second workflow seems to fail due to the matrix and name parameters but I can't see anything wrong in the script that would make them fail. I've tried looking at a yaml parser and couldn't see any null vaules and I tried the circle ci discussion forum with not a lot of luck.
I don't think that's the correct syntax. See the CircleCI documentation:
https://circleci.com/docs/2.0/configuration-reference/#matrix-requires-version-21
https://circleci.com/docs/2.0/using-matrix-jobs/
According to the above references, I believe it should be:
- nightly-android-test:
matrix:
alias: nightly
parameters:
system-image: ["system-images;android-30;google_apis;x86", "system-images;android-29;google_apis;x86", "system-images;android-28;google_apis;x86", "system-images;android-27;google_apis;x86"]
name: nightly-android-test-<<matrix.system-image>>

encountering this error on circle ci: Unable to parse YAML # mapping values are not allowed here # in 'string', line 3, column 5

Here's my code
version: 2.1
orbs:
flutter: circleci/flutter#1.1.0
jobs:
build:
working_directory: ~/project
docker:
- image: "cirrusci/flutter"
steps:
- checkout
- run: flutter main.dart
test:
working_directory: ~/project
docker:
- image: "cirrusci/flutter"
steps:
- checkout
- run: flutter widget_test.dart
workflows:
build_and_test:
jobs:
- build
- test:
requires:
- build
I got this error from the CircleCI:
Unable to parse YAML # mapping values are not allowed here # in
'string', line 3, column 5
You can use this YAML script to achieve your workflow:
# Use the latest 2.1 version of CircleCI pipeline process engine.
# See: https://circleci.com/docs/2.0/configuration-reference
version: 2.1
# Define a job to be invoked later in a workflow.
# See: https://circleci.com/docs/2.0/configuration-reference/#jobs
jobs:
build:
description: App Bundle Build
docker:
- image: cirrusci/flutter
steps:
- checkout
- run:
name: Build Flutter (Android)
command: |
flutter pub get
flutter clean
flutter build apk --release
- store_artifacts:
path: build/app/outputs/flutter-apk/app-release.apk
test:
description: Flutter App Test
docker:
- image: cirrusci/flutter
steps:
- checkout
- run:
name: Run Flutter Test and Analyze
command: |
flutter doctor
flutter pub get
flutter analyze
flutter test
workflows:
build_and_test:
jobs:
- build
- test:
requires:
- build

How can I create CI/CD pipeline with cloudbuild.yaml in GCP?

I am trying to create a simple CI/CD pipeline. After the client makes git push, it will start a trigger with the below cloudbuilder.yaml:
# steps:
# - name: 'docker/compose:1.28.2'
# args: ['up', '-d']
# - name: "docker/compose:1.28.2"
# args: ["build"]
# images: ['gcr.io/$PROJECT_ID/cloudbuild-demo-dockercompose']
# - name: 'gcr.io/cloud-builders/docker'
# id: 'backend'
# args: ['build','-t', 'gcr.io/$PROJECT_ID/cloudbuild-demo-dockercompose:latest','.']
# - name: 'gcr.io/cloud-builders/docker'
# args: ['push', 'gcr.io/$PROJECT_ID/cloudbuild-demo-dockercompose:latest']
# steps:
# - name: 'docker/compose:1.28.2'
# args: ['up', '-d']
# - name: "docker/compose:1.28.2"
# args: ["build"]
# images:
# - 'gcr.io/$PROJECT_ID/cloudbuild-demo-dockercompose'
# In this directory, run the following command to build this builder.
# $ gcloud builds submit . --config=cloudbuild.yaml
substitutions:
_DOCKER_COMPOSE_VERSION: 1.28.2
steps:
- name: 'docker/compose:1.28.2'
args:
- 'build'
- '--build-arg'
- 'DOCKER_COMPOSE_VERSION=${_DOCKER_COMPOSE_VERSION}'
- '-t'
- 'gcr.io/$PROJECT_ID/cloudbuild-demo-dockercompose:latest'
- '-t'
- 'gcr.io/$PROJECT_ID/cloudbuild-demo-dockercompose:${_DOCKER_COMPOSE_VERSION}'
- '.'
- name: 'gcr.io/$PROJECT_ID/cloudbuild-demo-dockercompose'
args: ['version']
images:
- 'gcr.io/$PROJECT_ID/cloudbuild-demo-dockercompose:latest'
- 'gcr.io/$PROJECT_ID/cloudbuild-demo-dockercompose:${_DOCKER_COMPOSE_VERSION}'
tags: ['cloud-builders-community']
It returns the error below: it cannot create images in the repository. How can I solve this issue?
ERROR: failed to pull because we ran out of retries.
ERROR
ERROR: build step 1 "gcr.io/internal-invoicing-solution/cloudbuild-demo-dockercompose" failed: error pulling build step 1 "gcr.io/internal-invoicing-solution/cloudbuild-demo-dockercompose": generic::unknown: retry budget exhausted (10 attempts): step exited with non-zero status: 1
```
Before pulling your image in the 2nd step, you need to push it. When you declare the images at the end of your yaml definition, the image are pushed automatically at the end on the pipeline. Here you need it in the middle.
EDIT 1
I just added a docker push step, copy and paste from your comments. Does it work?
substitutions:
_DOCKER_COMPOSE_VERSION: 1.28.2
steps:
- name: 'docker/compose:1.28.2'
args:
- 'build'
- '--build-arg'
- 'DOCKER_COMPOSE_VERSION=${_DOCKER_COMPOSE_VERSION}'
- '-t'
- 'gcr.io/$PROJECT_ID/cloudbuild-demo-dockercompose:latest'
- '-t'
- 'gcr.io/$PROJECT_ID/cloudbuild-demo-dockercompose:${_DOCKER_COMPOSE_VERSION}'
- '.'
- name: 'gcr.io/cloud-builders/docker'
args: ['push', 'gcr.io/$PROJECT_ID/cloudbuild-demo-dockercompose:latest']
- name: 'gcr.io/$PROJECT_ID/cloudbuild-demo-dockercompose'
args: ['version']
images:
- 'gcr.io/$PROJECT_ID/cloudbuild-demo-dockercompose:latest'
- 'gcr.io/$PROJECT_ID/cloudbuild-demo-dockercompose:${_DOCKER_COMPOSE_VERSION}'
tags: ['cloud-builders-community']

How to use docker image from build step in deploy step in CircleCI 2.0?

Struggling for a few last days to migrate from CircleCI 1.0 to 2.0 and while the build process is done, deployment is still a big issue. CircleCI documentation is not really of a big help.
Here is a similar config.yml to what I have:
version 2
jobs:
build:
docker:
- image: circleci/node:8.9.1
steps:
- checkout
- setup_remote_docker
- run
name: Install required stuff
command: [...]
- run:
name: Build
command: docker build -t project .
deploy:
docker:
- image: circleci/node:8.9.1
steps:
- checkout
- run:
name: Deploy
command: |
bash scripts/deploy/deploy.sh
docker tag project [...]
docker push [...]
workflows:
version: 2
build-deploy:
jobs:
- build
- deploy:
requires:
- build
filters:
branches:
only: develop
The issue is in deploy job. I have to specify the docker: -image point but I want to reuse the environment from build job where all the required stuff is installed already. Surely, I could just install them in deploy job, but having multiple deploy jobs leads to code duplication which is something I do not want.
you probably want to persist to workspace and attach it in your deploy job.
you wont need to use '- checkout' after that
https://circleci.com/docs/2.0/configuration-reference/#persist_to_workspace
jobs:
build:
docker:
- image: circleci/node:8.9.1
steps:
- checkout
- setup_remote_docker
- run
name: Install required stuff
command: [...]
- run:
name: Build
command: docker build -t project .
- persist_to_workspace:
root: ./
paths:
- ./
deploy:
docker:
- image: circleci/node:8.9.1
steps:
- attach_workspace:
at: ./
- run:
name: Deploy
command: |
bash scripts/deploy/deploy.sh
docker tag project [...]
docker push [...]
workflows:
version: 2
build-deploy:
jobs:
- build
- deploy:
requires:
- build
filters:
branches:
only: develop
If you label the image built by the build stage, you can then reference it in the deploy stage: https://docs.docker.com/compose/compose-file/#labels

Resources