I'm iterating on adding database migrations to a project. For the first step, I've made a repository that runs migrations. Now I need to make it so these migrations run against the stage/prod environment. I do not want this to happen on every commit. Does circle ci provide a way to have a button that I can click on to run a job?
I think ideally I'd have 2 buttons. One for running migrations on stage, one for running them on prod. Is this possible?
There is a manual approval process for workflows.
https://circleci.com/docs/2.0/workflows/#holding-a-workflow-for-a-manual-approval
workflows:
version: 2
build-test-and-approval-deploy:
jobs:
- build
- test1:
requires:
- build
- test2:
requires:
- test1
- hold:
type: approval
requires:
- test2
- deploy:
requires:
- hold
It's pretty limited. You can't use it to start a build.
Related
I am rewriting my CircleCI config. Everything was put in only one job and everything was working well, but for some good reasons I want more structure.
Now I have two jobs build and test, and I want the second job to reuse the machine exactly where the build job stopped.
I will later have a third and four job.
My desire would be a line that says I want to reuse the previous machine/executor, built-in from CircleCI.
Other options are Workspaces that save data on CircleCI machine, or building and deploying my own docker that represents the machine after the build job
What is the easiest way to achieve what I want to do ?
Currently, I have basically in my yaml:
jobs:
build:
docker:
- image: cypress/base:14.16.0
steps:
- checkout
- node/install:
install-yarn: true
node-version: '16.13'
- other-long-commands
test:
# NOT GOOD: need an executor
steps:
- run:
name: 'test'
command: 'npx cypress run'
environment:
TEST_SUITE: SMOKE
workflows:
build-and-test:
jobs:
- build
- smoke:
requires:
- build
Can't be done. Workspaces is the solution instead.
My follow up would be, why do you need two jobs? Depending on your use case, pulling steps out into reusable commands might help, or even an orb.
I have a monorepo project that is deployed to 3 environments - testing, staging and production. Deploys to testing come from the next branch, while staging and production from the master branch. Testing deploys should run automatically on every commit to next (but I'm also fine with having to trigger them manually), but deploys from the master branch should be triggered manually. In addition, every deploy may consist of a client push and server push (depending on the files changed). The commands to deploy to each of the hosts are exactly the same, the only thing changing is the host itself and the environment variables.
Therefore I have 2 questions:
Can I make Bitbucket prompt me the deployment target when I manually trigger the pipeline, thus basically letting me choose the set of the env variables to inject into the set sequence of commands? I've seen a screenshot for this in a tutorial, but I lost it and can't find it since.
Can I have parallel sequences of commands? I'd like the server and the client push to run simultaneously, but both of them have different steps. Or do I need to merge those into the same step with multiple scripts to achieve that?
Thank you for your help.
The answer to both of your questions is 'Yes'.
The feature that makes it possible is called custom pipelines. Here is a neat doc that demonstrates how to use them.
There is a parallel keyword which you can use to define parallel steps. Check out this doc for details.
If I'm not misinterpreting the description of your setup, your final pipeline should look very similar to this:
pipelines:
custom:
deploy-to-staging-or-prod: # As you say the steps are the same, only variable values will define the destination.
- variables: # List variable names under here, and Bitbucket will prompt you to supply their values.
- name: VAR1
- name: VAR2
- parallel:
- step:
- ./deploy-client.sh
- step:
- ./deploy-server.sh
branches:
next:
- step:
script:
- ./deploy-to-testing.sh
UPD
If you need to use Deployments instead of providing each variable separately, use can utilise manual type of trigger:
definitions:
steps:
- step: &RunTests
script:
- ./run-tests.sh
- step: &DeployFromMaster
script:
- ./deploy-from-master.sh
pipelines:
branches:
next:
- step:
script:
- ./deploy-to-testing.sh
master:
- step: *RunTests
- parallel:
- step:
<<: *DeployFromMaster
deployment: staging
trigger: manual
- step:
<<: *DeployFromMaster
deployment: production
trigger: manual
Key docs for understanding this pipeline is still this one and this one for yaml anchors. Keep in mind that I introduced a 'RunTests' step on purpose, as
Since a pipeline is triggered on a commit, you can't make the first step manual.
It will act as a stopper for the deploy step which can only be manual due to your requirements.
I have a CircleCI job that runs some visual snapshot tests, if the tests fail I'd like to show an approval job to let the developer run a second job that will update the snapshot tests and push them to the PR branch.
This is my current config:
jobs:
- visual-tests
- update-visual-tests-approval:
type: approval
requires:
- visual-tests
# Here I'd need something to only show this (and the subsequent step)
# only if the `visual-tests` step failed
- update-visual-tests:
requires:
- update-visual-tests-approval
What are my options?
this will do the trick:
https://support.circleci.com/hc/en-us/articles/360043188514-How-to-Retry-a-Failed-Step-with-when-Attribute-
you will need to apply this change on the update-visual-tests-approval job.
If I have a .circleci/config.yml file like so:
version: 2
jobs:
build-node8:
docker:
- image: oresoftware/lmx-circleci:8
steps:
- checkout
- run: ./scripts/circleci/run.sh
build-node9:
docker:
- image: oresoftware/lmx-circleci:9
steps:
- checkout
- run: ./scripts/circleci/run.sh
build-node10:
docker:
- image: oresoftware/lmx-circleci:10
steps:
- checkout
- run: ./scripts/circleci/run.sh
build-node11:
docker:
- image: oresoftware/lmx-circleci:11
steps:
- checkout
- run: ./scripts/circleci/run.sh
build-node12:
docker:
- image: oresoftware/lmx-circleci:12
steps:
- checkout
- run: ./scripts/circleci/run.sh
there are 5 jobs listed here, but when the builds start, only 4 jobs run in parallel. Is there a way to run more than 4 jobs in parallel is there a hard limit there?
My guess is that under workflows, I can change the parallelism level?
workflows:
version: 2
build_nodejs:
parallelism: 5
jobs:
- build-node8
- build-node9
- build-node10
- build-node11
- build-node12
perhaps this requires a paid account tho?
Short Answer:
CircleCi lets you run as many jobs in parallel as you want, as long as your payment plan has enough containers to serve each job.
I suspect that your plan only has 4 containers. You can check to see how many containers you have in the settings tab in CircleCi
In the below example I have a total of 2 containers available: 1 paid + 1 free. So right now at max I can only run 2 jobs in parallel. I can pay an extra $50 per month per container to add additional containers thoughs.
Additional Details:
This article gives a great overview of how to configure circle ci jobs to run in parallel (and it actually has an example where 5 jobs run in parallel). https://circleci.com/blog/decrease-your-build-times-by-running-jobs-in-parallel-with-workflows/
Regarding the config file code snippet you pasted in your question - It looks fine (Tho you don't need the parallelism: 5 flag, since circle will use all available plan capacity automatically)
Can you please check to see how many containers are in your plan and then report back?
FYI - CircleCi container and concurrent job plan info:
https://circleci.com/pricing/
What I want by default is for my branch to build, then wait for approval to deploy to dev. However, if I push to the dev branch, that should deploy without approval.
I have the following workflow:
workflows:
version: 2
build:
jobs:
- build
- approve-dev:
type: approval
requires:
- build
filters:
branches:
ignore: dev
- deploy-dev:
requires:
- approve-dev
The problem is that when the approve-dev job is skipped, the deploy-dev job loses its requirement, so the only possible step is build.
Is there a way around this?
I have worked out a way to do this, but it's quite verbose, basically you duplicate the workflow at that point, with one copy only for that branch, and one copy which ignores that branch, like so:
workflows:
version: 2
build:
jobs:
- build
- approve-dev:
type: approval
requires:
- build
filters:
branches:
ignore: dev
- deploy-dev:
requires:
- approve-dev
- deploy-dev-auto:
requires:
- build
filters:
branches:
only: dev
Using templates (<<deploy-dev-defaults: &deploy-dev-defaults) to define the tasks means you don't have to rewrite the job, just specify it twice with the two different names.