I use CircleCI and the pipeline is as follows:
build
test
build app & nginx Docker images and push them to a GitLab registry
deploy Docker stack to the development server (currently the Swarm manager)
I just pushed my develop branch to my repository and faced a "Symfony4 new Controller page" on the development server after a successful message from CircleCI.
I logged via SSH in it and executed (with output for the application service):
docker stack ps my-development-stack --format "{{.Name}} {{.Image}} {{.CurrentState}}"
my-stack_app.1 gitlab-image:latest-develop Running 33 minutes ago
On my GitLab repository's registry, the application image has been "Last Updated" 41 minutes ago. The service's image has apparently been refreshed before with the last version.
Is it a common issue/error ?
How could (or should) I fix this timing issue ?
Can CircleCI help about this ?
Perhaps it is best ( though not ideal ) to introduce a delay between build and deploy , you can refer to this example here CircelCI Delay Between Jobs
I found a workaround using a CircleCI scheduled workflow triggered by a CRON. I scheduled a nightly build workflow which will run every day at midnight.
Sample of my config.yml file
# Beginning of the config.yml
# ...
workflows:
version: 2
# Push workflow
# ...
# Nightly build workflow
nightly-dev-deploy:
triggers:
- schedule:
cron: "0 0 * * *"
filters:
branches:
only:
- develop
jobs:
- build
- test:
requires:
- build
- deploy-dev:
requires:
- test
Read more about scheduled workflow with a nightly build example in the CircleCI official documentation
Looks more like a workaround to me. I'd be glad to hear how do you avoid this issue, which could lead to a better answer to the question.
Related
I am rewriting my CircleCI config. Everything was put in only one job and everything was working well, but for some good reasons I want more structure.
Now I have two jobs build and test, and I want the second job to reuse the machine exactly where the build job stopped.
I will later have a third and four job.
My desire would be a line that says I want to reuse the previous machine/executor, built-in from CircleCI.
Other options are Workspaces that save data on CircleCI machine, or building and deploying my own docker that represents the machine after the build job
What is the easiest way to achieve what I want to do ?
Currently, I have basically in my yaml:
jobs:
build:
docker:
- image: cypress/base:14.16.0
steps:
- checkout
- node/install:
install-yarn: true
node-version: '16.13'
- other-long-commands
test:
# NOT GOOD: need an executor
steps:
- run:
name: 'test'
command: 'npx cypress run'
environment:
TEST_SUITE: SMOKE
workflows:
build-and-test:
jobs:
- build
- smoke:
requires:
- build
Can't be done. Workspaces is the solution instead.
My follow up would be, why do you need two jobs? Depending on your use case, pulling steps out into reusable commands might help, or even an orb.
I am trying to implement CI/CD. So far what I have is a CI using CircleCI, and
circleci/aws-ecr#6.15.0
orb, so when I push to master the image is created and pushed to ECR. That works perfect, but what I want to implement is the deployment.
I think that I am not searching well in google, but I can not find any tutorial or explanation of how to do it.
I read that the orb
circleci/aws-ecs#01.4.0
and seems to do the job but I really dont know how to proceed.
This is my yaml file:
version: 2.1
orbs:
node: circleci/node#4.1.0
aws-ecr: circleci/aws-ecr#6.15.0
aws-ecs: circleci/aws-ecs#01.4.0
workflows:
build_and_push_image:
jobs:
- aws-ecr/build-and-push-image:
repo: node.js-api
tag: "latest,v0.1.${CIRCLE_BUILD_NUM}"
dockerfile: "Dockerfile"
path: .
- aws-ecs/deploy-service-update:
requires:
- aws-ecr/build-and-push-image
family: "${AWS_RESOURCE_NAME_PREFIX}-service"
cluster-name: "${AWS_RESOURCE_NAME_PREFIX}-cluster"
container-image-name-updates: "container=${AWS_RESOURCE_NAME_PREFIX}-service,tag=${CIRCLE_SHA1}"
Error:
An error occurred (ClientException) when calling the
DescribeTaskDefinition operation: Unable to describe task definition.
What I want to achieve is an automated deploy, I choose CircleCI because I thought it would be a good option also with my bitbucket repo, but if there is another way to achieve this I am totally open to hear suggestions.
Thanks!!
I'm using the circleCI CLI locally to test my .circleci/config.yml. This is what it looks like:
version: 2.1
jobs:
test:
docker:
- image: circleci/node:4.8.2
steps:
- checkout
- run: echo 'test step'
workflows:
version: 2
workflow:
jobs:
- test
This fails with the following error:
* Cannot find a job named build to run in the jobs: section of your configuration file.
If you expected a workflow to run, check your config contains a top-level key called 'workflows:'
The 'hello world' workflow from the CLI docs works fine.
What am I missing here?
In the same CircleCI CLI documentation mentioned above it has in the 'limitations' section:
The CLI tool does not provide support for running workflows. By nature, workflows leverage running jobs concurrently on multiple machines allowing you to achieve faster, more complex builds. Because the CLI is only running on your machine, it can only run single jobs (which make up parts of a workflow).
So I guess running workflows with orbs works (as in the 'hello world' example), but running workflows with your own jobs does not work with the CLI.
Testing Jobs Locally
If you're looking to test your config locally like I was, you can still execute your individual jobs locally. In the same documentation linked above, under the title 'Running a Job' when using config with version 2.1+ you can explicitly call one of your jobs like so:
circleci config process .circleci/config.yml > process.yml
circleci local execute -c process.yml --job JOB_NAME
I'm looking for a way to run a "cleanup" job/pipeline/etc when a GitLab merge request is closed (either merged or not).
The issue is this - we create a feature deployment on our cluster anytime a merge request is opened. Currently, I have no mechanism of detecting when an MR is closed. Over time these old 'feature deployments' accumulate on the cluster.
I could write a manual cleanup script (look at all open features, remove no-longer-existing-ones) from the cluster but that is going to be a bit hairy and error-prone. Was hoping GitLab has a method to use the really easy/nice pipeline+jobs features for this type of cleanup
We use GitLab environments for review apps. Environments can be automatically stopped after x weeks with environment.auto_stop_in. This has the advantage that, also when MR's stay open for months as they might be forgotten, the data will be cleaned up after x weeks (we use 2 weeks).
In the script, you can do whatever you need to clean up things. Like in our case, a helm uninstall.
gitlab-ci.yaml
deploy_review:
stage: deploy
script: "./deploy_review.sh"
environment:
auto_stop_in: 2 weeks
name: review/$CI_COMMIT_REF_SLUG
on_stop: stop_review_app
deployment_tier: development
resource_group: deploy-review-$CI_COMMIT_REF_SLUG
stop_review_app:
stage: after_deploy
when: manual
only:
- branches
variables:
GIT_STRATEGY: none
script: "./stop_review_app.sh"
environment:
name: review/$CI_COMMIT_REF_SLUG
action: stop
Is it possible to have another job run in the context of another job? I have some jobs that have some steps in common, and I don't want to repeat these steps in the different jobs.
push-production-image:
docker:
- image: google/cloud-sdk:latest
working_directory: ~/app
steps:
- setup-gcp-docker
- run: docker push [image]
No you cannot however YAML itself has a way to solve this problem with what is called YAML Anchors and Aliases.
Here's a blog post I wrote on how to do specifically this: https://circleci.com/blog/circleci-hacks-reuse-yaml-in-your-circleci-config-with-yaml/