I have a backend API via Node/Express and am just starting to learn CI/CD. I am using CircleCI and have a general template set up but can't seem to figure out how to get CircleCi to create a build folder during the process.
Below is my starting circle.yml file:
version: 2.1
jobs:
build:
docker:
- image: circleci/node:12.9.1
steps:
- checkout
- run: echo "npm installing"
- run: npm install
- run: npm run build
test:
docker:
- image: circleci/node:12.9.1
steps:
- checkout
- run: echo "Running test suite (TODO not currently applicable)"
complete:
docker:
- image: circleci/node:12.9.1
steps:
- checkout
- run: echo "Completed build & test process (TODO Add linting checks)"
workflows:
version: 2.1
build_test:
jobs:
- build
I have a build script in my package.json with the following: "build": "tsc".
Running this outside of circleci, it compiles the build folder as I need.
Looking at the status on circleci, everything passes. But the branch does not have the actual build folder.
version: 2.1
jobs:
build:
docker:
- image: circleci/node:12.9.1
steps:
- checkout
- run: echo "npm installing"
- run: npm install
- run: npm run build
- persist_to_workspace:
root: ~/project
paths:
- build
test:
docker:
- image: circleci/node:12.9.1
steps:
- checkout
- run: echo "Running test suite (TODO not currently applicable)"
complete:
docker:
- image: circleci/node:12.9.1
steps:
- checkout
- run: echo "Completed build & test process (TODO Add linting checks)"
deploy:
docker:
- image: circleci/node:12.9.1
steps:
- attach_workspace:
at: ~/project
- run:
name: Deployment Step
command: |
#!/bin/sh
ls build
workflows:
version: 2.1
build_test:
jobs:
- build
- test:
requires:
- build
- complete:
requires:
- test
- deploy:
requires:
- complete
When the build script runs, creates a folder called build that you can persist in order to use in another job. In the new job called deploy, it calls the folder where the build folder is, so now you can for example send this folder to a server.
Related
Hi have Gitlab pipeline with multiple stages. I want to build a React project in a Nodejs container and then create another docker container with nginx on it for deployment on AWS.
Here is my gitlab-ci.yml file:
image:
name: node:14-alpine
cache:
key: '$CI_COMMIT_REF_SLUG'
paths:
- node_modules/
- .yarn
stages:
- install
- build
- deploy
job_install:
stage: install
script:
- echo 'yarn-offline-mirror ".yarn-cache/"' >> .yarnrc
- echo 'yarn-offline-mirror-pruning true' >> .yarnrc
- yarn install
cache:
key:
files:
- yarn.lock
paths:
- .yarn-cache/
tags:
- docker
job_build:
stage: build
cache:
key: '$CI_COMMIT_REF_SLUG'
paths:
- node_modules/
- .yarn
script:
- yarn build
artifacts:
paths:
- build
tags:
- docker
job_deploy:
stage: deploy
image: docker
services:
- docker:dind
cache:
key: "$CI_COMMIT_REF_SLUG"
script:
- apk update
- apk upgrade
- apk add bash
- chmod +x ./build-scripts/build_wrapper.sh
- ./build-scripts/build_wrapper.sh
artifacts:
paths:
- BUILDTAG.txt
- env.sh
- logs
when: always
resource_group: realizeui_deployment
tags:
- docker
Basically build_wrapper.sh file will exec docker build . for a Dockerfile in the code repo.This Dockerfile will install nginx and other tools in it.
I want to pass artifacts: /build from stage:build to stage:deploy for using in nginx container.
I have tried finding solution in documentation with no luck. Thanks in advance.
So I found the solution. I just needed to add previous job as dependency in deploy. This makes gitlab download all the artifacts from the dependency job and make them available for deploy job.
Here is how the deploy job looks like:
job_deploy:
stage: deploy
image: docker
services:
- docker:dind
cache:
key: "$CI_COMMIT_REF_SLUG"
dependencies:
- job_build
script:
- apk update
- apk upgrade
- apk add bash
- chmod +x ./build-scripts/build_wrapper.sh
- ./build-scripts/build_wrapper.sh
artifacts:
paths:
- BUILDTAG.txt
- env.sh
- logs
when: always
resource_group: realizeui_deployment
tags:
- docker
So I am trying to make my pipeline work, but I keep getting stuck.
I have a docker runner for my git-ci.yml file
I do this because my deploy stage errors with shell runners (but my build, test and sonarqube stage do work with the shell runner)
**git-ci.yml**
image: docker:latest
stages:
- build
- sonarqube-check
- test
- deploy
cache:
paths:
- .gradle/wrapper
- .gradle/caches
build:
stage: build
image: gradle:jre11-slim
script:
- chmod +x gradlew
- ./gradlew assemble
artifacts:
paths:
- build/libs/*.jar
expire_in: 1 week
only:
- master
sonarqube-check:
stage: test
image: gradle:jre11-slim
variables:
SONAR_USER_HOME: "${CI_PROJECT_DIR}/.sonar" # Defines the location of the analysis task cache
GIT_DEPTH: "0" # Tells git to fetch all the branches of the project, required by the analysis task
cache:
key: "${CI_JOB_NAME}"
paths:
- .sonar/cache
script: ./gradlew sonarqube
allow_failure: true
only:
- master
test:
stage: test
script:
- ./gradlew check
deploy:
stage: deploy
image: gradle:latest
script:
- apt-get update -qy
- apt-get install -y ruby-dev
- gem install dpl
- dpl --provider=heroku --app=heroku-coalition --api-key=$HEROKU_API_KEY
- echo "This job deploys something from the $CI_COMMIT_BRANCH branch."
only:
- master
after_script:
- echo "End CI"
First I got errors about my java home so I switched the image for the build stage. Now I did that and keep getting errors about the permission so I added the chmod +x gradlew
But I get this error when I add that line:
chmod: changing permissions of 'gradlew': Operation not permitted
And when I remove the chmod gradlew line I get:
/bin/bash: line 115: ./gradlew: Permission denied
So now I do not really know what to do.
In short: Which runner should I use to get this yml file to work, or how would I need to edit this yml file accordingly?
So after some research I came across the tags keyword in gitlab. This enables you to run 2 runners for 1 yml file. In this way I could use a shell runner for my build, test and sonarqube fase and a docker runner for my deploy fase!
I'm trying to set up continuous deployment on Circle CI.
I've successfully run my build script, which creates a build folder in the root directory. When I run the command locally to sync with s3, it works fine. But in Circle CI I can't get the path to the build folder.
I've tried ./build, adding working_directory: ~/circleci-docs in the deploy job, and printing the working directory in a test run, which was /home/circleci/project, so I tried manually using /home/circleci/project/build and that didn't work either.
This is my CircleCI config.yml file:
executors:
node-executor:
docker:
- image: circleci/node:10.8
python-executor:
docker:
- image: circleci/python:3.7
jobs:
build:
executor: node-executor
steps:
- checkout
- run:
name: Run build script
command: |
curl -o- -L https://yarnpkg.com/install.sh | bash
yarn install --production=false
yarn build
deploy:
executor: python-executor
steps:
- checkout
- run:
name: Install awscli
command: sudo pip install awscli
- run:
name: Deploy to S3
command: aws s3 sync build s3://{MY_BUCKET}
workflows:
version: 2
build-deploy:
jobs:
- build
- deploy:
requires:
- build
The error message was:
The user-provided path build does not exist.
Exited with code 255
I got it to work!
In the build job I used persist_to_workspace and the deploy job attach_workspace (both are under steps)
- persist_to_workspace:
root: ~/
paths:
- project/build
- attach_workspace:
at: ~/
I'm trying to setup a basic pipeline in Gitlab that does the following:
Run tests command, compile the client and deploy the application using docker-compose.
The problem comes when I'm trying to use npm install.
My .gitlab-ci.yml file looks like:
# This file is a template, and might need editing before it works on your
project.
# Official docker image.
image: docker:latest
services:
- docker:dind
stages:
- test
- build
- deploy
build:
stage: build
script:
- cd packages/public/client/
- npm install --only=production
- npm run build
test:
stage: test
only:
- develop
- production
script:
- echo run tests in this section
step-deploy-production:
stage: deploy
only:
- production
script:
- docker-compose up -d --build
environment: production
when: manual
And the error is:
Skipping Git submodules setup
$ cd packages/public/client/
$ npm install --only=production
bash: line 69: npm: command not found
ERROR: Job failed: exit status 1
I'm using the last docker image, so, I'm wondering whether I can define a new service on my build stage or should I use a different image for the whole process?
Thanks
A new service will not help you, you'll need to use a different image.
You can use a node image just for your build-stage like this:
build:
image: node:8
stage: build
script:
- cd packages/public/client/
- npm install --only=production
- npm run build
I have created a project in Angular cli. I want to do CI using circle ci. The project is uploaded in Bitbucket and is correctly picked by Circle CI. The build fails though. Following is the config.yml (picked CircleCI's sample.yml and changed it (added ng test). I assume that the package.json created by angularcli earlier would install AngularCLI.
version: 2
jobs:
build:
#working_directory: ~/mern-starter
# The primary container is an instance of the first list image listed. Your build commands run in this container.
docker:
- image: circleci/node:7.10.0
# The secondary container is an instance of the second listed image which is run in a common network where ports exposed on the primary container are available on localhost.
#- image: mongo:3.4.4
steps:
- checkout
- run:
name: Update npm
command: 'sudo npm install -g npm#latest'
- restore_cache:
key: dependency-cache-{{ checksum "package.json" }}
- run:
name: Install npm wee
command: npm install
- save_cache:
key: dependency-cache-{{ checksum "package.json" }}
paths:
- node_modules
test:
docker:
- image: circleci/node:7.10.0
#- image: mongo:3.4.4
steps:
- checkout
- run:
name: Test
command: ng test
#- run:
# name: Generate code coverage
# command: './node_modules/.bin/nyc report --reporter=text-lcov'
#- store_artifacts:
# path: test-results.xml
# prefix: tests
#- store_artifacts:
# path: coverage
# prefix: coverage
workflows:
version: 2
build_and_test:
jobs:
- build
- test:
requires:
- build
filters:
branches:
only: dev
Error
#!/bin/bash -eo pipefail
npm install
module.js:472
throw err;
^
Error: Cannot find module 'process-nextick-args'
at Function.Module._resolveFilename (module.js:470:15)
at Function.Module._load (module.js:418:25)
at Module.require (module.js:498:17)
at require (internal/module.js:20:19)
at Object.<anonymous> (/usr/local/lib/node_modules/npm/node_modules/readable-stream/lib/_stream_readable.js:26:23)
at Module._compile (mod
I see the following line after npm install step so I suppose process-nexttick-args is already installed.
process-nextick-args#1.0.7 node_modules/npm/node_modules/npm-registry-client/node_modules/concat-stream/node_modules/readable-stream/node_modules/process-nextick-arg
Following configuration worked for me. I used CircleCI 2.0. I am still refining it and might change the answer in future.
version: 2
jobs:
build:
working_directory: ~/angularcli
# The primary container is an instance of the first list image listed. Your build commands run in this container.
docker:
- image: circleci/node:6-browsers
environment:
CHROME_BIN: "/usr/bin/google-chrome"
steps:
- checkout
- run:
name: Install node_modules with npm
command: npm install
- save_cache:
key: dependency-cache-{{ checksum "package.json" }}
paths:
- ./node_modules
- run:
name: Install angularcli
command: sudo npm install -g #angular/cli#latest
- run:
name: Run unit tests with karma
command: ng test
- store_test_results:
path: test-results.xml
In addition to above script, set singleRun flag to true in karma.conf.js singleRun: true so that Karma exits after running all the test cases. Without this flag, Karma runs in continuous mode, the ng test stop doesn't end and test fails after timeout.