Hello i have simple configuration in my project:
version: 2
jobs:
build:
docker:
- image: circleci/node:7
steps:
- checkout
- run:
name: install-dependencies
command: npm install
- run:
name: tests
command: npm test
- deploy:
name: digital-ocean
command: ssh -o "StrictHostKeyChecking no" user#hostname "cd ~/profile-store; git pull; npm install; forever start app.js"
The problem is it need multiply command:
cd client
npm start
cd ..
(in second iteration should install packages from server and in the next run unit tests in client)
I tried these syntax:
command: ["cd client", "npm install", "cd .."]
But getting an error. The question is :
How can i write to execute 3 commands in one command instruction?
command: cd client && npm install && cd ..
For enhanced readability, you can use a folded block scalar (folds linebreaks into spaces):
command: >-
cd client &&
npm install &&
cd ..
Note that you do not really need the final cd .. since the shell instance executing the command is not re-used.
Related
I am downloading and unzipping binaryen in a run step.
- run: wget -c https://github.com/WebAssembly/binaryen/releases/download/version_101/binaryen-version_101-x86_64-linux.tar.gz -O - | tar -xz -C /tmp/
I am then updating the path in $BASH_ENV.
- run: echo "export PATH=/tmp/binaryen-version_101/bin/wasm-opt:\${PATH}" >> $BASH_ENV
However, I still get a command not found for wasm-opt.
How can I install the downloaded wasm-opt binary such that another run step can use it?
The main issue is that the PATH variable should contain a list of directories. You added the actual binary itself to the path instead of the directory it resides in.
So for example, instead of /tmp/binaryen-version_101/bin/wasm-opt you want /tmp/binaryen-version_101/bin/. Also, after you add a directory to the PATH you won't be able to run those binaries until the next step.
Here's an example config I made:
version: 2.1
workflows:
main:
jobs:
- build
jobs:
build:
docker:
- image: cimg/base:stable
steps:
- checkout
- run: curl -sSL "https://github.com/WebAssembly/binaryen/releases/download/version_101/binaryen-version_101-x86_64-linux.tar.gz" | tar -xz -C /tmp/
- run: echo 'export PATH=/tmp/binaryen-version_101/bin/:${PATH}' >> $BASH_ENV
- run: wasm-opt
I'm developing a Node.js program that connects to both Redis and memcached. I am testing my Node.js program with Jest, and before running the test I run docker-compose up. My Node.js program connects to the Docker Redis and memcached Docker containers fine, and my tests pass fine on my local machine.
However, I want the tests to run on Circle CI so that every time I git push, the CI environment will verify the program is buildable and that tests are passing.
When I try to do the same on Circle CI, it seems that the Docker containers spin up fine, however the tests aren't able to connect to the Redis or memcached servers in the containers, despite it working fine on my local PC.
My config.yml for Circle CI:
version: 2
jobs:
build:
docker:
- image: circleci/node
steps:
- checkout
- setup_remote_docker
- run:
name: Install Docker Compose
command: |
curl -L https://github.com/docker/compose/releases/download/1.28.2/docker-compose-`uname -s`-`uname -m` > ~/docker-compose
chmod +x ~/docker-compose
sudo mv ~/docker-compose /usr/local/bin/docker-compose
- run:
name: Start Container
command: |
docker-compose up -d
docker-compose ps
- restore_cache:
key: npm-cache-v1-{{ checksum "package-lock.json" }}
- run:
name: Install Dependencies
command: npm ci
- save_cache:
key: npm-cache-v1-{{ checksum "package-lock.json" }}
paths:
- /home/circleci/.npm
- run:
name: Ensure Test Parity
command: |
chmod +x ./validateTestCases.sh
./validateTestCases.sh
- run:
name: Run Tests
command: npm test
My docker-compose.yml:
services:
redis:
image: redis
container_name: redis-container
ports:
- 6379:6379
memcached:
image: memcached
container_name: memcached-container
ports:
- 11211:11211
My build failing test log in Circle CI:
#!/bin/bash -eo pipefail
npm test
> easy-cache#1.0.0 test
> jest
FAIL memcached/memcached.test.js
● Test suite failed to run
Error: connect ECONNREFUSED 127.0.0.1:11211
FAIL redis/redis.test.js
● Test suite failed to run
Timeout - Async callback was not invoked within the 5000 ms timeout specified by jest.setTimeout.Error: Timeout - Async callback was not invoked within the 5000 ms timeout specified by jest.setTimeout.
at mapper (node_modules/jest-jasmine2/build/queueRunner.js:27:45)
Test Suites: 2 failed, 2 total
Tests: 0 total
Snapshots: 0 total
Time: 36.183 s
Ran all test suites.
npm ERR! code 1
npm ERR! path /home/circleci/project
npm ERR! command failed
npm ERR! command sh -c jest
npm ERR! A complete log of this run can be found in:
npm ERR! /home/circleci/.npm/_logs/2021-02-05T20_29_26_896Z-debug.log
Exited with code exit status 1
CircleCI received exit code 1
Link to my current source code
I am not sure what to try next. I have tried moving the npm test block right after docker-compose up -d but that had no effect.
It turns out that Docker Compose is not required for what I'm trying to do. Instead, you can include multiple Docker images in Circle CI.
Here's my updated Circle CI yaml file, where my tests run successfully (connection to Redis and memcached works like on my local PC using Docker Compose):
version: 2
jobs:
build:
docker:
- image: circleci/node
- image: redis
- image: memcached
steps:
- checkout
# - setup_remote_docker
# - run:
# name: Install Docker Compose
# command: |
# curl -L https://github.com/docker/compose/releases/download/1.28.2/docker-compose-`uname -s`-`uname -m` > ~/docker-compose
# chmod +x ~/docker-compose
# sudo mv ~/docker-compose /usr/local/bin/docker-compose
# - run:
# name: Start Container
# command: |
# docker-compose up -d
# docker-compose ps
- restore_cache:
key: npm-cache-v1-{{ checksum "package-lock.json" }}
- run:
name: Install Dependencies
command: npm ci
- save_cache:
key: npm-cache-v1-{{ checksum "package-lock.json" }}
paths:
- /home/circleci/.npm
- run:
name: Ensure Test Parity
command: |
chmod +x ./validateTestCases.sh
./validateTestCases.sh
- run:
name: Run Tests
command: npm test
I am running this .yml file
kind: pipeline
type: docker
name: default
steps:
- name: test
image: ubuntu
commands:
- apt-get update
- apt-get install git -y
- ./build.sh bin
first, two commands are running correctly but in the last command it fails and the drone doesn't give me a message
![drone messages get while testing][1]
This fails after echo if1
PROJECT_PATH=$PWD
RUN=$1
ENVIRONMENT=${ENVIRONMENT:="local"}
if [ "$ENVIRONMENT" == "local" ]; then
echo if1
GIT_AUTHOR="$(git config --global --get user.name)"
GIT_AUTHOR_EMAIL="$(git config --global --get user.email)"
GIT_VERSION="$(git describe --tags --always --dirty)"
VERSION=$GIT_VERSION
echo endif
else
echo else1
DRONE_COMMIT_SHORT=${DRONE_COMMIT:0:8}
VERSION=${DRONE_TAG:=$DRONE_COMMIT_SHORT}
echo endelse
``
[1]: https://i.stack.imgur.com/ChrUo.png
You need shebang
#!/usr/bin/env bash
Your bash
I am building my project on CircleCI and I have a build job that looks like this:
build:
<<: *defaults
steps:
- checkout
- setup_remote_docker
- run:
name: Install pip
command: curl https://bootstrap.pypa.io/get-pip.py -o get-pip.py && sudo python get-pip.py
- run:
name: Install AWS CLI
command: curl "https://s3.amazonaws.com/aws-cli/awscli-bundle.zip" -o "awscli-bundle.zip" && unzip awscli-bundle.zip && sudo ./awscli-bundle/install -i /usr/local/aws -b /usr/local/bin/aws
- run:
name: Login to Docker Registry
command: aws ecr get-login --no-include-email --region us-east-1 | sh
- run:
name: Install Dep
command: curl https://raw.githubusercontent.com/golang/dep/master/install.sh | sh
- run:
name: Save Version Number
command: echo "export VERSION_NUM=${CIRCLE_TAG}.${CIRCLE_BUILD_NUM}" > deployment/dev/.env
- run:
name: Build App
command: source deployment/dev/.env && docker-compose -f deployment/dev/docker-compose.yml build
- run:
name: Test App
command: |
git config --global url."https://${GITHUB_PERSONAL_ACCESS_TOKEN} :x-oauth-basic#github.com/".insteadOf "https://github.com/"
dep ensure
go test -v ./...
- run:
name: Push Image
command: |
if [[ "${CIRCLE_TAG}" =~ ^v[0.9]+(\.[0-9]+)*-[a-z]*$ ]]; then
source deployment/dev/.env
docker-compose -f deployment/dev/docker-compose.yml push
else
echo 'No tag, not deploying'
fi
- persist_to_workspace:
root: .
paths:
- deployment/*
- tools/*
When I push a change to a branch, the build fails every time with Couldn't connect to Docker daemon at ... - is it running? when it reaches the Build App step of the build job.
Please help me figure out why branch builds are failing but tag builds are not.
I suspect you are hitting this docker-compose bug: https://github.com/docker/compose/issues/6050
The bug reports a misleading error (the one you're getting) when an image name in the docker-compose file is invalid.
If you use an environment variable for the image name or image tag, and that variable is set from a branch name, then it would fail on some branches, but not others.
The problem was occurring on the Save Version Number step. Sometimes that version would be .${CIRCLE_BUILD_NUM} since no tag was passed. Docker dislikes these tags starting with ., so I added a conditional check to see if CIRCLE_TAG was empty, and if it was, use some default version: v0.1.0-build.
I'm trying to setup codecov as code coverage tool in my repository. I referred to this link to pass reports through docker container -
Link - https://github.com/codecov/support/wiki/Testing-with-Docker
But travis ci fails and gives this error -
docker: Error parsing reference: "..." is not a valid repository/tag.
Here is my travis.yml
sudo: required
dist: trusty
language: node_js
node_js:
- 6
before_install:
- export CHROME_BIN=chromium-browser
- export DISPLAY=:99.0
- sh -e /etc/init.d/xvfb start
- docker run -v "$PWD/shared:/shared" ...
before_script:
- ng build
script:
- ng test --watch=false
- ng lint
- >
docker run -ti -v $(pwd):/app --workdir=/app coala/base coala --version
after_success:
- bash ./deploy.sh
- bash <(curl -s https://codecov.io/bash)
- mv -r coverage/ shared
cache:
bundler: true
directories:
- node_modules
- .coala-cache
services: docker
branches:
only:
- angular
How should I solve this? Thanks!
I assume you refer to Codecov Outside Docker. The current error message already tells you that the three dots ... need to be replaced with a real Docker repository name, e.g. node:6-alpine.
What you're still missing is the part of running the tests (including reports) inside the Docker container, so that you can mv the test reports to the shared folder. You could achieve that by adding a custom Dockerfile based on node, similar to the one below. I chose a more or less full base image including Chrome and other tools to make your use case work:
FROM markadams/chromium-xvfb-js:7
WORKDIR /proj
CMD npm install && \
node_modules/.bin/ng build && \
node_modules/.bin/ng test --watch=false && \
node_modules/.bin/ng lint && \
mkdir -p shared && \
mv coverage.txt shared
That custom image needs to be built and then run like this (assuming the Dockerfile to be in your project root directory):
docker build -t ci-build .
docker run --rm -v "$(pwd):/proj" ci-build
I suggest to change the .travis.yml like follows:
sudo: required
dist: trusty
language: node_js
node_js:
- 6
before_install:
- docker build -t ci-build .
script:
- >
docker run --rm -v $(pwd):/proj ci-build
- >
docker run -ti -v $(pwd):/app --workdir=/app coala/base coala --version
after_success:
- bash ./deploy.sh
- bash <(curl -s https://codecov.io/bash)
cache:
bundler: true
directories:
- node_modules
- .coala-cache
services: docker
branches:
only:
- angular
Another note: the coala/base image works similarly.