Bad substitution: Bringing key in env over to yml file - environment-variables

I am currently running tests from Cypress in CircleCI for my project. I have my record key for Cypress and I need to store it in an process.env file and bring it over to my config.yml file.
I can't really find anything too specific to my need.
I am getting an error as well:
/bin/bash: ${process.env.CYPRESS_RECORD_KEY}: bad substitution
Here is my code:
.circleci/config.yml/
version: 2
jobs:
build:
docker:
- image: cypress/browsers:chrome67
steps:
- checkout
- run:
name: Install Dependencies
command: npm ci
paths:
- ~/.npm
- ~/.cache
- run: npm install --save-dev cypress
- run: node_modules/.bin/cypress verify
- run: node_modules/.bin/cypress run --browser chrome --record ${process.env.CYPRESS_RECORD_KEY}
.env
CYPRESS_RECORD_KEY = <my_key_here>
Re-tried it by moving things around and still gets same error:
version: 2
jobs:
build:
docker:
- image: cypress/browsers:chrome67
steps:
- checkout
- run:
name: "setup custom environment variables"
command:
echo ${process.env.CYPRESS_RECORD_KEY} >> $BASH_ENV
- run:
name: Install Dependencies
command: npm ci
paths:
- ~/.npm
- ~/.cache
- run: npm install --save-dev cypress
- run: node_modules/.bin/cypress verify
- run: node_modules/.bin/cypress run --browser chrome --record $BASH_ENV
Not sure how to correctly bring it over.

As of 7/9/2019 Cypress still does NOT support headless Chrome (which is what runs in the command line only) being recorded. Even though I have the record key I cannot use it because they still have not been able to have headless Chrome support recording or anything like Electron can when in headless mode.
My only solution is that I run tests I want recorded locally but other than that I am just letting them pass in the command line and hoping Cypress just implements recording soon.

Related

How to run ./gradlew build with docker gitlab runner?

So I am trying to make my pipeline work, but I keep getting stuck.
I have a docker runner for my git-ci.yml file
I do this because my deploy stage errors with shell runners (but my build, test and sonarqube stage do work with the shell runner)
**git-ci.yml**
image: docker:latest
stages:
- build
- sonarqube-check
- test
- deploy
cache:
paths:
- .gradle/wrapper
- .gradle/caches
build:
stage: build
image: gradle:jre11-slim
script:
- chmod +x gradlew
- ./gradlew assemble
artifacts:
paths:
- build/libs/*.jar
expire_in: 1 week
only:
- master
sonarqube-check:
stage: test
image: gradle:jre11-slim
variables:
SONAR_USER_HOME: "${CI_PROJECT_DIR}/.sonar" # Defines the location of the analysis task cache
GIT_DEPTH: "0" # Tells git to fetch all the branches of the project, required by the analysis task
cache:
key: "${CI_JOB_NAME}"
paths:
- .sonar/cache
script: ./gradlew sonarqube
allow_failure: true
only:
- master
test:
stage: test
script:
- ./gradlew check
deploy:
stage: deploy
image: gradle:latest
script:
- apt-get update -qy
- apt-get install -y ruby-dev
- gem install dpl
- dpl --provider=heroku --app=heroku-coalition --api-key=$HEROKU_API_KEY
- echo "This job deploys something from the $CI_COMMIT_BRANCH branch."
only:
- master
after_script:
- echo "End CI"
First I got errors about my java home so I switched the image for the build stage. Now I did that and keep getting errors about the permission so I added the chmod +x gradlew
But I get this error when I add that line:
chmod: changing permissions of 'gradlew': Operation not permitted
And when I remove the chmod gradlew line I get:
/bin/bash: line 115: ./gradlew: Permission denied
So now I do not really know what to do.
In short: Which runner should I use to get this yml file to work, or how would I need to edit this yml file accordingly?
So after some research I came across the tags keyword in gitlab. This enables you to run 2 runners for 1 yml file. In this way I could use a shell runner for my build, test and sonarqube fase and a docker runner for my deploy fase!

When trying to deploy to S3 on Circle CI, error is `The user-provided path build does not exist.`

I'm trying to set up continuous deployment on Circle CI.
I've successfully run my build script, which creates a build folder in the root directory. When I run the command locally to sync with s3, it works fine. But in Circle CI I can't get the path to the build folder.
I've tried ./build, adding working_directory: ~/circleci-docs in the deploy job, and printing the working directory in a test run, which was /home/circleci/project, so I tried manually using /home/circleci/project/build and that didn't work either.
This is my CircleCI config.yml file:
executors:
node-executor:
docker:
- image: circleci/node:10.8
python-executor:
docker:
- image: circleci/python:3.7
jobs:
build:
executor: node-executor
steps:
- checkout
- run:
name: Run build script
command: |
curl -o- -L https://yarnpkg.com/install.sh | bash
yarn install --production=false
yarn build
deploy:
executor: python-executor
steps:
- checkout
- run:
name: Install awscli
command: sudo pip install awscli
- run:
name: Deploy to S3
command: aws s3 sync build s3://{MY_BUCKET}
workflows:
version: 2
build-deploy:
jobs:
- build
- deploy:
requires:
- build
The error message was:
The user-provided path build does not exist.
Exited with code 255
I got it to work!
In the build job I used persist_to_workspace and the deploy job attach_workspace (both are under steps)
- persist_to_workspace:
root: ~/
paths:
- project/build
- attach_workspace:
at: ~/

Gitlab CI with Docker and NPM

I'm trying to setup a basic pipeline in Gitlab that does the following:
Run tests command, compile the client and deploy the application using docker-compose.
The problem comes when I'm trying to use npm install.
My .gitlab-ci.yml file looks like:
# This file is a template, and might need editing before it works on your
project.
# Official docker image.
image: docker:latest
services:
- docker:dind
stages:
- test
- build
- deploy
build:
stage: build
script:
- cd packages/public/client/
- npm install --only=production
- npm run build
test:
stage: test
only:
- develop
- production
script:
- echo run tests in this section
step-deploy-production:
stage: deploy
only:
- production
script:
- docker-compose up -d --build
environment: production
when: manual
And the error is:
Skipping Git submodules setup
$ cd packages/public/client/
$ npm install --only=production
bash: line 69: npm: command not found
ERROR: Job failed: exit status 1
I'm using the last docker image, so, I'm wondering whether I can define a new service on my build stage or should I use a different image for the whole process?
Thanks
A new service will not help you, you'll need to use a different image.
You can use a node image just for your build-stage like this:
build:
image: node:8
stage: build
script:
- cd packages/public/client/
- npm install --only=production
- npm run build

Share result of a step between different jobs in CircleCi

I have this generic config.yml in CircleCi.
version: 2
jobs:
build:
docker:
- image: circleci/node:7.10
steps:
- checkout
- run: npm install
- run: npm run lint
deploy:
machine: true
steps:
- checkout
- run: npm install
- run: npm run build
As you can see, npm install is called twice, which is a duplication of tasks.
Is it possible to share the results of npm install between the 2 jobs?
The end goal is to install the package only one time.
What you're looking for is Workspaces: https://circleci.com/docs/2.0/workflows/#using-workspaces-to-share-data-among-jobs

npm install fails in circle ci (angular cli project)

I have created a project in Angular cli. I want to do CI using circle ci. The project is uploaded in Bitbucket and is correctly picked by Circle CI. The build fails though. Following is the config.yml (picked CircleCI's sample.yml and changed it (added ng test). I assume that the package.json created by angularcli earlier would install AngularCLI.
version: 2
jobs:
build:
#working_directory: ~/mern-starter
# The primary container is an instance of the first list image listed. Your build commands run in this container.
docker:
- image: circleci/node:7.10.0
# The secondary container is an instance of the second listed image which is run in a common network where ports exposed on the primary container are available on localhost.
#- image: mongo:3.4.4
steps:
- checkout
- run:
name: Update npm
command: 'sudo npm install -g npm#latest'
- restore_cache:
key: dependency-cache-{{ checksum "package.json" }}
- run:
name: Install npm wee
command: npm install
- save_cache:
key: dependency-cache-{{ checksum "package.json" }}
paths:
- node_modules
test:
docker:
- image: circleci/node:7.10.0
#- image: mongo:3.4.4
steps:
- checkout
- run:
name: Test
command: ng test
#- run:
# name: Generate code coverage
# command: './node_modules/.bin/nyc report --reporter=text-lcov'
#- store_artifacts:
# path: test-results.xml
# prefix: tests
#- store_artifacts:
# path: coverage
# prefix: coverage
workflows:
version: 2
build_and_test:
jobs:
- build
- test:
requires:
- build
filters:
branches:
only: dev
Error
#!/bin/bash -eo pipefail
npm install
module.js:472
throw err;
^
Error: Cannot find module 'process-nextick-args'
at Function.Module._resolveFilename (module.js:470:15)
at Function.Module._load (module.js:418:25)
at Module.require (module.js:498:17)
at require (internal/module.js:20:19)
at Object.<anonymous> (/usr/local/lib/node_modules/npm/node_modules/readable-stream/lib/_stream_readable.js:26:23)
at Module._compile (mod
I see the following line after npm install step so I suppose process-nexttick-args is already installed.
process-nextick-args#1.0.7 node_modules/npm/node_modules/npm-registry-client/node_modules/concat-stream/node_modules/readable-stream/node_modules/process-nextick-arg
Following configuration worked for me. I used CircleCI 2.0. I am still refining it and might change the answer in future.
version: 2
jobs:
build:
working_directory: ~/angularcli
# The primary container is an instance of the first list image listed. Your build commands run in this container.
docker:
- image: circleci/node:6-browsers
environment:
CHROME_BIN: "/usr/bin/google-chrome"
steps:
- checkout
- run:
name: Install node_modules with npm
command: npm install
- save_cache:
key: dependency-cache-{{ checksum "package.json" }}
paths:
- ./node_modules
- run:
name: Install angularcli
command: sudo npm install -g #angular/cli#latest
- run:
name: Run unit tests with karma
command: ng test
- store_test_results:
path: test-results.xml
In addition to above script, set singleRun flag to true in karma.conf.js singleRun: true so that Karma exits after running all the test cases. Without this flag, Karma runs in continuous mode, the ng test stop doesn't end and test fails after timeout.

Resources