In my .travis.yml I have this.
script:
- yarn lint
- yarn flow
- yarn test --runInBand
I was wondering is there a way to get them to run in parallel?
There's few suggestions in Travis docs you could use, i.e. split your build into multiple jobs: https://docs.travis-ci.com/user/speeding-up-the-build/
Another thing you could do is to employ GNU parallel:
addons:
apt_packages:
- parallel
script:
- parallel --gnu --keep-order ::: 'yarn lint' 'yarn flow' 'yarn test --runInBand'
The GNU parallel command has lots of options you might want to tweak to your needs. Read more about the tool on their website https://www.gnu.org/software/parallel/
To split Travis into several jobs you can either use stages or add the option env
This would run each script sequentially:
script:
- yarn lint
- yarn flow
- yarn test --runInBand
- yarn build
- yarn cypress
To get them to run in parallel jobs. You can update it to the code below (Though keep in mind this is limited by the number of concurrent jobs available. https://travis-ci.com/plans)
Using Build Stages
language: node_js
node_js:
- '9'
install:
- travis_retry yarn install
jobs:
include:
- stage: test
name: "Flow/Lint/Test"
script:
- yarn lint
- yarn flow
- yarn test
-
name: "Cypress"
script:
- yarn cypress
Using env
env:
- TEST_SUITE="yarn lint"
- TEST_SUITE="yarn flow"
- TEST_SUITE="yarn test --runInBand"
- TEST_SUITE="yarn build"
- TEST_SUITE="yarn cypress"
script: $TEST_SUITE
Another option would be to just have two concurrent builds.
env:
- TEST_SUITE="yarn lint && yarn flow && yarn test --runInBand && yarn build"
- TEST_SUITE="yarn cypress"
script: $TEST_SUITE
This might or might not improve the overall build times. For me majority of my build time was in cypress while lint + flow + test took a few minutes. So by separating cypress to be in its own job, I sped up my overall build time by a few minutes.
Related
I want to avoid perform install process times under multiple stages. My travis-ci config as follow, how to avoid run install process both in test and deploy stage? Beacause install is time-cost.
os: linux
language: android
licenses:
- android-sdk-preview-license-.+
- android-sdk-license-.+
- google-gdk-license-.+
android:
components:
- tools
- platform-tools
- build-tools-28.0.3
- android-28
jdk: oraclejdk8
sudo: false
install:
- git clone https://github.com/flutter/flutter.git -b stable --depth 1
- export PATH=$PATH:./flutter/bin
- flutter doctor
cache:
directories:
- "$HOME/.pub-cache"
stages:
- test
- name: depoly
if: tag =~ /^release-v\d+\.\d+\.\d+/ # tag match: release-v1.0.0
jobs:
include:
- stage: test
- script: flutter analyze lib/ test/
- script: flutter test test/
- stage: depoly
- script: flutter build apk
deploy:
provider: releases
api_key:
secure: $github_deploy_api_key
file: $APK_NAME
skip_cleanup: true
overwrite: true
on:
repo: stefanJi/Flutter4GitLab
branch: master
I have the following GitLab yml file and I have written it in stages assuming that every stage would retain the dependencies, but seems it is not to be the case!
# This file is a template, and might need editing before it works on your project.
# Official Java image. Look for the different tagged releases at
# https://hub.docker.com/r/library/java/tags/ . A Java image is not required
# but an image with a JVM speeds up the build a bit.
image: java:8
variables:
FILE_TARGET_PATH: $FILE_TARGET_PATH
stages:
- test
- run
cache:
key: "$CI_COMMIT_REF_NAME" # contains either the branch or the tag, so it's caching per branch
untracked: true
paths:
- "sbt-cache/.ivy.cache"
- "sbt-cache/.boot"
- "sbt-cache/.sbtboot"
- "sbt-cache/target"
before_script:
# Enable the usage of sources over https
- apt-get update -yqq
- apt-get install apt-transport-https -yqq
# Add keyserver for SBT
- echo "deb http://dl.bintray.com/sbt/debian /" | tee -a /etc/apt/sources.list.d/sbt.list
- apt-key adv --keyserver hkp://keyserver.ubuntu.com:80 --recv 2EE0EA64E40A89B84B2DF73499E82A75642AC823
# Install SBT
- apt-get update -yqq
- apt-get install sbt -yqq
Run unit Tests:
stage: test
tags:
- master
script:
# Execute your project's tests
- sbt -Denv=test clean test
Run Pipeline:
stage: run
tags:
- master
script:
# Execute the pipeline
- sbt -Denv=test run
How can I cache the dependencies in this multi stage set up? I have a local runner on my machine that runs the pipeline. Would artifacts help me?
If you are using a newer sbt version than 1.0.4 the caching won't work for you as the compiler will always invalidate all the files.
This compiler issue has already been reported here: https://github.com/sbt/sbt/issues/4168
My suggestion would be to downgrade sbt version to 1.0.4 for CI. If that doesn't help you can also check the reason why the cache is invalidated by adding to your build.sbt:
// Debug incremental zinc compiler
logLevel := Level.Debug
incOptions := incOptions.value.withApiDebug(true).withRelationsDebug(true)
I had the same issue for Bitbucket Pipelines CI and managed to successfully make it work here
I am trying to use the following pipeline in bitbucket to deploy to gcloud. The rest of the pipeline is missing but I know that works. The issue is when I get to step two the compiled CSS files are no longer existing. It appears to match what bitbucket suggest for using artifacts. I tried the following;
artifacts:
- **
-public/assets/css/generated/style.css
-public/assets/css/generated/*.css
-public/assets/css/generated/*
-public/*
-public/**
.
pipelines:
branches:
pipeline:
- step:
image: node:4.6.0
caches:
- node
script:
- cd web-application
- npm install
- npm install -g gulp
- gulp
- ls public/assets/css/generated
# - mv public/assets/css/generated/* ~/./
- cd ..
artifacts:
- public/assets/css/generated/style.css
- public/**
- step:
image: google/cloud-sdk:latest
script:
- cd web-application
- ls public/assets/css/generated
Artifacts showing in step 1
Trying to get travis to skip release stage for PR/non master branch builds but I can't seem to get the recipe right.
The travis config is listed here:
language: node_js
cache:
directories:
- node_modules
node_js:
- '8'
- '6'
before_install:
- npm install -g npm#5
- npm install -g greenkeeper-lockfile#1
install:
- yarn install --ignore-engines
before_script: greenkeeper-lockfile-update
after_script: greenkeeper-lockfile-upload
script:
- yarn coveralls
branches:
except:
- /^v\d+\.\d+\.\d+$/
jobs:
include:
- if: branch = master
- stage: release
node_js: lts/*
script: echo "Deploying to npm ..."
deploy:
provider: script
skip_cleanup: true
script:
- npx semantic-release
I also tried
jobs:
include:
- stage: release
if: branch = master
node_js: lts/*
script: echo "Deploying to npm ..."
deploy:
provider: script
skip_cleanup: true
script:
- npx semantic-release
and
jobs:
include:
-
if: branch = master
- stage: release
node_js: lts/*
script: echo "Deploying to npm ..."
deploy:
provider: script
skip_cleanup: true
script:
- npx semantic-release
but travis always executes the release stage - it doesn't run npx semantic-release but it still goes through initializing, running tests etc.
I am running on travis on 5 versions of nodeJS, .travis.yml is ....
language: node_js
node_js:
- 5.0
- 4.0
- 0.12.7
- 0.10.40
- 0.10.36
before_install:
- npm install -g grunt-cli
script:
- npm run travis
I want to set a travis environment variable for the run on nodeJS 5.0 only
something like this ...
language: node_js
node_js:
- 5.0
- env: POST_TO_COVERALLS=true
- 4.0
- 0.12.7
- 0.10.40
- 0.10.36
before_install:
- npm install -g grunt-cli
script:
- npm run travis
but this is invalid ... anyone know how to do this ...
1 - Preferably, via .travis.yml
2 - If not, through the travis web application
I KNOW HOW TO DO THIS VIA CODE - But can it be done through travis?
Thanks all
Newly introduced Travis build stages combined with some YML inheritance and and some hacking around assigning the Bash variable can be very handy for this:
jobs:
include:
- stage: Tests
script: # run tests
- script: # run more tests
- &deploy-stuff
stage: Deploy
if: branch != master
env:
- ENV=$(if [ "$SOMETHING" = "thing" ]; then echo ${TRAVIS_BRANCH//\//-}; else echo "staging"; fi)
script: # do some things before deploy
deploy:
- provider: script
skip_cleanup: true
script: echo $ENV
on:
all_branches: true
condition: $TRAVIS_BRANCH = "devel" || $SOMETHING = "thing"
- <<: *deploy-stuff
if: branch = master
env:
- ENV="production"
deploy:
- provider: script
skip_cleanup: true
script: echo $ENV
on:
branch: master
What about using the matrix to explicitly include the one "5.0" build where your environment variable is set to true (see documentation on explicitly including builds).
It would be something like the following
language: node_js
node_js:
- 4.0
- 0.12.7
- 0.10.40
- 0.10.36
env:
POST_TO_COVERALLS=false
matrix:
include:
- node_js: 5.0
env: POST_TO_COVERALLS=true
before_install:
- npm install -g grunt-cli
script:
- npm run travis