I have a project with two sub folders. Here's the structure:
src
|- api
|- ui
|-node_modules
bitbucket-pipelines.yml
Since my node modules folder is not in the same directory as the pipeline file, I tried creating a definition and included the path. Here's where I'm trying to cache it in my pipeline:
pipelines:
default:
- step:
name: "Frontend"
image: node:14.17.1
caches:
- frontendnode
script:
- cd src/ui
- npm install
- CI=false npm run build
definitions:
caches:
frontendnode: src/ui/node_modules
My guess is that my definition is wrong but I have tried multiple things and I'm getting the same error:
Assembling contents of new cache 'frontendnode'
Cache "frontendnode": Skipping upload for empty cache
Thanks!
Try to print all files of machine, call:
apt-get install -y tree
tree
Also cache will fill after success complete of pipeline
Or your src/ui/node_modules folder is empty
Related
From reading the documentation here I was able to get a working code-quality CI job added to my pipeline. The YAML looks like this
include:
- template: Code-Quality.gitlab-ci.yml
code_quality:
stage: test
variables:
SOURCE_CODE: "/src"
artifacts:
paths: [gl-code-quality-report.json]
However, even though I specify SOURCE_CODE: /src the job scans the entire project, not just the "src/" directory
How do I go about setting up .gitlab-ci.yml such that the Code-Quality CI job will only scan the "/src" directory? (The /src directory is located in the root of the project)
You need a pattern to match all the files inside the src directory.
SOURCE_CODE: src/** will match all the files in the src directory tree.
Example Matches:
src/foo/bar/xyz.java
src/abc.java
so, i need to integrate kcov in my gitlab-ci to see code coverage on a test executable executable. the documentation from kcov states that i need to run "kcov /path/to/outdir ./myexec" to generate a report in an html file. however, even if the command succedes, /path/to/outdir is still empty and i dont know why since the tests pass and kcov returns no errors
here is the .gitlab-ci.yml:
stage: coverage
dependencies:
- Build
script:
- mkdir build/test/kcov
- cd build/test
- kcov --include-path=../../src /kcov ./abuse-test
- cd kcov
- ls
artifacts:
paths:
- TP3/build
- TP3/src
my test exec is abuse-test, it is generated via cmake->make and is in a folder called TP3->build->test->abuse-test
the output of the console in the ci is the following:
on igl601-runner3 5d2b3c01
Using Docker executor with image depot.dinf.usherbrooke.ca:4567/e19-igl601/eq09/image_tp3 ...
Pulling docker image depot.dinf.usherbrooke.ca:4567/e19-igl601/eq09/image_tp3 ...
Using docker image sha256:c2cf0a7c10687670c7b28ee23ac06899de88ebb0d86e142bfbf65171147fc167 for depot.dinf.usherbrooke.ca:4567/e19-igl601/eq09/image_tp3 ...
Running on runner-5d2b3c01-project-223-concurrent-0 via dinf-prj-16...
Fetching changes...
Removing TP3/build/
HEAD is now at b2e1277 Update .gitlab-ci.yml
From https://depot.dinf.usherbrooke.ca/e19-igl601/eq09
b2e1277..7cf0af5 master -> origin/master
Checking out 7cf0af56 as master...
Skipping Git submodules setup
Downloading artifacts for Build (8552)...
Downloading artifacts from coordinator... ok id=8552 responseStatus=200 OK token=Pagxjp_C
$ cd TP3
$ mkdir build/test/kcov
$ cd build/test
$ kcov --include-path=../../src /kcov ./abuse-test
===============================================================================
All tests passed (3 assertions in 3 test cases)
$ cd kcov
$ ls
Uploading artifacts...
TP3/build: found 2839 matching files
TP3/src: found 211 matching files
Uploading artifacts to coordinator... ok id=8554 responseStatus=201 Created token=PxDHHjxf
Job succeeded
the kcov documentation states: "/path/to/outdir will contain lcov-style HTML output generated continuously while the application run"
and yet, when i browse the artefacts, i find nothing
Hi it looks like you're specifying /kcov as the outdir:
kcov --include-path=../../src /kcov ./abuse-test
Since you're working on a *nix based system, the / implies an absolute path from the root of your filesystem.
The cd kcov step assumes a relative path (down from your current directory) since it is missing the /.
So I guess changing your kcov command to:
kcov --include-path=../../src kcov ./abuse-test
Would fix your issue.
I'm using a bitbucket docker pipeline to validate my builds for an android app on push. One of my dependancies is a private package which I am hosting on another bitbucket repository. For builds on the user machine, I use Gradle's private maven repository plugin which can resolve my dependency with encrypted username and password.
This works well for developer machines, but I want to avoid hard coding usernames and passwords in the pipeline. Instead, since bitbucket support sshkeys across repositories for authentication, I have built in my pipeline script to clone the repository with the my private packages, and copy them over to the gradle cache. I have tried both:
/home/gradle/.gradle/caches/modules-2/files-2.1/com.mycompany
~/.gradle/caches/modules-2/files-2.1/com.mycompany
as caches. The clone and copy work just fine as I can see the files in their respective directories in the cache with by adding an ls in the pipeline, but gradlew still tries to go to the internet (the other bitbucket repository) to resolve the dependancies, as if there is no cache. Further, i'm using gradle docker image gradle:3.4.1 (the version of gradle in my project level build.gradle file), but gradle build fails with a google() is not a function.
Gradlew build fails trying to resolve my package.pom file, with a lack of a username (because there is no gradle.properties in the pipeline). But why doesn't it use the cache, instead of trying to go to the repository?
I have tried the standard java:8 docker image, the gradle docker image up to 5.1.1, and i have tried copying the package files into various gradle caches in the docker image. I have also tried altering permissions with chmod 775 to no avail. I have also tried gradlew assembleDebug with the same results as gradlew build. I'm a bit new to gradle, docker and bitbucket so i'm not sure what is causing the issue.
image: gradle:3.4.1
pipelines:
default:
- step:
caches:
- gradle
- android-sdk
script:
# Download and unzip android sdk
- wget --quiet --output-document=android-sdk.zip https://dl.google.com/android/repository/sdk-tools-linux-3859397.zip
- unzip -o -qq android-sdk.zip -d android-sdk
# Define Android Home and add PATHs
- export ANDROID_HOME="/opt/atlassian/pipelines/agent/build/android-sdk"
- export PATH="$ANDROID_HOME/tools:$ANDROID_HOME/tools/bin:$ANDROID_HOME/platform-tools:$PATH"
# Download packages.
- yes | sdkmanager "platform-tools"
- yes | sdkmanager "platforms;android-27"
- yes | sdkmanager "build-tools;27.0.3"
- yes | sdkmanager "extras;android;m2repository"
- yes | sdkmanager "extras;google;m2repository"
- yes | sdkmanager "extras;google;instantapps"
- yes | sdkmanager --licenses
# Build apk
- git clone git#bitbucket.org:myorg/myrepo.git
- scp -r myrepo/com/mycomp/* /home/gradle/.gradle/caches/modules-2/files-2.1/com.mycomp
#- gradle build
- ./gradlew build
#- ./gradlew assembleDebug
definitions:
caches:
android-sdk: android-sdk
Gradlew build error:
> Could not resolve all files for configuration ':app:debugCompileClasspath'.
> Could not resolve com.mycomp:mypackage:1.0.1.
Required by:
project :app
> Could not resolve com.mycomp:mypackage:1.0.1.
> Could not get resource 'https://bitbucket.org/myorg/myrepo/raw/releases/com/mycomp/mypackage/1.0.11/package-1.0.1.pom'.
> Username may not be null
I have a single BitBucket repository containing the code for an Angular app in a folder called ui and a Node API in a folder called api.
My BitBucket pipeline runs ng test for the Angular app, but the node_modules folder isn't being cached correctly.
This is my BitBucket Pipeline yml file:
image: trion/ng-cli-karma
pipelines:
default:
- step:
caches:
- angular-node
script:
- cd ui
- npm install
- ng test --watch=false
definitions:
caches:
angular-node: /ui/node_modules
When the builds runs it shows:
Cache "angular-node": Downloading
Cache "angular-node": Extracting
Cache "angular-node": Extracted
But when it performs the npm install step it says:
added 1623 packages in 41.944s
I am trying to speed the build up and I can't work out why npm needs to install the dependencies assuming they are already contained in the cache which has been restored.
my guess is, your cache position is not correct. there is a pre-configured node cache (named "node") that can just be activated. no need to do a custom cache for that. (the default cache fails, because your node build is in a sub folder of the clone directory, so you need a custom cache)
cache positons are relative to the clone directory. bitbucket clones into /opt/atlassian/pipelines/agent/build thats probably why your absolute cache-path did not work.
simply making the cache reference relative should do the trick
pipelines:
default:
- step:
caches:
- angular-node
script:
- cd ui
- npm install
- ng test --watch=false
definitions:
caches:
angular-node: ui/node_modules
that may fix your issue
I am running a CI pipeline to build firmware for ESP8266 using plaitformio and bitbucket pipelines, my code builds successfully and now I want to cache the directory that contains the platformio libraries (.piolibdeps). Here are the contains of my platform.ini file.
[env:nodemcuv2]
platform = espressif8266
board = nodemcuv2
framework = arduino
upload_port = 192.168.1.108
lib_deps =
ESPAsyncTCP#1.1.0
OneWire
Time
FauxmoESP
Blynk
DallasTemperature
ArduinoJson
Adafruit NeoPixel
How to cache this directory in BitBucket pipelines? Please see below the contents of bitbucket-pipelines.yml file, with this it is not caching the defined directory, what's wrong here?
image: eclipse/platformio
pipelines:
branches:
develop:
- step:
name: Build Project
caches: # caches the depende
- directories
script: # Modify the commands below to build your repository.
- pio ci --project-conf=./Code/UrbanAquarium.Firmware/platformio.ini ./Code/UrbanAquarium.Firmware/src
- pwd
definitions:
caches:
directories: ./Code/UrbanAquarium.Firmware/.piolibdeps
And here my folder structure.
in case you're still looking for an answer - I think you got it almost right, but probably need to specify a custom --build-dir (so that you can specify the same path for your cache) as well as --keep-build-dir (see https://docs.platformio.org/en/latest/userguide/cmd_ci.html). Also, I'm not sure why you specified a ./Code/UrbanAquarium.Firmware/ prefix.
That said, I've tried the above and it became quickly ugly - for now I'll only cache ~/.platformio, as well as the default pip cache:
image: python:2.7.16
pipelines:
default:
- step:
caches:
- pip
- pio
script:
- pip install -U platformio
- platformio update
- platformio ci src/ --project-conf=platformio.ini
definitions:
caches:
pio: ~/.platformio