I am trying to use Bitbucket Pipelines for my Android project.
There is my bitbucket-pipelines.yml :
image: javiersantos/android-ci:latest
pipelines:
default:
- step:
script:
- export GRADLE_USER_HOME=`pwd`/.gradle
- chmod +x ./gradlew
- ./gradlew assembleDebug
When i am running my pipeline, i have this error :
+ chmod +x ./gradlew
chmod: cannot access './gradlew': No such file or directory
What i missed in my pipeline configuration ?
You need to cd into the folder that contains your gradlew file, if it is not in the root folder of your Git repository. You can check where is your gradlew file in the Docker build machine, by checking the $BITBUCKET_CLONE_DIR environment variable, or listing the files:
pipelines:
default:
- step:
script:
- echo "The current folder is: $PWD"
- echo "The git repo root folder is: $BITBUCKET_CLONE_DIR"
- echo "The files in the git root directory are: $(ls -la)"
- echo "The folders in the git root directory are: $(echo */)"
- echo "The gradlew file is at location: $( find . -name "gradlew" -type f -print0 | xargs )"
- echo "Setting current working directory to subfolder"
- cd MyProject # This should contain your 'gradlew' file
- chmod +x ./gradlew
- ./gradlew assembleDebug
Related
I am trying to create curl request and the result of it i try to save to a file. All this happens during the pipeline.yml runs of bitbucket.
I am using this command:
- curl -X GET "https:myurl.com/file.json" -o ./filefolder/file.json
which works in local and i can even see it that it runs succesfully in pipeline, but the file is not created or it dissapears. I checked also in heroku folders and seems that the file is not there.
Configuration of .yml;
branches:
development:
- parallel:
- step:
name: Run build
caches:
- node
script:
- curl -X GET "https:myurl.com/file.json" -o ./filefolder/file.json
- export NODE_OPTIONS=--max-old-space-size=3072
- npm install
- npm run build
- npm test
- step:
name: "Zip the sources"
script:
- mkdir heroku
- tar -czf ../tmp/123.tar.gz .
- cd heroku
- mv ../../tmp/123.tar.gz .
artifacts:
- heroku/**
My Gilab CI script falls and exists with this error
Permission denied, please try again.
Permission denied, please try again.
$SSH_USER#$IPADDRESS: Permission denied (publickey,password).
This is my CI script:
image: alpine
before_script:
- echo "Before script"
- apk add --no-cache rsync openssh openssh-client
- mkdir -p ~/.ssh
- eval $(ssh-agent -s)
- echo "$SSH_KNOWN_HOSTS" > ~/.ssh/known_hosts
- chmod 644 ~/.ssh/known_hosts
- echo "${SSH_PRIVATE_KEY}" | tr -d '\r' | ssh-add - > /dev/null
- ssh -o 'StrictHostKeyChecking no' $SSH_USER#$IPADDRESS
- cd /var/www/preview.hidden.nl/test
building:
stage: build
script:
- git reset --hard
- git pull origin develop
- composer install
- cp .env.example .env
- php artisan key:generate
- php artisan migrate --seed
- php artisan cache:clear
- php artisan config:clear
- php artisan storage:link
- sudo chown -R deployer:www-data /var/www/preview.hidden.nl/test/
- find /var/www/preview.hidden.nl/test -type f -exec chmod 664 {} \;
- find /var/www/preview.hidden.nl/test -type d -exec chmod 775 {} \;
- chgrp -R www-data storage bootstrap/cache
- chmod -R ug+rwx storage bootstrap/cache
My setup is as follows.
Gitlab server > Gitlab.com
Gitlab runner > Hetzner server
Laravel Project > Same Hetzner server
I generated a new SSH-key (without password) pair for this runner, named gitlab & gitlab.pub. I added the content of "gitlab.pub" to the $KNOWN_HOST variable. I added the content of "gitlab" to the $SSH_PRIVATE_KEY variable.
The problem is, I don't really know what's going on. What I think is happening is the following. The GitLab Ci job is its own separate container. A container can't just ssh to a remote server. So the private key of my Hetzner server needs to be known to the docker container ( task: - echo "$SSH_KNOWN_HOSTS" > ~/.ssh/known_hosts ).
Because the key is then known, the docker container should not ask for a password. Yet it prompts for a password and also returns that the password is incorrect. I also have my own private key pair, besides the GitLab key pair, not sure if that causes the issue; but removing my own key there would block access to my server, so I did not test removing that.
Could someone help me in this manner? I've been working on this for two weeks and I can't even deploy a simple hello-world.txt file yet! :-)
I am downloading and unzipping binaryen in a run step.
- run: wget -c https://github.com/WebAssembly/binaryen/releases/download/version_101/binaryen-version_101-x86_64-linux.tar.gz -O - | tar -xz -C /tmp/
I am then updating the path in $BASH_ENV.
- run: echo "export PATH=/tmp/binaryen-version_101/bin/wasm-opt:\${PATH}" >> $BASH_ENV
However, I still get a command not found for wasm-opt.
How can I install the downloaded wasm-opt binary such that another run step can use it?
The main issue is that the PATH variable should contain a list of directories. You added the actual binary itself to the path instead of the directory it resides in.
So for example, instead of /tmp/binaryen-version_101/bin/wasm-opt you want /tmp/binaryen-version_101/bin/. Also, after you add a directory to the PATH you won't be able to run those binaries until the next step.
Here's an example config I made:
version: 2.1
workflows:
main:
jobs:
- build
jobs:
build:
docker:
- image: cimg/base:stable
steps:
- checkout
- run: curl -sSL "https://github.com/WebAssembly/binaryen/releases/download/version_101/binaryen-version_101-x86_64-linux.tar.gz" | tar -xz -C /tmp/
- run: echo 'export PATH=/tmp/binaryen-version_101/bin/:${PATH}' >> $BASH_ENV
- run: wasm-opt
Here is result tree on server after my script:
> pwd
/opt/atlassian/pipelines/agent/build
> tree -d
.
├── android-sdk-linux
│ ├── build-tools
│ │ └── 28.0.3
...
├── app
│ ├── build
...
└── readme
8005 directories
Here is my script from https://opatry.net/2017/11/06/bitbucket-pipelines-for-android/:
ci_install.sh
#!/usr/bin/env bash
set -eu
cur_dir=$(cd "$(dirname "$0")"; pwd)
origin_dir=$(cd "${cur_dir}/.."; pwd)
app_dir="${origin_dir}/android"
output_dir="${origin_dir}/artifacts"
default_android_sdk_zip_version="3859397"
android_sdk_zip_version=${1:-${default_android_sdk_zip_version}}
case $(uname -s) in
Linux)
os="linux"
;;
Darwin)
os="darwin"
;;
CYGWIN*|MINGW*)
os="windows"
;;
*)
echo "!! Unsupported OS $(uname -s)"
exit 1
;;
esac
export ANDROID_HOME="${origin_dir}/android-sdk-${os}"
if [ ! -f "${ANDROID_HOME}/tools/bin/sdkmanager" ]; then
# Download and unzip Android sdk
echo "Downloading Android SDK '${android_sdk_zip_version}' for '${os}'"
wget "https://dl.google.com/android/repository/sdk-tools-${os}-${android_sdk_zip_version}.zip"
unzip "sdk-tools-${os}-${android_sdk_zip_version}.zip" -d "${ANDROID_HOME}"
rm "sdk-tools-${os}-${android_sdk_zip_version}.zip"
fi
# Add Android binaries to PATH
export PATH="${ANDROID_HOME}/tools:${ANDROID_HOME}/tools/bin:${ANDROID_HOME}/platform-tools:${PATH}"
# Accept all licenses (source: http://stackoverflow.com/questions/38096225/automatically-accept-all-sdk-licences)
echo "Auto Accepting licenses"
mkdir -p "$ANDROID_HOME/licenses"
echo -e "\n8933bad161af4178b1185d1a37fbf41ea5269c55" > "${ANDROID_HOME}/licenses/android-sdk-license"
echo -e "\n84831b9409646a918e30573bab4c9c91346d8abd" > "${ANDROID_HOME}/licenses/android-sdk-preview-license"
# Update android sdk
echo "Downloading packages described by ${cur_dir}/package_file.txt"
cat "${cur_dir}/package_file.txt"
( sleep 5 && while [ 1 ]; do sleep 1; echo y; done ) | sdkmanager --package_file="${cur_dir}/package_file.txt"
package_file.txt
platform-tools
build-tools;26.0.2
platforms;android-26
bitbucket-pipelines.yml:
image: java:8
pipelines:
branches:
master:
- step:
caches:
- gradle
- android-sdk
script:
- bash ./build/ci_install.sh
- ANDROID_HOME=$PWD/android-sdk-linux bash ./build/android.sh
definitions:
caches:
android-sdk: android-sdk-linux
gradle: gradle
In result:
Build teardown
You already have a 'gradle' cache so we won't create it again
Assembling contents of new cache 'android-sdk'
But in Pipelines -> Caches->Dependency caches cache for android-sdk is not displayed:
And at next run:
Cache "android-sdk": Downloading
Cache "android-sdk": Not found
All works fine. My ci_install.sh file was in /MyProject/utils/pipelines/ci_install.sh so android-sdk-linux was created in /opt/atlassian/pipelines/agent/build/utils/android-sdk-linux folder
So I moved file ci_install.sh to /MyProject/pipelines/ci_install.sh and now android-sdk-linux is created in /opt/atlassian/pipelines/agent/build/android-sdk-linux folder
Removed folder utils from my project
I am using jhipster 5.1.0, I used "jhipster ci-cd" in order to generate the .gitlab-ci.yml file.
I am running Gitlab and Gitlab-CI on a private Ubuntu 18.04LTS server in my company. I configured the Gitlab Runner to execute the builds with docker.
My .gitlab-ci.yml file is as follows (I did not modify it much):
image: jhipster/jhipster:v5.1.0
cache:
key: "$CI_COMMIT_REF_NAME"
paths:
- .gradle/wrapper
- .gradle/caches
stages:
- build
before_script:
- export GRADLE_USER_HOME=`pwd`/.gradle
- ./gradlew yarn_install -PnodeInstall --no-daemon
gradle-build:
stage: build
script:
- ./gradlew compileJava -x check -PnodeInstall --no-daemon
- ./gradlew test -PnodeInstall --no-daemon
- ./gradlew yarn_test -PnodeInstall --no-daemon
- ./gradlew bootJar -Pprod -x check -PnodeInstall --no-daemon
artifacts:
paths:
- build/libs/*.jar
# Uncomment following to expire the artifacts after defined period, https://docs.gitlab.com/ee/ci/yaml/README.html#artifacts-expire_in
# expire_in: 90 day
Here is the output of the gitlab-ci runner:
...
Successfully extracted cache
$ export GRADLE_USER_HOME=`pwd`/.gradle
$ ./gradlew compileJava -x check -PnodeInstall --no-daemon
/bin/bash: line 60: ./gradlew: Permission denied
ERROR: Job failed: exit code 1
As the problem seems obvious, I tried to add " - chmod +x gradlew", before the ".gradlew" call in the "before_script" section. I thought it would be a good idea, because it was generated by the "jhipster ci-cd" command before 5.1.0, but not anymore. No success: Gitlab-CI output became as follows:
...
Successfully extracted cache
$ export GRADLE_USER_HOME=`pwd`/.gradle
$ chmod +x gradlew
chmod: changing permissions of 'gradlew': Operation not permitted
ERROR: Job failed: exit code 1
So I tried to switch to the docker image "openjdk:8" instead of "jhipster/jhipster:v5.1.0", in the .gitlab-ci.yml file. Much better, gradle runs the "yarn install" command, but it stops at some point, because that container does not contain "libpng-dev" (which was added recently into the jhipster container, no luck !):
...
[5/5] Building fresh packages...
error An unexpected error occurred:
"/builds/epigone/exportCCN/node_modules/pngquant-bin: Command failed.
Exit code: 1
Command: sh
Arguments: -c node lib/install.js
Directory: /builds/epigone/exportCCN/node_modules/pngquant-bin
Output:
⚠ The `/builds/epigone/exportCCN/node_modules/pngquant-bin/vendor/pngquant`
binary doesn't seem to work correctly
⚠ pngquant pre-build test failed
ℹ compiling from source
✔ pngquant pre-build test passed successfully
✖ Error: pngquant failed to build, make sure that libpng-dev is installed
at Promise.all.then.arr (/builds/epigone/exportCCN/node_modules/pngquant-bin/node_modules/bin-build/node_modules/execa/index.js:231:11)
at <anonymous>
at process._tickCallback (internal/process/next_tick.js:188:7)".
info If you think this is a bug, please open a bug report with the information provided in "/builds/epigone/exportCCN/yarn-error.log".
info Visit https://yarnpkg.com/en/docs/cli/install for documentation about this command.
:yarn_install FAILED
You need to modify the permissions on your git repo.
Run:
git update-index --chmod=+x gradlew
then commit and push.