I am trying to setup gitlab-runner in a aws instance.
My project is in a group. Hence setting up group-runner so that I can use it for other projects in the group.
# gitlab-runner version
Version: 11.7.0
Git revision: 8bb608ff
Git branch: 11-7-stable
GO version: go1.8.7
Built: 2019-01-22T11:46:13+0000
OS/Arch: linux/amd64
# docker
Docker version 18.09.1, build 4c52b90
docker-machine version 0.16.0, build 702c267f
Registered the runner with
sudo gitlab-runner register
Please enter the gitlab-ci coordinator URL (e.g. https://gitlab.com )
https://gitlab.com
Please enter the gitlab-ci token for this runner
<group runner token>
Please enter the gitlab-ci description for this runner
[hostame] my-runner
Please enter the executor: ssh, docker+machine, docker-ssh+machine, kubernetes, docker, parallels, virtualbox, docker-ssh, shell:
docker
Please enter the Docker image (eg. ruby:2.1):
ruby:2.5
I can see my-runner runner registered from gitlab UI.
Whenever I run/retry the pipeline, it always execute with gitlab's auto-scaling instance.
Running with gitlab-runner 11.7.0-rc1 (6e20bd76)
on docker-auto-scale ed2dce3a
Using Docker executor with image ruby:2.5 ...
Pulling docker image ruby:2.5 ...
Using docker image sha256:feea8cad6f9e7cc58f7ae793ac92bd80fa1ce4da54a381921f161447e978021f for ruby:2.5 ...
Running on runner-ed2dce3a-project-10682917-concurrent-0 via runner-ed2dce3a-srm-1549352595-5d0f29b8...
Cloning repository...
Cloning into '/builds/dr5nn/gitlab-ci-demo'...
Where and what I am missing to run gitlab-runner from my custom machine?
Do I need to add IP address somewhere or enable some port in my aws instance?
Bellow is my .gitlab-ci.yml
before_script:
- apt-get update -qq && apt-get install -y -qq sqlite3 libsqlite3-dev nodejs
- ruby -v
- which ruby
- gem install bundler -v 2.0.1
- bundle install --jobs $(nproc) "${FLAGS[#]}"
rubocop:
script:
- bundle exec rubocop
You are on right way - all you need is to disable shared gitlab runners for your group or particular project.
Registering group runner enables it for the group, but that doesn't actually disable all other runners - pipeline still chooses the most convenient one based on tags and other criteria.
Other way - use your private tags (not like docker and etc) to select runner. Runners pick their jobs according to the set of tags, specified for jobs. For example, if job has tags docker and linux, only runners with such tags can pick it up. So, you can simply mark jobs, which you want to execute on your group runner (and not on shared runners) with tag like private-runner and add this tag to your runner.
With suggestion from grapes. I'm able to run with bellow configurations.
While registering gitlab-runner from my aws intance
...
...
Please enter the gitlab-ci tags for this runner (comma separated):
my-tag
...
...
And changed gitlab-ci.yml file with
before_script:
- apt-get update -qq && apt-get install -y -qq sqlite3 libsqlite3-dev nodejs
- ruby -v
- which ruby
- gem install bundler -v 2.0.1
- bundle install --jobs $(nproc) "${FLAGS[#]}"
rubocop:
tags:
- my-tag
script:
- bundle exec rubocop
Related
I ran the following command, it seems to be stuck, neither error nor success：
docker run --rm ubuntu:22.04 /bin/bash -c "apt-get update && apt-get install -y subversion && svn co https://github.com/GPUOpen-LibrariesAndSDKs/AMF/trunk/amf/public/include --non-interactive amf-headers"
also I tried debug log,still no output:
docker run --rm ubuntu:22.04 /bin/bash -c "apt-get update && apt-get install -y subversion && svn co https://github.com/GPUOpen-LibrariesAndSDKs/AMF/trunk/amf/public/include --non-interactive --config-option servers:global:neon-debug-mask=1073741824 amf-headers"
so I tried ubuntu 20.04 and it at least gives an error:
docker run --rm ubuntu:20.04 /bin/bash -c "apt-get update && apt-get install -yqq subversion && svn co https://github.com/GPUOpen-LibrariesAndSDKs/AMF/trunk/amf/public/include --non-interactive amf-headers"
svn: E170013: Unable to connect to a repository at URL 'https://github.com/GPUOpen-LibrariesAndSDKs/AMF/trunk/amf/public/include'
svn: E230001: Server SSL certificate verification failed: issuer is not trusted
Why does svn have no response on Ubuntu 22.04?
If it is an ssl certificate problem like ubuntu 20.04, it should also give an error message instead of nothing
more detail：
GitHub repositories can be accessed from both Git and Subversion (SVN) clients doc
I do all the test with github action(ubuntu-latest/ubuntu-20.04),Included Software
Docker Compose v1 1.29.2
Docker Compose v2 2.1.1+azure-1
Docker-Buildx 0.7.0
Docker-Moby Client 20.10.11+azure-1
Docker-Moby
Server 20.10.11+azure-1
test workflow:
name: Test
on:
push:
branches: [ main ]
workflow_dispatch:
jobs:
Test:
runs-on: ubuntu-latest
strategy:
fail-fast: false
matrix:
version: ["22.04", "21.10", "21.04", "20.04"]
steps:
- name: Test
timeout-minutes: 5
shell: bash
run: |
docker run --rm ubuntu:${{ matrix.version }} /bin/bash -c "apt-get update && apt-get install -yqq subversion && svn co https://github.com/GPUOpen-LibrariesAndSDKs/AMF/trunk/amf/public/include --non-interactive amf-headers"
If you really want konw where I used this command,you can try using https://github.com/shinchiro/mpv-winbuild-cmake to build mpv with docker image ubuntu:22.04.The toolchain use svn to download some files.
This is a bug in current Ubuntu 22.04 development version filed as bug #1959717 - I guess we have to wait until it gets fixed.
I have my docker images in nexus. When i am trying to run that playbook i am getting error
Playbook Code
---
- hosts: localhost
become: True
tasks:
- name: pull an image
docker_image:
name: 10.220.110.10:8083/halosys:f5a950f
source: pull
...
ERROR while running my playbook
TASK [pull an image] **************************************************************************************************************
fatal: [localhost]: FAILED! => {"changed": false, "msg": "Unsupported parameters for (docker_image) module: source Supported parameters include: api_version, archive_path, buildargs, cacert_path, cert_path, container_limits, debug, docker_host, dockerfile, force, http_timeout, key_path, load_path, name, nocache, path, pull, push, repository, rm, ssl_version, state, tag, timeout, tls, tls_hostname, tls_verify, use_tls"}
Please help to fix this error
Option source was added in Ansible 2.8 - see documentation. The error message states it is missing, so you're using older version. To fix this you need to either update your Ansible installation to at least version 2.8 or check the documentation for your version.
For example, Ansible 2.7 documentation states that to pull an image you need to do:
- name: pull an image
docker_image:
name: 10.220.110.10:8083/halosys:f5a950f
To check what version of Ansible you're running use command
ansible --version
EDIT:
I can’t guarantee it’ll work, but I would try removing Ansible installed from system packages:
sudo yum remove ansible
followed by installing Python 3:
sudo yum install python36 python36-virtualenv python36-pip
then installing docker-py using pip:
python3 -m pip install docker-py
And finally installing Ansible with Python3 support:
python3 -m pip install ansible
Here’s Ansible documentation about Python3 support.
Also if package python36 is not available you can run
sudo yum list | grep python3
to find what version is available. As per documentation mentioned above you need at least python35 to make Ansible work.
I am trying to setup bitbucket pipeline for a php based (Laravel-Lumen) app intended to be deployed on nanobox.io. I want this pipeline to deploy my app as soon as code changes are committed.
My bitbucket-pipelines.yml looks like this
image: php:7.1.29
pipelines:
branches:
staging:
- step:
name: Publish to staging version
deployment: staging
caches:
- composer
script:
- apt-get update && apt-get install -y unzip
- curl -sS https://getcomposer.org/installer | php -- --install-dir=/usr/local/bin --filename=composer
- composer install
# - vendor/bin/phpunit
- bash -c "$(curl -fsSL https://s3.amazonaws.com/tools.nanobox.io/bootstrap/ci.sh)"
- nanobox deploy
This gives Following error
+ nanobox deploy
Failed to validate provider - missing docker - exec: "docker": executable file not found in $PATH
Using nanobox with native requires tools that appear to not be available on your system.
docker
View these requirements at docs.nanobox.io/install
I then followed this page and changed second last line to look like this
sudo bash -c "$(curl -fsSL https://s3.amazonaws.com/tools.nanobox.io/bootstrap/ci.sh)"
when done that, I am getting following error
+ sudo bash -c "$(curl -fsSL https://s3.amazonaws.com/tools.nanobox.io/bootstrap/ci.sh)"
bash: sudo: command not found
I ran out of tricks here, also I don't have experience in this area. Any help is very much appreciated.
First, you can't use sudo in pipelines, but that's probably not relevant here. The issue is that nanobox cli wan't to execute docker, which isn't installed. You should enable the docker service for your step.
image: php:7.1.29
pipelines:
branches:
staging:
- step:
name: Publish to staging version
deployment: staging
# Enable docker service
services:
- docker
caches:
- composer
script:
- docker version
You might wan't to have a look at Pipelines docs as well: Run Docker commands in Bitbucket Pipelines
I have successfully installed Gitlab CI and also linked it with my gitlab. I have also configured shared runners using docker with a ruby-2.2 image with mysql service.
Following was what I executed to configure a runner by referring https://about.gitlab.com/2015/04/17/unofficial-gitlab-ci-runner/:
$ gitlab-ci-multi-runner register \
--non-interactive \
--url "https://my.gitlab.ip/" \
--registration-token "REGISTRATION_TOKEN" \
--description "ruby-mysql" \
--executor "docker" \
--docker-image ruby:2.2 --docker-mysql latest
I have a sample Ruby / Rails application and for some reason the runner doesn't run the build. Here is my .gitlab-ci.yml:
image: ruby:2.2
services:
- mysql:latest
before_script:
- ruby -v
- gem install bundler
- cp config/database.yml.example config/database.yml
- cp config/secrets.yml.example config/secrets.yml
- bundle install
spec:
script:
- bundle exec rspec
tags:
- ruby-mysql
Try removing the first line image: ruby:2.2 of your .gitlab-ci.yml
I had a similar issue where the CI reported a success, yet didn't do any work.
I used the lint provided on http://my.domain/lint
It flagged "image" and "stage/stages" as not correct.
That's why I think removing the first line will help you.
I think the problem is that the community version doesn't recognize the keywords yet.
I am seeing a build failure on travis-ci, which I cannot reproduce on my local machine. Are there instructions somewhere for setting up a VM that is identical to the travis-ci linux build environment? I'm glad to have travis-ci already reveal a new bug, but less excited to debug it by sending in commits that add debug code.
For container-based builds, there are now instructions on how to setup a docker image locally.
Unfortunately, quite a few steps are still manual. Here are the commands you need to get it up and running:
# change the image according to the language chosen in .travis.yml
$ docker run -it -u travis quay.io/travisci/travis-jvm /bin/bash
# now that you are in the docker image, switch to the travis user
sudo su - travis
# Install a recent ruby (default is 1.9.3)
rvm install 2.3.0
rvm use 2.3.0
# Install travis-build to generate a .sh out of .travis.yml
cd builds
git clone https://github.com/travis-ci/travis-build.git
cd travis-build
gem install travis
travis # to create ~/.travis
ln -s `pwd` ~/.travis/travis-build
bundle install
# Create project dir, assuming your project is `me/project` on GitHub
cd ~/builds
mkdir me
cd me
git clone https://github.com/me/project.git
cd project
# change to the branch or commit you want to investigate
travis compile > ci.sh
# You most likely will need to edit ci.sh as it ignores matrix and env
bash ci.sh
You can use Travis Build which is a library (which means you've to place it in ~/.travis/) to generate a shell based build script (travis compile) which can be then uploaded to the VMs using SSH and executed.
Below steps are just guidance in order to get you into the right track (if anything is missing, let me know).
Docker
Example command to run container (which can be found at Docker Hub):
docker run -it travisci/ubuntu-ruby:18.04 /bin/bash
Run your container, clone your repository then test it manually.
See: Running a Container Based Docker Image Locally
SSH access
Check out this answer. Basically you need to setup bounce host, then configure your build to run SSH tunnel.
Here is the example .travis.yml:
sudo: required
dist: trusty
language: python
python: "2.7"
script:
- echo travis:$sshpassword | sudo chpasswd
- sudo sed -i 's/ChallengeResponseAuthentication no/ChallengeResponseAuthentication yes/' /etc/ssh/sshd_config
- sudo service ssh restart
- sudo apt-get install sshpass
- sshpass -p $sshpassword ssh -R 9999:localhost:22 -o StrictHostKeyChecking=no travisci#$bouncehostip
Local setup
Here are the steps to test it on your local environment:
cd ~
git clone https://github.com/travis-ci/travis-build.git
ln -s ~/travis-build/ ~/.travis/travis-build
sudo gem install bundler
bundle install --gemfile ~/.travis/travis-build/Gemfile
cd repo-dir/
travis login -g <github_token>
vim .travis.yaml
travis lint # to validate script
travis compile # to transform into shell script
Vagrant/VM
After you did travis compile which would produce the bash script as result of your .travis.yml, you can use use vagrant to run this script into virtualized environment using provided Vagrantfile and the following steps:
vagrant up
vagrant ssh
cd /vagrant
bundle exec rspec spec
You probably need to install more tools in order to test it.
Here is some git hint which avoids you to generates unnecessary commits when doing trial & errors commits for Travis CI testing:
Fork the repo (or use separate branch).
After initial commit, keep adding --amend to replace your previous commit:
git commit --amend -m 'Same message.' -a
Push the amended commit by force (e.g. into already opened PR):
git push fork -f
Now Travis CI would re-check the same commit over and over again.
See also: How to run travis-ci locally.
I'm facing the same issue right now. I used to use CircleCI before, where you could just login to VM via ssh, but this doesn't work with Travis-CI VMs.
I was able to debug it (to a certain point) by setting up Travis-ci VM clone via Travis-Cookbooks. You would need to install VirtualBox and Vagrant on your computer first before cloning this repository.
Once you have Travis-Cookbooks cloned, open the folder, launch command prompt|terminal and type vagrant up. Once Vagrant finishes setting up VM (may take a long time) on your machine, you can connect to it via ssh by running vagrant ssh.
From there, you would need to clone your own repository (or just copy the code to VM) and apply the steps from your .travis.yml file.
Eregon's answer failed for me at travis compile, there error looks like:
/home/travis/.rvm/rubies/ruby-2.3.0/lib/ruby/2.3.0/rubygems/core_ext/kernel_require.rb:55:in `require': cannot load such file -- travis/support (LoadError)
I got it working with the following adjustments: (Adjustments marked with # CHANGED. I'm using the node environment)
# change the image according to the language chosen in .travis.yml
# Find images at https://quay.io/organization/travisci
docker run -it quay.io/travisci/travis-node-js /bin/bash
# now that you are in the docker image, switch to the travis user
su travis
# Install a recent ruby (default is 1.9.3) to make bundle install work
rvm install 2.3.0
rvm use 2.3.0
# Install travis-build to generate a .sh out of .travis.yml
sudo mkdir builds # CHANGED
cd builds
sudo git clone https://github.com/travis-ci/travis-build.git
cd travis-build
gem install travis
travis # to create ~/.travis
ln -s `pwd` ~/.travis/travis-build
bundle install
bundler add travis # CHANGED
sudo mkdir bin # CHANGED
sudo chmod a+w bin/ # CHANGED
bundler binstubs travis # CHANGED
# Create project dir, assuming your project is `me/project` on GitHub
cd ~/builds
mkdir me
cd me
git clone https://github.com/me/project.git
cd project
# change to the branch or commit you want to investigate
~/.travis/travis-build/bin/travis compile > ci.sh # CHANGED
# You most likely will need to edit ci.sh as it ignores matrix and env
# In particular I needed to edit --branch=’’ to the branch name
bash ci.sh