SonarScanner fails with apt-get not found - docker

I have installed SonarQube on a ubuntu machine via a docker image. All working fine and I'm able to log in without issues.
Have connected to our GitLab installation and see all available projects, when I try to configure the existing pipeline with the following, I got stuck.
I have the following pipeline.yml in use (partially shown here):
sonarqube-check:
stage: sonarqube-check
image: mcr.microsoft.com/dotnet/core/sdk:latest
variables:
SONAR_USER_HOME: "${CI_PROJECT_DIR}/.sonar" # Defines the location of the analysis task cache
GIT_DEPTH: "0" # Tells git to fetch all the branches of the project, required by the analysis task
cache:
key: "${CI_JOB_NAME}"
paths:
- .sonar/cache
script:
- "apt-get update"
- "apt-get install --yes openjdk-11-jre"
- "dotnet tool install --global dotnet-sonarscanner"
- "export PATH=\"$PATH:$HOME/.dotnet/tools\""
- "dotnet sonarscanner begin /k:\"my_project_location_AYDMUbUQodVNV6NM7qxd\" /d:sonar.login=\"$SONAR_TOKEN\" /d:\"sonar.host.url=$SONAR_HOST_URL\" "
- "dotnet build"
- "dotnet sonarscanner end /d:sonar.login=\"$SONAR_TOKEN\""
allow_failure: true
only:
- master
All looking good, but when it runs it gives me this error:
$ apt-get update
bash: apt-get: command not found
I just don't know how to fix this and can't find a solution on the internet somewhere

dotnet/core/sdk image has apt (not apt-get):
$ docker run -ti --rm mcr.microsoft.com/dotnet/core/sdk:latest sh
# apt update
Following SonarCube documentation, you can use their docker image with the CLI already installed:
image:
name: sonarsource/sonar-scanner-cli:latest
variables:
SONAR_TOKEN: "your-sonarqube-token"
SONAR_HOST_URL: "http://your-sonarqube-instance.org"
SONAR_USER_HOME: "${CI_PROJECT_DIR}/.sonar" # Defines the location of the analysis task cache
GIT_DEPTH: 0 # Tells git to fetch all the branches of the project, required by the analysis task
cache:
key: ${CI_JOB_NAME}
paths:
- .sonar/cache
sonarqube-check:
stage: test
script:
- sonar-scanner -Dsonar.qualitygate.wait=true
allow_failure: true
only:
- master

Apt /apt-get command not found - Problem fixed:
I think in your /usr/bin have no the apt and apt-get, you can download it and install it on that https://packages.debian.org/stretch/apt, like this
wget http://ftp.cn.debian.org/debian/pool/main/a/apt/apt_1.4.9_amd64.deb
dpkg -i apt_1.4.9_amd64.deb

Related

Svn checkout has no response in the docker image of Ubuntu 22.04

I ran the following command, it seems to be stuck, neither error nor success:
docker run --rm ubuntu:22.04 /bin/bash -c "apt-get update && apt-get install -y subversion && svn co https://github.com/GPUOpen-LibrariesAndSDKs/AMF/trunk/amf/public/include --non-interactive amf-headers"
also I tried debug log,still no output:
docker run --rm ubuntu:22.04 /bin/bash -c "apt-get update && apt-get install -y subversion && svn co https://github.com/GPUOpen-LibrariesAndSDKs/AMF/trunk/amf/public/include --non-interactive --config-option servers:global:neon-debug-mask=1073741824 amf-headers"
so I tried ubuntu 20.04 and it at least gives an error:
docker run --rm ubuntu:20.04 /bin/bash -c "apt-get update && apt-get install -yqq subversion && svn co https://github.com/GPUOpen-LibrariesAndSDKs/AMF/trunk/amf/public/include --non-interactive amf-headers"
svn: E170013: Unable to connect to a repository at URL 'https://github.com/GPUOpen-LibrariesAndSDKs/AMF/trunk/amf/public/include'
svn: E230001: Server SSL certificate verification failed: issuer is not trusted
Why does svn have no response on Ubuntu 22.04?
If it is an ssl certificate problem like ubuntu 20.04, it should also give an error message instead of nothing
more detail:
GitHub repositories can be accessed from both Git and Subversion (SVN) clients doc
I do all the test with github action(ubuntu-latest/ubuntu-20.04),Included Software
Docker Compose v1 1.29.2
Docker Compose v2 2.1.1+azure-1
Docker-Buildx 0.7.0
Docker-Moby Client 20.10.11+azure-1
Docker-Moby
Server 20.10.11+azure-1
test workflow:
name: Test
on:
push:
branches: [ main ]
workflow_dispatch:
jobs:
Test:
runs-on: ubuntu-latest
strategy:
fail-fast: false
matrix:
version: ["22.04", "21.10", "21.04", "20.04"]
steps:
- name: Test
timeout-minutes: 5
shell: bash
run: |
docker run --rm ubuntu:${{ matrix.version }} /bin/bash -c "apt-get update && apt-get install -yqq subversion && svn co https://github.com/GPUOpen-LibrariesAndSDKs/AMF/trunk/amf/public/include --non-interactive amf-headers"
If you really want konw where I used this command,you can try using https://github.com/shinchiro/mpv-winbuild-cmake to build mpv with docker image ubuntu:22.04.The toolchain use svn to download some files.
This is a bug in current Ubuntu 22.04 development version filed as bug #1959717 - I guess we have to wait until it gets fixed.

How to run bitbucket pipeline to deploy php based app on nanobox

I am trying to setup bitbucket pipeline for a php based (Laravel-Lumen) app intended to be deployed on nanobox.io. I want this pipeline to deploy my app as soon as code changes are committed.
My bitbucket-pipelines.yml looks like this
image: php:7.1.29
pipelines:
branches:
staging:
- step:
name: Publish to staging version
deployment: staging
caches:
- composer
script:
- apt-get update && apt-get install -y unzip
- curl -sS https://getcomposer.org/installer | php -- --install-dir=/usr/local/bin --filename=composer
- composer install
# - vendor/bin/phpunit
- bash -c "$(curl -fsSL https://s3.amazonaws.com/tools.nanobox.io/bootstrap/ci.sh)"
- nanobox deploy
This gives Following error
+ nanobox deploy
Failed to validate provider - missing docker - exec: "docker": executable file not found in $PATH
Using nanobox with native requires tools that appear to not be available on your system.
docker
View these requirements at docs.nanobox.io/install
I then followed this page and changed second last line to look like this
sudo bash -c "$(curl -fsSL https://s3.amazonaws.com/tools.nanobox.io/bootstrap/ci.sh)"
when done that, I am getting following error
+ sudo bash -c "$(curl -fsSL https://s3.amazonaws.com/tools.nanobox.io/bootstrap/ci.sh)"
bash: sudo: command not found
I ran out of tricks here, also I don't have experience in this area. Any help is very much appreciated.
First, you can't use sudo in pipelines, but that's probably not relevant here. The issue is that nanobox cli wan't to execute docker, which isn't installed. You should enable the docker service for your step.
image: php:7.1.29
pipelines:
branches:
staging:
- step:
name: Publish to staging version
deployment: staging
# Enable docker service
services:
- docker
caches:
- composer
script:
- docker version
You might wan't to have a look at Pipelines docs as well: Run Docker commands in Bitbucket Pipelines

Cache dependencies between GitLab CI build stages

I have the following GitLab yml file and I have written it in stages assuming that every stage would retain the dependencies, but seems it is not to be the case!
# This file is a template, and might need editing before it works on your project.
# Official Java image. Look for the different tagged releases at
# https://hub.docker.com/r/library/java/tags/ . A Java image is not required
# but an image with a JVM speeds up the build a bit.
image: java:8
variables:
FILE_TARGET_PATH: $FILE_TARGET_PATH
stages:
- test
- run
cache:
key: "$CI_COMMIT_REF_NAME" # contains either the branch or the tag, so it's caching per branch
untracked: true
paths:
- "sbt-cache/.ivy.cache"
- "sbt-cache/.boot"
- "sbt-cache/.sbtboot"
- "sbt-cache/target"
before_script:
# Enable the usage of sources over https
- apt-get update -yqq
- apt-get install apt-transport-https -yqq
# Add keyserver for SBT
- echo "deb http://dl.bintray.com/sbt/debian /" | tee -a /etc/apt/sources.list.d/sbt.list
- apt-key adv --keyserver hkp://keyserver.ubuntu.com:80 --recv 2EE0EA64E40A89B84B2DF73499E82A75642AC823
# Install SBT
- apt-get update -yqq
- apt-get install sbt -yqq
Run unit Tests:
stage: test
tags:
- master
script:
# Execute your project's tests
- sbt -Denv=test clean test
Run Pipeline:
stage: run
tags:
- master
script:
# Execute the pipeline
- sbt -Denv=test run
How can I cache the dependencies in this multi stage set up? I have a local runner on my machine that runs the pipeline. Would artifacts help me?
If you are using a newer sbt version than 1.0.4 the caching won't work for you as the compiler will always invalidate all the files.
This compiler issue has already been reported here: https://github.com/sbt/sbt/issues/4168
My suggestion would be to downgrade sbt version to 1.0.4 for CI. If that doesn't help you can also check the reason why the cache is invalidated by adding to your build.sbt:
// Debug incremental zinc compiler
logLevel := Level.Debug
incOptions := incOptions.value.withApiDebug(true).withRelationsDebug(true)
I had the same issue for Bitbucket Pipelines CI and managed to successfully make it work here

CircleCI branch build failing but tag build succeeds

I am building my project on CircleCI and I have a build job that looks like this:
build:
<<: *defaults
steps:
- checkout
- setup_remote_docker
- run:
name: Install pip
command: curl https://bootstrap.pypa.io/get-pip.py -o get-pip.py && sudo python get-pip.py
- run:
name: Install AWS CLI
command: curl "https://s3.amazonaws.com/aws-cli/awscli-bundle.zip" -o "awscli-bundle.zip" && unzip awscli-bundle.zip && sudo ./awscli-bundle/install -i /usr/local/aws -b /usr/local/bin/aws
- run:
name: Login to Docker Registry
command: aws ecr get-login --no-include-email --region us-east-1 | sh
- run:
name: Install Dep
command: curl https://raw.githubusercontent.com/golang/dep/master/install.sh | sh
- run:
name: Save Version Number
command: echo "export VERSION_NUM=${CIRCLE_TAG}.${CIRCLE_BUILD_NUM}" > deployment/dev/.env
- run:
name: Build App
command: source deployment/dev/.env && docker-compose -f deployment/dev/docker-compose.yml build
- run:
name: Test App
command: |
git config --global url."https://${GITHUB_PERSONAL_ACCESS_TOKEN} :x-oauth-basic#github.com/".insteadOf "https://github.com/"
dep ensure
go test -v ./...
- run:
name: Push Image
command: |
if [[ "${CIRCLE_TAG}" =~ ^v[0.9]+(\.[0-9]+)*-[a-z]*$ ]]; then
source deployment/dev/.env
docker-compose -f deployment/dev/docker-compose.yml push
else
echo 'No tag, not deploying'
fi
- persist_to_workspace:
root: .
paths:
- deployment/*
- tools/*
When I push a change to a branch, the build fails every time with Couldn't connect to Docker daemon at ... - is it running? when it reaches the Build App step of the build job.
Please help me figure out why branch builds are failing but tag builds are not.
I suspect you are hitting this docker-compose bug: https://github.com/docker/compose/issues/6050
The bug reports a misleading error (the one you're getting) when an image name in the docker-compose file is invalid.
If you use an environment variable for the image name or image tag, and that variable is set from a branch name, then it would fail on some branches, but not others.
The problem was occurring on the Save Version Number step. Sometimes that version would be .${CIRCLE_BUILD_NUM} since no tag was passed. Docker dislikes these tags starting with ., so I added a conditional check to see if CIRCLE_TAG was empty, and if it was, use some default version: v0.1.0-build.

Passing reports through docker in codecov gives error

I'm trying to setup codecov as code coverage tool in my repository. I referred to this link to pass reports through docker container -
Link - https://github.com/codecov/support/wiki/Testing-with-Docker
But travis ci fails and gives this error -
docker: Error parsing reference: "..." is not a valid repository/tag.
Here is my travis.yml
sudo: required
dist: trusty
language: node_js
node_js:
- 6
before_install:
- export CHROME_BIN=chromium-browser
- export DISPLAY=:99.0
- sh -e /etc/init.d/xvfb start
- docker run -v "$PWD/shared:/shared" ...
before_script:
- ng build
script:
- ng test --watch=false
- ng lint
- >
docker run -ti -v $(pwd):/app --workdir=/app coala/base coala --version
after_success:
- bash ./deploy.sh
- bash <(curl -s https://codecov.io/bash)
- mv -r coverage/ shared
cache:
bundler: true
directories:
- node_modules
- .coala-cache
services: docker
branches:
only:
- angular
How should I solve this? Thanks!
I assume you refer to Codecov Outside Docker. The current error message already tells you that the three dots ... need to be replaced with a real Docker repository name, e.g. node:6-alpine.
What you're still missing is the part of running the tests (including reports) inside the Docker container, so that you can mv the test reports to the shared folder. You could achieve that by adding a custom Dockerfile based on node, similar to the one below. I chose a more or less full base image including Chrome and other tools to make your use case work:
FROM markadams/chromium-xvfb-js:7
WORKDIR /proj
CMD npm install && \
node_modules/.bin/ng build && \
node_modules/.bin/ng test --watch=false && \
node_modules/.bin/ng lint && \
mkdir -p shared && \
mv coverage.txt shared
That custom image needs to be built and then run like this (assuming the Dockerfile to be in your project root directory):
docker build -t ci-build .
docker run --rm -v "$(pwd):/proj" ci-build
I suggest to change the .travis.yml like follows:
sudo: required
dist: trusty
language: node_js
node_js:
- 6
before_install:
- docker build -t ci-build .
script:
- >
docker run --rm -v $(pwd):/proj ci-build
- >
docker run -ti -v $(pwd):/app --workdir=/app coala/base coala --version
after_success:
- bash ./deploy.sh
- bash <(curl -s https://codecov.io/bash)
cache:
bundler: true
directories:
- node_modules
- .coala-cache
services: docker
branches:
only:
- angular
Another note: the coala/base image works similarly.

Resources