Packer fails on Gitlab-CI with {message:401 Unauthorized}: command not found - docker

I am trying to use packer with GCP in gitlab-ci but every time I get to the packer build it will fail with the following error:
starting remote command: chmod +x /tmp/script_5147.sh; DEBIAN_FRONTEND='noninteractive' PACKER_BUILDER_TYPE='googlecompute' PACKER_BUILD_NAME='my_vm' /tmp/script_5147.sh
==> googlecompute.my_vm: /tmp/script_5147.sh: line 1: {message:401 Unauthorized}: command not found
2022/11/13 13:26:05 [INFO] 0 bytes written for 'stdout'
2022/11/13 13:26:05 packer-plugin-googlecompute_v1.0.16_x5.0_linux_amd64 plugin: 2022/11/13 13:26:05 [ERROR] Remote command exited with '127': chmod +x /tmp/script_5147.sh; DEBIAN_FRONTEND='noninteractive' PACKER_BUILDER_TYPE='googlecompute' PACKER_BUILD_NAME='my_vm' /tmp/script_5147.sh
2022/11/13 13:26:05 packer-plugin-googlecompute_v1.0.16_x5.0_linux_amd64 plugin: 2022/11/13 13:26:05 [INFO] RPC endpoint: Communicator ended with: 127
The script
#!/bin/bash
set -e
if [ "$EUID" -ne 0 ]
then echo "Please run as root"
exit
fi
apt update
apt install -y curl
curl -fsSL https://deb.nodesource.com/setup_18.x | bash -
apt upgrade -y iptables
# The iptables-persistent must be installed in order to create the /etc/iptables/rules.v4 file
apt install -y nginx libzmq3-dev nodejs ipset iptables-persistent net-tools libre2-dev
npm install -g yarn
rm /etc/nginx/sites-enabled/default
The packer is working locally (also with the image) and I created a VM in GCP all of them are working, only in gitlab it fails.
I created a custom image of packer and it still failed for me on gitlab.
I will try to move it to my own runner but it will take a few days for me.
Would love if someone could help me figure it out.

Ok, I figured out the problem.
I curled some files and the token was not good so when it got to packer it failed because the files were not as expected.
I printing the 401 of the curl.

Related

"logname: no login name" inside Docker container when running dpkg -i

I need to install an SDK package inside an Ubuntu 18.04 Docker container, but am constantly running into this problem:
theuser#e9fa4f39e0f0:/src/spinnaker$ sudo dpkg -i libspinnaker_2.2.0.48_arm64.deb
(Reading database ... 52013 files and directories currently installed.)
Preparing to unpack libspinnaker_2.2.0.48_arm64.deb ...
Unpacking libspinnaker (2.2.0.48) over (2.2.0.48) ...
logname: no login name
dpkg: warning: old libspinnaker package post-removal script subprocess returned error exit status 1
dpkg: trying script from the new package instead ...
logname: no login name
dpkg: error processing archive libspinnaker_2.2.0.48_arm64.deb (--install):
new libspinnaker package post-removal script subprocess returned error exit status 1
logname: no login name
dpkg: error while cleaning up:
new libspinnaker package post-removal script subprocess returned error exit status 1
Errors were encountered while processing:
libspinnaker_2.2.0.48_arm64.deb
I've tried all manner of workarounds, setting USER, SUDO_USER, LOGNAME, running the container with the "-u" switch to my uid/gid and all get the same logname error. Is there a work around for this?
I had the same problem with the latest spinnaker api release.
The issue is that postinst call logname to find out where your home directory is, to install some config files. In the docker build context, there is no logged in user.
My egregious hack was to overwrite the logname executable with "echo root".
e.g.:
# Install spinnaker sdk https://www.flir.com/support-center/iis/machine-vision/downloads/spinnaker-sdk-and-firmware-download/
COPY external/spinnaker/* spinnaker/
# Pre-answer the apt install prompts
COPY spinnaker.dat .
RUN cat spinnaker.dat >> /var/cache/debconf/config.dat
# Fake out logname (no login context in docker build)
RUN echo "echo root" > /usr/bin/logname
# Install other postinst dependencies
RUN DEBIAN_FRONTEND=noninteractive apt install -y iputils-ping wget
RUN DEBIAN_FRONTEND=noninteractive apt install -y --no-install-recommends ./spinnaker/lib*.deb && rm -rv spinnaker
The contents of spinnaker.dat (to avoid being prompted from the preinst script) are:
Name: libspinnaker/accepted-flir-eula
Template: libspinnaker/accepted-flir-eula
Value: true
Owners: libspinnaker
Flags: seen
Name: libspinnaker/error-flir-eula
Template: libspinnaker/error-flir-eula
Owners: libspinnaker
Name: libspinnaker/present-flir-eula
Template: libspinnaker/present-flir-eula
Value:
Owners: libspinnaker
Flags: seen

fastlane - error at google cloud build: "OCI runtime create failed: container_linux.go:345"

I'm using fastlane container that stores at google container registry to upload APK to google play store using Google Cloud Build.
APK has been succesfully created.However, when processing last step (fastlane), it face errors:
Step #2: 487ea6dabc0c: Pull complete
Step #2: a7ae4fee33c9: Pull complete
Step #2: Digest: sha256:2e31d5ae64984a598856f1138c6be0577c83c247226c473bb5ad302f86267545
Step #2: Status: Downloaded newer image for gcr.io/myapp789-app/fastlane:latest
Step #2: gcr.io/myapp789-app/fastlane:latest
Step #2: docker: Error response from daemon: OCI runtime create failed: container_linux.go:345: starting container process caused "exec: \"supply\": executable file not found in $PATH": unknown.
Step #2: time="2019-08-29T23:22:55Z" level=error msg="error waiting for container: context canceled"
Finished Step #2
ERROR
ERROR: build step 2 "gcr.io/myapp789-app/fastlane" failed: exit status 127
Note:
1) Docker Source file was taken from https://hub.docker.com/r/fastlanetools/fastlane and then I build my own image.
2) Docker Image Build on Google Cloud VM using Debian GNU/Linux 9 (stretch)
Docker Source File for fastlane:
# Final image #
###############
FROM circleci/ruby:latest
MAINTAINER milch
ENV PATH $PATH:/usr/local/itms/bin
# Java versions to be installed
ENV JAVA_VERSION 8u131
ENV JAVA_DEBIAN_VERSION 8u131-b11-1~bpo8+1
ENV CA_CERTIFICATES_JAVA_VERSION 20161107~bpo8+1
# Needed for fastlane to work
ENV LANG C.UTF-8
ENV LC_ALL C.UTF-8
# Required for iTMSTransporter to find Java
ENV JAVA_HOME /usr/lib/jvm/java-8-openjdk-amd64/jre
USER root
# iTMSTransporter needs java installed
# We also have to install make to install xar
# And finally shellcheck
RUN echo 'deb http://archive.debian.org/debian jessie-backports main' > /etc/apt/sources.list.d/jessie-
backports.list \
&& apt-get -o Acquire::Check-Valid-Until=false update \
&& apt-get install --yes \
make \
shellcheck \
&& apt-get clean \
&& rm -rf /var/lib/apt/lists/*
USER circleci
COPY --from=xar_builder /tmp/xar /tmp/xar
RUN cd /tmp/xar \
&& sudo make install \
&& sudo rm -rf /tmp/*
CloudBuild.yaml:
- name: 'gcr.io/$PROJECT_ID/fastlane'
args: ['supply', '--package_name','${_ANDROID_PACKAGE_NAME}', '--track', '${_ANDROID_RELEASE_CHANNEL}', '--json_key_data', '${_GOOGLE_PLAY_UPLOAD_KEY_JSON}', '--apk', '/workspace/${_REPO_NAME}/build/app/outputs/bundle/release/app.aab']
timeout: 1200s
Any Idea to solve this?
I solve this by building docker image using docker source from Google Cloud Official other than fastlane on hub.docker.com (where's it never update since 5 month ago)

Running sbt in docker as non-root user

Trying to create a docker image that has sbt installed and can build sbt projects but, when building, will not be running as the root user (this is all in the context of running Jenkins inside docker).
Dockerfile sets up sbt
ENV SBT_VERSION=1.1.6
RUN \
curl -L -o sbt-$SBT_VERSION.deb http://dl.bintray.com/sbt/debian/sbt-$SBT_VERSION.deb && \
dpkg -i sbt-$SBT_VERSION.deb && \
rm sbt-$SBT_VERSION.deb && \
apt-get update && \
apt-get install sbt && \
sbt sbtVersion
And if I then run sbt as the root user, all works ok
docker exec -u root myjenkins sbt sbtVersion
produces
[warn] No sbt.version set in project/build.properties, base directory: /
[info] Set current project to root (in build file:/)
[info] 1.1.6
But when I run sbt as the jenkins user, it tries to download sbt 1.1.6 again and eventually fails when it tries to modify an apt system file.
docker exec -u jenkins myjenkins sbt sbtVersion
produces:
Getting org.scala-sbt sbt 1.1.6 (this may take some time)...
downloading https://repo1.maven.org/maven2/org/scala-sbt/sbt/1.1.6/sbt-1.1.6.jar ...
[SUCCESSFUL ] org.scala-sbt#sbt;1.1.6!sbt.jar (68ms)
.
.
.
[warn] No sbt.version set in project/build.properties, base directory: /
[error] java.io.FileNotFoundException: /var/cache/apt/archives/lock (Permission denied)
I understand that all of the "RUN" commands in your Dockerfile are as a root user.
SBT downloading Scala: Check where it is downloading. SBT by default downloads dependencies on ~/.ivy2 (and/or ~/.m2). If you change user, your home also changes, so it will look for dependencies in /home/jenkins/.ivy2, then on .ivy2 (double-check on this), which do not have those dependencies downloaded already, so it tries to download them.
About the var/cache/apt/archives/lock, it is trying to install via SBT via apt with your jenkins user, when you need to be privileged user to use apt. Your app-user should not need to install anything (or anything that requires root access), but rather build an image with all required installs and then use it as a separate user. Also, if apt gives you headaches, you can just install via download into folder, something like:
RUN \
curl -fsL http://downloads.typesafe.com/scala/$SCALA_VERSION/scala-$SCALA_VERSION.tgz | tar xfz - -C /usr/local && \
ln -s /usr/local/scala-$SCALA_VERSION/bin/* /usr/local/bin/
PS: You may want to run your container always as jenkins user, in that case you can use USER jenkins after you finished installations and do any additional unprivileged operations there.

Docker: transaction check error

I am trying to install docker and following https://get.docker.com/
I am on CentOS7
-bash-4.2$ cd
-bash-4.2$ wget -qO- https://get.docker.com/ | sh
+ sudo -E sh -c 'sleep 3; yum -y -q install docker-engine'
Transaction check error:
file /usr/lib/systemd/system/blk-availability.service from install of device-mapper-7:1.02.107-5.el7_2.5.x86_64 conflicts with file from package lvm2-7:2.02.105-14.el7.x86_64
file /usr/sbin/blkdeactivate from install of device-mapper-7:1.02.107-5.el7_2.5.x86_64 conflicts with file from package lvm2-7:2.02.105-14.el7.x86_64
file /usr/share/man/man8/blkdeactivate.8.gz from install of device-mapper-7:1.02.107-5.el7_2.5.x86_64 conflicts with file from package lvm2-7:2.02.105-14.el7.x86_64
Error Summary
-------------
Anyone know what this error means and how to resolve it?
Looks like device-mapper-7 is incompatible with lvm2-7 on your system.
You could try deinstalling device-mapper-7. Then try to run the Docker installation one more time.
I've updated with:
sudo yum update
Now docker command works now

Running docker-compose on a docker gitlab-ci-multi-runner

I have a project running on Docker with docker-compose for dev environment.
I want to get it running on GitLabCI with a gitlab-ci-multi-runner "Docker mode" instance.
Here is my .gitlab-ci.yml file:
image: soullivaneuh/docker-bash
before_script:
- apk add --update bash curl
- curl --silent --location https://github.com/docker/compose/releases/download/1.5.2/docker-compose-`uname -s`-`uname -m` > /usr/local/bin/docker-compose
- chmod +x /usr/local/bin/docker-compose
- ./configure
- docker-compose up -d
Note that soullivaneuh/docker-bash image is just a docker image with bash installed.
The script fails on docker-compose up -d command:
gitlab-ci-multi-runner 0.7.2 (998cf5d)
Using Docker executor with image soullivaneuh/docker-bash ...
Pulling docker image soullivaneuh/docker-bash:latest ...
Running on runner-1ee5079f-project-3-concurrent-1 via sd-59984...
Fetching changes...
Removing app/config/parameters.yml
Removing docker-compose.env
HEAD is now at 5c5e7ff remove docker service
From https://git.dummy.net/project/project
5c5e7ff..45e643d docker-ci -> origin/docker-ci
Checking out 45e643dd as docker-ci...
Previous HEAD position was 5c5e7ff... remove docker service
HEAD is now at 45e643d... Remove docker info commands
$ apk add --update bash curl
fetch http://dl-4.alpinelinux.org/alpine/v3.2/main/x86_64/APKINDEX.tar.gz
OK: 10 MiB in 28 packages
$ curl --silent --location https://github.com/docker/compose/releases/download/1.5.2/docker-compose-`uname -s`-`uname -m` > /usr/local/bin/docker-compose
$ chmod +x /usr/local/bin/docker-compose
$ ./configure
$ docker-compose up -d
bash: line 30: /usr/local/bin/docker-compose: No such file or directory
ERROR: Build failed with: exit code 1
I have absolutly no idea why this is failing.
Thanks for help.
The No such file or directory is misleading. I've received that many times while trying to run dynamically linked binaries using alpine linux (which it appears you are using).
The problem (as I understand it) is that the binary was compiled and linked against glibc, but alpine uses musl, not glibc.
You could use ldd /usr/local/bin/docker-compose to tell you which libraries are missing (or run it with strace if all else fails).
To get it working, it might be easier to install from python source (https://docs.docker.com/compose/install/#install-using-pip), which is what the official compose image does (https://github.com/docker/compose/blob/master/Dockerfile.run).
Or you could use an image built on debian or some other distro that uses glibc.

Resources