I am trying to build an image using singularity. in one step I have to run a R script to do so, in the recipe file I need to install R and I did using the following command:
apt-get install -y systemd systemd-sysv gdebi-core procps libssl1.1 ed wget curl libqt5webkit5 libqt5core5a libxml2-dev r-cran-xml wget libssl-dev curl libcurl4-openssl-dev libnetcdf-dev netcdf-bin libcairo2-dev libxt-dev default-jre texlive-latex-base libhdf5-dev r-base r-base-dev
curl https://download1.rstudio.org/rstudio-xenial-1.1.463-amd64.deb > /rstudio-1.1.463-amd64.deb
apt-get -y install /rstudio-1.1.463-amd64.deb
wget -O /rstudio-server-stretch-1.1.463-amd64.deb \
https://download2.rstudio.org/rstudio-server-stretch-1.1.463-amd64.deb
gdebi -n /rstudio-server-stretch-1.1.463-amd64.deb
and I run the recipe file using this command:
sudo singularity build nanos.sif Singularity.recipe
but after running it, at some point it asks me that which time zones I am located at and here is the message:
Please select the geographic area in which you live. Subsequent configuration
questions will narrow this down by presenting a list of cities, representing
the time zones in which they are located.
1. Africa 6. Asia 11. System V timezones
2. America 7. Atlantic Ocean 12. US
3. Antarctica 8. Europe 13. None of the above
4. Australia 9. Indian Ocean
5. Arctic Ocean 10. Pacific Ocean
Geographic area:
I chose one of them using name and numbers but building did not proceed. do you know how I can fix this problem?
You can set this via the TZ environment variable in %post. e.g., export TZ=UTC. You may also need to set TZ=UTC (or the desired timezone) in %environment as well.
See additional info on timezones in R and a related question: How to change the default time zone in R?
To me the following spell worked
%environment
TZ=UTC
DEBIAN_FRONTEND=noninteractive
%post
export TZ=UTC
export DEBIAN_FRONTEND=noninteractive
apt-get update
apt-get install -y .....
Replace ..... with your packages. No worries about the specific timzone. Normally it should be taken from the host system, so UTC will be overridden unless you instruct singularity otherwise at the run time.
Related
This question already has answers here:
How to fill user input for interactive command for "RUN" command?
(2 answers)
Closed 6 months ago.
While trying to install tzdata into an ubuntu docker image the job would get stuck at this stage.
Unpacking tzdata (2022a-0ubuntu0.18.04) ...
Setting up tzdata (2022a-0ubuntu0.18.04) ...
debconf: unable to initialize frontend: Dialog
debconf: (Dialog frontend will not work on a dumb terminal, an emacs shell buffer, or without a controlling terminal.)
debconf: falling back to frontend: Readline
Configuring tzdata
------------------
Please select the geographic area in which you live. Subsequent configuration
questions will narrow this down by presenting a list of cities, representing
the time zones in which they are located.
1. Africa 4. Australia 7. Atlantic 10. Pacific 13. Etc
2. America 5. Arctic 8. Europe 11. SystemV
3. Antarctica 6. Asia 9. Indian 12. US
Geographic area:
After a lot of trial and error and research i finally managed to fix it by adding the following to the scripts tag in the .gitlab-ci.yml
- echo 'debconf debconf/frontend select Noninteractive' | sudo debconf-set-selections
- DEBIAN_FRONTEND=noninteractive sudo apt-get update && sudo apt-get -y --no-install-recommends install tzdata
- sudo cp /usr/share/zoneinfo/Australia/Brisbane /etc/localtime
I am using this GitHub Action: https://github.com/roles-ansible/check-ansible-ubuntu-focal-action
name: Ansible check ubuntu:focal
on: [push, pull_request]
jobs:
build:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout#v2
- name: ansible check with ubuntu:focal
uses: roles-ansible/check-ansible-ubuntu-focal-action#master
with:
targets: "local.yml"
group: "workstations"
hosts: "localhost"
It hangs and when I cancel the job, I see this in the log:
Setting up tzdata (2021a-0ubuntu0.20.04) ...
debconf: unable to initialize frontend: Dialog
debconf: (TERM is not set, so the dialog frontend is not usable.)
debconf: falling back to frontend: Readline
Configuring tzdata
------------------
Please select the geographic area in which you live. Subsequent configuration
questions will narrow this down by presenting a list of cities, representing
the time zones in which they are located.
1. Africa 4. Australia 7. Atlantic 10. Pacific 13. Etc
2. America 5. Arctic 8. Europe 11. SystemV
3. Antarctica 6. Asia 9. Indian 12. US
Geographic area:
Error: The operation was canceled.
So that's a case of a docker container that waits for interactive input to configure tzdata.
How do I resolve this in my own code? The only solution I can think of so far, is to fork the upstream repo of the GitHub Action, and add this line to the Dockerfile before the RUN apt-get commands:
ARG DEBIAN_FRONTEND=noninteractive
But I want to avoid contributing to upstream if I can solve it on my own.
Any ideas?
For reference, this is the current Dockerfile of the upstream repo of the GitHub Action: https://github.com/roles-ansible/check-ansible-ubuntu-focal-action/blob/master/Dockerfile
FROM ubuntu:focal
LABEL "maintainer"="L3D <l3d#c3woc.de>"
LABEL "repository"="https://github.com/roles-ansible/check-ansible-ubuntu-focal-action.git"
LABEL "homepage"="https://github.com/roles-ansible/check-ansible-ubuntu-focal-action"
LABEL "com.github.actions.name"="check-ansible-ubuntu-focal"
LABEL "com.github.actions.description"="Check ansible role or playbook with Ubuntu focal"
LABEL "com.github.actions.icon"="aperture"
LABEL "com.github.actions.color"="green"
RUN apt-get update -y && apt-get install -y \
software-properties-common \
build-essential \
libffi-dev \
libssl-dev \
python3-dev \
python3-pip \
git \
systemd
RUN pip3 install setuptools && pip3 install ansible
RUN ansible --version
ADD ansible-docker.sh /ansible-docker.sh
ENTRYPOINT ["/ansible-docker.sh"]
There is no way to do it in my own code, the change must be done upstream.
I made a pull request: https://github.com/roles-ansible/check-ansible-ubuntu-focal-action/pull/1/files
This is the diff:
## -9,6 +9,8 ## LABEL "com.github.actions.description"="Check ansible role or playbook with Ubun
9 9 LABEL "com.github.actions.icon"="aperture"
10 10 LABEL "com.github.actions.color"="green"
11 11
12 + ARG DEBIAN_FRONTEND=noninteractive
13 +
12 14 RUN apt-get update -y && apt-get install -y \
13 15 software-properties-common \
14 16 build-essential \
Fortunately my PR got approved and merged within only 3 minutes.
You can try to provide the answer for the package. Every answer to package questions is stored in debian 'selection' database. You can see answers for already installed packets with debconf-get-selections, and you can add answers for packages (which are even not installed yet) using debcinf-set-selections.
In this case you may want to configure tzdata to some reasonable value (like UTC timezone).
This question already has answers here:
How to fill user input for interactive command for "RUN" command?
(2 answers)
Closed 6 months ago.
I am trying to create a docker image with base image as Ubuntu, NodeJS, Git & Google chrome.
This is my dockerfile.
FROM ubuntu:20.04
USER root
WORKDIR /home/app
COPY ./package.json /home/app/package.json
RUN apt-get update
RUN apt-get -y install curl gnupg
RUN apt-get install g++ build-essential --yes
RUN curl -sL https://deb.nodesource.com/setup_15.x | bash -
RUN apt-get -y install nodejs
RUN apt-get install git --yes
# Install Google Chrome
RUN apt-get install wget
RUN wget https://dl.google.com/linux/direct/google-chrome-stable_current_amd64.deb
RUN apt-get install ./google-chrome*.deb --yes
When I build the image, I keep getting stuck the this step.
=> [12/13] RUN apt-get install ./google-chrome*.deb --yes 409.6s
=> => # 1. Africa 6. Asia 11. System V timezones
=> => # 2. America 7. Atlantic Ocean 12. US
=> => # 3. Antarctica 8. Europe 13. None of the above
=> => # 4. Australia 9. Indian Ocean
=> => # 5. Arctic Ocean 10. Pacific Ocean
=> => # Geographic area:
Has anyone had a similar experience and was able to resolve this.
I have finally gotten some clarity on this issue and can provide a work-around for anyone who runs into this issue in the future.
Problem: The geographical area will be requested within the standard ubuntu 20.04 image from Docker because that image has been stripped of the locale settings. Those settings can usually be found by using locale or localectl status in the terminal; however, both of those commands require the presence of the systemd service (which is not included with the standard ubuntu 20.04 image you are using in your FROM statement). In addition to this, you will not be able to easily add systemd to that image as you'll run into an issue with it not being PID1 during start of your docker container.
Solution: The easiest solution I found is to simply change to using an Ubuntu image that already contains systemd. One such image that I am using now is jrei/systemd-ubuntu. So what you could do to prevent that question for region selection on Chrome or any other app that may request it is to replace your current FROM statement with this one: FROM jrei/systemd-ubuntu:20.04.
I solved that issue by configuring the timezone before installing Chrome
ENV TZ=Europe/Madrid RUN echo "Preparing geographic area ..."
RUN ln -snf /usr/share/zoneinfo/$TZ /etc/localtime && echo $TZ > /etc/timezone
I have an AWS EC2 instance that I have scheduled to open at 11:50 PM MST and close at 11:59 PM PST. I have set the timezone of the instance to MST so that I can run a cron job that executes a .sh file at 11:55 PM MST. The cron job is pretty simple: 55 23 * * * sudo bash docker run --mount type=bind,source="/home/ec2-user/environment/Project",target="/Project" myubuntu. The docker will mount to a local folder "Project" that contains a .cpp file that web scrapes data from Steam's user information page. The code within the .cpp file is very reliant on the current time/date, hence why I have gone through so much work to get everything running in MST so that everything is standard throughout. However, even with everything running on MST, when the docker container is running it is not in MST despite the dockerfile stating to run with ENV TZ="America/Salt Lake City", I have since changed it from Salt Lake City to Phoenix just to try it out but it still doesn't run the docker in MST. For example, when I run the docker at 9:22 PM MST Nov 24th, the date within the docker is 04:22 AM UTC Nov 25th. This slight date and time change is greatly affecting the code I am trying to run.
To kind of explain what the code does, Steam has a .json URL that holds about 48-62 hours worth of data in "[unix epoch time, # users logged in]". The goal is automation so I figured if I had the code cut out any data that did not match the date the code was run at, it would not be included in the data collection. So I am collecting 24 hours worth of data at a time by running the code at the very end of the day every single day. The difference in date/time between the MST time that both I and my EC2 instance are running on and the UTC time my docker is running on is causing data collection issues.
I was given the dockerfile by my professor, and it supposedly is set up to run on MST but it is not from what I can tell. I have tried to run my command within my .sh file with the included -v /etc/timezone:/etc/timezone but that does not seem to fix the timezone issue either. The dockerfile I was given is below:
# This image will be based on the ubuntu image. Build it using the
# command in the comment below. You may omit the ubuntu pull if you already
# did it. Before running the build, change into the directory containing
# this Dockerfile.
FROM ubuntu
# Set a default shell.
SHELL ["/bin/bash", "-c"]
# The timezone library stops the docker build and waits for user input to
# select a timezone. This breaks the build. To get around this,
# set up timezone information prior to installing that library. This
# Docker code does that. Composited from two sources:
# https://rtfm.co.ua/en/docker-configure-tzdata-and-timezone-during-build/
# https://serverfault.com/questions/949991/how-to-install-tzdata-on-a-ubuntu-docker-image
ARG DEBIAN_FRONTEND=noninteractive
ENV TZ="America/Phoenix"
# Install a handful of useful libraries. Note that this is generally against
# best practices -- containers should be lean, and not include 'things you
# might use'. In our case, we're using the containers for development, so
# this may be reasonable here.
RUN apt-get -y update && apt-get -y install \
apt-utils \
emacs-nox \
g++
# Copy in the files from the current folder (recursively) For our purposes,
# put them in /cs3505
COPY . /cs3505
RUN apt-get -y install wget
Is there something I, or my professor, has done wrong in the setup of the docker to cause this timezone issue? How can I go about fixing my docker so that every time it runs at 11:55 PM MST it opens up with MST as the timezone?
Edit: I do not know if this makes a difference but running cat /etc/timezone returns "United States/Mountain" and running emacs /etc/timezone shows the same thing.
This is a dockerfile I customized based on Debian, you can refer to it：
FROM debian:stable-slim
ARG ARG_TIMEZONE=Asia/Shanghai
ENV ENV_TIMEZONE ${ARG_TIMEZONE}
# install base dependence
RUN echo 'debconf debconf/frontend select Noninteractive' | debconf-set-selections \
&& apt-get update && apt-get install -y -q \
dialog apt-utils \
locales systemd cron \
vim wget curl exuberant-ctags tree \
tzdata ntp ntpstat ntpdate \
&& apt-get clean && apt-get autoremove -y && rm -rf /var/lib/apt/lists/* \
&& localedef -i en_US -c -f UTF-8 -A /usr/share/locale/locale.alias en_US.UTF-8
# sync timezone
RUN echo '$ENV_TIMEZONE' > /etc/timezone \
&& ln -fsn /usr/share/zoneinfo/$ENV_TIMEZONE /etc/localtime \
&& dpkg-reconfigure --frontend noninteractive tzdata
We use Docker to well define the build environment and help with deterministic builds but on my machine I get a tiny change in the build results using Docker but not when not using Docker.
I did pretty extensive testing and am out of ideas :(
I tested on the following systems:
A: My new PC without Docker
AD1: My new PC with Docker, using our Dockerfile based on ubuntu:18.04 compiled "a year ago"
AD2: My new PC with Docker, using our Dockerfile based on ubuntu:19:10 compiled now
B: My laptop (that I had copied the disk from to my new PC) without Docker
BD: My laptop with Docker
CD1: Co-worker's laptop with Docker, using our Dockerfile based on ubuntu:18.04 compiled "a year ago"
CD2: Co-worker's laptop with Docker, using our Dockerfile based on ubuntu:19:10 compiled now
DD: A Digital Ocean VPS with our Dockerfile based on ubuntu:18.04 compiled now
In all scenarios we got either of two build results I will name variant X and Y.
We got variant X using A, B, CD1, CD2 and DD.
We got variant Y using AD1, AD2 and BD.
The issue keeps being 100% reproducible since several releases of our Android app. It did not go away when I updated my Docker from 19.03.6 to 19.03.8 to match my co-worker's version. We both had Ubuntu 19.10 back then and I now keep getting the issue with Ubuntu 20.04.
I always freshly cloned our project into a new folder, used disorderfs to eliminate file system sorting issues and mounted the folder into the docker container.
I doubt it's relevant but we are using this Dockerfile:
FROM ubuntu:18.04
RUN dpkg --add-architecture i386 && \
apt-get update -y && \
apt-get install -y software-properties-common && \
apt-get update -y && \
apt-get install -y wget \
openjdk-8-jre-headless=8u162-b12-1 \
openjdk-8-jre=8u162-b12-1 \
openjdk-8-jdk-headless=8u162-b12-1 \
openjdk-8-jdk=8u162-b12-1 \
git unzip && \
rm -rf /var/lib/apt/lists/* && \
apt-get autoremove -y && \
apt-get clean
# download and install Android SDK
ARG ANDROID_SDK_VERSION=4333796
ENV ANDROID_HOME /opt/android-sdk
RUN mkdir -p /opt/android-sdk && cd /opt/android-sdk && \
wget -q https://dl.google.com/android/repository/sdk-tools-linux-${ANDROID_SDK_VERSION}.zip && \
unzip *tools*linux*.zip && \
rm *tools*linux*.zip && \
yes | $ANDROID_HOME/tools/bin/sdkmanager --licenses
Also here are the build instructions I run and get different results. The diff itself is can be found here.
Edit: I also filed it as a bug on the docker repo.
Docker is not fully architecture-independent. For different architecture you may have more or less minute differences. Usually it should not affect anything important but may change some optimisation decisions of a compiler and such things. It is more visible if you try very different CPUs like AMD64 vs ARM. For Java it should not matter but it seems that at least sometimes it matters.
Another thing is network and DNS. When you do apt-get, wget and other such things then it downloads code or binary from network. It may differ depending on which DNS you use (which may lead to different server or different repo url) and there can be some minute differences. Theoretically there should be no difference but practically there can be difference sometimes like when they roll out new version and it's visible only on some nodes or something bad happened or you have some cache/proxy in between and connect through that and it caches etc.
Also the latter can create differences that appear in time. Like app is compiled on one month and someone tries to verify few weeks or months later and apt-get installs other versions of libraries and in effect there are minute differences.
I'm not sure which applies here but I have some ideas:
may try to make some small changes to the app so in effect it will again build same on most of popular CPU's, do extensive testing, and then list architectures on which it can be verified
make verification process a little more complex and non-free so users should have to run a server instance (on AWS or Google or Azure or Rackspace or other) with specified architecture and build and verify there - may try and specify on which types of machines exacly result will be the same and what are minimal requirements (as it may or may not run on free-plan instances)
check diff of created images content (not only apk but full system image), maybe there will be something important that differs between docker images on different machines producing different results
try to find as small as possible initial image and don't allow apt-get or other things automatically install dependencies with newest version but specify all dependencies and their versions