I have CircleCI pipeline setup for my test flow using Jest snapshot and one of my snapshot tests keeps failing. I use Javascript to generate a Date object (new Date("YYYY-MM-DD")) and locally it yields MM/DD/YYYY but in the docker image (node:8) it yields YYYY-MM-DD instead so the snapshot test fails. I have tried to set up locales by:
docker:
- image: circleci/node:8
environment:
TZ: "America/Los_Angeles"
LANG: en_US.UTF-8
LANGUAGE: en_US.UTF-8
LC_ALL: en_US.UTF-8
But it complains it cannot set the default locale so I added:
- run:
name: Reconfigure Locale
command: sudo dpkg-reconfigure locales
which seemed to be a solution for most of the people that had the same problem but not my case.
I also tried to have the same local docker image and test it there and it worked fine with these commands:
RUN apt-get update && DEBIAN_FRONTEND=noninteractive apt-get install -y locales
RUN sed -i -e 's/# en_US.UTF-8 UTF-8/en_US.UTF-8 UTF-8/' /etc/locale.gen && \
dpkg-reconfigure --frontend=noninteractive locales && \
update-locale LANG=en_US.UTF-8
So I tried these in circleci and sed commands complained about permissions even when it is called with sudo.
Okay just FYI, it was the node version caused the date format issue. I installed full-icu npm package which handles locale for the node application. To re-visit my problem, I had successfully installed locale and set it to be the same as the local machine but Node won't pick the locale from the system but from the browser being used. I hope this info helps.
Related
I created a custom image with the following Dockerfile:
FROM apache/airflow:2.1.1-python3.8
USER root
RUN apt-get update \
&& apt-get -y install gcc gnupg2 \
&& curl https://packages.microsoft.com/keys/microsoft.asc | apt-key add - \
&& curl https://packages.microsoft.com/config/debian/10/prod.list > /etc/apt/sources.list.d/mssql-release.list
RUN apt-get update \
&& ACCEPT_EULA=Y apt-get -y install msodbcsql17 \
&& ACCEPT_EULA=Y apt-get -y install mssql-tools
RUN echo 'export PATH="$PATH:/opt/mssql-tools/bin"' >> ~/.bashrc \
&& echo 'export PATH="$PATH:/opt/mssql-tools/bin"' >> ~/.bashrc \
&& source ~/.bashrc
RUN apt-get -y install unixodbc-dev \
&& apt-get -y install python-pip \
&& pip install pyodbc
RUN echo -e “AIRFLOW_UID=$(id -u) \nAIRFLOW_GID=0” > .env
USER airflow
The image creates successfully, but when I try to run it, I get this error:
"airflow command error: the following arguments are required: GROUP_OR_COMMAND, see help above."
I have tried supplying a group ID with the --user, but I can't figure it out.
How can I start this custom Airflow Docker image?
Thanks!
First of all this line is wrong:
RUN echo -e “AIRFLOW_UID=$(id -u) \nAIRFLOW_GID=0” > .env
If you are running it with Docker Compose (I presume you took it from https://airflow.apache.org/docs/apache-airflow/stable/start/docker.html), this is something you should run on "Host" machine, not in the image. Remove that line, it has no effect.
Secondly - it really depends what "command" you run. The "GROUP_OR_COMMAND" message you got is the output of "airflow" command. You have not copied the whole output of your command but this is a message you get when you try to run airflow without telling it what to do. When you run the image you will run by default the airflow command which has a number of subcommands that can be executed. So the "see help above" message tells you the very thing you should do - look at the help and see what subcommand you wanted to run (and possibly run it).
docker run -it apache/airflow:2.1.2
usage: airflow [-h] GROUP_OR_COMMAND ...
positional arguments:
GROUP_OR_COMMAND
Groups:
celery Celery components
config View configuration
connections Manage connections
dags Manage DAGs
db Database operations
jobs Manage jobs
kubernetes Tools to help run the KubernetesExecutor
pools Manage pools
providers Display providers
roles Manage roles
tasks Manage tasks
users Manage users
variables Manage variables
Commands:
cheat-sheet Display cheat sheet
info Show information about current Airflow and environment
kerberos Start a kerberos ticket renewer
plugins Dump information about loaded plugins
rotate-fernet-key
Rotate encrypted connection credentials and variables
scheduler Start a scheduler instance
sync-perm Update permissions for existing roles and optionally DAGs
version Show the version
webserver Start a Airflow webserver instance
optional arguments:
-h, --help show this help message and exit
airflow command error: the following arguments are required: GROUP_OR_COMMAND, see help above.
when you extend the official image, it will pass the parametor to "airflow" command which causing this problem. Check this out: https://airflow.apache.org/docs/docker-stack/entrypoint.html#entrypoint-commands
I have an AWS EC2 instance that I have scheduled to open at 11:50 PM MST and close at 11:59 PM PST. I have set the timezone of the instance to MST so that I can run a cron job that executes a .sh file at 11:55 PM MST. The cron job is pretty simple: 55 23 * * * sudo bash docker run --mount type=bind,source="/home/ec2-user/environment/Project",target="/Project" myubuntu. The docker will mount to a local folder "Project" that contains a .cpp file that web scrapes data from Steam's user information page. The code within the .cpp file is very reliant on the current time/date, hence why I have gone through so much work to get everything running in MST so that everything is standard throughout. However, even with everything running on MST, when the docker container is running it is not in MST despite the dockerfile stating to run with ENV TZ="America/Salt Lake City", I have since changed it from Salt Lake City to Phoenix just to try it out but it still doesn't run the docker in MST. For example, when I run the docker at 9:22 PM MST Nov 24th, the date within the docker is 04:22 AM UTC Nov 25th. This slight date and time change is greatly affecting the code I am trying to run.
To kind of explain what the code does, Steam has a .json URL that holds about 48-62 hours worth of data in "[unix epoch time, # users logged in]". The goal is automation so I figured if I had the code cut out any data that did not match the date the code was run at, it would not be included in the data collection. So I am collecting 24 hours worth of data at a time by running the code at the very end of the day every single day. The difference in date/time between the MST time that both I and my EC2 instance are running on and the UTC time my docker is running on is causing data collection issues.
I was given the dockerfile by my professor, and it supposedly is set up to run on MST but it is not from what I can tell. I have tried to run my command within my .sh file with the included -v /etc/timezone:/etc/timezone but that does not seem to fix the timezone issue either. The dockerfile I was given is below:
# This image will be based on the ubuntu image. Build it using the
# command in the comment below. You may omit the ubuntu pull if you already
# did it. Before running the build, change into the directory containing
# this Dockerfile.
FROM ubuntu
# Set a default shell.
SHELL ["/bin/bash", "-c"]
# The timezone library stops the docker build and waits for user input to
# select a timezone. This breaks the build. To get around this,
# set up timezone information prior to installing that library. This
# Docker code does that. Composited from two sources:
# https://rtfm.co.ua/en/docker-configure-tzdata-and-timezone-during-build/
# https://serverfault.com/questions/949991/how-to-install-tzdata-on-a-ubuntu-docker-image
ARG DEBIAN_FRONTEND=noninteractive
ENV TZ="America/Phoenix"
# Install a handful of useful libraries. Note that this is generally against
# best practices -- containers should be lean, and not include 'things you
# might use'. In our case, we're using the containers for development, so
# this may be reasonable here.
RUN apt-get -y update && apt-get -y install \
apt-utils \
emacs-nox \
g++
# Copy in the files from the current folder (recursively) For our purposes,
# put them in /cs3505
COPY . /cs3505
RUN apt-get -y install wget
Is there something I, or my professor, has done wrong in the setup of the docker to cause this timezone issue? How can I go about fixing my docker so that every time it runs at 11:55 PM MST it opens up with MST as the timezone?
Edit: I do not know if this makes a difference but running cat /etc/timezone returns "United States/Mountain" and running emacs /etc/timezone shows the same thing.
This is a dockerfile I customized based on Debian, you can refer to it：
FROM debian:stable-slim
ARG ARG_TIMEZONE=Asia/Shanghai
ENV ENV_TIMEZONE ${ARG_TIMEZONE}
# install base dependence
RUN echo 'debconf debconf/frontend select Noninteractive' | debconf-set-selections \
&& apt-get update && apt-get install -y -q \
dialog apt-utils \
locales systemd cron \
vim wget curl exuberant-ctags tree \
tzdata ntp ntpstat ntpdate \
&& apt-get clean && apt-get autoremove -y && rm -rf /var/lib/apt/lists/* \
&& localedef -i en_US -c -f UTF-8 -A /usr/share/locale/locale.alias en_US.UTF-8
# sync timezone
RUN echo '$ENV_TIMEZONE' > /etc/timezone \
&& ln -fsn /usr/share/zoneinfo/$ENV_TIMEZONE /etc/localtime \
&& dpkg-reconfigure --frontend noninteractive tzdata
My container has locale settep up to POSIX and I want to change it. After I do that, I exit and reenter the container and the locale is back to POSIX.
I don't want to build a new image or run a new container because we have a lot of containers in several machines.
Running this:
DEBIAN_FRONTEND=noninteractive apt-get install -y locales
sed -i -e 's/# pt_PT ISO-8859-1/pt_PT ISO-8859-1/' /etc/locale.gen
dpkg-reconfigure --frontend=noninteractive locales
export LANGUAGE=pt_PT
export LANG=pt_PT
export LC_ALL=pt_PT
Works great in running container but exiting and reentering the container makes the changes lost.
Already tried this code in container Entrypoint but the export doesn't have any effect.
Those settings are shell-session bound, not OS-bound. To make it OS-bound, you should write it in OS files, but when the service restarts it will apply the image without those changes.
So, that has to be set in Dockerfile, to be image-bound, something like:
RUN DEBIAN_FRONTEND=noninteractive apt-get install -y locales && \
sed -i -e 's/# pt_PT ISO-8859-1/pt_PT ISO-8859-1/' /etc/locale.gen && \
dpkg-reconfigure --frontend=noninteractive locales
ENV LANG pt_PT
ENV LANGUAGE pt_PT
ENV LC_ALL pt_PT
changes can't be stored in the container. I think the best way is to commit your changes into the container and create a new one.
you can use "docker commit" for this purpose.
docker commit
Ref: https://docs.docker.com/engine/reference/commandline/commit/
I'm trying to build a Docker image based on oracle/database:11.2.0.2-xe (which is based on Oracle Linux based on RHEL) and want to change the system locale in this image (using some RUN command inside a Dockerfile).
According to this guide I should use localectl set-locale <MYLOCALE> but this command is failing with Failed to create bus connection: No such file or directory message. This is a known Docker issue for commands that require SystemD to be launched.
I tried to start the SystemD anyway (using /usr/sbin/init as first process as well as using -v /sys/fs/cgroup:/sys/fs/cgroup:ro -v /run thanks to this help) but then the localectl set-locale failed with Could not get properties: Connection timed out message.
So I'm now trying to avoid the usage of localectl to change my system globale locale, how could I do this?
According to this good guide on setting locale on Linux, I should use
localedef -c -i fr_FR -f ISO-8859-15 fr_FR.ISO-8859-15
But this command failed with
cannot read character map directory `/usr/share/i18n/charmaps': No such file or directory`
This SO reply indicated one could use yum reinstall glibc-common -y to fix this and it worked.
So my final working Dockerfile is:
RUN yum reinstall glibc-common -y && \
localedef -c -i fr_FR -f ISO-8859-15 fr_FR.ISO-8859-15 && \
echo "LANG=fr_FR.ISO-8859-15" > /etc/locale.conf
ENV LANG fr_FR.ISO-8859-15
On a normal server e.g. a Linode VPS I would normally do:
localectl set-locale LANG=<locale>.utf8
timedatectl set-timezone <timezone>
But since systemd is not present or does not work on containers I get:
Failed to create bus connection: No such file or directory
Now, my goal is just to change these settings without using systemd but such approach seems to go undocumented. Is there a reference for non-systemd alternatives to config tools?
Some documentation about locale setting in arch wiki: https://wiki.archlinux.org/index.php/locale
In Dockerfile, adjust LANG to your desired locale. You can add more than one locale in /etc/locale.gen to have a choice later.
Works on debian, arch, but locale-gen misses on fedora:
ENV LANG=en_US.utf8
RUN echo "$LANG UTF-8" >> /etc/locale.gen
RUN locale-gen
RUN update-locale --reset LANG=$LANG
More general is localedef, works on fedora, too:
ENV LANG=en_US.UTF-8
localedef --verbose --force -i en_US -f UTF-8 en_US.UTF-8
Put this in your Dockerfile
ENV TZ=America/Denver
RUN ln -snf /usr/share/zoneinfo/$TZ /etc/localtime && echo $TZ > /etc/timezone
Edit .bash_profile or .bashrc from root and add the following.
TZ='Asia/Kolkata'
export TZ
Save file and commit image after its done.
Based on a technique used in sti-base, I came up with the following workaround for https://github.com/ncoghlan/fedbuildenv/blob/09a18d91e7af64a45394669bac2595a4b628960d/Dockerfile#L26:
# Set a useful default locale
RUN echo "export LANG=en_US.utf-8" > /opt/export_LANG.sh
ENV BASH_ENV=/opt/export_LANG.sh \
ENV=/opt/export_LANG.sh \
PROMPT_COMMAND="source /opt/export_LANG.sh"
BASH_ENV covers non-interactive bash sessions, ENV covers sh sessions, and PROMPT_COMMAND covers interactive bash sessions.
this seems to be the debians's equivalent of locale-gen:
RUN localedef -v -c -i fr_FR -f UTF-8 fr_FR.UTF-8 || true