I tried to run a simple python UDP echo-server listening on Port 507 inside a docker container that uses a non-root user.
The Dockerfile looks like this:
FROM docker.io/centos
RUN yum -y install iputils iproute
COPY echo-server.py /tmp/
USER 1000
CMD ["python", "/tmp/echo-server.py"]
Since 507 is a well-known port, I also added the capability NET_BIND_SERVICE when issuing docker run but I still get an error:
# docker run --cap-add=NET_BIND_SERVICE 4d1c2301b166
Traceback (most recent call last):
File "/tmp/echo-server.py", line 12, in <module>
s.bind(('', port))
File "/usr/lib64/python2.7/socket.py", line 224, in meth
return getattr(self._sock,name)(*args)
socket.error: [Errno 13] Permission denied
When inspecting the capabilities I could see that the effective capabilities are not set when using a non-root user.
[root#srv-tcn-01 ha-service]# docker run --cap-add=NET_BIND_SERVICE 4d1c2301b166 grep Cap /proc/self/status
CapInh: 00000000a80425fb
CapPrm: 0000000000000000
CapEff: 0000000000000000
CapBnd: 00000000a80425fb
Does anyone know how to run a program in a Docker container with a non-root user and certain capabilities?
Related
I’m trying to run CNN training from Graphcore’s examples repo as a non-root user from Graphcore’s TensorFlow 1.5 Docker image, but it’s throwing:
2020-04-23 11:17:32.960014: I tensorflow/compiler/jit/xla_compilation_cache.cc:250] Compiled cluster using XLA! This line is logged at most once for the lifetime of the process.Saved checkpointto ./logs/RN152_bs1x16p_GN32_16.16_v1.1.11_6LT/ckpt-0
2020-04-23 11:19:07.615030: W tensorflow/core/framework/op_kernel.cc:1651] OP_REQUIRES failed at xla_ops.cc:361 : Unknown: [Error][Build graph] could not get temporary file for model 'MappedCodelet_%%%%%%%%%%%%%%.cpp': Permission denied
Traceback (most recent call last):
File "/usr/local/lib/python3.6/dist-packages/tensorflow_core/python/client/session.py", line 1365, in _do_call
return fn(*args)
File "/usr/local/lib/python3.6/dist-packages/tensorflow_core/python/client/session.py", line 1350, in _run_fn target_list,run_metadata)
File "/usr/local/lib/python3.6/dist-packages/tensorflow_core/python/client/session.py", line 1443, in _call_tf_sessionrun run_metadata) tensorflow.python.framework.errors_impl.UnknownError: [Error][Build graph] could not get temporary file for model 'MappedCodelet_%%%%%%%%%%%%%%.cpp': Permission denied
[[{{node cluster}}]]
The program works fine when I’m running it as root user, but when I create a new user it starts throwing this error. Does this mean Graphcore’s Docker images only work if you’re using root?
It is possible to run IPU programs as a non-root user. The reason you're seeing this behaviour is because switching user within a running Docker container (and any Ubuntu based environment) causes environment variables to be reset. These environment variables contain important IPU configuration settings required to attach to and run a program on an IPU. You can avoid this behaviour by instead doing your user management in a Dockerfile. Below is a sample snippet (where examples is a clone of https://github.com/graphcore/examples/):
FROM graphcore/tensorflow:1
ENV LC_ALL=C.UTF-8
ENV LANG=C.UTF-8
RUN adduser [username]
ADD examples examples
RUN chown [username] -R examples
Then you can build the image with:
docker image build . -t graphcore-examples
Now you have 3 options to run the CNN training as a non-root user:
Run the CNN training directly:
gc-docker -- -ti -u [username] graphcore-examples python3 /examples/applications/tensorflow/cnns/training/train.py
Launch the container into a bash shell as the non-root user and then run the training from there:
gc-docker -- -ti -u [username] graphcore-examples
$ python3 /examples/applications/tensorflow/cnns/training/train.py
Launch the container as root and then preserve the environment when switching user:
gc-docker -- -ti graphcore-examples
$ su --preserve-environment - [username]
$ python3 /examples/applications/tensorflow/cnns/training/train.py
I’d recommend using option 1 or 2 where possible. You can find more information about the gc-docker command line tool here.
I cannot, for the love of it, start Jupyter from inside a Docker container. My OS:
Software:
System Software Overview:
System Version: macOS 10.13.6 (17G5019)
Kernel Version: Darwin 17.7.0
Boot Volume: Macintosh HD
Boot Mode: Normal
Secure Virtual Memory: Enabled
System Integrity Protection: Enabled
Time since boot: 1:10
Dockerfile:
FROM tensorflow/tensorflow:latest-py3-jupyter
LABEL maintainer="xxxx"
ADD Mask_RCNN/ /Mask_RCNN/
ADD startup.sh /
RUN apt-get -y update
WORKDIR /
RUN pip3 install -r /Mask_RCNN/requirements.txt
RUN cd /Mask_RCNN/ && python3 setup.py install
CMD /startup.sh
The file startup.sh is simply
#!/bin/sh
/bin/bash -c "jupyter notebook --allow-root --no-browser --NotebookApp.token='sometoken'"
The Docker image is built without an hassle, with the following command:
docker image build --build-arg http_proxy=someproxy --build-arg https_proxy=someproxy --build-arg no_proxy=localhost -t mask-rcnn:v20190308 .
I run the container with
docker container run -e http_proxy=someproxy -e https_proxy=someproxy -e no_proxy=localhost --rm -it --name mask-rcnn -p 6067:8888 mask-rcnn:v20190308
and I get the following error:
[I 10:44:10.991 NotebookApp] Writing notebook server cookie secret to /root/.local/share/jupyter/runtime/notebook_cookie_secret
Traceback (most recent call last):
File "/usr/local/bin/jupyter-notebook", line 10, in <module>
sys.exit(main())
File "/usr/local/lib/python3.5/dist-packages/jupyter_core/application.py", line 266, in launch_instance
return super(JupyterApp, cls).launch_instance(argv=argv, **kwargs)
File "/usr/local/lib/python3.5/dist-packages/traitlets/config/application.py", line 657, in launch_instance
app.initialize(argv)
File "</usr/local/lib/python3.5/dist-packages/decorator.py:decorator-gen-7>", line 2, in initialize
File "/usr/local/lib/python3.5/dist-packages/traitlets/config/application.py", line 87, in catch_config_error
return method(app, *args, **kwargs)
File "/usr/local/lib/python3.5/dist-packages/notebook/notebookapp.py", line 1628, in initialize
self.init_webapp()
File "/usr/local/lib/python3.5/dist-packages/notebook/notebookapp.py", line 1407, in init_webapp
self.http_server.listen(port, self.ip)
File "/usr/local/lib/python3.5/dist-packages/tornado/tcpserver.py", line 143, in listen
sockets = bind_sockets(port, address=address)
File "/usr/local/lib/python3.5/dist-packages/tornado/netutil.py", line 168, in bind_sockets
sock.bind(sockaddr)
OSError: [Errno 99] Cannot assign requested address
What's happening? I mapped the Jupyter notebook port (8888) to a container port (6067), thus I don't understand what's the problem.
Apparently, just modifying the startup.sh script to
/bin/bash -c "jupyter notebook --ip 0.0.0.0 --allow-root --no-browser --NotebookApp.token='sometoken'"
fixed everything.
I have a spark and airflow cluster which is built with docker swarm. Airflow container cannot contain spark-submit as I expect.
I am using following images which exist in github
Spark: big-data-europe/docker-hadoop-spark-workbench
Airflow: puckel/docker-airflow (CeleryExecutor)
I prepared a .py file and add it under dags folder.
from airflow import DAG
from airflow.contrib.operators.spark_submit_operator import SparkSubmitOperator
from datetime import datetime, timedelta
args = {'owner': 'airflow', 'start_date': datetime(2018, 9, 24) }
dag = DAG('spark_example_new', default_args=args, schedule_interval="#once")
operator = SparkSubmitOperator(task_id='spark_submit_job', conn_id='spark_default', java_class='Main', application='/SimpleSpark.jar', name='airflow-spark-example',conf={'master':'spark://master:7077'},
dag=dag)
I also configure the connection as folows in web site:
Master is the hostname of spark master container.
But it does not find the spark-submit, it produces following error:
[2018-09-24 08:48:14,063] {{logging_mixin.py:95}} INFO - [2018-09-24 08:48:14,062] {{spark_submit_hook.py:283}} INFO - Spark-Submit cmd: ['spark-submit', '--master', 'spark://master:7077', '--conf', 'master=spark://master:7077', '--name', 'airflow-spark-example', '--class', 'Main', '--queue', 'root.default', '/SimpleSpark.jar']
[2018-09-24 08:48:14,067] {{models.py:1736}} ERROR - [Errno 2] No such file or directory: 'spark-submit': 'spark-submit'
Traceback (most recent call last):
File "/usr/local/lib/python3.6/site-packages/airflow/models.py", line 1633, in _run_raw_task
result = task_copy.execute(context=context)
File "/usr/local/lib/python3.6/site-packages/airflow/contrib/operators/spark_submit_operator.py", line 168, in execute
self._hook.submit(self._application)
File "/usr/local/lib/python3.6/site-packages/airflow/contrib/hooks/spark_submit_hook.py", line 330, in submit
**kwargs)
File "/usr/local/lib/python3.6/subprocess.py", line 709, in __init__
restore_signals, start_new_session)
File "/usr/local/lib/python3.6/subprocess.py", line 1344, in _execute_child
raise child_exception_type(errno_num, err_msg, err_filename)
FileNotFoundError: [Errno 2] No such file or directory: 'spark-submit': 'spark-submit'
As far as I know puckel/docker-airflow uses Python slim image(https://hub.docker.com/_/python/). This image does not contain the common packages and only contains the minimal packages needed to run python. Hence, you will need to extend the image and install spark-submit on your container.
Edit: Airflow does need spark binaries in the container to run SparkSubmitOperator as documented here.
The other approach you can use is to use SSHOperator to run spark-submit command on an external VM by SSHing into a remote machine. But here as well SSH should be available which isn't available in Puckel Airflow.
This is late answer
you should install apache-airflow-providers-apache-spark
So you should create a file called 'requirements.txt'
add apache-airflow-providers-apache-spark in the file requirements.txt
Create a Dockerfile like this
FROM apache/airflow:2.2.3
# Install OpenJDK-11
RUN apt update && \
apt-get install -y openjdk-11-jdk && \
apt-get install -y ant && \
apt-get clean;
# Set JAVA_HOME
ENV JAVA_HOME /usr/lib/jvm/java-11-openjdk-amd64/
RUN export JAVA_HOME
USER airflow
COPY requirements.txt .
RUN pip install -r requirements.txt
in the docker-compose.yml comment the line :
# image: ${AIRFLOW_IMAGE_NAME:-apache/airflow:2.2.3}
and uncomment the line build .
Finally run
docker-compose build
docker-compose up
I have the following Python script, example.py:
import redis
r = redis.Redis()
r.set('a','b')
and the following Dockerfile:
FROM ubuntu:18.04
# Install system-wide dependencies
RUN apt-get -yqq update
RUN apt-get -yqq install python3-dev python3-pip
RUN apt-get -yqq install redis-tools redis-server
RUN pip3 install redis
# Copy application code
ADD . /usr/local/example
WORKDIR /usr/local/example
# Start application
CMD /etc/init.d/redis-server restart \
&& python3 example.py
After I build the container (docker build -t redis-example .) and run it (docker run -P -it -d redis-example), the following stack trace is printed:
Stopping redis-server: redis-server.
Starting redis-server: redis-server.
Traceback (most recent call last):
File "/usr/local/lib/python3.6/dist-packages/redis/connection.py", line 484, in connect
sock = self._connect()
File "/usr/local/lib/python3.6/dist-packages/redis/connection.py", line 541, in _connect
raise err
File "/usr/local/lib/python3.6/dist-packages/redis/connection.py", line 529, in _connect
sock.connect(socket_address)
OSError: [Errno 99] Cannot assign requested address
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/local/lib/python3.6/dist-packages/redis/client.py", line 667, in execute_command
connection.send_command(*args)
File "/usr/local/lib/python3.6/dist-packages/redis/connection.py", line 610, in send_command
self.send_packed_command(self.pack_command(*args))
File "/usr/local/lib/python3.6/dist-packages/redis/connection.py", line 585, in send_packed_command
self.connect()
File "/usr/local/lib/python3.6/dist-packages/redis/connection.py", line 489, in connect
raise ConnectionError(self._error_message(e))
redis.exceptions.ConnectionError: Error 99 connecting to localhost:6379. Cannot assign requested address.
I'm not sure why this very simple example is failing in Docker. I'd like to run an instance of a Redis server local to the Docker container.
It does not need to be reachable outside of the container and should be unique to the container.
How can I resolve this exception?
Running Raku (previously aka Perl 6) kernel in Jupyter notebook would be great for reproducibility and ease of use (personal view).
I wanted to run the Perl 6 notebook in a docker container and access it in my web browser. For this I created this docker image.
The code to create the docker image was:
FROM sumankhanal/rakudo:2019.07.1
RUN apt-get update \
&& apt-get install -y ca-certificates python3-pip && pip3 install jupyter notebook \
&& zef install Jupyter::Kernel --force-test
ENV TINI_VERSION v0.6.0
ADD https://github.com/krallin/tini/releases/download/${TINI_VERSION}/tini /usr/bin/tini
RUN chmod +x /usr/bin/tini
ENTRYPOINT ["/usr/bin/tini", "--"]
EXPOSE 8888
CMD ["jupyter", "notebook", "--port=8888", "--no-browser", "--ip=0.0.0.0"]
I am on Windows 10 64 bit and IP address of my docker is 192.168.99.100.
When I tried to run the container with this code:
docker run -it -p 8888:8888 sumankhanal/raku-notebook in the docker terminal
I get this error:
$ docker run -it -p 8888:8888 sumankhanal/raku-notebook
[I 14:26:43.598 NotebookApp] Writing notebook server cookie secret to /root/.local/share/jupyter/runtime/notebook_cookie_secret
Traceback (most recent call last):
File "/usr/local/bin/jupyter-notebook", line 11, in <module>
sys.exit(main())
File "/usr/local/lib/python3.5/dist-packages/jupyter_core/application.py", line 267, in launch_instance
return super(JupyterApp, cls).launch_instance(argv=argv, **kwargs)
File "/usr/local/lib/python3.5/dist-packages/traitlets/config/application.py", line 657, in launch_instance
app.initialize(argv)
File "<decorator-gen-7>", line 2, in initialize
File "/usr/local/lib/python3.5/dist-packages/traitlets/config/application.py", line 87, in catch_config_error
return method(app, *args, **kwargs)
File "/usr/local/lib/python3.5/dist-packages/notebook/notebookapp.py", line 1296, in initialize
self.init_webapp()
File "/usr/local/lib/python3.5/dist-packages/notebook/notebookapp.py", line 1120, in init_webapp
self.http_server.listen(port, self.ip)
File "/usr/local/lib/python3.5/dist-packages/tornado/tcpserver.py", line 142, in listen
sockets = bind_sockets(port, address=address)
File "/usr/local/lib/python3.5/dist-packages/tornado/netutil.py", line 197, in bind_sockets
sock.bind(sockaddr)
OSError: [Errno 99] Cannot assign requested address
Any help ?
You need to add --allow-root in the CMD of your Dockerfile. Also you need to link the kernel with jupyter in your dockerfile
jupyter-kernel.p6 --generate-config
Once you do that you will be able to see the dockerfile. I also noticed that your images size is very huge, you should try and find a better base image for jupyter rather than the one you have.
For more details about installing kernels refer to the below link
https://jupyter.readthedocs.io/en/latest/install-kernel.html