This is my application structure:
project/
├── Dockerfile
├── example_api
│ └── main.py
└── requirements.txt
Here is the Dockerfile
FROM python:3.9-slim
COPY ./example_api /app/src
COPY ./requirements.txt /app
WORKDIR /app
RUN pip3 install -r requirements.txt
EXPOSE 8000
CMD ["uvicorn", "example_api.main:app", "--host=0.0.0.0", "--reload"]
building with
docker build -t siddesh-fastapi:0.1 .
running with
docker run -p 8000:8000 --name siddesh-test-api siddesh-fastapi:0.1
error message
INFO: Will watch for changes in these directories: ['/app']
INFO: Uvicorn running on http://0.0.0.0:8000 (Press CTRL+C to quit)
INFO: Started reloader process [1] using watchgod
Process SpawnProcess-1:
Traceback (most recent call last):
File "/usr/local/lib/python3.9/multiprocessing/process.py", line 315, in _bootstrap
self.run()
File "/usr/local/lib/python3.9/multiprocessing/process.py", line 108, in run
self._target(*self._args, **self._kwargs)
File "/usr/local/lib/python3.9/site-packages/uvicorn/subprocess.py", line 76, in subprocess_started
target(sockets=sockets)
File "/usr/local/lib/python3.9/site-packages/uvicorn/server.py", line 68, in run
return asyncio.run(self.serve(sockets=sockets))
File "/usr/local/lib/python3.9/asyncio/runners.py", line 44, in run
return loop.run_until_complete(main)
File "uvloop/loop.pyx", line 1501, in uvloop.loop.Loop.run_until_complete
File "/usr/local/lib/python3.9/site-packages/uvicorn/server.py", line 76, in serve
config.load()
File "/usr/local/lib/python3.9/site-packages/uvicorn/config.py", line 448, in load
self.loaded_app = import_from_string(self.app)
File "/usr/local/lib/python3.9/site-packages/uvicorn/importer.py", line 24, in import_from_string
raise exc from None
File "/usr/local/lib/python3.9/site-packages/uvicorn/importer.py", line 21, in import_from_string
module = importlib.import_module(module_str)
File "/usr/local/lib/python3.9/importlib/__init__.py", line 127, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
File "<frozen importlib._bootstrap>", line 1030, in _gcd_import
File "<frozen importlib._bootstrap>", line 1007, in _find_and_load
File "<frozen importlib._bootstrap>", line 972, in _find_and_load_unlocked
File "<frozen importlib._bootstrap>", line 228, in _call_with_frames_removed
File "<frozen importlib._bootstrap>", line 1030, in _gcd_import
File "<frozen importlib._bootstrap>", line 1007, in _find_and_load
File "<frozen importlib._bootstrap>", line 984, in _find_and_load_unlocked
ModuleNotFoundError: No module named 'example_api'
I guess I am missing something in the Dockerfile?
Related
tldr-version: I have no idea whats causing this error. im pretty sure its not the line endings because i changed them manually with notepad++ (unless i need to change more than entrypoint.sh because thats all i changed the line endings on).
original post below.
I have no idea what is causing this error caused when i do docker-compose -f docker-compose-deploy.yml up --build into my command line i get the following output
Use 'docker scan' to run Snyk tests against images to find vulnerabilities and learn how to fix them
Starting mygoattheend-copy_app_1 ... done
Starting mygoattheend-copy_proxy_1 ... done
Attaching to mygoattheend-copy_app_1, mygoattheend-copy_proxy_1
app_1 | exec /scripts/entrypoint.sh: no such file or directory
proxy_1 | /docker-entrypoint.sh: /docker-entrypoint.d/ is not empty, will attempt to perform configuration
proxy_1 | /docker-entrypoint.sh: Looking for shell scripts in /docker-entrypoint.d/
proxy_1 | /docker-entrypoint.sh: Launching /docker-entrypoint.d/10-listen-on-ipv6-by-default.sh
proxy_1 | 10-listen-on-ipv6-by-default.sh: info: can not modify /etc/nginx/conf.d/default.conf (read-only file system?)
proxy_1 | /docker-entrypoint.sh: Launching /docker-entrypoint.d/20-envsubst-on-templates.sh
proxy_1 | /docker-entrypoint.sh: Launching /docker-entrypoint.d/30-tune-worker-processes.sh
proxy_1 | /docker-entrypoint.sh: Configuration complete; ready for start up
mygoattheend-copy_app_1 exited with code 1
proxy_1 | 2022/11/03 18:51:39 [emerg] 1#1: host not found in upstream "app" in /etc/nginx/conf.d/default.conf:9
proxy_1 | nginx: [emerg] host not found in upstream "app" in /etc/nginx/conf.d/default.conf:9
mygoattheend-copy_proxy_1 exited with code 1
Other examples of errors that appear when i search for nginx: [emerg] host not found in upstream "app" in /etc/nginx/conf.d/default.conf:9 online suggest the problem is due to missing -depends_on: so ive included my docker-compose file below but i followed the tutorial perfectly and his worked fine. And my docker-compose-deploy has its
-depends_on:
my full docker compose is below
version: '3.7'
services:
app:
build:
context: .
ports:
- "8000:8000"
volumes:
- ./app:/app
command: sh -c "python manage.py runserver 0.0.0.0:8000"
environment:
- DEBUG=1
My full docker-compose-deploy.yml is below
version: '3.7'
services:
app:
build:
context: .
volumes:
- static_data:/vol/web
environment:
- SECRET_KEY=samplesecretkey123
- ALLOWED_HOSTS=127.0.0.1,localhost
proxy:
build:
context: ./proxy
volumes:
- static_data:/vol/static
ports:
- "8080:8080"
depends_on:
- app
volumes:
static_data:
the image below is my full directory
The error does mention that it can't find app, which i copy with the main dockerfile (not the one in the proxy folder)
My main dockerfile is below.
FROM python:3.8-alpine
ENV PATH="/scripts:${PATH}"
COPY ./requirements.txt /requirements.txt
RUN apk add --update --no-cache --virtual .tmp gcc libc-dev linux-headers
RUN pip install -r /requirements.txt
RUN apk del .tmp
RUN mkdir /app
COPY ./app /app
WORKDIR /app
COPY ./scripts /scripts
RUN chmod +x /scripts/*
RUN mkdir -p /vol/web/media
RUN mkdir -p /vol/web/
RUN adduser -D user
RUN chown -R user:user /vol
RUN chmod -R 755 /vol/web
USER user
CMD ["entrypoint.sh"]
what could be causing this error?
what other info do you need to work it out?
im following this tutorial. im at the very end https://www.youtube.com/watch?v=nh1ynJGJuT8
update 1 (adding extra info)
my proxy/default.conf is below
server {
listen 8080;
location /static {
alias /vol/static;
}
location / {
uwsgi_pass app:8000;
include /etc/nginx/uwsgi_params;
}
}
my proxy/dockerfile is below
FROM nginxinc/nginx-unprivileged:1-alpine
COPY ./default.conf /etc/nginx/conf.d/default.conf
COPY ./uwsgi_params /etc/nginx/uwsgi_params
USER root
RUN mkdir -p /vol/static
RUN chmod 755 /vol/static
USER nginx
update 2
this is my whole project uploaded to github https://github.com/tgmjack/help
update 3
editing the line endings in vscode didnt appear to work.
update 4
new dockerfile trying dos2unix
FROM python:3.8-alpine
ENV PATH="/scripts:${PATH}"
COPY ./requirements.txt /requirements.txt
RUN apk add --update --no-cache --virtual .tmp gcc libc-dev linux-headers dos2unix
RUN pip install -r /requirements.txt
RUN apk del .tmp
RUN mkdir /app
COPY ./app /app
WORKDIR /app
COPY ./scripts /scripts
RUN chmod +x /scripts/*
RUN mkdir -p /vol/web/media
RUN mkdir -p /vol/web/
RUN adduser -D user
RUN chown -R user:user /vol
RUN chmod -R 755 /vol/web
USER user
CMD ["dos2unix", "entrypoint.sh"]
but i still get the same error.
update 5
ok so i changed the eol of entrypoint.sh manually with notepad++ but i still get the same error.
do i need to apply this to more than just entrypoint.sh?
There are two problems in this setup.
DOS line endings
The first problem is the use of DOS line endings on the entrypoint. Here's what I see when I use od to get a character-by-character dump of the files:
$ od -c scripts/entrypoint.sh | head -n 2
0000000 # ! / b i n / s h \r \n \r \n s e t
0000020 - e \r \n \r \n p y t h o n m a
The issue is that after #!/bin/sh, we have \r\n, when we should have just \n. This is a DOS-style line ending, but Linux expects just a \n. More information
I used a program called dos2unix to replace those lines:
$ dos2unix scripts/entrypoint.sh
dos2unix: converting file scripts/entrypoint.sh to Unix format...
(VS code can do this too - here's how.)
When I run it, I get a new error:
app_1 | Traceback (most recent call last):
app_1 | File "manage.py", line 21, in <module>
app_1 | main()
app_1 | File "manage.py", line 17, in main
app_1 | execute_from_command_line(sys.argv)
app_1 | File "/usr/local/lib/python3.8/site-packages/django/core/management/__init__.py", line 401, in execute_from_command_line
app_1 | utility.execute()
app_1 | File "/usr/local/lib/python3.8/site-packages/django/core/management/__init__.py", line 345, in execute
app_1 | settings.INSTALLED_APPS
app_1 | File "/usr/local/lib/python3.8/site-packages/django/conf/__init__.py", line 76, in __getattr__
app_1 | self._setup(name)
app_1 | File "/usr/local/lib/python3.8/site-packages/django/conf/__init__.py", line 63, in _setup
app_1 | self._wrapped = Settings(settings_module)
app_1 | File "/usr/local/lib/python3.8/site-packages/django/conf/__init__.py", line 142, in __init__
app_1 | mod = importlib.import_module(self.SETTINGS_MODULE)
app_1 | File "/usr/local/lib/python3.8/importlib/__init__.py", line 127, in import_module
app_1 | return _bootstrap._gcd_import(name[level:], package, level)
app_1 | File "<frozen importlib._bootstrap>", line 1014, in _gcd_import
app_1 | File "<frozen importlib._bootstrap>", line 991, in _find_and_load
app_1 | File "<frozen importlib._bootstrap>", line 975, in _find_and_load_unlocked
app_1 | File "<frozen importlib._bootstrap>", line 671, in _load_unlocked
app_1 | File "<frozen importlib._bootstrap_external>", line 843, in exec_module
app_1 | File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
app_1 | File "/app/app/settings.py", line 31, in <module>
app_1 | ALLOWED_HOSTS.extend(ALLOWED_HOSTS.split(',')) # .split(",") = incase multiple so we seperate them with a comma
app_1 | AttributeError: 'list' object has no attribute 'split'
Wrong variable referenced in settings file
I edited the file app/app/settings.py, and changed this line
ALLOWED_HOSTS.extend(ALLOWED_HOSTS.split(','))
to
ALLOWED_HOSTS.extend(ALLOWED_HOSTS_ENV.split(','))
After this, I found this worked.
Nginx
The nginx configuration turned out to be fine. The issue was that the app died, so nginx couldn't do a DNS lookup to find the IP address of the app container. Fixing the app container also fixes nginx.
Im trying to host web app inside docker container and for it i need docker-compose in docker. I installed docker-compose inside ubuntu image and that solution worked! (im running container with privileged mode)
Image file:
FROM ubuntu:20.04
RUN apt-get update && \
apt-get -qy full-upgrade && \
apt-get -qy install sudo && \
apt-get install -qy curl && \
apt-get install -qy nginx && \
apt-get install -qy ca-certificates curl gnupg lsb-release && \
mkdir -p /etc/apt/keyrings && \
curl -fsSL https://download.docker.com/linux/ubuntu/gpg | gpg --dearmor -o /etc/apt/keyrings/docker.gpg && \
echo "deb [arch=$(dpkg --print-architecture) signed-by=/etc/apt/keyrings/docker.gpg] https://download.docker.com/linux/ubuntu $(lsb_release -cs) stable" | tee /etc/apt/sources.list.d/docker.list > /dev/null && \
apt-get update && \
apt-get install -qy docker-ce docker-ce-cli containerd.io && \
curl -L "https://github.com/docker/compose/releases/download/1.28.6/docker-compose-$(uname -s)-$(uname -m)" -o /usr/local/bin/docker-compose && \
chmod +x /usr/local/bin/docker-compose && \
rm /etc/nginx/sites-enabled/default
WORKDIR /home
COPY ****secret****
COPY ****secret****
COPY ****secret****
COPY docker /home/docker
COPY ****secret****
COPY ****secret****
COPY ****secret****
EXPOSE 3000 7777
COPY dockerized_build/entrypoint.sh /entrypoint.sh
RUN ["chmod", "+x", "/entrypoint.sh"]
ENTRYPOINT ["/entrypoint.sh"]
Entrypoint:
#!/bin/bash
sudo service docker start
sudo nginx -g daemon off;
sudo service nginx restart
cd /home/docker && sudo docker-compose up
All working fine after container creation, but if i want to restart container with docker restart or sudo docker restart it throws this error(its full container log):
mount: /sys/fs/cgroup/cpuset: permission denied.
mount: /sys/fs/cgroup/cpu: permission denied.
mount: /sys/fs/cgroup/cpuacct: permission denied.
mount: /sys/fs/cgroup/blkio: permission denied.
mount: /sys/fs/cgroup/memory: permission denied.
mount: /sys/fs/cgroup/devices: permission denied.
mount: /sys/fs/cgroup/freezer: permission denied.
mount: /sys/fs/cgroup/net_cls: permission denied.
mount: /sys/fs/cgroup/perf_event: permission denied.
mount: /sys/fs/cgroup/net_prio: permission denied.
mount: /sys/fs/cgroup/hugetlb: permission denied.
mount: /sys/fs/cgroup/pids: permission denied.
mount: /sys/fs/cgroup/rdma: permission denied.
mount: /sys/fs/cgroup/misc: permission denied.
* Starting Docker: docker [ OK ]
nginx: invalid option: "off"
* Restarting nginx nginx [ OK ]
Traceback (most recent call last):
File "urllib3/connectionpool.py", line 677, in urlopen
File "urllib3/connectionpool.py", line 392, in _make_request
File "http/client.py", line 1277, in request
File "http/client.py", line 1323, in _send_request
File "http/client.py", line 1272, in endheaders
File "http/client.py", line 1032, in _send_output
File "http/client.py", line 972, in send
File "docker/transport/unixconn.py", line 43, in connect
ConnectionRefusedError: [Errno 111] Connection refused
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "requests/adapters.py", line 449, in send
File "urllib3/connectionpool.py", line 727, in urlopen
File "urllib3/util/retry.py", line 410, in increment
File "urllib3/packages/six.py", line 734, in reraise
File "urllib3/connectionpool.py", line 677, in urlopen
File "urllib3/connectionpool.py", line 392, in _make_request
File "http/client.py", line 1277, in request
File "http/client.py", line 1323, in _send_request
File "http/client.py", line 1272, in endheaders
File "http/client.py", line 1032, in _send_output
File "http/client.py", line 972, in send
File "docker/transport/unixconn.py", line 43, in connect
urllib3.exceptions.ProtocolError: ('Connection aborted.', ConnectionRefusedError(111, 'Connection refused'))
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "docker/api/client.py", line 214, in _retrieve_server_version
File "docker/api/daemon.py", line 181, in version
File "docker/utils/decorators.py", line 46, in inner
File "docker/api/client.py", line 237, in _get
File "requests/sessions.py", line 543, in get
File "requests/sessions.py", line 530, in request
File "requests/sessions.py", line 643, in send
File "requests/adapters.py", line 498, in send
requests.exceptions.ConnectionError: ('Connection aborted.', ConnectionRefusedError(111, 'Connection refused'))
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "docker-compose", line 3, in <module>
File "compose/cli/main.py", line 81, in main
File "compose/cli/main.py", line 198, in perform_command
File "compose/cli/command.py", line 70, in project_from_options
File "compose/cli/command.py", line 153, in get_project
File "compose/cli/docker_client.py", line 43, in get_client
File "compose/cli/docker_client.py", line 170, in docker_client
File "docker/api/client.py", line 197, in __init__
File "docker/api/client.py", line 222, in _retrieve_server_version
docker.errors.DockerException: Error while fetching server API version: ('Connection aborted.', ConnectionRefusedError(111, 'Connection refused'))
[128] Failed to execute script docker-compose
Any suggestions? I really got stuck on that problem :(
Its working finally!! For everyone else solution was to shutdown docker inside docker before container shutdown.
Entrypoint:
#!/usr/bin/env bash
sudo service docker start
sudo service nginx restart
cd /home/docker && sudo docker-compose up -d
onContainerStop() {
echo "Container stopped, performing docker restart procedure..."
# shellcheck disable=SC2046
sudo docker stop $(docker ps -aq)
sudo service docker stop
sudo service nginx stop
}
trap 'onContainerStop' SIGTERM
while true; do :; done
I'm trying this:
https://docs.docker.com/compose/environment-variables/#substitute-environment-variables-in-compose-files
and as stated by the documentation, this is my docker-compose.yml:
services:
web:
image: "${IMAGE}"
This is my .env.dev file
IMAGE=node:12
And this is the command that I'm running:
docker-compose --env-file ./.env.dev up
But I'm getting this error:
WARNING: The IMAGE variable is not set. Defaulting to a blank string.
Traceback (most recent call last):
File "bin/docker-compose", line 3, in <module>
File "compose/cli/main.py", line 67, in main
File "compose/cli/main.py", line 126, in perform_command
File "compose/cli/main.py", line 1070, in up
File "compose/cli/main.py", line 1066, in up
File "compose/project.py", line 615, in up
File "compose/service.py", line 350, in ensure_image_exists
File "compose/service.py", line 376, in image
File "docker/utils/decorators.py", line 17, in wrapped
docker.errors.NullResource: Resource ID was not provided
[50006] Failed to execute script docker-compose
Am I doing something wrong?
I've docker-compose 1.27.4
I found in Ubuntu docker-compose I have to go to the directory directly where all files (docker-compose.yml ... and some.env) are located.
Afterwards it did work.
Solution was only to cd into the directory:
cd my-folder-with-compose
docker-compose --env-file all.env ...
I am running my nodejs and blockchain code . And I made a Docker Container with dockerfile and docker compose but i am getting error while running the docker-compose up.
This is the traceback on console :
Traceback (most recent call last):
File "docker-compose", line 6, in <module>
File "compose/cli/main.py", line 68, in main
File "compose/cli/main.py", line 121, in perform_command
File "compose/cli/main.py", line 938, in up
File "compose/project.py", line 430, in up
File "compose/service.py", line 317, in ensure_image_exists
File "compose/service.py", line 918, in build
File "site-packages/docker/api/build.py", line 238, in build
File "site-packages/docker/api/build.py", line 285, in _set_auth_headers
File "site-packages/docker/auth.py", line 97, in resolve_authconfig
File "site-packages/docker/auth.py", line 125, in _resolve_authconfig_credstore
File "site-packages/dockerpycreds/store.py", line 25, in get
File "site-packages/dockerpycreds/store.py", line 57, in _execute
File "subprocess.py", line 711, in __init__
File "subprocess.py", line 1343, in _execute_child
UnicodeDecodeError: 'ascii' codec can't decode byte 0xe2 in position 0: ordinal not in range(128)
Failed to execute script docker-compose
and my docker-compose.yml is :
version: '2'
services:
node-app:
build: app
ports:
- "4000:8080"
volumes:
- ./app/src:/myapp/src
depends_on:
- ethernet_server
environment:
- ETHEREUM_CLIENT_IP=http://192.168.178.22
- ETHEREUM_CLIENT_PORT=8545
ethernet_server:
build: testrpc
I don't know what went wrong.can someone help me to fix it?
Paste the below mentioned lines in your docker file. And it will fix the unicode error.
RUN apt-get update -y
RUN apt-get install --reinstall -y locales
# uncomment chosen locale to enable it's generation
RUN sed -i 's/# pl_PL.UTF-8 UTF-8/pl_PL.UTF-8 UTF-8/' /etc/locale.gen
# generate chosen locale
RUN locale-gen pl_PL.UTF-8
# set system-wide locale settings
ENV LANG pl_PL.UTF-8
ENV LANGUAGE pl_PL
ENV LC_ALL pl_PL.UTF-8
Try building the containers directly with docker:
docker build app
docker build testrpc
then run your docker-compose again.
I was encountering a similar report from docker-compose of UnicodeDecodeError: 'ascii' codec can't decode byte 0xe2 in position 0 and found that using docker to rebuild my containers resolved the problem.
In my case, while saving docker commands i had saved it as a .odt file initially and then changed the file extension in finder app to .yml manually, resulted in above issue.
Used VS code (you can use any other tool or simply select all files as format type while saving) to save the file with .yml extension and then tried executing docker-compose again worked for me.
Running Raku (previously aka Perl 6) kernel in Jupyter notebook would be great for reproducibility and ease of use (personal view).
I wanted to run the Perl 6 notebook in a docker container and access it in my web browser. For this I created this docker image.
The code to create the docker image was:
FROM sumankhanal/rakudo:2019.07.1
RUN apt-get update \
&& apt-get install -y ca-certificates python3-pip && pip3 install jupyter notebook \
&& zef install Jupyter::Kernel --force-test
ENV TINI_VERSION v0.6.0
ADD https://github.com/krallin/tini/releases/download/${TINI_VERSION}/tini /usr/bin/tini
RUN chmod +x /usr/bin/tini
ENTRYPOINT ["/usr/bin/tini", "--"]
EXPOSE 8888
CMD ["jupyter", "notebook", "--port=8888", "--no-browser", "--ip=0.0.0.0"]
I am on Windows 10 64 bit and IP address of my docker is 192.168.99.100.
When I tried to run the container with this code:
docker run -it -p 8888:8888 sumankhanal/raku-notebook in the docker terminal
I get this error:
$ docker run -it -p 8888:8888 sumankhanal/raku-notebook
[I 14:26:43.598 NotebookApp] Writing notebook server cookie secret to /root/.local/share/jupyter/runtime/notebook_cookie_secret
Traceback (most recent call last):
File "/usr/local/bin/jupyter-notebook", line 11, in <module>
sys.exit(main())
File "/usr/local/lib/python3.5/dist-packages/jupyter_core/application.py", line 267, in launch_instance
return super(JupyterApp, cls).launch_instance(argv=argv, **kwargs)
File "/usr/local/lib/python3.5/dist-packages/traitlets/config/application.py", line 657, in launch_instance
app.initialize(argv)
File "<decorator-gen-7>", line 2, in initialize
File "/usr/local/lib/python3.5/dist-packages/traitlets/config/application.py", line 87, in catch_config_error
return method(app, *args, **kwargs)
File "/usr/local/lib/python3.5/dist-packages/notebook/notebookapp.py", line 1296, in initialize
self.init_webapp()
File "/usr/local/lib/python3.5/dist-packages/notebook/notebookapp.py", line 1120, in init_webapp
self.http_server.listen(port, self.ip)
File "/usr/local/lib/python3.5/dist-packages/tornado/tcpserver.py", line 142, in listen
sockets = bind_sockets(port, address=address)
File "/usr/local/lib/python3.5/dist-packages/tornado/netutil.py", line 197, in bind_sockets
sock.bind(sockaddr)
OSError: [Errno 99] Cannot assign requested address
Any help ?
You need to add --allow-root in the CMD of your Dockerfile. Also you need to link the kernel with jupyter in your dockerfile
jupyter-kernel.p6 --generate-config
Once you do that you will be able to see the dockerfile. I also noticed that your images size is very huge, you should try and find a better base image for jupyter rather than the one you have.
For more details about installing kernels refer to the below link
https://jupyter.readthedocs.io/en/latest/install-kernel.html