Library wkhtmltopdf is not working inside Docker - docker

I have a code in Python Flask where I generate pdf files using an HTML template. The code works just fine when I run it alone, but when I try to run it inside a Docker container, as soon as I call the endpoint that generates the report the docker crashes and resets. It just stays loading then it returns an error (in Postman which I'm using to test).
The code for the PDF is as follows:
def create_report(download_uuid):
transactions = get_transaction_for_report(download_uuid)
config = pdfkit.configuration(wkhtmltopdf=environ.get('WKHTMLTOPDF'))
file_obj = io.BytesIO()
with zipfile.ZipFile(file_obj, 'w') as zip_file:
for transaction in transactions:
html = render_template("report.html", transaction=transaction)
pdf = pdfkit.from_string(html, False, configuration=config)
data = zipfile.ZipInfo('{}.pdf'.format(transaction['control_number']))
data.compress_type = zipfile.ZIP_DEFLATED
zip_file.writestr(data, pdf)
file_obj.seek(0)
return send_file(file_obj, attachment_filename="forms.zip", as_attachment=True)
It is returning a zip file, but inside the zip file are pdf files. Furthermore, if I remove the pdf generating part, the zip file returns just fine. This is my Dockerfile:
FROM madnight/docker-alpine-wkhtmltopdf as wkhtmltopdf_image
FROM python:3.9-alpine
RUN adduser -D custom
WORKDIR /home/Project
COPY requirements.txt requirements.txt
RUN python -m venv venv
RUN venv/bin/pip install --upgrade pip
RUN apk add make automake gcc g++ subversion python3-dev jpeg-dev zlib-dev libffi-dev musl-dev openssl-dev freetype freetype-dev ttf-freefont libxrender qt5-qtbase-dev
RUN venv/bin/pip install -r requirements.txt
RUN venv/bin/pip install gunicorn
COPY Project Project
COPY boot.sh app.py .env run.py create_database.py config.py ./
COPY templates templates
COPY static static
COPY --from=wkhtmltopdf_image /bin/wkhtmltopdf /usr/local/bin/wkhtmltopdf
RUN chmod +x boot.sh
ENV FLASK_APP app.py
USER root
RUN chown -R custom ./
USER custom
EXPOSE 9001
ENTRYPOINT ["./boot.sh"]
I should say that this is the last iteration of many, MANY attempts to try to get this to work. Essentially, I've tried getting wkhtmltox by curl, I've tried putting wkhtmltopdf in different directories. So far nothing has worked. I don't know what I'm missing. This is basically what I need to fix in order to finish this project so any help at all will be immensely appreciated.
EDIT: docker-compose.yml
version: '2'
services:
app:
build: .
networks:
- custom
ports:
- "9001:9001"
volumes:
- "./static:/home/EventismedEquipmentAPI/static"
external_links:
- eventismed-equipment:db
networks:
custom:
external: true

Let's fix this.
I've managed to run wkhtmltopdf isolated on a docker container.
Dockerfile:
# https://stackoverflow.com/a/62737156/152016
# Create image based on the official openjdk 8-jre-alpine image from the dockerhub
FROM openjdk:8-jre-alpine
# Install wkhtmltopdf
# https://stackoverflow.com/a/56925361/152016
RUN apk add --no-cache wkhtmltopdf ttf-dejavu
ENTRYPOINT ["sh"]
docker-compose.yml:
version: '3.8'
services:
wkhtmltopdf:
image: wkhtmltopdf
container_name: wkhtmltopdf
build:
dockerfile: Dockerfile
context: .
Then:
docker-compose build
docker run -ti --rm -v /tmp:/tmp wkhtmltopdf
Inside the container:
$ cd /tmp
$ wkhtmltopdf https://www.google.com test.pdf
Then you will see the pdf on your mac at /tmp/test.pdf
First let me know if this works.

Related

How to install a golang package in a docker file?

I'm new in docker and I want to setting-up a docker-compose for my django app. in the backend of my app, I have golang packages too and run that in djang with subprocess library.
But, when I want to install a package using go install github.com/x/y#latest and then copy its binary to the project directory, it gives me the error: package github.com/x/y#latest: cannot use path#version syntax in GOPATH mode
I searched a lot in the internet but didn't find a solution to solve my problem. Could you please tell me where I'm wrong?
here is my Dockerfile:
FROM golang:1.18.1-bullseye as go-build
# Install go package
RUN go install github.com/hakluke/hakrawler#latest \
&& cp $GOPATH/bin/hakrawler /usr/local/bin/
# Install main image for backend
FROM python:3.8.11-bullseye
# set environment variables
ENV PYTHONDONTWRITEBYTECODE 1
ENV PYTHONUNBUFFERED 1
# Install Dist packages
RUN apt-get update \
&& apt-get -y install --no-install-recommends software-properties-common libpq5 python3-dev musl-dev git netcat-traditional golang \
&& rm -rf /var/lib/apt/lists/
# Set work directory
WORKDIR /usr/src/redteam_toolkit/
# Install dependencies
RUN pip install --upgrade pip
COPY ./requirements.txt .
RUN pip install -r requirements.txt
# Copy project, and then the go package
COPY . .
COPY --from=go-build /usr/local/bin/hakrawler /usr/src/redteam_toolkit/toolkit/scripts/webapp/
docker-compose.yml:
version: '3.3'
services:
webapp:
build: .
command: python manage.py runserver 0.0.0.0:4334
container_name: toolkit_webapp
volumes:
- .:/usr/src/redteam_toolkit/
ports:
- 4334:4334
env_file:
- ./.env
depends_on:
- db
db:
image: postgres:13.4-bullseye
container_name: database
volumes:
- postgres_data:/var/lib/postgresql/data/
environment:
- POSTGRES_USER=user
- POSTGRES_PASSWORD=password
- POSTGRES_DB=redteam_toolkit_db
volumes:
postgres_data:
the get.py file inside /usr/src/redteam_toolkit/toolkit/scripts/webapp/ directory, to just run the go package, and list files in this dir:
import os
import subprocess
BASE_DIR = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
print(f"Current path is: {BASE_DIR}")
def go(target_url):
run_go_package = subprocess.getoutput(
f"echo {target_url} | {BASE_DIR}/webapp/hakrawler -t 15 -u"
)
list_files = subprocess.getoutput(f"ls {BASE_DIR}/webapp/")
print(run_go_package)
print(list_files)
go("https://example.org")
and then I just run:
$ docker-compose up -d --build
$ docker-compose exec webapp python toolkit/scripts/webapp/get.py
The output is:
Current path is: /usr/src/redteam_toolkit/toolkit/scripts
/bin/sh: 1: /usr/src/redteam_toolkit/toolkit/scripts/webap/hakrawler: not found
__init__.py
__pycache__
scr.py
gather.py
This looks like a really good candidate for a multi-stage build:
FROM golang:1.18.0 as go-build
# Install packages
RUN go install github.com/x/y#latest \
&& cp $GOPATH/bin/pacakge /usr/local/bin/
FROM python:3.8.11-bullseye as release
...
COPY --from=go-build /usr/local/bin/package /usr/src/toolkit/toolkit/scripts/webapp/
...
Your compose file also needs to be updated, it is masking the entire /usr/src/redteam_toolkit folder with the volume mount. Delete that volume mount to see the content of the image.
GOPATH mode does not work with Golang modules, in your Dockerfile file, add:
RUN unset GOPATH
use RUN go get <package_repository>

Exposing Docker Volumes to Nginx

I'm trying to connect a Json file which resides in a docker volume of the following container to my main docker container which is running a django project.
Since I am using Caprover my Docker Compose options are very limited.
So Docker Composer is not really an option. I want to instead just expose the json file over the web with a link.
Something like domain.com/folder/jsonfile.json
Can somebody tell me if this is possible inside this dockerfile?
The image I am using is crucial to the container so can I just add an nginx image or do I need any other changes to make this work?
Or is nginx not even necessary?
FROM ubuntu:devel
ENV TZ=Etc/UTC
ARG APP_HOME=/app
WORKDIR ${APP_HOME}
ENV DEBIAN_FRONTEND=noninteractive
RUN ln -snf /usr/share/zoneinfo/$TZ /etc/localtime
RUN echo $TZ > /etc/timezone
RUN apt-get update && apt-get upgrade -y
RUN apt-get install gnumeric -y
RUN mkdir -p /etc/importer/data
RUN mkdir /voldata
COPY config.toml /etc/importer/
COPY datasets/* /etc/importer/data/
VOLUME /voldata
COPY importer /usr/bin/
RUN chmod +x /usr/bin/importer
COPY . ${APP_HOME}
CMD sleep 999d
Using the same volume in 2 containers
docker-compose:
volumes:
shared_vol:
services:
service1:
volumes:
- 'shared_vol:/path/to/file'
service2:
volumes:
- 'shared_vol:/path/to/file'
the mechanism above replaces the volumes_from since v3, but this works for v2 as well:
volumes:
shared_vol:
services:
service1:
volumes:
- 'shared_vol:/path/to/file'
service2:
volumes_from:
- service1
If you want to avoid unintentional altering add :ro for readonly to the target service:
service1:
volumes:
- 'shared_vol:/path/to/file'
service2:
volumes:
- 'shared_vol:/path/to/file:ro'
http-server
Surely you can provide the file via http (or other protocol). There are two oppertunities:
Including a http-service to your container (quite easy depending on what is already given in the container) e.g. using nodejs you can use this https://www.npmjs.com/package/http-server very easy. Size doesn't matter? So just install:
RUN apt-get install -y nodejs npm
RUN npm install -g http-server
EXPOSE 8080
CMD ["http-server", "--cors", "-p8080", "/path/to/your/json"]
docker-compose (Runs per default on 8080, so open this):
existing_service:
ports:
- '8080:8080'
Run a stand alone http-server (nginx, apache httpd,..) in another container, but then you depend again on using the same volume for two services, so for local solutions quite an overkill.
Base image
If you don't have good reasons i'll would never use something like :devel, :rolling or :latest as base image. Stick to a LTS version instead like ubuntu:22.04
Testing for http-server
Dockerfile
FROM ubuntu:20.04
ENV TZ=Etc/UTC
RUN ln -snf /usr/share/zoneinfo/$TZ /etc/localtime && echo $TZ > /etc/timezone
RUN apt-get update
RUN apt-get install -y nodejs npm
RUN npm install -g http-server#13.1.0 # Issue with JSON-File in V14: https://github.com/http-party/http-server/issues/634
COPY ./test.json ./usr/wwwhttp/test.json
EXPOSE 8080
CMD ["http-server", "--cors", "-p8080", "/usr/wwwhttp/"]
# docker build -t test/httpserver:latest .
# docker run -p 8080:8080 test/httpserver:latest
Disclaimer:
I am not that familiar with node-docker-images, this is just to give a quick working solution and go on from there. I'm not using nodeJS in production, but I'm sure it can be optimized from being fat to.. well.. being rather fat. But for quick prototyping size doesn't matter.
If you want to just have two containers access the same file, just use a volume with --mount.

Why docker-compose don't let me create a volume?

I am writing this request today because I will like to create my first Docker container. I watched a lot of tutorials, and there I come across a problem that I cannot solve, I must have missed a piece of information.
My program is quite basic, I would like to create a volume so as not to lose the information retrieved each time the container is launched.
Here is my docker-compose
version: '3.3'
services:
homework-logger:
build: .
ports:
- '54321:1235'
volumes:
- ./app:/app
image: 'cinabre/homework-logger:latest'
networks:
- homeworks
networks:
homeworks:
name: homeworks-logger
and here is my DockerFile
FROM debian:9
WORKDIR /app
RUN apt-get update -yq && apt-get install wget curl gnupg git apt-utils -yq && apt-get clean -y
RUN apt-get install python3 python3-pip -y
RUN git clone http://192.168.5.137:3300/Cinabre/Homework-Logger /app
VOLUME /app
RUN ls /app
RUN python3 -m pip install bottle beaker bottle-cork requests
CMD ["python3", "main.py"]
I did an "LS" in the container to see if the / app folder was empty: it is not
Any ideas?
thanks in advance !
Volumes are there to hold your application data, not its code. You don't usually need the Dockerfile VOLUME directive and you should generally avoid it unless you understand exactly what it does.
In terms of workflow, it's commonplace to include the Dockerfile and similar Docker-related files in the source repository yourself. Don't run git clone in the Dockerfile. (Credential management is hard; building a non-default branch can be tricky; layer caching means Docker won't re-pull the branch if it's changed.)
For a straightforward application, you should be able to use a near-boilerplate Dockerfile:
FROM python:3.9 # unless you have a strong need to hand-install it
WORKDIR /app
# Install packages first. Unless requirements.txt changes, Docker
# layer caching won't repeat this step. Do not list out individual
# packages in the Dockerfile; list them in Python-standard setup.py
# or Pipfile.
COPY requirements.txt .
# ...in the "system" Python space, not a virtual environment.
RUN pip3 install -r requirements.txt
# Copy the rest of the application in.
COPY . .
# Set the default command to run the container, and other metadata.
EXPOSE 1235
CMD ["python3", "main.py"]
In your application code you need to know where to store the data. You might put this in an environment variable:
import os
DATA_DIR = os.environ.get('DATA_DIR', '.')
with open(f"${DATA_DIR}/output.txt", "w") as f:
...
Then in your docker-compose.yml file, you can specify an alternate data directory and mount that into your container. Do not mount a volume over the /app directory containing your application's source code.
version: '3.8'
services:
homework-logger:
build: .
image: 'cinabre/homework-logger:latest' # names the built image
ports:
- '54321:1235'
environment:
- DATA_DIR=/data # (consider putting this in the Dockerfile)
volumes:
- homework-data:/data # (could bind-mount `./data:/data` instead)
# Use the automatic `networks: [default]`
volumes:
homework-data:

Docker error: Service 'app' failed to build

I am new to docker, currently following book to learn Django.
Is it necessary to be in virtual environment when running the below
command?
I have gone through docker basic videos which says it saves each apps as images. But where these images are saved?.
Does this line make the current pc root directory or dockers Image '
WORKDIR /usr/src/app'
ADD is placed before RUN in the Dockerfile.
$ sudo docker-compose build
But I got these errors.
ERROR: Service 'app' failed to build: ADD failed: stat /var/lib/docker/tmp/docker-builder912263941/config/requirements.txt: no such file or directory
Dockerfile
FROM python:3
RUN apt-get update \
&& apt-get install -y --no-install-recommends \
mysql-client default-libmysqlclient-dev
WORKDIR /usr/src/app
ADD config/requirements.txt ./
RUN pip3 install --upgrade pip; \
pip3 install -r requirements.txt
RUN django-admin startproject myproject .;\
mv ./myproject ./origproject
docker-compose.yml
version: '2'
services:
db:
image: 'mysql:5.7'
app:
build: .
command: python3 manage.py runserver 0.0.0.0:8000
volumes:
- './project:/usr/src/app/myproject'
- './media:/usr/src/app/media'
- './static:/usr/src/app/static'
- './templates:/usr/src/app/templates'
- './apps/external:/usr/src/app/external'
- './apps/myapp1:/usr/src/app/myapp1'
- './apps/myapp2:/usr/src/app/myapp2'
ports:
- '8000:8000'
links:
- db
requirements.txt
Pillow~=5.2.0
mysqlclient~=1.3.0
Django~=2.1.0
Is it necessary to be in virtual environment when running the below
command?
No, the docker build environment is isolated from the host. Any virtualenv on the host is ignored on the build context and the resulting image.
I have gone through docker basic videos which says it saves each apps
as images. But where these images are saved?.
The images are stored somewhere in /var/lib/docker but isn't meant to be browsed manually. You can send the images somewhere with docker push <image:tag> or save them with docker save <image:tag> -o <image>.tar
Does this line make the current pc root directory or dockers Image ' WORKDIR > /usr/src/app'
That line change the current workdir on the image.
ERROR: Service 'app' failed to build: ADD failed: stat /var/lib/docker/tmp/docker-builder912263941/config/requirements.txt: no such file or directory
This error means that you do not have config/requirements.txt in your current directory where build is run. Adjust your path on the Dockerfile properly.
$ docker-compose up -d
This will download the necessary Docker images and create a container for the web service.

File not found from Dockerfile, docker-compose

I have these codes in my Dockerfile.
FROM python:3
# Create user named "airport".
RUN adduser --disabled-password --gecos "" airport
# Login as the newly created "airport" user.
RUN su - airport
# Change working directory.
WORKDIR /home/airport/mount_point/
# Install Python packages at system-wide level.
RUN pip install -r requirements.txt
# Make sure to migrate all static files to the root of the project.
RUN python manage.py collectstatic --noinput
# This utility.sh script is used to reset the project environment. This includes
# removing unecessary .pyc and __pycache__ folders. This is optional and not
# necessary, I just prefer to have my environment clean before Docking.
RUN utility_scripts/utility.sh
When I called docker-compose build it returns /bin/sh: 1: requirements.txt: not found. Despite I have load the necessary volume in my docker-compose.yml. I am sure that requirements.txt is in ./
web:
build:
context: ./
dockerfile: Dockerfile
command: /home/airport/mount_point/start_web.sh
container_name: django_airport
expose:
- "8080"
volumes:
- ./:/home/airport/mount_point/
- ./timezone:/etc/timezone
How can I solve this problem?
Before running RUN pip install -r requirements.txt, you need to add the requirements.txt file to the image.
...
ADD requirements.txt requirements.txt
RUN pip install -r requirements.txt
...
For a sample on how to dockerize a django application, check https://docs.docker.com/compose/django/ . You need to add the requirements.txt and the code to the image.

Resources