I'm attempting to add some necessary latext style files to the pandoc/latex docker container (which itself is built over Alpine) and the underlying latex to pdf conversion does is unable to find some of the style files I've added. Here's my Dockerfile, which adds the packages I need.
FROM pandoc/latex:latest
RUN apk --no-cache add texlive-xetex texmf-dist-pictures texmf-dist-latexextra poppler-utils && texhash
I can see that the style file I need is installed.
$ docker run --rm -it --entrypoint /bin/sh -v $PWD:/data mypandoc:latest
/data # find / -name tikzpagenodes.sty
/usr/share/texmf-dist/tex/latex/tikzpagenodes/tikzpagenodes.sty
But, when I try to do the actual build, I get the following error.
Error producing PDF.
! LaTeX Error: File `tikzpagenodes.sty' not found.
Type X to quit or <RETURN> to proceed,
or enter new name. (Default extension: sty)
Enter file name:
! Emergency stop.
<read *>
l.103 \usepackage
I had hoped that the call to texhash in the container build would fix this, but it did not. Is there some trick to getting the texmf-dist directories into the search path? The problem appears to be with all of the styles I've added from texmf-dist, not just this one. Thanks.
Use the tlmgr from TeXLive to install more packages. The pandoc/latex Docker container does not use the TeX installation from Alpine in order to give users more flexibility and to reduce image size – the Alpine packages are far less fine-grained.
Usually, one will add the following to the Dockerfile
# Install additional LaTeX packages
RUN tlmgr update --self && tlmgr install \
pgf # <list of packages goes here>
Alternatively, you could use the pandoc/core image and build your own LaTeX container using the Alpine system packages.
Related
I am trying to follow the 2 steps mentioned below:
1) Downloaded source code of
https://sourceforge.net/projects/hunspell/files/Hyphen/2.8/hyphen-2.8.8.tar.gz/download
2) Compiled it and you will get binary named example:
hyphen-2.8.8$ ./example ~/dev/smc/hyphenation/hi_IN/hyph_hi_IN.dic
~/hi_sample.text
I have downloaded and uncompressed the tar file. My question is how to create a dockerfile to automate this?
There are only 3 commands involved:
./configure
make all-recursive
make install
I can select the official python image as a base container. But how do I write the commands in a docker file?
You can do that with a RUN command:
FROM python:<version number here>
RUN ./configure && make-recursive && make install
CMD ['<some command here>']
what you use for <some command here> depends on what the image is meant to do. Remember that docker containers only run as long as that command is executing, so if you put the configure/make/install steps in a script and use that as your entry point, it's going to build your program, and then the container will halt.
Also you need to get the downloaded files into the container. That can be done using a COPY or an ADD directive (before the RUN of course). If you have the tar.gz file saved locally, then ADD will both copy the file into the container and expand it into a directory automatically. COPY will not expand it, so if you do that, you'll need to add a tar -zxvf or similar to the RUN.
If you want to download the file directly into the container, that could be done with ADD <source URL>, but in that case it won't expand it, so you'll have to do that in the RUN. COPY doesn't allow sourcing from a URL. This post explains COPY vs ADD in more detail.
You can have the three commands in a shell script and then use the following docker commands
COPY ./<path to your script>/<script-name>.sh /
ENTRYPOINT ["/<script-name>.sh"]
CMD ["run"]
For reference, you can create your docker file as they have created for one of the projects I worked on Apache Artemis Active Mq:
https://github.com/apache/activemq-artemis/blob/master/artemis-docker/Dockerfile-ubuntu
If I build a cmake file, create an executeble with make and delete everything except the executable, the executable is still functional. Can I,
build the file but the only output is the file that can be executed with ./project
or
have all of the files build, create the executable with make, then delete everything except the executable afterwards
and if so, how do I?
If I am getting this correctly, you want to create a stand-alone binary that cannot be executed even if the docker image does not has any dependencies then you need to use static option during the build - i am not expert in this - maybe as described in the following answer of Compiling a static executable with CMake.
Next you might use a multi-stage builds in docker which will makes you able to have a final minimal image with your executable file only without any build dependencies, just the needed packages for your run-time environment. I have an example not with make, it was created using g++ but achieving the similar concept as below:
FROM gcc:5 as builder
COPY ./hello_world_example.cc /hello_world_example.cc
RUN g++ -o hello_world_binary -static hello_world_example.cc && chmod +x hello_world_binary
FROM debian:jessie
COPY --from=builder /hello_world_binary /hello_world_binary
CMD ["/hello_world_binary"]
And the final result when you run the container:
$ docker run --rm -it helloworldimage:latest
Hello from Dockerized image
Why do you need that?
You can add install() command to your CMakeLists.txt and then call make install to copy your executable into CMAKE_INSTALL_PREFIX directory. If you set CMAKE_INSTALL_PREFIX to an empty dir, you'd end with a directory containing only your executable file.
If a Dockerfile is written with mistakes for example:
CMD ["service", "--config", "/etc/service.conf] (missing quote)
Is there a way to lint it to detect such mistake before building?
Try:
Either the Haskell Dockerfile Linter ("hadolint"), also available online. hadolint parses the Dockerfile into an AST and performs checking and validation based on best practice Docker images rules. It also uses Shellcheck to lint the Bash code on RUN commands.
Or dockerlinter (node.js-based).
I've performed a simple test against of a simple Docker file with RUN, ADD, ENV and CMD. dockerlinter was smart about grouping the same violation of rules together but it was not able to inspect as thorough as hadolinter possibly due to the lack of Shellcheck to statically analyze the Bash code.
Although dockerlinter falls short in the scope it can lint, it does seem to be much easier to install. npm install -g dockerlinter will do, while compiling hadolinter requires a Haskell compiler and build environment that takes forever to compile.
$ hadolint ./api/Dockerfile
L9 SC2046 Quote this to prevent word splitting.
L11 SC2046 Quote this to prevent word splitting.
L8 DL3020 Use COPY instead of ADD for files and folders
L10 DL3020 Use COPY instead of ADD for files and folders
L13 DL3020 Use COPY instead of ADD for files and folders
L18 DL3020 Use COPY instead of ADD for files and folders
L21 DL3020 Use COPY instead of ADD for files and folders
L6 DL3008 Pin versions in apt get install. Instead of `apt-get install <package>` use `apt-get install <package>=<version>`
L6 DL3009 Delete the apt-get lists after installing something
L6 DL3015 Avoid additional packages by specifying `--no-install-recommends`
$ dockerlint ./api/Dockerfile
WARN: ADD instruction used instead of COPY on line 8, 10, 13, 18, 21
ERROR: ./api/Dockerfile failed.
Update in 2018. Since hadolint has the official Docker repository now, you can get the executable quickly:
id=$(docker create hadolint/hadolint:latest)
docker cp "$id":/bin/hadolint .
docker rm "$id"
or you can use this command
docker container run --rm -i hadolint/hadolint hadolint - < Dockerfile
This is a statically compiled executable (according to ldd hadolint), so it should run regardless of installed libraries. A reference on how the executable is built: https://github.com/hadolint/hadolint/blob/master/docker/Dockerfile.
If you have a RedHat subscription, you can access the "Linter for Dockerfile" application directly at https://access.redhat.com/labs/linterfordockerfile/; information about the application is located at https://access.redhat.com/labsinfo/linterfordockerfile
This Node.js application is also available on GitHub https://github.com/redhataccess/dockerfile_lint if you prefer to run it locally.
I use very successfully in my CI pipeline npm's dockerfile_lint. You can add or extend rules. Using the package.json you can create different configs for the different jobs. There are both
Docker CLI
docker run -it --rm --privileged -v `pwd`:/root/ \
projectatomic/dockerfile-lint \
dockerfile_lint [-f Dockerfile]
docker run -it --rm --privileged -v `pwd`:/root/ \
-v /var/run/docker.sock:/var/run/docker.sock \
projectatomic/dockerfile-lint \
dockerfile_lint image <imageid>
and Atomic CLI available
atomic run projectatomic/dockerfile-lint
atomic run projectatomic/dockerfile-lint image <imageid>
Also you can lint your images for tagging.
I created dockerfile-validator as an extension for VS Code, which uses the dockerfile-lint mentioned in a previous answer. By default it uses dockerfile-lint default rules, but in VS code User Settings (dockerfile-validator.rulefile.path) you can specify a path to a custom rule file with your own coding standards.
Recently, I cam across dockerfilelint which is NodeJS based.
dockerfilelint Dockerfile
Supports following rules and rudimentary CMD checks
required_params
uppercase_commands
from_first
invalid_line
sudo_usage
apt-get_missing_param
apt-get_recommends
apt-get-upgrade
apt-get-dist-upgrade
apt-get-update_require_install
apkadd-missing_nocache_or_updaterm
apkadd-missing-virtual
invalid_port
invalid_command
expose_host_port
label_invalid
missing_tag
latest_tag
extra_args
missing_args
add_src_invalid
add_dest_invalid
invalid_workdir
invalid_format
apt-get_missing_rm
deprecated_in_1.13
Hadolint seems like a better option but this may suffice for simple needs. Also, Github's super-linter uses this.
I'm not too familiar with go but it looks like you can simply call the Parse method as is done in the test suite here. If that does not return an err then your lint passes. I'm assuming that's trivial to expose to a script or something to call during development.
I have a requirement that before an application runs, some part of it needs to read the environmental variable. For this I have the following docker file
FROM nodesource/jessie:0.12.7
# install gettext for envsubst
RUN apt-get update
RUN apt-get install -y gettext-base
# cache package.json and node_modules to speed up builds
ADD package.json package.json
RUN npm install
# Add source files
ADD src src
# Substiture value for backend endpoint env var
RUN envsubst < src/js/envapp.js > src/js/app.js
ADD node_modules node_modules
EXPOSE 8000
CMD ["npm","start"]
The above envsubst line reads (should read) an env variable $MYENV and substitutes it. But when I open the file app.js, its empty.
I checked if the environmental variable exists in the container and it does. Any reason its value is not read and substituted?
I also tried the same command in teh container and it works. It only does not work when I run the image
This is likely because $MYENV is not available for envsubst when you run the image.
Each RUN command runs on its own shell.
From the Docker documentations:
RUN (the command is run in a shell - /bin/sh -c - shell form)
You need to source your profile as well, for example if the $MYENV environment variable is available in the .bashrc file, you can modify your Dockerfile like this:
RUN source ~/.bashrc && envsubst < src/js/envapp.js > src/js/app.js
I encountered the same issues, and after much research and fishing through the internet. I managed to find a few work arounds to this issue. Below I'll list them and identifiable risks at the time of this "Answer post"
Solutions:
1.) apt-get install -y gettext its a standard GNU package language library, one of these libraries that it includes is envsubst` and I can confirm that it works for docker UBUNTU:latest and it should work for every flavored version.
2.) npm install envsub dependent on the "use case" - this approach would be better supported by node based projects.
3.) enstub cli project lib in my opinion it seems a bit overkill to downloading a custom cli from a random stranger but, it's also another option.
Risk:
apt-get install -y gettext:
1.) gettext - this approach would NOT be ideal for VM's as with any package library, it requires maintenance and updates as time passes. However, this isn't necessary for docker because once an a container is initialized and up and running we can create a bashscript to add the package, substitute env vars and then remove the package.
2.) It's a bad idea for VM's because it can be used to execute arbitrary code
npm install ensub
1.) envsub - updating packages and this approach wouldn't be ideal if your dealing with a different stack and not using nodejs.
NOTE:
There's also a PHP version for those developing a PHP application and it seems to work PHP's cli if you need a custom environment.
Resources:
GetText package library info: https://www.gnu.org/software/gettext/
GetText Risk - https://ubuntu.com/security/notices/USN-3815-2
PHP-GetText - apt-get install -y php-gettext
Custom ensubst cli: https://github.com/a8m/envsubst
I suggest that since your are using Node, you use the npm envsub module.
This module is well tested and is developed with docker in mind.
It avoids the need for relying on other dependencies when you already have the full Node arsenal at your fingertips.
envsub is described as
envsub is envsubst for NodeJS
NodeJS global CLI module providing file-level environment variable substitution via Handlebars
I am the author of the package. I think you will enjoy it.
I had some issues with envsubst in Docker.
For some reasons envsubst doesn't work when I try to copy the output in the same file. For example, this is not working:
RUN envsubst < file.conf > file.conf
But when I when I tried to use a temp file the issue disappeared:
RUN envsubst < file.conf > file.conf.temp && cp -f file.conf.temp file.conf
I have a makefile for building debian and rpm packages. I have two Jenkins environments, one for Ubuntu and one for CentOS. The debian package works no problem, and the rpm make command works on my machine, but not on Jenkins. Jenkins returns the following error:
cp: cannot stat /root/rpmbuild/SOURCES/myfile.file': No such file or directory
error: Bad exit status from /var/tmp/rpm-tmp.mII8KL (%install)
I was getting similar errors when developing the package but eventually figured everything out, and all was good. I think the problem may lie with $RPM_BUILD_ROOT, %{buildroot}, or _topdir options. Nothing I have tried has led me anywhere however.
Here is my (modified) Makefile:
# a list of tools we depend on and must install if they're missing
DEBTOOLS=/usr/bin/debuild-pbuilder
RPMTOOLS=/usr/bin/rpmbuild
# convenience target for "make deb"
deb: my-package_1.0_all.deb
# convenience target for "make rpm".
rpm: my-package-1.0-Public.x86_64.rpm
# the target package (on Ubuntu at least)
my-package_1.0_all.deb: $(DEBTOOLS)
cd my-package; debuild-pbuilder -us -uc
my-package-1.0-Public.x86_64.rpm: $(RPMTOOLS)
cd rpmbuild; rpmbuild -bb SPECS/my-package.spec
/usr/bin/debuild-pbuilder:
apt-get -y install pbuilder
/usr/bin/rpmbuild:
yum -y install rpm-build
This is my spec file:
Summary: My Package
Name: my-package
Version: 1.0
Release: Public
Group: Applications/System
License: Public
Requires: external-package
Source1: myfile.file
%description
blah blah
%files
%config /etc/myfile.file
%install
mkdir -p $RPM_BUILD_ROOT/etc/
cp %{SOURCE1} %{buildroot}/etc/myfile.file
%post
ln -sf /etc/myfile.file /etc/external-package.conf
The problem was in fact that the file wasn't being found (obviously). For me this had a lot to do with the confusing nature of building rpm files. When the make command is executed, and the rpmbuild command is called, I needed to be able to specify the directory. When reading the documentation, it was stated you could use rpmbuild -D '_topdir .' -bb path/to/spec.spec to set the _topdir variable to the local directory you call from. This made sense as . represents this in linux.
However the actual call needs to be
rpmbuild -D "_topdir `pwd`" -bb path/to/spec.spec
This doesn't look all that different except it is crucial to use double-quotes. Using this command will run the build within the directory you call it from. After this rpmbuild will copy and handle the files for you as it should (which is confusing in itself).