I have developed a web app (maven, tomcat) in Intellij and managed to create a container through 'Services' tab in Intellij which was straightforward thanks to easy deployment config. During the process, I encountered that the cache size was not enough so I manually changed the context.xml (added <Resources cacheMaxSize="51200" />) file of tomcat manually locally after which the app ran smoothly.
To summarize the container creation in Intellij under services tab (see the bottom for the image):
- pulling an image: tomcat:9.0.65-jre8
- container name
- bind ports: 127.0.0.1:8080:8080
- bind mounts: mount host path which contains the war WITH /usr/local/tomcat/webapps
Though not sure, I guess the war file I created took already into account the change I made in the context.xml file since my application server is the tomcat I downloaded and made the change on its context.xml.
However, I also need to create a container with a dockerfile:
My dockerfile is:
FROM maven:3.8.4-jdk-8 as maven_builder
COPY . /usr/src/maven_pdfparse
WORKDIR /usr/src/maven_pdfparse
RUN mvn clean install -f /usr/src/maven_pdfparse && mkdir /usr/src/wars/
RUN find /usr/src/maven_pdfparse/ -iname '*.war' -exec cp {} /usr/src/wars/ \;
ADD pom.xml .
FROM tomcat:9.0.65-jre8
COPY --from=maven_builder /usr/src/wars/* /usr/local/tomcat/webapps
When I ran this on docker, I again got the 'insufficient cache' issue.
So how can I make the same change on context.xml when creating a dockerfile?
Or is there a way to get the dockerfile automatically when I create the container through deployment configuration?
You have a few options to choose from if you wish to add to or modify the context.xml file.
You can modify the context.xml file already within the image as part of the image build. Add a RUN command using a command line tool like sed to add the required <Resources> element to the file.
You could also have a pre-modified version of the file and just copy it into the image to overwrite the existing one.
You could add a custom startup command that modifies the context.xml (e.g. using sed as per option 1) before invoking the usual tomcat startup script. Using this mechanism you could also get the cacheMaxSize value to use from an environment variable and thus allow run-time control of the value.
Please check this answer
Tomcat 8 throwing - org.apache.catalina.webresources.Cache.getResource Unable to add the resource
Where you'll need to update context.xml and rebuild your image with copying context.xml
Related
I have a very strange issue with copying the contents of subdirectories to a Docker container.
This is the directory structure:
Note: There are two Dockerfiles, I use the one on the upper level for test purposes. Ignore the one in the WebApp folder.
I want to copy the directories Bilder and JSON to the container, including all contents, but it doesn't work. The folders in the container will be empty. However, copying the Testdir does indeed work.
This is part of my Dockerfile:
FROM python:3.7-buster
# -- Init --
RUN mkdir -p /app/src
WORKDIR /app/src
ADD WebApp/Testdir ./Testdir #works
ADD WebApp/Bilder ./Bilder #doesn't work
CMD ["sleep", "50"] #to check contents
I build the image as part of a docker-compose.yml file with
docker-compose build test
Does anyone have a clue what's happening here? I've been searching for a solution for quite some time...
If anyone is interested by why this was a problem: it actually had nothing to do with Docker. I was working on a cluster that was not synchronizing my local files to the server correctly, so I solved this issue by checking every time whether the files were actually copied from my local machine to the cluster. Just in case someone has a similar issue, you might be advised to check whether the file accessibility could be the problem.
I am trying to follow the 2 steps mentioned below:
1) Downloaded source code of
https://sourceforge.net/projects/hunspell/files/Hyphen/2.8/hyphen-2.8.8.tar.gz/download
2) Compiled it and you will get binary named example:
hyphen-2.8.8$ ./example ~/dev/smc/hyphenation/hi_IN/hyph_hi_IN.dic
~/hi_sample.text
I have downloaded and uncompressed the tar file. My question is how to create a dockerfile to automate this?
There are only 3 commands involved:
./configure
make all-recursive
make install
I can select the official python image as a base container. But how do I write the commands in a docker file?
You can do that with a RUN command:
FROM python:<version number here>
RUN ./configure && make-recursive && make install
CMD ['<some command here>']
what you use for <some command here> depends on what the image is meant to do. Remember that docker containers only run as long as that command is executing, so if you put the configure/make/install steps in a script and use that as your entry point, it's going to build your program, and then the container will halt.
Also you need to get the downloaded files into the container. That can be done using a COPY or an ADD directive (before the RUN of course). If you have the tar.gz file saved locally, then ADD will both copy the file into the container and expand it into a directory automatically. COPY will not expand it, so if you do that, you'll need to add a tar -zxvf or similar to the RUN.
If you want to download the file directly into the container, that could be done with ADD <source URL>, but in that case it won't expand it, so you'll have to do that in the RUN. COPY doesn't allow sourcing from a URL. This post explains COPY vs ADD in more detail.
You can have the three commands in a shell script and then use the following docker commands
COPY ./<path to your script>/<script-name>.sh /
ENTRYPOINT ["/<script-name>.sh"]
CMD ["run"]
For reference, you can create your docker file as they have created for one of the projects I worked on Apache Artemis Active Mq:
https://github.com/apache/activemq-artemis/blob/master/artemis-docker/Dockerfile-ubuntu
Can Docker's COPY or RUN cp be used in a Dockerfile to overwrite a default config file with a docker-specific version of the file?
In a Rails project, our config folder has multiple versions of database.yml for different environments:
# projectname/config/
database.yml # an unused default placeholder
database_for_docker_2.yml
database_for_vagrant.yml
For different dev environments (vagrant+virtualbox vs docker) during initialization of the machine/container we copy the appropriate version of the .yml to database.yml
In the Dockerfile, after this section:
WORKDIR /my_app
RUN bundle install
COPY . /my_app
we tried:
RUN cp ./config/database_docker_2.yml /my_app/config/database.yml
but the file does not seem to be copied, the default version of database.yml is used when we spin up the container.
we then tried:
COPY ./config/database_docker_2.yml /my_app/config/database.yml
the file still does not seem to be copied, the default version of the file gets used when we spin up the container.
What DOES work is adding another entry to the volume section of docker-compose.yml specifically for that one file:
volumes:
- .:/my_app
- ./config/database_docker_2.yml:/my_app/config/database.yml
but we prefer to manage the placement of env-specific versions of files in the Dockerfile (as opposed to littering the docker-compose.yml with such env-specific files)
The command COPY ./config/database_docker_2.yml /my_app/config/database.yml probably works, there is no reason it shouldn't assuming the source exists.
What I suspect happens, is that when you are testing it, you already have a volume with .:/my_app, which then shows you the local folder, and not the in-container folder.
Run it without the volume, and I believe you will in fact see that it copied it into the container, as you intended.
On a side note:
If you are not yet locked in your way of handling this multiple database config, I would consider re-evaluating your situation, and trying to find a solution that does not require you to change database.yml for each environment. One way, would be to have the database.yml use an environment variable (usually DATABASE_URL) and then you have one docker-compose for all, and one database.yml for all, and you only configure environment with environment variables.
I'm working on building a website in Go, which is hosted on my home server via docker.
What I'm trying to do:
I make changes to my website/server locally, then push them to github. I'd like to write a dockerfile such that it pulls this data from my github, builds the image, which my docker-compose file will then use to create the container.
Unfortunately, all of my attempts have been somewhat close but wrong.
FROM golang:1.8-onbuild
MAINTAINER <my info>
RUN go get <my github url>
ENV webserver_path /website/
ENV PATH $PATH: webserver_path
COPY website/ .
RUN go build .
ENTRYPOINT ./website
EXPOSE <ports>
This file is kind of a combination of a few small guides I found through google searches, but none quite gave me the information I needed and it never quite worked.
I'm hoping somebody with decent docker experience can just put a Dockerfile together for me to use as a guide so I can find what I'm doing wrong? I think what I'm looking for can be done in only a few lines, and mine is a little more verbose than needed.
ADDITIONAL BUT PROBABLY UNNECESSARY INFORMATION BELOW
Project layout:
Data: is where my go files are Sidenote: This was throwing me errors when trying to build image, something about not being in the environment path. Not sure if that is helpful
Static: CSS, JS, Images
TPL: go template files
Main.go: launches server/website
There are several strategies:
Using of pre-build app. Build your app using
go build command according to target system architecture and OS (using GOOS and GOARCH system variable for example) then use COPY docker command to move this builded file (with assets and templates) to your WORKDIR and finally run it via CMD or ENTRYPOINT (last is preferable). Dockerfile for this example will look like:
FROM scratch
ENV PORT 8000 EXPOSE $PORT
COPY advent / CMD ["/advent"]
Build by dockerfile. Typical Dockerfile:
# Start from a Debian image with the latest version of Go installed
# and a workspace (GOPATH) configured at /go.
FROM golang
# Copy the local package files to the container's workspace.
ADD . /go/src/github.com/golang/example/outyet
# Build the outyet command inside the container.
# (You may fetch or manage dependencies here,
# either manually or with a tool like "godep".)
RUN go install github.com/golang/example/outyet
# Run the outyet command by default when the container starts.
ENTRYPOINT /go/bin/outyet
# Document that the service listens on port 8080.
EXPOSE 8080
Using GitHub. Build your app and pull to dockerhub as ready to use image.
Github supports Webhooks which can be used to do all sorts of things automagically when you push to a git repo. Since you're already running a web server on your home box, why don't you have Github send a POST request to that when it receives a commit on master and have your home box re-download the git repo and restart web services from that?
I was able to solve my issue by just creating an automated build through docker hub, and just using this for my dockerfile:
FROM golang-onbuild
EXPOSE <ports>
It isn't exactly the correct answer to my question, but it is an effective workaround. The automated build connects with my github repo the way I was hoping my dockerfile would.
I used to list the tests directory in .dockerignore so that it wouldn't get included in the image, which I used to run a web service.
Now I'm trying to use Docker to run my unit tests, and in this case I want the tests directory included.
I've checked docker build -h and found no option related.
How can I do this?
Docker 19.03 shipped a solution for this.
The Docker client tries to load <dockerfile-name>.dockerignore first and then falls back to .dockerignore if it can't be found. So docker build -f Dockerfile.foo . first tries to load Dockerfile.foo.dockerignore.
Setting the DOCKER_BUILDKIT=1 environment variable is currently required to use this feature. This flag can be used with docker compose since 1.25.0-rc3 by also specifying COMPOSE_DOCKER_CLI_BUILD=1.
See also comment0, comment1, comment2
from Mugen comment, please note
the custom dockerignore should be in the same directory as the Dockerfile and not in root context directory like the original .dockerignore
i.e.
when calling
DOCKER_BUILDKIT=1
docker build -f /path/to/custom.Dockerfile ...
your .dockerignore file should be at
/path/to/custom.Dockerfile.dockerignore
At the moment, there is no way to do this. There is a lengthy discussion about adding an --ignore flag to Docker to provide the ignore file to use - please see here.
The options you have at the moment are mostly ugly:
Split your project into subdirectories that each have their own Dockerfile and .dockerignore, which might not work in your case.
Create a script that copies the relevant files into a temporary directory and run the Docker build there.
Adding the cleaned tests as a volume mount to the container could be an option here. After you build the image, if running it for testing, mount the source code containing the tests on top of the cleaned up code.
services:
tests:
image: my-clean-image
volumes:
- '../app:/opt/app' # Add removed tests
I've tried activating the DOCKER_BUILDKIT as suggested by #thisismydesign, but I ran into other problems (outside the scope of this question).
As an alternative, I'm creating an intermediary tar by using the -T flag which takes a txt file containing the files to be included in my tar, so it's not so different than a whitelist .dockerignore.
I export this tar and pipe it to the docker build command, and specify my docker file, which can live anywhere in my file hierarchy. In the end it looks like this:
tar -czh -T files-to-include.txt | docker build -f path/to/Dockerfile -
Another option is to have a further build process that includes the tests. The way I do it is this:
If the tests are unit tests then I create a new Docker image that is derived from the main project image; I just stick a FROM at the top, and then ADD the tests, plus any required tools (in my case, mocha, chai and so on). This new 'testing' image now contains both the tests and the original source to be tested. It can then simply be run as is or it can be run in 'watch mode' with volumes mapped to your source and test directories on the host.
If the tests are integration tests--for example the primary image might be a GraphQL server--then the image I create is self-contained, i.e., is not derived from the primary image (it still contains the tests and tools, of course). My tests use environment variables to tell them where to find the endpoint that needs testing, and it's easy enough to get Docker Compose to bring up both a container using the primary image, and another container using the integration testing image, and set the environment variables so that the test suite knows what to test.
Sadly it isn't currently possible to point to a specific file to use for .dockerignore, so we generate it in our build script based on the target/platform/image. As a docker enthusiast it's a sad and embarrassing workaround.