I am not sure I am understanding Kubernetes Init containers properly. What i want to do is run an initialization on the pod so that it pip installs some additional libraries that are not in my app container image. Specifically I am wanting to install the Azure storage queue such that i can use it with the standard TensorFlow image.
I set up my init container with the command "pip install azure-storage-queue" and that ran fine however my app container tells me "No module named azure"
Is this not how an init container can be used?
NOTE: I realize i could create a new image with all my prerequisites installed however this is just for development purposes
That's not really how init containers work... Init containers are meant to initialize the pod and the image isn't really shared with other containers that will later run on that pod.
The best solution is to create a new container image including the Python modules you need.
An alternative is to use a command to run in your container that first installs the modules using pip and later runs the script that needs them, that way you can avoid creating a new container image.
Related
I'm in the process of learning about Docker by trying to containerify a Strapi CMS. The default Docker image (https://github.com/strapi/strapi-docker) works well enough as a starting point, but I'm trying to add a couple packages to the Strapi instance for my needs (adding in Azure storage account support using https://www.npmjs.com/package/strapi-provider-upload-azure-storage). As I'm new to Docker, I'm having a hard time figuring how to make the container install that package as part of the Docker run process.
I see that the strapi/base image Dockerfile contains this line referencing a package.json file:
COPY ./package.json ./
I'm assuming that's where I would add a reference to the packages I'm wanting to install so that later on they are installed by npm, but I'm not sure where that package.json file is located, let alone how to modify it.
Any help on figuring out how to install that package during the Docker run process is greatly appreciated!
I figured out that strapi-docker uses a script to build images and not just the Docker files in the repo (bin/build.js). I also discovered that docker-entrypoint.sh is where the dependency installation is happening, so I added a couple of npm install statements after the check for the node_modules directory. Doing this allowed me to successfully add the desired packages to my Docker container.
I followed some of the advice from the Docker team here:
https://www.docker.com/blog/keep-nodejs-rockin-in-docker/
After performing the initial setup with docker-compose and the strapi/strapi image, I was able to install additional dependencies directly inside the container using docker-compose run <service name> yarn add <package>.
I opted for this route since I was having trouble installing the sharp library - it has different dependencies/binaries for Linux and Mac OS. This approached worked well for me, but the downside is that you can't mount your node_modules folder as a volume and it may take a little longer to install packages in the container.
I'm looking at a way to run a docker container with all the requirements already provided to avoid to wait on the download of the requirements.
I'm debugging python lambda locally.
I use the sam-cli integration in PyCharm.
To specify the requirements I have them all listed in a requirements.txt file.
When I run the debug sam build is executed with the user container setting.
This goes an fetches all the requirements from the internet into the container and then executes it.
When I'm working offline or with slow internet I would like to be able to use a container that has all the requirements. This will also be great to speed up the debugging process.
How can I setup my environment so it uses a pre-built docker container?
Build a new container base on old container and add a RUN instruction to install all you requirements to new layer on new image.
I’m struggling to install some python dependencies separate from an official docker image (Odoo:12.0). I’m trying to learn docker but I’m not sure what to do in this case. I’ve tried:
Rebuilding the image adding the dependencies in the Dockerfile
(Somehow a bunch of dependencies fail to install via this method…)
Manually entering the docker container and downloading the dependencies there (Odoo doesn’t recognize the dependencies and
tells me the dependency is missing.)
I’ve read that one could make a sort of separate volume with those extra dependencies but I haven’t achieved much. Any ideas on how to proceed?
Cheers
I am again and again downloading and install a package whenever a pod restarts so how can I automate it.
I wanted the pod to automatically install or download a package when the pod restarts.how can I do it in Kubernetes?
Option 1 (best practice):
If you want the package to be downloaded and installed in the container inside of the Pod when it starts, then you must add those download and install instructions to the Dockerfile using the RUN directive.
If you’re using a public image, and not your own custom image, then you’ll want to create a Dockerfile and use the public image as the base image to create and push your own custom Docker image to a image repository. You do this by adding the FROM directive as the first line of the Dockerfile.
There are a ton of tutorials out there on how to build images with Dockerfiles and push them to a repository for use by Kubernetes. Here’s the official documentation from Docker which explains the aforementioned directives and everything else you need to know to create Dockerfiles, build them into Docker images, and push them to a image repository like Docker Hub.
In short, you’ll end up with a Dockerfile that looks something like this, which adds the instructions to download and install the package to a base image:
FROM <base image (i.e. the image you’re currently using)>
RUN <download command>
RUN <install command>
The linked documentation tells you how to build and push a Docker image to a repo, and then it’s just a matter of updating the image field in your Pod manifest.
Now whenever your Pod restarts it will already have the package installed.
Option 2 (anti-pattern, not recommended):
Caution: I'm including this option because OP does not want to use Option 1, so I'm including one possible, theoretical alternative.
You could use an init container to accomplish this. The init container would need to download and install a package for your app container and put it in a mounted emptyDir volume shared by the init container and app container. This work would be done using the command property in the init container.
After your init container runs, your app container can access the package via the mounted shared emptyDir volume.
More about sharing data between containers.
There are several reasons I could think of off the top of my head for why this is an anti-pattern:
It slows down Pod startup time.
Your emptyDir volume is mutable and if you somehow delete or corrupt the package, and your app container (not Pod) crashes, your Pod will crash loop as your init container only runs on Pod restarts, not container restarts.
It unnecessarily complicates your Pod manifest.
A pod should be complete - it shouldn't need to download and install a package. You are essentially trying to do a 'build step' during pod startup, this is almost certainly an anti-pattern.
If the image you are using doesn't contain a package then you should build your own image and deploy it to a docker registry or look for a different docker image that contains your required package.
ericstaples has done a good job of explaining how to roll your own docker image and push it do a docker repository.
If you edit your question and provide the image repo:tag that you are currently using and also the package you want to install then you may receive better responses.
Let's say I create a docker image called foo that contains the apt package foo. foo is a long running service inside the image, so the image isn't restarted very often. What's the best way to go about updating the package inside the container?
I could tag my images with the version of foo that they're running and install a specific version of the package inside the container (i.e. apt-get install foo=0.1.0 and tag my container foo:0.1.0) but this means keeping track of the version of the package and creating a new image/tag every time the package updates. I would be perfectly happy with this if there was some way to automate it but I haven't seen anything like this yet.
The alternative is to install (and update) the package on container startup, however that means a varying delay on container startup depending on whether it's a new container from the image or we're starting up an existing container. I'm currently using this method but the delay can be rather annoying for bigger packages.
What's the (objectively) best way to go about handling this? Having to wait for a container to start up and update itself is not really ideal.
If you need to update something in your container, you need to build a new container. Think of the container as a statically compiled binary, just like you would with C or Java. Everything inside your container is a dependency. If you have to update a dependency, you recompile and release a new version.
If you tamper with the contents of the container at startup time you lose all the benefits of Docker: That you have a traceable build process and each container is verifiably bit-for-bit identical everywhere and every time you copy it.
Now let's address why you need to update foo. The only reason you should have to update a dependency outside of the normal application delivery cycle is to patch a security vulnerability. If you have a CVE notice that ubuntu just released a security patch then, yep, you have to rebuild every container based on ubuntu.
There are several services that scan and tell you when your containers are vulnerable to published CVEs. For example, Quay.io and Docker Hub scan containers in your registry. You can also do this yourself using Clair, which Quay uses under the hood.
For any other type of update, just don't do it. Docker is a 100% fossilization strategy for your application and the OS it runs on.
Because of this your Docker container will work even if you copy it to 1000 hosts with slightly different library versions installed, or run it alongside other containers with different library versions installed. You container will continue to work 2 years from now, even if the dependencies can no longer be downloaded from the internet.
If for some reason you can't rebuild the container from scratch (e.g. it's already 2 years old and all the dependencies went missing) then yes, you can download the container, run it interactively, and update dependencies. Do this in a shell and then publish a new version of your container back into your registry and redeploy. Don't do this at startup time.