I am very new to the realm of dockers. I want to make sure I have understood the safety part of it correctly.
Imagine the following case:
I create an app that consists of multiple scripts and models.
I dockerize my app.
I host the dockerized app by using a cloud platform on their servers.
The app has an UI that can be accessed by anyone online, for instance through a web link.
The question is:
Can a person from the outside world access to the contents of this app in any way - or may I sleep in peace and be sure no one can see the stuff inside it?
As part of dockerizing your application, you exposed ports that allow interaction with the container (typically in your Dockerfile). If everything is configured correctly, then external visitors can only access the contents of the container via that port or ports.
Running your container at a well-known provider is a great start, but not a guarantee of a secure configuration.
A few things to consider:
Whatever runs on the port or ports that you expose, can provide whatever info from the container. The service there should be secure itself, regardless of Docker.
You host your Docker image in a registry, where the platform starts it from. That registry should also be configured to not allow unauthorized access to the image.
You should have no secrets in Docker images anyway. If the image needs some kind of a secret, that should be provided at runtime (eg. via environment variables), or even better, downloaded from a secret vault.
Related
We have a couple of single page apps that we want to host on a single web server. I'm only talking about the frontend part (Angular, React). The APIs run elsewhere. Each app is basically just a directory with a collection of static files (js, html, css, etc.) generated by the CI process. In fact, the build process creates one Docker image per app. Each image basically just contains a directory that contains the build artifacts.
All apps should appear in different folders on the same website:
/app1
/app2
/app3
What would be the best practice for deploying the apps? We've come up with a few strategies.
1. A single image / container
We could build a final web server image (e.g. Apache) and merge all the directories from the app images into it.
Cons: Versioning sounds like hell. Each new version of an app causes a new version of the final image. What if we want to revert to an older version of an app while a newer version of another app has already been deployed?
2. Multiple containers with a front-end reverse proxy
We could build each app image with its own built-in web server. And then route them all together with a front-end reverse proxy (nginx, traefik, etc.).
Cons: Waste of resources running multiple web servers.
3. One web server container and multiple data-only containers for the apps
Deploy each app in a separate container that provides it's app directory as a volume but does nothing else. Then there is a separate web server container that shares the same volumes in order to have access to all the files.
So far I like the 3rd variant best. Whenever a new version of an app needs to be deployed, we simply do a Docker pull on a new version of its image. But it still seems hacky. Volumes must be deleted manually. Otherwise the volume will not be seeded with the new content. Also having containers that do nothing isn't the Docker way, isn't it?
A Docker container wraps a process, but your compiled front-end applications are static files. That is, the setup you're describing here doesn't really match Docker's model.
Without Docker you could imagine deploying these to a single directory
/var/www/
app1/
index.html
css/app.css
app2/
index.html
css/app2.css
js/main.js
and serve these with a single HTTP server; you would not typically run a separate server for each front-end application.
A totally reasonable option, in fact, is to completely ignore Docker here. Even if your back-end applications are being served from containers, you can publish your front-end code (again, compiled to static files) via whatever hosting service you have conveniently available. Things like Webpack's file hashing can help support deploying updated versions of the application without breaking existing clients.
If I was using Docker I'd use either of your first two options but not the third. Running a combined all-the-front-ends HTTP server is the same pattern already discussed, except the HTTP server is in a container instead of the host. Running a dedicated HTTP server for each front-end application lets you use Docker's image versioning, and the incremental cost of an additional HTTP server isn't that expensive.
I would avoid any approach that involves named volumes or "data-only containers". Nothing ever automatically copies content into a volume, except for one specific corner case (on native Docker only, using named volumes but not any other kind of mount, only the first time you use a volume but never updating the volume content), and so you'd have to manually write code to copy content out of an image into a shared hosting location; that's more complicated and doesn't really gain you anything over directly running Webpack on the host.
This is related to docker container which is deployed in microk8s cluster. The container when deployed thru k8s with host volume mounted inside it. when the container runs, it makes few keys and token generation to establish a secure tunnel with another container outside of this node. The container creates those keys inside the provided mount path. The keys and token which are generated are created as plain files (like public.key, private.key, .crt, .token etc) under the mounted path inside container. Also the tokens are refreshed in some time interval.
Now I want to secure those tokens/keys which are generated post container runs so that it can't be accessed by outsiders to harm the system/application. Something kind of vault store, but I want to maintain inside container or outside the container on host in some encrypted way. So that whenever the container application wants the files, it can decrypt from that path/location and use it.
Is there any way this can be achieved inside docker container system based on Ubuntu 18 host OS and k8s v1.18. Initially I thought of linux keyrings or some gpg encrypt mechanism. But I am bot sure whether it can affect the container runtime performance or not. I am fine to implement any code in python/c to encrypt/decrypt the files for the application inside container. But the encryption mechanism should be fips compliant or industry standard.
Also anyway we can encrypt the directory where those keys are generated and use it decrypting when needed by the application..or some directory level permission we can set so that it can't read by other users to make those files secure.
thanks for reading this long post. However I donot have a clear solution for this as of now. any pointers and suggestion in this regard is much appreciated.
thanks
I have created a simple PHP api application that works with a mysql database to store data. I have been experimenting with Kubernetes on my Windows 10 machine through Minikube.
I have just about got my head round the ideas involved, yet I’m not sure about how to implement this properly. So far I have used Kompose to create a set of yaml files from an existing docker-compose file. This has been half successful.
To get my application code into a pod hosting PHP, I have been using hostPath to share from my local machine. I mount to the minikube machine and share from there. I was having trouble sharing by other means. The application code is hosted in a github repo.
My questions are:
Is mounting my application code into a pod (assuming this is similar to what happens in docker) the correct way to do this? I’m not clear exactly what information is held on an image retrieved from the docker hub. Although I have read up on containers isolating the build environment from your machine.
How does this approach to translate into a production environment hosted on a cloud? I see there are various storage types. I had for example, wanted to try deploying on AWS just to see how this would work in practice.
I’m really looking for guidance to go from the tutorials found on the web working on my machine, to something that could be done for a customer hosted on the cloud. This might scale up to a more microservices style architecture over time.
The approach you are describing is mostly for development setups, where you want to mount your code into the container as a volume so you don't have to rebuild every time your code changes. Typically done with a docker-compose file.
For production setups, you want the docker image to correctly work and only mount volumes to data you want to persist, typically databases are the core example. For this EKS is deeply integrated into the AWS infrastructure and will create EBS volumes on demand. You don't need to provision any volume or even care for most cases (unless you need multiple read-write volumes needed for scaling).
For a PHP application you really should not persist any data in the pod, because it will create other issues when you need to scale the application. Also, a good approach for managing files that need to persist is S3 (AWS simple storage service).
So generally speaking, you need a deployment per application a service to access each pod on that application and then an ingress object to route traffic from the internet to each pod.
Your application docker image is really the core. You just build it with your code inside. Make sure to pass configuration using environment variable or configuration file so you can connect to the database.
Now for kubernetes, for each compoment (e.g. PHP application, MySQL) you will most likely create a deployment k8s manifest that points to the docker image and add some configuration environment variables.
For production, you will need persistence volume. On aws you can simply use EBS-backed volumes
To get traffic from Internet to your PHP application, you will need to add one or more k8s components:
K8s Service manifest that exposes your PHP deployment/pod on a stable address. If you only have q or very few services, you can use LoadBalancer which on cloud like AWS will create an ALB/ELB (might need to add annotation to your service)
An ingress which is just a reverse proxy (contour, nginx, traefik). On cloud environment it will map to an ALB/ELB. The advantage of this is that you can have a single ALB for all your services i.e. save money. Also you can configure routing path or TLS termination in one place.
I'm using Docker to build an nginx enviornment. I'm wondering if it's possible to expose to publish the ports (80, 443) during build so letsencrypt can run at build time (it needs network access to a server in the (intermediate) container).
Is this possible?
I have never seen that and i think that is not possible by design.
You should not place the secret key in the image
You might need to re-assure the license after 2 months and would need to rebuild the whole image
in general, this is done using a companion letsencrypt docker image, sometimes called sidekick. You basically have your app (and its containers) and a letsencrypt container, exposing a volume which nginx then mounts using volume_from this volume is were the letsencrypt container puts the fetched certificates. This happens during image-startup, not during image creation. You use a docker-compose file to configure anything needed.
E.g. you can have a look here
a) https://github.com/rancher/community-catalog/blob/master/templates/letsencrypt/2/docker-compose.yml
b) or http://letsencrypt.readthedocs.io/en/latest/using.html#running-with-docker
a) lets you defined the domains you are going to need using ENV variables, which will suite a docker-compose way very well, not providing any files like a configuration on the host ( keeps it portable ).
You can still put all this on the nginx-server, but its just not best practise, out of many reasons ( e.g. the need to configure nginx ).
If you want to stick to "build time", an alternative is using the DNS verify mode, so instead of verifying using connect-back on a port, you rather verify using a DNS-entry, some links for that
- https://github.com/lukas2511/letsencrypt.sh/wiki/Examples-for-DNS-01-hooks
- the a) container does this
For this scenario you might want to pick http://cloudflare.com - AFAIK it is the only DNS service with free API access for unlimited domains, anything else either costs money or has limits.
I am new to both Docker and Consul, and am trying to get a feel for how containerized apps could use Consul for both service registry and KV pair config management ("configuration").
My understanding was that I could:
Create an image that runs Consul server, so something like this; then
Spin up three of these Docker-Consul containers (thus forming a cluster/quorum) on myvm01.example.com (an Ubuntu VM); then
Refactor my app to use Consul and create a Docker image that runs my app and Consul agent, with the agent configured to join the 3-node quorum at startup. On startup, my app uses the local Consul agent to pull down all of its configurations, stored as KV pairs. It also pulls in registered/healthy services, and uses a local load balancing tool to balance the services it integrates with.
Run my app's containers on, say, myvm02.example.com (another Ubuntu VM).
So to begin with, if any of this seems like I am misunderstanding the normal/proper uses of Docker and Consul (sans Registrator), please begin by correcting me!
Assuming I'm more or less correct, I recently stumbled across Registrator and am now even more confused. Registrator seems to be some middleman between your app containers and your Consul (or whatever registry you use) servers.
After reading their Quickstart tutorial, it sounds like what you're supposed to do is:
Deploy my Consul cluster/quorum containers to myvm01.example.com like before
Instead of "Dockerizing" my app to use Consul directly, I simply integrate it with Registrator
Then I deploy a Registrator container somewhere, and configure it to integrate with Consul
Then I deploy my app containers. They integrate with Registrator, and Registrator in turn integrates with Consul.
My concerns:
Is my understanding here correct or way off base? If so, how?
What is actually gained by the addition of Registrator. It doesn't seem (to the untrained eye at least) like anything more than a layer of indirection between the app and the service registry.
Will I still be able to leverage Consul's KV config service through Registrator?
Is my understanding here correct or way off base? If so, how?
It seems to me, that it's not a good solution, to have all cluster/quorum members running inside the same VM. It's not so bad if you use it for development or tetsing or something, where you don't care much about reliability, but not for production.
Once your VM dies, you'll loose all the advantages you have by creating a cluster. And even more, you can loose all the data you have in K/V store, because you are running Consul servers inside a docker containers, which should be additionaly configured to share the configuration between runs.
As for the rest, I see it the same as you.
What is actually gained by the addition of Registrator.
From my point of view, the main thing is, that you don't have to provide an instance of Consul Agent in every container you run. And the container with the image you run is responsible only for their main functions, not for registering itself somewhere. You may simply pull an image and just run a container with it, to make it's service available, without making additional work.
Will I still be able to leverage Consul's KV config service through Registrator?
Unfortunately, no. At least, we didn't find a solution to use it this way, when we were looking for something to make service discovering and configuration management. We came to conclusion, that Registrator is not a proxy for K/V store and is used only to automate service discovery. So you have to use some other logic to access consul's K/V store.
Update: furthermore, here is 2 articles: "Automatic Docker Service Announcement with Registrator" and "Automatic container registration with Consul and Registrator", I found usefull to understand Registrator role in service discovery process.