I'm working on integrating Docker into our TeamCity build process so that I can create a task that runs a "docker build" to create an image from our code. Right now, all our build agents run on either Windows Server 2008 or Windows Server 2012, neither of which can run Docker. There's a chance we can get a license for one Windows Server 2016 build machine, but I'm wondering if there's a way to run Docker Engine on that machine while issuing docker commands from other build agents.
Here's what I've considered so far:
Docker Toolkit: This is a way to run Docker on legacy systems, but it spins up a local VirtualBox VM running Linux thus it can only run Linux containers. I need to be able to build and run Windows containers.
Docker Machine: This is a way to talk to a remote Docker engine. However, according to this open bug, it appears Docker Machine is only capable to talking to remote engines on Linux hosts due to security implementations; It's an old issue but I can't find any indication this limitation has been removed.
Docker itself uses a client/server architecture, but I couldn't find any documentation on how to talk to a remote engine without using something like Docker Machine.
Anything else I'm missing, or am I just pretty much out of luck unless we upgrade all our build agents to Windows 10 or Windows Server 2016?
You can start using the remote Windows Server 2016 instance from other build agents.
Docker allows to expose the Docker Engine (aka Daemon) via tcp. In that case and especially when the host is publicly reachable you should consider configuring authentication using client/server certificates. Details can be found in the official documentation at https://docs.docker.com/engine/security/https/, but you may find the Windows Server specific article at https://stefanscherer.github.io/protecting-a-windows-2016-docker-engine-with-tls/ more helpful.
Regarding your aspect of using a client to connect to a remote Docker Engine, please use the -H tls://<host>:<port> argument like described at https://docs.docker.com/engine/reference/commandline/cli/ (or see the example provided at https://stefanscherer.github.io/protecting-a-windows-2016-docker-engine-with-tls/#testtlsconnection).
Related
I am new to the containers topic and would appreciate if this forum is the right place to ask this question.
I am learning dockers and containers and I now have some skills using the docker commands and dealing with containers. I understand that docker has two main parts, the docket client (docker.exe) and the docker server (dockerd.exe). Now in the development life both are installed on my local machine (I am manually installed them on windows server 2016) followed Nigel Poulton tutorial here https://app.pluralsight.com/course-player?clipId=f1f27565-e2bf-4e58-96f3-bc2c3b160ec9. Now when it comes to the real production life, then, how would I configure my docker client to communicate with a remote docker server. I tried to make some research on the internet but honestly could not find a simple answer for this question. I installed docker for desktop on my windows 10 machine and noticed that it created a hyper-v machine which might be Linux machine, my understanding is that this machine has the docker server that my docker client interacts with but do not understand how is this interaction gets done.
I would appreciate if I get some guidance or clear answer to my inquiries.
In production environments you never have a remote Docker daemon. Generally you interact with Docker either through a dedicated orchestrator (Kubernetes, Docker Swarm, Nomad, AWS ECS), or through a general-purpose system automation tool (Chef, Ansible, Salt Stack), or if you must by directly ssh'ing to the system and running docker commands there.
Remote access to the Docker daemon is something of a security disaster. If you can access the Docker daemon at all, you can edit any file on the host system as root, and pretty trivially take over the whole thing. (Google "Docker cryptojacking" for some real-world examples.) In principle you can secure it with mutual TLS, but this is a tricky setup.
The other important best practice is that Docker images should be self-contained. Don't try to deploy a Docker image to production, and also separately copy your application code. The same Ansible setup that can deploy a Docker container can also install Node directly on the target system, avoiding a layer; it's tricky to copy application code into a Kubernetes volume, especially when Kubernetes pods can restart outside your direct control. Deploy (and test!) your images with all of the code COPYd in a Dockerfile, minimizing the use of bind mounts.
By any chance can Pycharm on Windows be configured to run docker inside a linux VM (VirtualBox) ?
Meaning that i would like to take advantage of the Pycharm Docker tools of professional version but avoid installing docker for windows, since im using Linux VMs that have Docker installed.
Tried to set the tcp://xx.xx.xx.xx:2375 of the guest host vm but failing. Not sure if this can be done at all.
There's no native support, but you can configure PyCharm to work with remote Docker daemon. Basically you need a deployment configuration alongside the Docker-based remote interpreter to transfer the project files to remote machine where they can be mounted inside a container.
See the detailed instruction in PyCharm's bug tracker https://youtrack.jetbrains.com/issue/PY-33489 and please vote for the ticket if you want this support to be a first-class citizen in PyCharm.
Practically I want to play with .NET Core within Docker.
So as I understand it from this post to give myself the best flexibility I would install "Docker For Windows". Means I can ultimately deploy my .Core app to a container that is either a Windows or a Linux container. However the Linux container is still a Hyper-V managed Linux container.
1) Is there a way to instead use the Windows Subsystem for Linux (WSL) to do this in the Windows 10 Creators Update? Seems like less overhead than have Windows/Docker manage a separate Linux VM for me?
No, running Docker containers in WSL is not supported (link mine):
The docker engine is not a supported scenario in the short term. I would suggest hitting our User Voice page and upvoting Docker if you're looking to run the docker engine.
The docker client however should be running in build 14342. I have been able to run the docker client and connect to a docker engine running in a VM.
As to why it's not supported:
WSL is a clean-room kernel reimplementation. So it can't, for both technical and legal reasons, simply take the kernel components of Docker and "make it work". They would need to reverse-engineer years of ongoing kernel development and reimplement it. (Or take some other nontrivial approach.)
we use VMWare vSphere for VMs in our company.
To automatically create docker hosts we use one simple command:
docker-machine --driver vmwarevsphere .... vm params(cpu,memory,network,name, etc)
It automatically creates new VM machine in our VM cluster, installs docker and then we add it to swarm or create new.
Right now I need to create windows docker hosts to run windows containers.
Docker-machine installs boot2docker.iso after creating VM.
But instead I need VM with microsoft servercore or nano.
How do I do it?
Thanks a lot.
Anton
On a Windows machine with Docker for Windows installed you could run the following command to pull the official images for server or nanoserver
docker pull microsoft/nanoserver
or
docker pull microsoft/windowsservercore
I'm not exactly sure how you're automating this - are you using a dockerfile or docker compose?
Are you talking about setting up the Windows host that runs Docker engine? If so, Docker for Windows CE is meant to be desktop software so not recommended for server side workload. Also, Windows EE Server requires Windows Server 2016 or later. If you would really like to use Windows server core mode, Windows Server 1709 offers that. Still, it quite bit new, so you should not set high expectations just yet.
As per the instruction to install the engine, MS has this.
https://learn.microsoft.com/en-us/virtualization/windowscontainers/quick-start/quick-start-windows-server
Or, equivalent one from Docker here.
https://docs.docker.com/engine/installation/windows/docker-ee/
you are talking about hosting a windows container on VMware vSphere? I don't think this is possible right now, may be in the future. I have no documentation or link to verify my answer but in our company we have a similar situation and use vSphere for VMs and Linux container and Hyper-V in parallel for VMs and windows container.
I've been reading a lot about Docker and its uses. Currently we are using TFS for building and deploying our in house app (.net c#). Are there any benefits for transitioning to Docker? Meaning having it set up so that after TFS runs the build, it will deploy to a container?
Currently there is no easy way to run the build ourselves besides doing bunch of manual steps: How to: Build Team Projects on a Desktop.
Can you run docker on Windows Server?
Would we be able to setup our VMs with docker so that we can deploy/mimic deployment process locally (to our vms)?
And have the same process set up for our production environment?
Yes, you can run docker on Windows Server, but it cannot run natively on Windows for now.
Because Docker relies on Linux-specific features, you can’t run Docker
natively in Windows. Instead, you must install the Docker Toolbox
application. The application installs a VirtualBox Virtual Machine
(VM), Docker itself, and the Docker Toolbox management tool. These
three things allow you to run Docker on Windows.
For the deployment with docker, refer to this link for details: http://www.colinsalmcorner.com/post/continuous-deployment-with-docker-and-build-vnext
In the meanwhile Microsoft has released their Windows Container feature within Windows Server 2016 and Windows 10 Pro and up. One can start running container natively on Windows. Mind you, you can only run Windows-based container images. So this cannot be leveraged if you want to run Linux-based container images, which currently is still the majority. Unfortunately at this moment one cannot easily filter within Docker Hub on the desired container-base.
Refer to https://learn.microsoft.com/en-us/virtualization/windowscontainers/