There was a project thrown my way recently that involves the orchestration of several (Linux capable) embedded devices, deploying software to them, and allowing for the applications to be updated when the code base updates in a git repo.
The initial thought was to make a standard image for each device, and I set out, attempting to install docker on an UDOO Quad and an Intel Edison to start, but without any success up to this point.
My thinking is that it seems to be a good idea to install Docker on embedded devices--but if that's the case, surely it would have been ported by now. The only group out there that seems to be making these efforts is Resin.io.
Is there something I'm missing, or is there a clear reason why Docker doesn't make sense on embedded devices? If there isn't a reason, and it does make sense to run Docker on embedded systems, is there something I've overlooked out there: are there any sources of discussion on porting, or how-to's that cover this?
I have considered running docker on embedded devices (a mips system), but didn't go that way. There are some problems with it, in my humble view:
Docker is implemented in Golang. There is currently no available tool chain for mips to compile go. You will need to create the tool chain yourself using gcc-go.
The size of docker is larger than lxc. In a desktop computer this is not a problem, but the embedded device has limited flash storage.
Docker uses some quite up-to-date feature of linux kernel. Sometimes the kernel version on embedded devices are not so new and back-port is needed to make it work.
The docker image has to be built on the same architecture as the run time environment. It means that if you want to run a docker container on Raspberry Pi, the docker image has to be built on an ARM-architecture system. QEMU can be used to build docker image in the cloud, but it doesn't support all CPU architectures used in embedded system. (for example, it currently doesn't support MIPS)
In the end, lxc was chosen for the specific task of running a container on embedded device. It has limited features compared to docker, but currently it suits the requirement of the project.
As of year 2019, I would like to update this answer since I did port docker to embedded system with ARM cpu. With the price of flash usage, memory usage, by using docker you will have container management, image management, and many ready to run images from docker hub. So the decision is a balance between cost and features.
Here is an update for 2018:
You can work with Docker on embedded devices such as Raspberry Pi and Orange Pi quite easily now because of advancements in the development of Raspbian and Armbian operating system images. Specifically, both types of devices and their respective OS images now support kernels that are of sufficiently high enough versions to install Docker without any problems (at least version 3.10, though both now offer 4.x+ versions).
Your desire for faster rates of change can definitely be realized by using embedded Docker. I can say from experience that I have tested and regularly run the approach you describe. Basically, you start with a base operating system image such as Raspbian or Armbian, tweak that operating system enough that it's secure and has Docker installed, and then you use Docker for handling development iteration and application updates.
As an aside, if you are interested in running Docker on embedded Linux devices, then I recommend you check out a free, open-source, MIT-licensed command line tool I wrote to help developers work with embedded Docker on multiple devices at once: https://github.com/ForwardLoopLLC/floopcli .
Even if you are not interested in the tool itself, the documentation for the tool describes several patterns for working with Dockerized applications across multiple devices in multiple languages: https://docs.forward-loop.com/floopcli/master/index.html . The materials there should serve as a starting point for porting applications to Docker and then deploying them on embedded devices. The documentation also addresses some embedded device subtleties, such as differences between ARMv6 and ARMv7. Hopefully this helps you get started!
There is a great article on LinkedIn describing his experience with that
https://www.linkedin.com/pulse/whale-jar-when-running-docker-embedded-linux-good-thing-fletcher#pulse-comments-urn:li:article:7736487387895237975
Often embedded systems have a very slow rate of change. Docker works well on a minimum build then layering on top. If you want to sacrifice the overhead of running docker on a minimum embedded system for docker's ability to have a build system and steady rate of change then you could explorer it.
Related
Let's say I have old, unmaintained application that lives on a VPS (i.e. Symfony 3 PHP app that relies on PHP 5).
If some changes are needed I have to clone this app to my desktop, build it, change and re-deploy. As time goes, recreating desktop dev environment gets harder - in this example I can't simply build the app as I use PHP7 in my CLI that breaks building process.
I tried to dockerize the app, so I added Ubuntu 18 to my docker-compose file... and it doesn't work as latest Ubuntu that has PHP5 support is 14.04. 14.04 is also the oldest (official) version available on DockerHub. But will it be still available in 3 years? If not, Docker won't build a container.
So, my question is: is Docker a right tool to solve described problem at all?
If so, should I backup docker images described that my build relies on?
If not, beside proper maintenance, what tool is better?
You can install PHP5 in newer ubuntu versions, but it means adding an external repository.
You could also create your own docker image, containing only the libraries you want. If so, I'd advise to try and use alpine as a base image. There is a bit of a learning curve to adapt, but once you do it you'll have a small image tailored to your needs.
Given that containers allow you to isolate processus and conf with minimal footprint compared to a VM, I think it is the best option. Tailoring and maintaining your own image is not that expensive in terms of maintenance if you document it correctly, and it will allow you to always have a system 'maintaining' all your precise requirements.
Ive taken an app and built a docker image for windows server 2016 using microsoft/aspnetcore:2.0 base image.
My question is...what machines/OS's will I be able to run the container on?
I know it cant run on Linux.....but could it run on (e.g.) ANY version of windows server 2016? How about windows server 2019?
The architecture is AMD64....does that mean the container will only run on machines with that exact architecture?
Im trying to figure out why containers are considered beneficial
I don't have any experience with Docker Windows containers, but I have a ton of experience with Docker containers in general, and the concepts between Windows and Linux containers should be mostly the same.
When you run your built app, no matter if you run it on Windows Server 2016, Windows Server 2019, or even Windows 10 Pro, the app should function exactly the same. Under the covers, Docker provides an isolated application environment. From your applications perspective, it only knows/experiences/sees itself and the Windows Kernel that it's running on. If you had, say, an IIS instance also running on that server, your app would have no idea. The point here is that Docker provides a means to:
Run multiple versions of an app on the same machine, in complete isolation.
Have a more clean running environment for every app.
Be much more resource efficient than running discrete VMs
Another huge benefit of Docker is that it provides a means to ephemeral environments. Which means you should expect to have the exact same behavior from an app running on machine #1 as you do on machine #2. It eliminates the "works on my machine" mentality, especially when some other 3rd party dependency is not installed/forgotten, because these will be bundled into the container as part of the build.
Lastly, about architecture. The app you built is designed to run against the architecture of the Windows Kernel it was built with. In your case AMD64, from my understanding, this implies the x86_64 architecture. This should mean that your container will run on any 64-bit x86 machine (AMD or Intel). Your container will not run on any other architecture: x86 (32 bit), 386, 486, ARM, ARM 64, etc. I think in the case of Windows this isn't as important of an issue, because 90% of the time you're running on x86_64. But with Linux you end up with everything from SPARC to ARM, and so that architecture distinction is important.
I too had a lot of the same questions when I started using docker. While the product "Docker" has been hit-or-miss on occasion, the concept "containers" and the benefits they provide when used correctly are very powerful and I use the for almost every project I work on.
Let's say that I make an image for an OS that uses a kernel of version 10. What behavior does Docker exhibit if I run a container for that image on a host OS running a kernel of version 9? What about version 11?
Does the backward compatibility of the versions matter? I'm asking out of curiosity because the documentation only talks about "minimum Linux kernel version", etc. This sounds like it doesn't matter what kernel version the host is running beyond that minimum. Is this true? Are there caveats?
Let's say that I make an image for an OS that uses a kernel of version 10.
I think this is a bit of a misconception, unless you are talking about specific software that relies on newer kernel features inside your Docker image, which should be pretty rare. Generally speaking a Docker image is just a custom file/directory structure, assembled in layers via FROM and RUN instructions in one or more Dockerfiles, with a bit of meta data like what ports to open or which file to execute on container start. That's really all there is to it. The basic principle of Docker is very much like a classic chroot jail, only a bit more modern and with some candy on top.
What behavior does Docker exhibit if I run a container for that image on a host OS running a kernel of version 9? What about version 11?
If the kernel can run the Docker daemon it should be able to run any image.
Are there caveats?
As noted above, Docker images that include software which relies on bleeding edge kernel features will not work on kernels that do not have those features, which should be no surprise. Docker will not stop you from running such an image on an older kernel as it simply does not care whats inside an image, nor does it know what kernel was used to create the image.
The only other thing I can think of is compiling software manually with aggressive optimizations for a specific cpu like Intel or Amd. Such images will fail on hosts with a different cpu.
Docker's behaviour is no different: it doesn't concern itself (directly) with the behaviour of the containerized process. What Docker does do is set up various parameters (root filesystem, other mounts, network interfaces and configuration, separate namespaces or restrictions on what PIDs can be seen, etc.) for the process that let you consider it a "container," and then it just runs the initial process in that environment.
The specific software inside the container may or may not work with your host operating system's kernel. Using a kernel older than the software was built for is not infrequently problematic; more often it's safe to run older software on a newer kernel.
More often, but not always. On a host with kernel 4.19 (e.g. Ubuntu 18.04) try docker run centos:6 bash. You'll find it segfaults (exit code 139) because that old build of bash does something that greatly displeases the newer kernel. (On a 4.9 or lower kernel, docker run centos:6 bash will work fine.) However, docker run centos:6 ls will not die in the same way because that program is not dependent on particular kernel facilities that have changed (at least, not when run with no arguments).
This sounds like it doesn't matter what kernel version the host is running beyond that minimum. Is this true?
As long as your kernel meets Docker's minimum requirements (which mostly involve having the necessary APIs to support the isolated execution environment that Docker sets up for each container), Docker doesn't really care what kernel you're running.
In many way, this isn't entirely a Docker question: for the most part, user-space tools aren't tied particularly tightly to specific kernel versions. This isn't unilaterally true; there are some tools that by design interact with a very specific kernel version, or that can take advantage of APIs in recent kernel versions for improved performance, but for the most part your web server or database just doesn't care.
Are there caveats?
The kernel version you're running may dictate things like which storage drivers are available to Docker, but this doesn't really have any impact on your containers.
Older kernel versions may have security vulnerabilities that are fixed in more recent versions, and newer versions may have fixes that offer improved performance.
For CI purposes I have a need to set up a cluster of build slaves capable of building iOS apps. For now I'm relying on a single MacMini -with the aim to deploy several more in the future- and I'd like to virtualize several slaves on top of it. Some of these virtual slaves will build the iOS app, others will be smaller Linux slaves for miscellaneous purposes.
I'm completely new to Docker, so my main question is whether it's possible to dockerize Xcode 9.2 and/or MacOS in order to virtualize my iOS build slaves. I've seen very little literature out there on whether this can be achieved and I've found some images in hub.docker.com but they're not documented and don't appear to be very popular.
I'm going through a Docker tutorial right now and eventually will be attempting this -and if I'm successful I'll be answering my own question here for the benefit of others- but given the lack of information I have doubts on whether it is even possible or where I should even start.
Any tips or pointers on this would be greatly appreciated.
Or if anyone knows for fact that this is not possible and can explain why, that would also save me a lot of time.
OS X does not use the Linux kernel, so it cannot run in a Docker container
XCode is not open-sourced and does not have a Linux installer, so it cannot be used in a Linux Docker image.
It seems like your best bet is to build a Packer template using something like packer-macos osx-vm-templates and integrate that into your pipeline.
Look at Docker-OSX which runs macOS with Xcode support inside Docker.
You can connect to that macOS via SSH or VNC. It might be possible to use the same approach in CI/CD.
Related link from readme: "I want to use Docker-OSX for CI/CD-related purposes (sign into Xcode, Transporter)"
my question is little vague but I tried looking for the answer here and there but could not understand if I can leverage docker for my work. My requirements
I usually try different versions of java, python and other software like different versions of eclipse, Linux package and other tools. This at the end make my Ubuntu installation a complete mess and some time completely broken. Then I started using Vm it solve most of the problem but make my pc very slow for frequent switching.
So my question can I achieve my work using docker without affecting my os? Can I run gui application, install different package without affecting underlying OS.
Switch actively between different docker container and underlying os.
Clean/remove unused/broken install of docker instance (containers?) etc. Any pointer to similar use case or how to would be helpful.
Thanks.
Ps- if it doesn't fit for SO then please move it to where it is best fitted. Sorry for non programming question.
Can it be done?
yes, there are examples of docker images that run graphical application, but running those containers might be a bit tricky. See for instance Can you run GUI apps in a docker container?
Is Docker the right tool for your problem ?
Maybe a package manager such as Nix would be better suited, as graphical software installed with it won't have any issue. With Nix you can install side-by-side many versions of a single software without interference.