Like virtualization using VMs, we know that new circuits add extensions to handle that kind of virtualization. I am wondering if with containers such extensions exist, or maybe is there any study for a potential hardware support for containerization even if it happens in the OS level by default.
thank you.
Containers are not using hardware virtualization. Instead, they leverage built-in Linux functionality: namespaces and control groups.
Namespaces provided by the Linux kernel to allow separation of process ID space (PID),
and control groups allow accurate resource control to limit CPU or memory usage for each container.
So there is little need for special hardware support to run containers in 64-bit Linux.
(Windows and Mac are a different story)
Related
I need to perform some machine learning tasks using a Tensor Flow based Neuronal network architecture (PointNet https://github.com/charlesq34/pointnet). I would like to use cloud infrastructure to do this, because I do not have the physical resources needed. The demands of the customer are, that they would like to get the whole set up machine I used for the training afterward and not only the final model. This is because they are researchers and would like to use the machine themselves, play around and understand what I did but they do not want to do the setup/installation work on their own. Unfortunately they can not provide a (physical or virtual) machine themselves right now.
The question is: Is it possible/reasonable to set up a machine on a cloud infrastructure provider like google cloud or AWS, install the needed software (which uses Nvidia Cuda) and export this machine after a while when suitable hardware is available, import it to a virtualisation tool (like Virtual Box) and continue the usage on ones own system? Will the installed GPU/Cuda-related software like TensorFlow etc. still work?
I guess it's possible, but it will be needed to configure the specific hardware to make it work on the local environment.
For Google Cloud Platform,
the introduction to Deep Learning Containers, will you allow to create portable environments.
Deep Learning Containers are a set of Docker containers with key data science frameworks, libraries, and tools pre-installed. These containers provide you with performance-optimized, consistent environments that can help you prototype and implement workflows quickly. Learn more.
In addition, please check Running Instances with GPU accelerators
Google provides a seamless experience for users to run their GPU workloads within Docker containers on Container-Optimized OS VM instances so that users can benefit from other Container-Optimized OS features such as security and reliability as well.
To configure Docker with Virtualbox, please check this external blog.
As I understand, ACPI defines a generic hardware programming model where operating system relies on the OEM firmware provided AML (ACPI machine language) code to manipulate the hardware.
In order to execute the AML code, operating system has to incorporate an AML interpreter.
So, it looks to me that firmware developers use AML to provide a control interface between platform hardware and operating system.
But do we really need AML?
I think ultimately the hardware can only be configured through the native instruction of the platform. So the AML interpreter must translate the AML into native instructions otherwise it cannot be executed on the platform.
But what's the point of using an intermediate language like AML? I mean though the AML is said to be platform-independent, which means I can use AML to describe my platform in a non-native way.
But the AML is part of the platform firmware in practice. And the entire firmware has already been built into the target platform's native instructions. So what good can it be to make such a little part of the firmware as platform-independent? Why not just use native instructions? There must be some way to let OS use it as well. And this way operating system doesn't need the AML interpreter at all. A lot of complexity can be avoided.
One of the big goals of ACPI over its predecessor APM was to give the OS more viability and control over power state transitions.
APM was a black box. The OS knew nothing about the power management implementation. It would just call a BIOS function and the BIOS handled all of the magic. Did it work? Did the system sleep properly? Did the system freeze? Was a user application able to handle the BIOS implementation? The sad truth was that many systems had power management that was downright broken, and Microsoft wanted to provide a better power management experience for the growing laptop industry.
Now, the BIOS hands the ASL/AML code over to the OS and the OS executes it not the BIOS. If the BIOS code does something dumb (like messing with registers it shouldn't), Windows can detect that by parsing the code and block it. AML is 100% decompilable unlike C.
Remember that ACPI is not x86 specific. At the time it was developed, Itanium and Xscale were around. Intel and Microsoft needed a language that would work on all platforms, both 32 and 64 bit.
Lastly, ASL is more than just a list of executable functions. It is also number of static configuration tables. The ASL code has tables to define the non PnP hardware built onto your motherboard. It has tables of supported power states. A traditional programming language like C isn't really setup for that.
If ACPI was invented today, they would probably use something like XML to provide the info to the OS.
Originally, hardware for "80x86 PC" was cloned from IBM's PC, and this created an effective de-facto standard for hardware to follow. However it didn't take long before manufacturers wanted to add features that didn't previously exist, where there was no (official or de-facto) standard to follow.
This led to a major problem for operating system software (how do you support "non-standard chaos"). Some standards were created for some things (APM, etc) but they didn't really cover everything needed and became out-of-date. ACPI was created to fix this.
Ideally, what was (and still is) needed is standards that allow operating system to detect and use supported features of the motherboard. For example, a "standardised case temperature and fan control" device (with support for detecting how many fans, temperature sensors, etc), or a "standardised CPU speed/power consumption", a "PCI slot IRQ routing for IO APICs" standard, a "hot-plug PCI controller device" standard, etc.
However, ACPI didn't provide useful standards that hardware manufacturers and operating systems can use. Instead, ACPI provided an over-engineered mess (AML) to allow an OS to cope with ACPI's failure to standardise the hardware.
Essentially; we "need" AML now because it's the only viable way for an OS to work-around the "non-standard chaos" problem that ACPI failed to fix.
The problem with providing native code instead of AML is that different operating systems use CPUs in different ways (e.g. native 64-bit 80x86 code in firmware would be useless for an older "32-bit only" OS). AML provides portability between different types of CPUs and between the same CPU/s in different modes.
Also; native code is considered a major security problem (rootkits, etc); and people tend to think an interpreted language mitigates that problem. Of course in practice AML needs far too much access to the underlying hardware and does it in a way that an OS can't check, and there's isn't even a way for an OS to determine if the AML has been maliciously modified before the OS booted. For these reasons AML is still a major security problem despite using interpreted language.
I recently deployed a tiny Haskell app with docker, using "scratch-haskell" as a base image.
Then I read about Unikernels and HALVM. And I got a little confused.
My docker container is about 6MB large. A Unikernel (with the same haskell app) would be roughly the same size I guess.
The Unikernel runs directly on the Xen hypervisor, whereas the docker Image (or general LXC) runs on a normal Linux distribution, which runs on bare metal.
Now I have the "choice" of running Linux with multiple minimal containers OR a Xen machine with multiple small Unikernels.
But what are the advantages and disadvantages of those two solutions? Is one more secure than the other? And are there any significant performance differences between them?
from http://wiki.xenproject.org/wiki/Unikernels
What do Unikernels Provide?
Unikernels normally generate a singular runtime environment meant to
enable single applications built solely with that environment.
Generally, this environment lacks the ability to spawn subprocesses,
execute shell commands, create multiple threads, or fork processes.
Instead, they provide a pure incarnation of the language runtime
targetted, be it OCaml, Haskell, Java, Erlang, or some other
environment.
Unikernels Versus Linux Containers
Much has been made recently of the advantages of Linux Container
solutions over traditional VMs. It is said by container advocates that
their lightweight memory footprint, quick boot time, and ease of
packaging makes containers the future of virtualization. While these
aspects of containers are certainly notable, they do not spell the end
of the world of the hypervisor. In fact, Unikernels may reduce the
long-term usefulness of containers.
Unikernels facilitate the very same desirable attributes described by
the container proponents, with the addition of an absolutely splendid
security story which few other solutions can match.
So if you want just run Haskell application Unikernels may work for you, and they should have even less overhead than docker (and docker overhead is very small anyway), but if your application will need some prepared environment, need to communicate with non Unikernels software docker is a better choice. I guess it is too early to say will Unikernels be useful or widespread or not, only time will tell.
Unikernals are great for things that are stateless. When you start needing disk access you are better off using Docker.
That's why all the "killer" apps for unikernals are statically configured kernels, like static web pages or software defined networking stacks.
There are many good explations heres a simple one :
Unikernel are VMs but specialized and optimized for the particular application.
I am doing a class project comparing different programming languages. Is it possible to use BIOS interrupts in the Forth language? I can't seem to find any such information on this. If so what would be an example?
I think you're under a mistaken idea that there's a single all-encompassing "Forth" out there. There isn't. There are many Forth implementations. Those that run "bare bones" (without an OS) or under DOS can certainly be coaxed to access the BIOS APIs. Those that run under a 32 or 64 bit operating system like Windows or Linux are unlikely to provide such functionality, since the operating system makes it hard to run BIOS APIs to start with.
When running under Windows, using 16-bit BIOS APIs (as opposed to reading data without running BIOS code) is cumbersome. Modern BIOSes also offer 32-bit APIs, but in all cases you're limited to what hardware you can access (none) - this is enforced by the OS, not by the BIOS code.
Generally speaking, the BIOS APIs are cumbersome and there's no point to using them when you have a full-blown operating system available to you.
Now if you don't care much whether the BIOS calls access real hardware or emulated hardware, you can certainly use Forth to access something like DOSBox and run the real BIOS on emulated hardware. Heck, DOSBox provides its own BIOS implementation :)
I completely newbie in device drivers, so I hope my question is in place, but I need to develop a driver to control some equipment. I was thinking on using Linux as the host OS, but not sure if it is such good idea. I've heard some horror stories about the mess of developing device drivers under Linux. Is there a better alternative under the *Nix world? Or maybe should I check other OSes?
Linux documentation is basically non-existent (similar to other platforms). However, there are a few books which do cover enough information to get started, and the trickier kernel bits can borrowed from other drivers (yay for Open Source).
However, it is one of the easiest current platforms to develop drivers for. There are cleaner models, such as QNX, but that product is sadly near the end (and doesn't support 1/10th as much as hardware as Linux)
What type of device is the driver targetting? Many times you can avoid writing in-kernel drivers (for instance, using libusb in userspace, or the user space IO framework)