I am trying to install Confluent Platform on my Windows machine. As far as I know, installing Confluent Platform will give me access to KSQL which is not available in Apache Kafka package. The other hurdle is : KSQL can't run on windows directly, it requires Unix environment. As I am on Windows so my options are limited.
I explored below options :
I tried to use Windows Subsystem for Linux, but installing anything from Windows App Store is
restricted in my environment. So, it's not possible to install Ubuntu from app store.
As I have docker installed on my system. I am planning to pull Ubuntu image and run Kafka inside it. I pulled Ubuntu image from Docker hub. Now, I need to download confluent-platform. I am planning to download it using WGET but not sure about the URL or path that I need to provide to wget.
Please suggest me the path to download confluent package.
You could use this instead of the bare bones ubuntu image.
It runs a Debian base, so apt-get will still work if you want to extend it
Please suggest me the path to download confluent package
Try the Confluent website??
As far as I know, installing Confluent Platform will give me access to KSQL which is not available in Apache Kafka package.
First, it's now referred to as ksqlDB. Second, it works with any Kafka provider. Confluent provides Apache Kafka as part of their distribution
You can use ksqlDB container with Apache Kafka running on Windows
Or you can run everything in containers, as shown in the quickstart - https://ksqldb.io/quickstart.html
Related
I'm trying to migrate .NET legacy application to AWS ECS/Fargate. I'm following this article that explains how to create a custom Windows Docker image with MSBuild tools used in AWS CodePipeline/CodeBuild project. I also need to be able to install a Docker deamon and AWS CLIV2 into that custom image so that I could execute docker and AWS CLI commands in buildspec.yaml file in CodeBuild. So far I've been able to use this code in my custom image Dockerfile which installs Docker in Docker but the Docker service never gets started even though it understands docker --version command. I was also trying to modify this PowerShell script to install AWS CLI but also stuck with having little to no progress.
I'd appreciate any help in installing Docker in Docker and AWS CLI.
When I had to use docker in docker, I instead used the host docker socket by mounting that in the container.
I had to mount 2 files in linux.
/usr/bin/docker (executable)
/var/run/docker.sock (service socket)
Update - Above would work for linux, for windows, a double slash is required. Below socket would have to be mounted for windows. I couldn't personally test as I don't have windows.
"\\.\pipe\docker_engine:\\.\pipe\docker_engine"
I found a very good GUI tool explained this
Ref: https://tomgregory.com/running-docker-in-docker-on-windows/
By any chance can Pycharm on Windows be configured to run docker inside a linux VM (VirtualBox) ?
Meaning that i would like to take advantage of the Pycharm Docker tools of professional version but avoid installing docker for windows, since im using Linux VMs that have Docker installed.
Tried to set the tcp://xx.xx.xx.xx:2375 of the guest host vm but failing. Not sure if this can be done at all.
There's no native support, but you can configure PyCharm to work with remote Docker daemon. Basically you need a deployment configuration alongside the Docker-based remote interpreter to transfer the project files to remote machine where they can be mounted inside a container.
See the detailed instruction in PyCharm's bug tracker https://youtrack.jetbrains.com/issue/PY-33489 and please vote for the ticket if you want this support to be a first-class citizen in PyCharm.
What I mean is if I can run for example the official docker image DEBIAN and on top of that
run the official docker image NGINX with both same supported architecture e.g. Linux x86-64?
Will it work like I would install NGINX package for DEBIAN operating systems in non-docker way?
Because I'm learning docker and I've came across that NGINX was build and run from official NGINX repository for DEBIAN OS on top of the official docker image DEBIAN?
Is that a clue that docker images are not cross-platform compatible?
I've also came across this helpful question.
If by cross-platform you mean whether a docker image built on an x86_64 machine will run on a ppcle64 machine, then the answer is no (there are ways around it by using an emulator, but generally speaking the answer is no).
If you mean, whether an Ubuntu container can be run on a Debian host, then yes (provided host kernel version is compatible, which it will be, since you were able to install docker).
As for the question of why NGINX official image is Debian, the developers might have their own reasons. In fact, the official repo has Alpine flavour image as well. You can modify the Dockerfile to use Ubuntu image, make the necessary modifications (such as the ubuntu version of the installer) and build it on a Debian host. It will produce an Ubuntu image which will run an Ubuntu container on any Linux, Unix, MacOS or Windows (using Linux VM) . You can build that Dockerfile as is on an Ubuntu host and it will create the same nginx:latest image as you would download from dockerhub. This can be verified using the checksum.
we use VMWare vSphere for VMs in our company.
To automatically create docker hosts we use one simple command:
docker-machine --driver vmwarevsphere .... vm params(cpu,memory,network,name, etc)
It automatically creates new VM machine in our VM cluster, installs docker and then we add it to swarm or create new.
Right now I need to create windows docker hosts to run windows containers.
Docker-machine installs boot2docker.iso after creating VM.
But instead I need VM with microsoft servercore or nano.
How do I do it?
Thanks a lot.
Anton
On a Windows machine with Docker for Windows installed you could run the following command to pull the official images for server or nanoserver
docker pull microsoft/nanoserver
or
docker pull microsoft/windowsservercore
I'm not exactly sure how you're automating this - are you using a dockerfile or docker compose?
Are you talking about setting up the Windows host that runs Docker engine? If so, Docker for Windows CE is meant to be desktop software so not recommended for server side workload. Also, Windows EE Server requires Windows Server 2016 or later. If you would really like to use Windows server core mode, Windows Server 1709 offers that. Still, it quite bit new, so you should not set high expectations just yet.
As per the instruction to install the engine, MS has this.
https://learn.microsoft.com/en-us/virtualization/windowscontainers/quick-start/quick-start-windows-server
Or, equivalent one from Docker here.
https://docs.docker.com/engine/installation/windows/docker-ee/
you are talking about hosting a windows container on VMware vSphere? I don't think this is possible right now, may be in the future. I have no documentation or link to verify my answer but in our company we have a similar situation and use vSphere for VMs and Linux container and Hyper-V in parallel for VMs and windows container.
How do I run Datalab locally when it requires Docker (and Docker Toolbox is not supported as documented here: https://cloud.google.com/datalab/docs/quickstarts/quickstart-local)? The Docker website says Docker requires Windows 10 Professional or Enterprise 64-bit, and most corporate environments don't run Windows 10.
Docker is highly preferred over Docker Toolbox, as its a simpler, self-contained installation, with simpler configuration (since you don't have additional virtualization software to deal with, as you do with Docker Toolbox - namely boot2docker and its underlying functionality). However if you have a setup to run docker on your end, you should theoretically be able to use that for running the Datalab docker container by adapting the instructions.
You do have the option of running everything on a GCE VM.
I was facing the same problem, what I found more comfrotable in the end is to install Ubuntu on Virtual Box. This is free and fairly easy, and from the virtual machine you can use all the Docker and the Google guide to run Datalab locally.