The setup below is all done on Google Cloud.
I have a Docker setup running on a VM instance having a Hyperledger Fabric Network of 3 Orderers and 2 peers of a single organization.
I have a Kubernetes Cluster running on GKE having just 2 peer pods.
What I want to achieve is:
I want the peer pod of Kubernetes to be able to communicate with the network setup in Docker.
i.e. I want at least 1 peer of the Kubernetes to be able to join the network channel on the Docker
Network and be able to perform some transaction/query.
Till now what I have done is exposed the services of the 2 peer pods in Kubernetes i.e. ClusterIP -> NodePort.
I have done some reading as to what options are there such as having an Nginx proxy and other things, but I am new to this and still in a beginner phase so a few things are still confusing to me.
Any help would be appreciated.
Since you have two separate container setups (and two different container technologies), containers in one space have to communicate with the other as though they're outside the cluster. You can't join arbitrary Docker containers on another instance into the Kubernetes network environment, nor can you join Kubernetes pods into a single-host Docker network environment.
Using a NodePort or LoadBalancer-type Service as you already have is close to the best you can do, particularly if the underlying protocol isn't HTTP. If you need a multi-host setup, migrating the whole thing into GKE might be a better long-term answer.
Related
I am currently working with a Hyperledger Fabric v1.4 deployment over k8s. The chaincode containers that are generated are basically create by the container running within the peer pods and k8s as such has no knowledge and control of the chaincode containers. In such a scenario where there is a Docker container running along with k8s and k8s has no knowledge of a particular docker container, is it possible for the Docker container to in some way gain access to the k8s master API and gain access to the whole k8s cluster consequently?
My intention with asking this question is to figure out if there is a way to use an container external to any pods in k8s, to cause any undesirable impact to the k8s cluster by gaining unauthorized access to k8s. The chaincode container that I talked about, is created using a trusted template image and the only possible malicious component in the container is a single golang, java or nodejs script that is provided by the user. So my real question here is, "Is it possible using these user scripts gain unauthorized access to the k8s cluster?" And I am primarily focusing on a manager k8s service like Azure Kubernetes Service.
Your question totally changed the meaning so I'll try to rewrite the answer.
You have to remember that the pod you are running the code on by default is limited to just the namespace it's running on. If you didn't gave it any higher privileges. Also the code is not running as root.
You can read about Pod Security Policies and Configure a Security Context for a Pod or Container.
TLDR.
As long as you don't give it any special privileges or rights it should be fairly save for your cluster.
I'm currently working on making a game where the servers need to be replicated on pods inside the Minikube cluster (I'm using Minikube since it's just a school project). However, I cannot seem to figure out how to make these game servers accessible from other networks. I have tried making a NodePort service, that points to the deployment of my server image, and when using the Minikube cluster IP + my service's port I can connect fine.
What would be the best approach to solve this issue?
By default, the Pod is only accessible by its internal IP address within the Kubernetes cluster. To make the your container accessible from outside the Kubernetes virtual network, you have to expose the Pod as a Kubernetes Service in this example as a type LoadBalancer.
On cloud providers that support load balancers, an external IP address would be provisioned to access the Service. On Minikube, the LoadBalancer type makes the Service accessible through the minikube service command.
Minikube is great for local setups but not for real clusters. It spins up only a one-node cluster for development and testing. It is useless if you want to use multiple servers environment.
Taking under consideration that you want to play fast with scalability across the nodes it is useful to create Kubernetes cluster - using Kubespray or Kubeadm.
If you want to run an actual local Kubernetes, use a local VM and then create K8s cluster using these tools for automated deployment.
Kubeadm
The kubeadm tool is good if you are new in cloud technologies -run k8s cluster for the first time. You can install and use kubeadm on various machines: your laptop, a set of cloud servers and more. Whether you’re deploying into the cloud or on-premises, you can integrate kubeadm into provisioning systems such as Ansible or Terraform.
Kubeadm provides domain Knowledge of Kubernetes clusters' life cycle management, including self-hosted layouts, dynamic discovery services and so on.
Kubespray
Kubespray supports deployments on AWS, Google Compute Engine, Microsoft Azure, OpenStack, and bare metal. It enables deployment of Kubernetes highly available clusters. It supports a variety of Linux distributions and CI. Kubespray supports kubeadm.
There isn't really a good way to do this. Minikube is for local development. If you want to run an actual local Kubernetes, start a local VM directly and use something like kubeadm and kubespray.
I have a question regarding Kubernetes networking.
I know that in Docker swarm if I want to run difference containers on difference servers, I need to create an overlay network, and then all the containers (from all the servers) will be attached to this network and they can communicate with each other (for example, I can ping from container A to container B).
I guess that in Kubernetes there isn't an overlay network - but another solution. For example, I would like to create 2 linux containers on 2 servers (server 1: ubuntu, server 2: centos7), so how do the pods communicate with each other if there isn't an overlay network?
And another doubt - can I create a cluster which consists of windows and linux machines with kubernetes?I mean, a multi platform kubernetes which all the pods communicate with each other.
Thanks a lot!!
In kubernetes, pods communicate with each other through service. To access any pod within cluster, it must be exposed using clusterIP service. So if you created service before creating pods, you will have env variable for each available service within container. Using that you can ping or access services and in turn pods.
For example:
Suppose you have two pods U1 and C1 and those are exposed by service named U-SVC and C-SVC respectively.
So if you want to access C1 from U1, you will have C-SVC service env variables(C-SVC_SERVICE_HOST,C-SVC_SERVICE_PORT) within container which you can use for access.
Also if DNS server set for your cluster, you can access service without env varibles.
I have been playing around with Hyperledger to make it run on Kubernetes. And I was successful to do so. The only thing which I was not happy with the solution/work-around for the container that was spun up when chaincode is instantiated by the peer.
Kubernetes is simply not aware of this container as it was not started by Kubernetes and by the peer. And to make the peer and chaincode talk to each other I had to update the docker daemon running on the kubernetes node with dns server ip address of the kube-dns service.
Is it possible to instantiate a chaincode in a way where kubernetes is aware of the container of the chaincode.
And also chaincode container is able to talk to peer in a seamless fashion rather than updating docker daemon process of the node within kubernetes cluster
I have been investigating the same issue you are having. One alternative to using the docker daemon on your kubernetes node is spinning up a new container in your Pod using DnD (Docker in docker) technique. In this way you can successfully instantiate the chaincode container in a natural way (you will be able to use KubeDNS for example) as it will be sharing the same network space as the kubernetes Pod. I couldn't find any tutorial on the internet showing the implementation of this theory but if you find one (or do it yourself) please share it on this thread.
Thank you
Reference:
https://medium.com/kokster/simpler-setup-for-hyperledger-fabric-on-kubernetes-using-docker-in-docker-8346f70fbe80
I'm trying to figure out whether Docker Swarm or Kubernetes are a good choice for my use case.
Basically, I want to build a small cluster of forward proxies (via squid, nginx or a custom nodejs script), and be able to deploy/start/stop/purge them all together.
I should be able to access the proxy cluster via a single IP address, manager should be able to load-balance the request to a node, and each proxy node must use a unique outgoing IP address.
I'm wondering:
Are Docker Swarm and/or Kubernetes the right way to go about it?
If so, should I set-up Docker Swarm and/or Kubernetes and its worker nodes (running the proxy) on a single dedicated server or separate virtual servers?
Is it also possible for all the cluster nodes to share a file system storage for caching, common config etc.
Any other tips to get this working.
Thanks!
Docker running in swarm mode should work well for this
Run docker on a single dedicated server; I see no need for virtual servers. You could also run the swarm across multiple dedicated servers.
https://docs.docker.com/engine/swarm/secrets/ work well for some settings and configurations. If you require significant storage, simply add a database service to your cluster
Docker swarm mode fits your requirements quite well; requests are automatically balanced across your swarm and each service instance can be configured to have a unique address. You should check out the swarm mode tutorial: https://docs.docker.com/engine/swarm/swarm-tutorial/