AWS + Kubeadm (k8s 1.4)
I tried following the README at:
https://github.com/kubernetes/contrib/tree/master/ingress/controllers/nginx
but that doesnt seem to work. I asked around in slack, and it seems the yamls are out-dated, which i had to modify as such
first i deployed default-http-backend using yaml found on git:
https://github.com/kubernetes/contrib/blob/master/ingress/controllers/nginx/examples/default-backend.yaml
Next, the ingress-RC i had to modify:
https://gist.github.com/lilnate22/5188374
(note the change to get path to healthz to reflect default-backend as well as the port change to 10254 which is apparently needed according to slack)
Everything is running fine
kubectl get pods i see the ingress-controller
kubectl get rc i see 1 1 1 for the ingress-rc
i then deploy the simple echoheaders application (according to git readme):
kubectl run echoheaders --image=gcr.io/google_containers/echoserver:1.4 --replicas=1 --port=8080
kubectl expose deployment echoheaders --port=80 --target-port=8080 --name=echoheaders-x
next i created a simple ingress :
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: test-ingress
spec:
backend:
serviceName: echoheaders-x
servicePort: 80
both get ing and describe ing gives be a good sign:
Name: test-ingress
Namespace: default
Address: 172.30.2.86 <---this is my private ip
Default backend: echoheaders-x:80 (10.38.0.2:8080)
Rules:
Host Path Backends
---- ---- --------
* * echoheaders-x:80 (10.38.0.2:8080)
but attempting to go to nodes public ip doesnt seem to work, as i am getting "unable to reach server`
Unfortunately it seems that using ingress controllers with Kubernetes clusters set up using kubeadm doesn't is not supported at the moment.
The reason for this is that the ingress controllers specify a hostPort in order to become available on the public IP of the node, but the cluster created by kubeadm uses the CNI network plugin which does not support hostPort at the moment.
You may have better luck picking a different way to set up the cluster which does not use CNI.
Alternatively, you can edit your ingress-rc.yaml to declare "hostNetwork: true" under the "spec:" section. Specifying hostNetwork will cause the containers to run using the host's network namespace, giving them access to the network interfaces, routing tables and iptables rules of the host. Think of this as equivalent to "docker run" with the option --network="host".
Ok for all those that came here wondering the same thing..here is how i solved it.
PRECURSOR: the documentation is ambiguous such that reading the docs, i was under the impression, that running through the README would allow me to visit http://{MY_MASTER_IP} and get to my services...this is not true.
in order to get ingress_controller, I had to create a service for ingress-controller, and then expose that service via nodePort. this allowed me to access the services (in the case of README, echoheaders) via http://{MASTER_IP}: {NODEPORT}
there is an "issue" with nodePort that you get a random port#, which somewhat defeats the purpose of ingress... to solve that i did the following:
First: I needed to edit kube-api to allow a lower nodePort IP.
vi /etc/kubernetes/manifests/kube-apiserver.json
then in the kube-api containers arguments section add: "--service-node-port-range=80-32767",
this will allow nodePort to be from 80-32767.
** NOTE: i would probably not recommend this for production...**
Next, i did kubectl edit svc nginx-ingress-controller and manually edited nodePort to port 80.
this way, i can go to {MY_MASTER_IP} and get to echoheaders.
now what i am able to do is, have different Domains pointed to {MY_MASTER_IP} and based on host (similar to README)
you can just use the image nginxdemos/nginx-ingress:0.3.1 ,you need not build yourself
#nate's answer is right
https://kubernetes.github.io/ingress-nginx/deploy/baremetal/#over-a-nodeport-service
has a bit more details.
They do not recommend setting the service's node port range though
This question is the first in the search results of Google, I will add my solution.
kubeadm v1.18.12
helm v3.4.1
Yes, the easiest way is to use helm. Also I use standard ingress https://github.com/kubernetes/ingress-nginx
Add the repository
helm repo add ingress-nginx https://kubernetes.github.io/ingress-nginx
Install ingress
helm install ingress --namespace ingress --create-namespace --set rbac.create=true,controller.kind=DaemonSet,controller.service.type=ClusterIP,controller.hostNetwork=true ingress-nginx/ingress-nginx
Daedmonset makes ingress readily available on every node in your cluster.
hostNetwork=true specify uses the node public IP address.
After that, you need to configure the rules for ingress and set the necessary DNS records.
Related
I installed minikube on my Mac and I'd like to deploy elasticsearch on this k8s cluster. I followed this instruction: https://www.elastic.co/guide/en/cloud-on-k8s/current/k8s-deploy-elasticsearch.html
The file I created is:
apiVersion: elasticsearch.k8s.elastic.co/v1
kind: Elasticsearch
metadata:
name: quickstart
spec:
version: 7.10.0
nodeSets:
- name: default
count: 1
config:
node.store.allow_mmap: false
when I run kubectl apply -f es.yaml, I got this error: error: unable to recognize "es.yaml": no matches for kind "Elasticsearch" in version "elasticsearch.k8s.elastic.co/v1"
It says kind is not matched. I wonder how I can make it work. I searched k8s doc and it seems kind can be service, pod, deployment. But why the above instruction uses Elasticsearch as the kind? What value of kind should I specify?
I think you might have missed the step of installing CRD and the operator for ElasticSearch. Have you followed this step https://www.elastic.co/guide/en/cloud-on-k8s/current/k8s-deploy-eck.html?
Service, Pod, Deployment etc are Kubernetes native resources. Kubernetes provides a way to write custom resources also, using CRDs. Elasticsearch is one such example, so you have to define custom resource before using it for Kubernetes to understand that.
I'm using Window and learning Kubernetes, I'm trying to install ingress-nginx by running this command
kubectl apply -f https://raw.githubusercontent.com/kubernetes/ingress-nginx/controller-v0.35.0/deploy/static/provider/cloud/deploy.yaml.
It didn't work at 1st, so I copied the yaml file locally and run "kubectl apply -f" on that local file but then I got this
unable to recognize "ingress.yaml": Get https://kubernetes.docker.internal:6443/api?timeout=32s: dial tcp 127.0.0.1:6443: connectex: No connection could be made because the target machine actively refused it.
Please help me (I don't use minikube)
This is the issue with kubectl itself that doesn't see your ~/.kube/config.
Most probably you will see nothing if run
kubectl config view
apiVersion: v1
clusters: []
contexts: []
current-context: ""
kind: Config
preferences: {}
users: []
Make sure KUBECONFIG looks to your config file(~/.kube/config)
You mat also want to visit Trouble installing applications in Kubernetes github issue for more information
I have created a namespace on my physical K8 cluster.
Now I'm trying to spin-up resources with the help of *dep.yaml and namespace mentioned in the file.
Also created secrets under the same namespace.
But Status showing 'ContainerCreating'.
application-dep.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
name: service-name
namespace: namespace-service
labels:
module: testmodule
...
Note: It's working in the default namespace.
If a Pod is stuck in state ContainerCreating, it may be good to check the Events with kubectl describe pod and check the Events in the bottom.
There is usually an Event that explain why the Pod fails to reach state Running.
Can you share the rest of the deployment file?
If you are currently running a service (assuming that it exposes a specific NodePort) in the default namespace, you cannot expose the same port again on a different namespace.
Also, as mentioned, you can always get more information using the "describe" command -
kubectl describe <resource> (can be a pod, deployment, service, etc)
I've been following tutorial videos and trying to understand to build a small minimalistic application. The videos I followed are pulling containers from the registries while I'm trying to test, build and deploy everything locally at the moment if possible. Here's my setup.
I've the latest docker installed with Kubernetes enabled on mac OS.
A helloworld NodeJS application running with Docker and Docker Compose
TODO: I'd like to be able to start my instances, let's say 3 in the kubernetes cluster
Dockerfile
FROM node:alpine
COPY package.json package.json
RUN npm install
COPY . .
CMD ["npm", "start"]
docker-compose.yml
version: '3'
services:
user:
container_name: users
build:
context: ./user
dockerfile: Dockerfile
Creating a deployment file with the help of this tutorial and it may have problems since I'm merging information both from youtube as well as the web link.
Creating a miminalistic yml file for to be able to get up and running, will study other aspects like readiness and liveness later.
apiVersion: v1
kind: Service
metadata:
name: user
spec:
selector:
app: user
ports:
- port: 8080
type: NodePort
Please review the above yml file for correctness, so the question is what do I do next?
The snippets you provide are regrettably insufficient but you have the basics.
I had a Google for you for a tutorial and -- unfortunately -- nothing obvious jumped out. That doesn't mean that there isn't one, just that I didn't find it.
You've got the right idea and there are quite a few levels of technology to understand but, I commend your approach and think we can get you there.
Let's start with a helloworld Node.JS tutorial
https://nodejs.org/en/docs/guides/getting-started-guide/
Then you want to containerize this
https://nodejs.org/de/docs/guides/nodejs-docker-webapp/
For #3 below, the last step here is:
docker build --tag=<your username>/node-web-app .
But, because you're using Kubernetes, you'll want to push this image to a public repo. This is so that, regardless of where your cluster runs, it will be able to access the container image.
Since the example uses DockerHub, let's continue using that:
docker push <your username>/node-web-app
NB There's an implicit https://docker.io/<your username>/node-web-app:latest here
Then you'll need a Kubernetes cluster into which you can deploy your app
I think microk8s is excellent
I'm a former Googler but Kubernetes Engine is the benchmark (requires $$$)
Big fan of DigitalOcean too and it has Kubernetes (also $$$)
My advice is (except microk8s and minikube) don't ever run your own Kubernetes clusters; leave it to a cloud provider.
Now that you have all the pieces, I recommend you just:
kubectl run yourapp \
--image=<your username>/node-web-app:latest \
--port=8080 \
--replicas=1
I believe kubectl run is deprecated but use it anyway. It will create a Kubernetes Deployment (!) for you with 1 Pod (==replica). Feel free to adjust that value (perhaps --replicas=2) if you wish.
Once you've created a Deployment, you'll want to create a Service to make your app accessible (top of my head) this command is:
kubectl expose deployment/yourapp --type=NodePort
Now you can query the service:
kubectl get services/yourapp
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
yourapp NodePort 10.152.183.27 <none> 80:32261/TCP 7s
NB The NodePort that's been assigned (in this case!) is :32261 and so I can then interact with the app using curl http://localhost:32261 (localhost because I'm using microk8s).
kubectl is powerful. Another way to determine the NodePort is:
kubectl get service/yourapp \
--output=jsonpath="{.spec.ports[0].nodePort}"
The advantage of the approach of starting from kubectl run is you can then easily determine the Kubernetes configuration that is needed to recreate this Deployment|Service by:
kubectl get deployment/yourapp \
--format=yaml \
> ./yourapp.deployment.yaml
kubectl get service/yourapp \
--format=yaml \
> ./yourapp.service.yaml
These commands will interrogate the cluster, retrieve the configuration for you and pump it into the files. It will include some instance data too but the gist of it shows you what you would need to recreate the deployment. You will need to edit this file.
But, you can test this by first deleting the deployment and the service and then recreating it from the configuration:
kubectl delete deployment/yourapp
kubectl delete service/yourapp
kubectl apply --filename=./yourapp.deployment.yaml
kubectl apply --filename=./yourapp.service.yaml
NB You'll often see multiple resource configurations merged into a single YAML file. This is perfectly valid YAML but you only ever see it used by Kubernetes. The format is:
...
some: yaml
---
...
some: yaml
---
Using this you could merge the yourapp.deployment.yaml and yourapp.service.yaml into a single Kubernetes configuration.
As far as I'm concerned, this is more of a development question than a server question, but it lies very much on the boundary of the two, so feel free to migrate to serverfault.com if that's the consensus).
I have a service, let's call it web, and it is declared in a docker-compose.yml file as follows:
web:
image: webimage
command: run start
build:
context: ./web
dockerfile: Dockerfile
In front of this, I have a reverse-proxy server running Apache Traffic Server. There is a simple mapping rule in the url remapping config file
map / http://web/
So all incoming requests are mapped onto the web service described above. This works just peachily in docker-compose, however when I move the service to kubernetes with the following service description:
apiVersion: v1
kind: Service
metadata:
labels:
io.kompose.service: web
name: web
spec:
clusterIP: None
ports:
- name: headless
port: 55555
targetPort: 0
selector:
io.kompose.service: web
status:
loadBalancer: {}
...traffic server complains because it cannot resolve the DNS name web.
I can resolve this by slightly changing the DNS behaviour of traffic server with the following config change:
CONFIG proxy.config.dns.search_default_domains INT 1
(see https://docs.trafficserver.apache.org/en/7.1.x/admin-guide/files/records.config.en.html#dns)
This config change is described as follows:
Traffic Server can attempt to resolve unqualified hostnames by expanding to the local domain. For example if a client makes a request to an unqualified host (e.g. host_x) and the Traffic Server local domain is y.com, then Traffic Server will expand the hostname to host_x.y.com.
Now everything works just great in kubernetes.
However, when running in docker-compose, traffic-server complains about not being able to resolve web.
So, I can get things working on both platforms, but this requires config changes to do so. I could fire a start-up script for traffic-server to establish if we're running in kube or docker and write the config line above depending on where we are running, but ideally, I'd like the DNS to be consistent across platforms. My understanding of DNS (and in particular, DNS default domains/ local domains) is patchy.
Any pointers? Ideally, a local domain for docker-compose seems like the way to go here.
The default kubernetes local domain is
default.svc.cluster.local
which means that the fully qualified name of the web service under kubernetes is web.default.svc.cluster.local
So, in the docker-compose file, under the trafficserver config section, I can create an alias for web as web.default.svc.cluster.local with the following docker-compose.yml syntax:
version: "3"
services:
web:
# ...
trafficserver:
# ...
links:
- "web:web.default.svc.cluster.local"
and update the mapping config in trafficserver to:
map / http://web.default.svc.cluster.local/
and now the web service is reachable using the same domain name across docker-compose and kubernetes.
I found the same problem but solved it in another way, after much painful debugging.
With CONFIG proxy.config.dns.search_default_domains INT 1 Apache Traffic Server will append the names found under search in /etc/resolve.conf one by one until it gets a hit.
In my case resolve.conf points to company.intra so I could name my services (all services used from Apache Traffic Server) according to this
version: '3.2'
services:
# this hack is ugly but we need to name this
# (and all other service called from ats), with
# the same name as found under search in /etc/resolve.conf)
web.company.intra:
image: web-image:1.0.0
With this change I don't need to make any changes to remap.config at all, the URL used can still be only "web", since it gets expanded to a name that matches both environments,
web.company.intra in docker.compose
web.default.svc.local.cluster in kubernetes