I have configured only elasticsearch, kibana, fleet-server, elastic-agent and apm-server. I have not configured metricbeat. And we don't want metricbeat but inside kibana log ( Observability > Logs > Stream ), It is showing
[elastic_agent.metricbeat][error] Error watching for docker events: Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?
From a link (https://discuss.elastic.co/t/metricbeat-not-using-docker-but-getting-error-is-the-docker-daemon-running/226696) I have found that I have to comment these lines from metricbeat.yaml:
add_docker_metadata: ~
add_kubernetes_metadata: ~
I have not installed metricbeat, but still I have found a metricbeat.yml at /var/lib/elastic-agent/data/elastic-agent-b79a5d/components/metricbeat.yml
And inside that file I have commented add_docker_metadata: ~ and add_kubernetes_metadata: ~
But still, It is showing the same error.
please help me with any solution.
Note:
We don't want metricbeat.
when metricbeat is installed automatically, it is not
installed as systemd, so systemctl stop/restart metricbeat is not
working.
ls /var/lib/elastic-agent/data/elastic-agent-b79a5d/components | grep metricbeat is showing
metricbeat
metricbeat.reference.yml
metricbeat.spec.yml
metricbeat.yml
Please let me know how to solve this error to show or how to stop metricbeat. Thanks in advance.
I'm trying to install Geonode on Manjaro. I've also seen that it can be installed with docker but it's new to me and I can't find any tutorial that works for me. My Linux distribution is Arch Linux, for which the package manager is pacman and not apt-get as it is the case in Ubuntu, is it possible to install Geonode on Arch Linux?
I have also used this command triying to install it on postgres:
docker-compose -f docker-compose.yml -f docker
compose.override.localhost.yml up --build
But I get an error like this:
enter image description here
ERROR: for nginx4geonode Cannot start service geonode: driver failed
programming external connectivity on endpoint nginx4geonode
(e0ef6b5b7bd9ad55b195996232e7cb3ce90408379c587f3c0240b6bf06fef544):
Error starting userland proxy: listen tcp 0.0.0.0:80: bind: address
already in use
ERROR: for geonode Cannot start service geonode: driver failed
programming external connectivity on endpoint nginx4geonode
(e0ef6b5b7bd9ad55b195996232e7cb3ce90408379c587f3c0240b6bf06fef544):
Error starting userland proxy: listen tcp 0.0.0.0:80: bind: address
already in use ERROR: Encountered errors while bringing up the
project.
And this command shows the same error:
docker-compose up --build
enter image description here
In a docker container I want to run k8s.
When I run kubeadm join ... or kubeadm init commands I see sometimes errors like
\"modprobe: ERROR: ../libkmod/libkmod.c:586 kmod_search_moddep() could
not open moddep file
'/lib/modules/3.10.0-1062.1.2.el7.x86_64/modules.dep.bin'.
nmodprobe:
FATAL: Module configs not found in directory
/lib/modules/3.10.0-1062.1.2.el7.x86_64",
err: exit status 1
because (I think) my container does not have the expected kernel header files.
I realise that the container reports its kernel based on the host that is running the container; and looking at k8s code I see
// getKernelConfigReader search kernel config file in a predefined list. Once the kernel config
// file is found it will read the configurations into a byte buffer and return. If the kernel
// config file is not found, it will try to load kernel config module and retry again.
func (k *KernelValidator) getKernelConfigReader() (io.Reader, error) {
possibePaths := []string{
"/proc/config.gz",
"/boot/config-" + k.kernelRelease,
"/usr/src/linux-" + k.kernelRelease + "/.config",
"/usr/src/linux/.config",
}
so I am bit confused what is simplest way to run k8s inside a container such that it consistently past this getting the kernel info.
I note that running docker run -it solita/centos-systemd:7 /bin/bash on a macOS host I see :
# uname -r
4.9.184-linuxkit
# ls -l /proc/config.gz
-r--r--r-- 1 root root 23834 Nov 20 16:40 /proc/config.gz
but running exact same on a Ubuntu VM I see :
# uname -r
4.4.0-142-generic
# ls -l /proc/config.gz
ls: cannot access /proc/config.gz
[Weirdly I don't see this FATAL: Module configs not found in directory error every time, but I guess that is a separate question!]
UPDATE 22/November/2019. I see now that k8s DOES run okay in a container. Real problem was weird/misleading logs. I have added an answer to clarify.
I do not believe that is possible given the nature of containers.
You should instead test your app in a docker container then deploy that image to k8s either in the cloud or locally using minikube.
Another solution is to run it under kind which uses docker driver instead of VirtualBox
https://kind.sigs.k8s.io/docs/user/quick-start/
It seems the FATAL error part was a bit misleading.
It was badly formatted by my test environment (all on one line.
When k8s was failing I saw the FATAL and assumed (incorrectly) that was root cause.
When I format the logs nicely I see ...
kubeadm join 172.17.0.2:6443 --token 21e8ab.1e1666a25fd37338 --discovery-token-unsafe-skip-ca-verification --experimental-control-plane --ignore-preflight-errors=all --node-name 172.17.0.3
[preflight] Running pre-flight checks
[WARNING FileContent--proc-sys-net-bridge-bridge-nf-call-iptables]: /proc/sys/net/bridge/bridge-nf-call-iptables does not exist
[preflight] The system verification failed. Printing the output from the verification:
KERNEL_VERSION: 4.4.0-142-generic
DOCKER_VERSION: 18.09.3
OS: Linux
CGROUPS_CPU: enabled
CGROUPS_CPUACCT: enabled
CGROUPS_CPUSET: enabled
CGROUPS_DEVICES: enabled
CGROUPS_FREEZER: enabled
CGROUPS_MEMORY: enabled
[WARNING SystemVerification]: this Docker version is not on the list of validated versions: 18.09.3. Latest validated version: 18.06
[WARNING SystemVerification]: failed to parse kernel config: unable to load kernel module: "configs", output: "modprobe: ERROR: ../libkmod/libkmod.c:586 kmod_search_moddep() could not open moddep file '/lib/modules/4.4.0-142-generic/modules.dep.bin'\nmodprobe: FATAL: Module configs not found in directory /lib/modules/4.4.0-142-generic\n", err: exit status 1
[discovery] Trying to connect to API Server "172.17.0.2:6443"
[discovery] Created cluster-info discovery client, requesting info from "https://172.17.0.2:6443"
[discovery] Failed to request cluster info, will try again: [the server was unable to return a response in the time allotted, but may still be processing the request (get configmaps cluster-info)]
There are other errors later, which I originally though were a side-effect of the nasty looking FATAL error e.g. .... "[util/etcd] Attempt timed out"]} but I now think root cause is Etcd part times out sometimes.
Adding this answer in case someone else puzzled like I was.
Each time when starting docker-compose up on Laradock, receives Permission error for workspace laradock on Windows 10. I have tried restarting but not working.
PS D:\projects\sites\laradock> docker-compose up -d nginx mysql phpmyadmin redis workspace
laradock_redis_1 is up-to-date
laradock_mysql_1 is up-to-date
laradock_docker-in-docker_1 is up-to-date
laradock_phpmyadmin_1 is up-to-date
Starting laradock_workspace_1 ... error
ERROR: for laradock_workspace_1 Cannot start service workspace: driver failed programming external connectivity on endpoint laradock_workspace_1 (cc289cd2758bb1c99d3f8aab7243220a94078e19b2a1f4a08e4b0b01fbba64a3): Error starting userland proxy: Bind for 0.0.0.0:2222: unexpected error Permission denied
ERROR: for workspace Cannot start service workspace: driver failed programming external connectivity on endpoint laradock_workspace_1 (cc289cd2758bb1c99d3f8aab7243220a94078e19b2a1f4a08e4b0b01fbba64a3): Error starting userland proxy: Bind for 0.0.0.0:2222: unexpected error Permission denied
ERROR: Encountered errors while bringing up the project.
I think you can find your answer here .In short you should execute this command netcfg -d this will clean up all networking devices, and requires a reboot
I'm trying to get my head around something that's been working on a Centos+Vagrant, but not on our providers RHEL (Red Hat Enterprise Linux Server release 6.5 (Santiago)). A sudo service docker restart hands this:
Stopping docker: [ OK ]
Starting cgconfig service: Error: cannot mount cpuset to /cgroup/cpuset: Device or resource busy
/sbin/cgconfigparser; error loading /etc/cgconfig.conf: Cgroup mounting failed
Failed to parse /etc/cgconfig.conf [FAILED]
Starting docker: [ OK ]
The service starts okey enough, but images cannot run. A mounting failed error is shown when I try. And the startup-log also gives a warning or two. Regarding the kernelwarning, centos gives the same and has no problems as Epel should resolve this:
WARNING: You are running linux kernel version 2.6.32-431.17.1.el6.x86_64, which might be unstable running docker. Please upgrade your kernel to 3.8.0.
2014/08/07 08:58:29 docker daemon: 1.1.2 d84a070; execdriver: native; graphdriver:
[1233d0af] +job serveapi(unix:///var/run/docker.sock)
[1233d0af] +job initserver()
[1233d0af.initserver()] Creating server
2014/08/07 08:58:29 Listening for HTTP on unix (/var/run/docker.sock)
[1233d0af] +job init_networkdriver()
[1233d0af] -job init_networkdriver() = OK (0)
2014/08/07 08:58:29 WARNING: mountpoint not found
Anyone had any success overcoming this problem or should I throw in the towel and wait for the provider to update to RHEL 7?
I have the same issue.
(1) check cgconfig status
# /etc/init.d/cgconfig status
if it stopped, restart it
# /etc/init.d/cgconfig restart
check cgconfig is running
(2) check cgconfig is on
# chkconfig --list cgconfig
cgconfig 0:off 1:off 2:off 3:off 4:off 5:off 6:off
if cgconfig is off, turn it on
(3) if still does not work, may be some cgroups modules is missing. In the kernel .config file, make menuconfig, add those modules into kernel and recompile and reboot
after that, it should be OK
I ended up asking the same question at Google Groups and in the end finding a solution with some help. What worked for me was this:
umount cgroup
sudo service cgconfig start
The project of making Docker work was put on halt all the same. Later a problem of network connection for the containers. This took to much time to solve and had to give up.
So I spent the whole day trying to rig docker to work on my vps. I was running into this same error. Basically what it came down to was the fact that OpenVZ didn't support docker containers up until a couple months ago. Specifically this RHEL update:
https://openvz.org/Download/kernel/rhel6/042stab105.14
Assuming this is your problem, or some variation of it, the burden of solving it is on your host. They will need to follow these steps:
https://openvz.org/Docker_inside_CT
In my case
/etc/rc.d/rc.cgconfig start
was generating
Starting cgconfig service: Error: cannot mount cpu,cpuacct,memory to
/cgroup/cpu_and_mem: Device or resource busy /usr/sbin/cgconfigparser;
error loading /etc/cgconfig.conf: Cgroup mounting failed Failed to
parse /etc/cgconfig.conf
i had to use:
/etc/rc.d/rc.cgconfig restart
and it automagicly umouted and mounted groups
Stopping cgconfig service: Starting cgconfig service:
it seems like the cgconfig service not running,so check it!
# /etc/init.d/cgconfig status
# mkdir -p /cgroup/cpuacct /cgroup/memory /cgroup/devices /cgroup/freezer net_cls /cgroup/blkio
# cat /etc/cgconfig.conf |tail|grep "="|awk '{print "mount -t cgroup -o",$1,$1,$NF}'>cgroup_mount.sh
# sh ./cgroup_mount.sh
# /etc/init.d/cgconfig restart
# /etc/init.d/docker restart
This situation occurs when the kernel is booted with cgroup_disable=memory and /etc/cgconfig.conf contains memory = /cgroup/memory;
This causes only /cgroup/cpuset to be mounted instead of the full set.
Solution: either remove cgroup_disable=memory from your kernel boot options or comment out memory = /cgroup/memory; from cgconfig.conf.
The cgconfig service startup uses mount and umount which requires an extra privilege bump from docker.
See the --privileged=true flag here for more info.
I was able to overcome this issue by starting my container with:
docker run -it --privileged=true my-image.
Tested in Centos6, Centos6.5.