memsql/cluster-in-a-box latest image failed to start container - docker

after pulling the latest image this issue accures
on this version everything works fine
memsql/cluster-in-a-box:centos-7.3.12-2d93725f98-3.2.11-1.11.7
~/workdir/js/master *16 !1 ▓▒░ docker-compose up memsql ░▒▓ 2 ✘  15s   2.7.2   08:24:30 
Recreating platform-js_memsql_1 ... done
Attaching to platform-js_memsql_1
memsql_1 | 2021-07-26 05:24:41.431505 Starting Cluster
memsql_1 | Latest errors from MemSQL tracelog:
memsql_1 | 13651863 2021-07-26 05:24:55.333 FATAL: Thread 115111: jumpToUpgradeStep: This node is not managed by a supported tool. Please use a toolbox version at least as new as 1.11.3.
memsql_1 | : Failed to connect to MemSQL: process exited: exit status 1
memsql_1 | Traceback (most recent call last):
memsql_1 | File "/startup", line 122, in <module>
memsql_1 | start_cluster()
memsql_1 | File "/startup", line 86, in start_cluster
memsql_1 | ctl("start-node", "--all")
memsql_1 | File "/startup", line 18, in ctl
memsql_1 | subprocess.check_output(["memsqlctl", "-yj"] + list(args)))
memsql_1 | File "/usr/lib64/python2.7/subprocess.py", line 575, in check_output
memsql_1 | raise CalledProcessError(retcode, cmd, output=output)
memsql_1 | subprocess.CalledProcessError: Command '['memsqlctl', '-yj', 'start-node', '--all']' returned non-zero exit status 1
this is my docker-compose setup using memsql/cluster-in-a-box
memsql:
image: memsql/cluster-in-a-box
volumes:
- "./init.sql"
ports:
- "3307:3306"
- "8080:8080"
environment:
START_AFTER_INIT: Y
ROOT_PASSWORD: 'root'
LICENSE_KEY:*************************
OS: macOs bigSur v11.4
Docker : v20.10.7
also tries as suggested by Volodymyr Tkachuk
to run it directly with docker and not working
docker run -i --init --name memsql -e LICENSE_KEY=$LICENSE -e ROOT_PASSWORD=root -p 3306:3306 -p 8080:8080 memsql/cluster-in-a-box:latest docker start memsql
'Please use a toolbox version at least as new as 1.11.3.'
isn't toolbox part of the image?
this issue related to 3rd party dependencies or container issues?

Unfortunately the recent release of SingleStore 7.5 broke upgrade for this Docker image. We added an upgrade step to the release which requires running sdb-upgrade. We will be fixing this, but in the meantime you have two choices:
If you don't care about the data in this image (or you can recreate the data) run docker-compose up -V to start 7.5 with an empty data directory.
If you do care about the data in this image, modify the entrypoint to run sdb-upgrade which should upgrade the data volume, and then you can run the container like normal. You should test this process before running it on your actual image since it is potentially destructive.
Sorry that you ran into this, we will fix the bug soon.

Related

Error: unable to authenticate user when running kibana, elasticsearch and APM from docker

I'm following this guide to run APM with kibana and elasticsearch:
and when I run this command: ./scripts/compose.py start master --no-kibana
this is the output I get:
Kibana will be launched to configure APM integration and stopped after that.
Starting/Building stack services..
Pulling elasticsearch ... done
Pulling kibana ... done
Pulling apm-server ... done
Pulling wait-service ... done
Starting localtesting_8.4.1_elasticsearch ... done
Starting localtesting_8.4.1_kibana ... done
ERROR: for wait-service Container "3897cf4e31e4" is unhealthy.
ERROR: for apm-server Container "3897cf4e31e4" is unhealthy.
ERROR: Encountered errors while bringing up the project.
Traceback (most recent call last):
File "/home/omar/Downloads/apm-integration-testing/./scripts/compose.py", line 31, in <module>
main()
File "/home/omar/Downloads/apm-integration-testing/./scripts/compose.py", line 17, in main
setup()
File "/home/omar/Downloads/apm-integration-testing/scripts/modules/cli.py", line 213, in __call__
self.args.func()
File "/home/omar/Downloads/apm-integration-testing/scripts/modules/cli.py", line 590, in start_handler
self.build_start_handler("start")
File "/home/omar/Downloads/apm-integration-testing/scripts/modules/cli.py", line 782, in build_start_handler
self.run_docker_compose_process(docker_compose_cmd + up_params)
File "/home/omar/Downloads/apm-integration-testing/scripts/modules/cli.py", line 476, in run_docker_compose_process
subprocess.check_call(docker_compose_cmd)
File "/usr/lib/python3.10/subprocess.py", line 369, in check_call
raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['docker-compose', '-f', '/home/omar/Downloads/apm-integration-testing/docker-compose.yml', 'up', '-d']' returned non-zero exit status 1.
and when I run docker ps I get this:
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
3897cf4e31e4 docker.elastic.co/kibana/kibana:8.4.1-SNAPSHOT "/bin/tini -- /usr/l…" 2 days ago Up 7 minutes (unhealthy) 127.0.0.1:5601->5601/tcp localtesting_8.4.1_kibana
4eef98f32f5d docker.elastic.co/elasticsearch/elasticsearch:8.4.1-SNAPSHOT "/bin/tini -- /usr/l…" 2 days ago Up 8 minutes (healthy) 127.0.0.1:9200->9200/tcp, 9300/tcp localtesting_8.4.1_elasticsearch
So I try running kibana locally with yarn start in the kibana folder, but I encounter this error:
[ERROR][elasticsearch-service] Unable to retrieve version information from Elasticsearch nodes. security_exception: [security_exception] Reason: unable to authenticate user [kibana_system] for REST request [/_nodes?filter_path=nodes.*.version%2Cnodes.*.http.publish_address%2Cnodes.*.ip]
I tried adding the kibana_system user to kibana.yml like this:
elasticsearch.username: 'kibana_system'
but with no luck, If someone encountered a similar issue or got apm-integration-testing to work locally with kibana, I'd be grateful for the help

podman-compose failing to compose

I realize podman-compose is still under development. I'm going to be replacing my docker stack with podman once I replace Debian with CentOS8 on my Poweredge server as part of my homelab. Right now I'm just testing out/playing around with podman on my Fedora machine.
OS: Fedora 32
KERNEL:5.6.12-300.fc32.x86_64
PODMAN: 1.9.2
PODMAN-COMPOSE: 0.1.5
PROBLEM: podman-compose is failing and I'm unable to ascertain the reason why.
Here's my docker-compose.yml:
version: "2.1"
services:
deluge:
image: linuxserver/deluge
container_name: deluge
network_mode: host
environment:
- PUID=1000
- PGID=1000
- TZ=America/New_York
# - UMASK_SET=022 #optional
# - DELUGE_LOGLEVEL=error #optional
volumes:
- /home/mike/test/deluge:/config
- /home/mike/Downloads:/downloads
restart: unless-stopped
When I run podman-compose up Here is the output:
[mike#localhost test]$ podman-compose up
podman pod create --name=test --share net
ce389be26589efe4433db15d875844b2047ea655c43dc84dbe49f69ffabe867e
0
podman create --name=deluge --pod=test -l io.podman.compose.config-hash=123 -l io.podman.compose.project=test -l io.podman.compose.version=0.0.1 -l com.docker.compose.container-number=1 -l com.docker.compose.service=deluge --network host -e PUID=1000 -e PGID=1000 -e TZ=America/New_York --mount type=bind,source=/home/mike/test/deluge,destination=/config --mount type=bind,source=/home/mike/Downloads,destination=/downloads --add-host deluge:127.0.0.1 --add-host deluge:127.0.0.1 linuxserver/deluge
Trying to pull registry.fedoraproject.org/linuxserver/deluge...
manifest unknown: manifest unknown
Trying to pull registry.access.redhat.com/linuxserver/deluge...
name unknown: Repo not found
Trying to pull registry.centos.org/linuxserver/deluge...
manifest unknown: manifest unknown
Trying to pull docker.io/linuxserver/deluge...
Getting image source signatures
Copying blob a54f3db92256 done
Copying blob c114dc480980 done
Copying blob d0d29aaded3d done
Copying blob fa1dff0a3a53 done
Copying blob 5076df76a29a done
Copying blob a40b999f3c1e done
Copying config 31fddfa799 done
Writing manifest to image destination
Storing signatures
Error: error checking path "/home/mike/test/deluge": stat /home/mike/test/deluge: no such file or directory
125
podman start -a deluge
Error: unable to find container deluge: no container with name or ID deluge found: no such container
125
Then finally when I quite via ctrl-c :
^CTraceback (most recent call last):
File "/home/mike/.local/bin/podman-compose", line 8, in <module>
sys.exit(main())
File "/home/mike/.local/lib/python3.8/site-packages/podman_compose.py", line 1093, in main
podman_compose.run()
File "/home/mike/.local/lib/python3.8/site-packages/podman_compose.py", line 625, in run
cmd(self, args)
File "/home/mike/.local/lib/python3.8/site-packages/podman_compose.py", line 782, in wrapped
return func(*args, **kw)
File "/home/mike/.local/lib/python3.8/site-packages/podman_compose.py", line 914, in compose_up
thread.join(timeout=1.0)
File "/usr/lib64/python3.8/threading.py", line 1005, in join
if not self._started.is_set():
File "/usr/lib64/python3.8/threading.py", line 513, in is_set
def is_set(self):
KeyboardInterrupt
I'm not experienced enough to be able to read through this and figure out what the problem is so I'm hoping to learn from you all.
Thanks!
There is an error in your path:
volumes:
/home/mike/test/deluge:/config
/home/mike/test/deluge: no such file or directory
Check the folder path.

How to resolve exceptions.OSError: [Errno 1] Operation not permitted (docker container)?

I am trying to scan BLE devices with bluepy. My scan.py code is --
from bluepy.btle import Scanner, DefaultDelegate
class ScanDelegate(DefaultDelegate):
def __init__(self):
DefaultDelegate.__init__(self)
def handleDiscovery(self, dev, isNewDev, isNewData):
if isNewDev:
print "Discovered device", dev.addr
elif isNewData:
print "Received new data from", dev.addr
# prepare scanner
scanner = Scanner().withDelegate(ScanDelegate())
# scan for 5 seconds
devices = scanner.scan(5.0)
for dev in devices:
print "Device %s (%s), RSSI=%d dB" % (dev.addr, dev.addrType, dev.rssi)
for (adtype, desc, value) in dev.getScanData():
print " %s = %s" % (desc, value)
According to the documentation (mentioned at the very last as Note) --
(1) LE scanning must be run as root
That means we need to run the script with sudo. I run it as --
sudo python scan.py
Basically bluepy-helper requires the sudo to scan. It is required set the capabilities for blupe-helper to run the code without sudo. According to the solution, I did --
sudo setcap 'cap_net_raw,cap_net_admin+eip' /usr/local/lib/python2.7/site-packages/bluepy/bluepy-helper
From the Terminal, the scan code is now run without sudo like --
python scan.py
Finally, I made a Dockerfile --
FROM arm32v7/python:2.7.15-jessie
WORKDIR /usr/app/gfi_ble
COPY . /usr/app/gfi_ble
RUN chmod 755 ./setcap_for_bluepy_helper.sh
RUN pip install -r requirements.txt
CMD ["./setcap_for_bluepy_helper.sh", "--", "python", "src/scan.py"]
The content of the setcap_for_bluepy_helper.sh is --
#!/bin/bash
cmd="$#"
>&2 setcap 'cap_net_raw,cap_net_admin+eip' /usr/local/lib/python2.7/site-packages/bluepy/bluepy-helper
exec $cmd
The image is created successfully but when I run the container I am getting the error like --
Creating con_gfi_ble ... done
Attaching to con_gfi_ble
con_gfi_ble | 2019-01-12 23:06:24+0000 [-] Unhandled Error
con_gfi_ble | Traceback (most recent call last):
con_gfi_ble | File "/usr/app/gfi_ble/src/scan.py", line 17, in new_devices
con_gfi_ble | devices = scanner.scan(5.0)
con_gfi_ble | File "/usr/local/lib/python2.7/site-packages/bluepy/btle.py", line 852, in scan
con_gfi_ble | self.start(passive=passive)
con_gfi_ble | File "/usr/local/lib/python2.7/site-packages/bluepy/btle.py", line 789, in start
con_gfi_ble | self._startHelper(iface=self.iface)
con_gfi_ble | File "/usr/local/lib/python2.7/site-packages/bluepy/btle.py", line 284, in _startHelper
con_gfi_ble | preexec_fn = preexec_function)
con_gfi_ble | File "/usr/local/lib/python2.7/subprocess.py", line 394, in __init__
con_gfi_ble | errread, errwrite)
con_gfi_ble | File "/usr/local/lib/python2.7/subprocess.py", line 1047, in _execute_child
con_gfi_ble | raise child_exception
con_gfi_ble | exceptions.OSError: [Errno 1] Operation not permitted
con_gfi_ble |
Question: What does exceptions.OSError: [Errno 1] Operation not permitted?
My code is fine when I run it from Terminal. What's wrong with the container? Any idea!
Docker containers run with reduced capabilities. This prevents root inside a container from escaping the container by running kernel commands without namespaces, and accessing parts of the host outside of the container, like raw network interfaces or physical devices. You need to add capabilities to the container externally if you need them, but understand this reduces the security provided by docker's default settings.
From docker run, this looks like:
docker run --cap-add=NET_ADMIN --cap-add=NET_RAW ...
https://docs.docker.com/engine/reference/run/#runtime-privilege-and-linux-capabilities
In a compose file, this looks like:
version: '2'
services:
app:
image: your_image
cap_add:
- NET_ADMIN
- NET_RAW
Ref: https://docs.docker.com/compose/compose-file/
This will not work with swarm mode. Work is ongoing for adding the ability to run commands with added capabilities within swarm mode. There are ugly workarounds if you need this.
Note that you should not be running sudo inside of a container. Doing so means everything has access to promote itself to root and defeats the purpose of running anything as a user. Instead you should start the container as root and drop to a regular user as soon as possible, which is a one way operation.

PyHive Thrift transport exception: read 0 bytes

I'm trying to connect to Hive server-2 running inside docker container (from outside the container) via python (PyHive 0.5, python 2.7) using DB-API (asynchronous) example
from pyhive import hive
conn = hive.connect(host='172.17.0.2', port='10001', auth='NOSASL')
However, I'm getting following error
Traceback (most recent call last):
File "py_2.py", line 4, in <module>
conn = hive.connect(host='172.17.0.2', port='10001', auth='NOSASL')
File "/home/foodie/anaconda2/lib/python2.7/site-packages/pyhive/hive.py", line 64, in connect
return Connection(*args, **kwargs)
File "/home/foodie/anaconda2/lib/python2.7/site-packages/pyhive/hive.py", line 164, in __init__
response = self._client.OpenSession(open_session_req)
File "/home/foodie/anaconda2/lib/python2.7/site-packages/TCLIService/TCLIService.py", line 187, in OpenSession
return self.recv_OpenSession()
File "/home/foodie/anaconda2/lib/python2.7/site-packages/TCLIService/TCLIService.py", line 199, in recv_OpenSession
(fname, mtype, rseqid) = iprot.readMessageBegin()
File "/home/foodie/anaconda2/lib/python2.7/site-packages/thrift/protocol/TBinaryProtocol.py", line 148, in readMessageBegin
name = self.trans.readAll(sz)
File "/home/foodie/anaconda2/lib/python2.7/site-packages/thrift/transport/TTransport.py", line 60, in readAll
chunk = self.read(sz - have)
File "/home/foodie/anaconda2/lib/python2.7/site-packages/thrift/transport/TTransport.py", line 161, in read
self.__rbuf = BufferIO(self.__trans.read(max(sz, self.__rbuf_size)))
File "/home/foodie/anaconda2/lib/python2.7/site-packages/thrift/transport/TSocket.py", line 132, in read
message='TSocket read 0 bytes')
thrift.transport.TTransport.TTransportException: TSocket read 0 bytes
The docker image that I'm using is this (tag: mysql_corrected).
It runs following services (as outputted by jps command)
992 Master
1810 RunJar
259 DataNode
2611 Jps
584 ResourceManager
1576 RunJar
681 NodeManager
137 NameNode
426 SecondaryNameNode
1690 RunJar
732 HistoryServer
I'm launching the container using
docker run -it -p 8088:8088 -p 8042:8042 -p 4040:4040 -p 18080:18080 -p 10002:10002 -p 10000:10000 -e 3306 -e 9084 -h sandbox -v /home/foodie/docker/w1:/usr/tmp/test rohitbarnwal7/spark:mysql_corrected bash
Furthermore, I perform following steps to launch Hive server inside docker container
Start mysql service: service mysqld start
Switch to directory /usr/local/hive: cd $HIVE_HOME
Launch Hive metastore server: nohup bin/hive --service metastore &
Launch Hive server 2: hive --service hive-server2 (note that thrift-server port is already changed to 10001 in /usr/local/hive/conf/hive-site.xml)
Launch beeline shell: beeline
Connect beeline shell with Hive server-2: !connect jdbc:hive2://localhost:10001/default;transportMode=http;httpPath=cliservice
I've already tried the following things without any luck
Making python 2.7.3 as default python version inside docker container (original default is python 2.6.6, python 2.7.3 is installed inside container but isn't default)
Changing Hive server port to it's' default value: 10000
Trying to connect with Hive server by running same python script inside the container (it still gives the same error)

libGL error: failed to load driver swrast in docker container

I have a docker container from which I am trying to run a pyqt app. Everything works well except a chunk of the GUI is not able to render. The docker logs throw this out:
libGL error: failed to load driver: swrast
X Error: GLXBadContext 169
Extension: 154 (Uknown extension)
Minor opcode: 6 (Unknown request)
Resource id: 0x6400003
X Error: BadValue (integer parameter out of range for operation) 2
Extension: 154 (Uknown extension)
Minor opcode: 3 (Unknown request)
Resource id: 0x0
...
QGLContext::makeCurrent(): Failed.
In my Dockerfile, I tried installing pretty much all the packages I could find that might be related, including mesa-utils.
In terms of the docker-compose file, here's what it looks like:
version: '2'
services:
gui:
build: .
volumes:
- .:/usr/src
- /tmp/.X11-unix:/tmp/.X11-unix
command: /bin/bash -c "python start.py"
environment:
- DISPLAY=unix$DISPLAY
- QT_X11_NO_MITSHM=1
devices:
- "/dev/snd:/dev/snd"
- "/dev/dri:/dev/dri"
privileged: true
Any ideas what I might be missing?
Figured it out. I had to build the gui with hardware accelerated OpenGL support. Theres a repo (https://github.com/gklingler/docker3d) that contains docker images with nvidia or other graphics drivers support.
The other catch was, it didn't work for me unless the host and the container had the exact same driver. In order to resolve this, you can run the following shell script if you're running on linux:
#!/bin/bash
version="$(glxinfo | grep "OpenGL version string" | rev | cut -d" " -f1 | rev)"
wget http://us.download.nvidia.com/XFree86/Linux-x86_64/"$version"/NVIDIA-Linux-x86_64-"$version".run
mv NVIDIA-Linux-x86_64-"$version".run NVIDIA-DRIVER.run

Resources