How to tell docker-compose exec to read from stdin? - docker

docker-compose exec is failing when reading from stdin:
$ echo "use mydb; show measurements;" | docker-compose exec influxdb influx
Connected to http://localhost:8086 version 1.2.4
InfluxDB shell version: 1.2.4
> Traceback (most recent call last):
File "/usr/local/bin/docker-compose", line 9, in <module>
load_entry_point('docker-compose==1.9.0', 'console_scripts', 'docker-compose')()
File "/usr/local/lib/python2.7/dist-packages/compose/cli/main.py", line 65, in main
command()
File "/usr/local/lib/python2.7/dist-packages/compose/cli/main.py", line 117, in perform_command
handler(command, command_options)
File "/usr/local/lib/python2.7/dist-packages/compose/cli/main.py", line 462, in exec_command
pty.start()
File "/usr/local/lib/python2.7/dist-packages/dockerpty/pty.py", line 338, in start
io.set_blocking(pump, flag)
File "/usr/local/lib/python2.7/dist-packages/dockerpty/io.py", line 32, in set_blocking
old_flag = fcntl.fcntl(fd, fcntl.F_GETFL)
ValueError: file descriptor cannot be a negative integer (-1)
Using the -T flag gets rid of the problem, but it seems to suppress reading from stdin:
$ echo "use mydb; show measurements;" | docker-compose exec -T influxdb influx
(I need to kill that command, since it shows nothing and is not able to read from stdin)
This seems like a docker-compose bug, maybe I should file this?
$ docker-compose --version
docker-compose version 1.9.0, build 2585387
EDIT
Related:
https://github.com/docker/compose/issues/3352
https://github.com/docker/compose/issues/4290

In https://github.com/docker/compose/issues/3352 endzyme suggests using the docker command for doing this. I created a tiny bash function I have in my bash profile based on his suggestion:
# Like `docker-compose exec` but forwarding stdin to the container
# See https://github.com/docker/compose/issues/3352
function docin() {
local service="$1"; shift
docker exec -i "$(docker-compose ps -q $service)" "$#"
}
In your case you would use it like:
echo "use mydb; show measurements;" | docin influxdb influx
It's a good enough work-around until they fix this in docker-compose.

Related

/usr/bin/sudo: Permission denied when calling sudo from sh script via telegra-cli with lua script

Im trying to run my .sh scipt status.sh via a telegram message:
Ubuntu 20.04.1 LTS server
Telegram-cli with a lua script to action status.sh script
when i send the message "status" to my server via telegram it actions the status.sh script, in this script i have a bunch of stuff that gathers info for me and sends it back to telegram so i can see what the status of my server is, however (i recently did a fresh install of the server) for some reason if the script has a line of code starting with sudo i get:
line 38: /usr/bin/sudo: Permission denied
if i run the script from the command line ./status.sh it runs without any problem!? so im thinking its because it is being called from telegram or lua!?
example of code that generates the error: sudo ifconfig enp0s25 >> file
on the other hand this line works without a problem:
sudo echo Time: $(date +"%H:%M:%S") > file
/usr/bin has 0755 permission set
sudo has 4755 permission set
The following command
sudo ifconfig enp0s25 >> file
would not work if file requires root privilege to be modified.
sudo affects ifconfig but not the redirection.
To fix it:
sudo sh -c 'ifconfig enp0s25 >> file'
As mentioned in Egor Skriptunoff's answer, sudo only affects the command being run with sudo, and not the redirect.
Perhaps nothing is being written to file in your case because ifconfig is writing the output you are interested in to stderr instead of to stdout.
If you want to append both stdout and stderr to file as root, use this command:
sudo sh -c 'ifconfig enp0s25 >> file 2>&1'
Here, sh is invoked via sudo so that the redirect to file will be done as root.
Without the 2>&1, only ifconfig's stdout will be appended to file. The 2>&1 tells the shell to redirect stderr to stdout.
If file can be written to without root, this may simplify to
sudo ifconfig enp0s25 >> file 2>&1

ModuleNotFoundError: No module named 'maskrcnn_benchmark'

I am doing MLperf, object_detection project test.
https://github.com/mlperf/training/tree/master/object_detection
Question 1:
When doing:
nvidia-docker run -v .:/workspace -t -i --rm --ipc=host mlperf/object_detection \
"cd mlperf/training/object_detection && ./install.sh"
It responses:
docker: Error response from daemon: create .: volume name is too short, names should be at least two alphanumeric characters.
I need to change -v .: to -v $(pwd):/workspace
Question 2:
When applied to the modification above, I got a new error:
docker: Error response from daemon: OCI runtime create failed: container_linux.go:345: starting container process caused "exec: \"cd mlperf/training/object_detection && ./install.sh\": stat cd mlperf/training/object_detection && ./install.sh: no such file or directory": unknown.
It seems docker can't accept a string with space, ex: "cd xxxxxxx && ./install.sh"
If I modified the string to single command (./install.sh)
nvidia-docker run -v $(pwd):/workspace -t -i --rm --ipc=host mlperf/object_detection \
"./install.sh"
This will work, it doesn't look like an incorrect path problem, I tested to use an absoluted path it got the same error.
Question 3:
After followed the steps in the webpage, I always got an error:
ModuleNotFoundError: No module named 'maskrcnn_benchmark'
root#nvme:/markkang/mlperf/training/object_detection# nvidia-docker run -v $(pwd):/workspace -t -i --rm --ipc=host mlperf/object_detection "./run_and_time.sh"
/workspace/pytorch /workspace
Traceback (most recent call last):
File "tools/train_mlperf.py", line 8, in <module>
from maskrcnn_benchmark.utils.env import setup_environment # noqa F401 isort:skip
ModuleNotFoundError: No module named 'maskrcnn_benchmark'
Edit train_mlperf.py and insert the following path code before the invocation of maskrcnn_benchmark.utils.env, e.g.
import sys
sys.path.append('/workspace/pytorch/')
from maskrcnn_benchmark.utils.env import setup_environment # noqa F401 isort:skip

Cannot start a Jupyter notebook from inside this docker container

I cannot, for the love of it, start Jupyter from inside a Docker container. My OS:
Software:
System Software Overview:
System Version: macOS 10.13.6 (17G5019)
Kernel Version: Darwin 17.7.0
Boot Volume: Macintosh HD
Boot Mode: Normal
Secure Virtual Memory: Enabled
System Integrity Protection: Enabled
Time since boot: 1:10
Dockerfile:
FROM tensorflow/tensorflow:latest-py3-jupyter
LABEL maintainer="xxxx"
ADD Mask_RCNN/ /Mask_RCNN/
ADD startup.sh /
RUN apt-get -y update
WORKDIR /
RUN pip3 install -r /Mask_RCNN/requirements.txt
RUN cd /Mask_RCNN/ && python3 setup.py install
CMD /startup.sh
The file startup.sh is simply
#!/bin/sh
/bin/bash -c "jupyter notebook --allow-root --no-browser --NotebookApp.token='sometoken'"
The Docker image is built without an hassle, with the following command:
docker image build --build-arg http_proxy=someproxy --build-arg https_proxy=someproxy --build-arg no_proxy=localhost -t mask-rcnn:v20190308 .
I run the container with
docker container run -e http_proxy=someproxy -e https_proxy=someproxy -e no_proxy=localhost --rm -it --name mask-rcnn -p 6067:8888 mask-rcnn:v20190308
and I get the following error:
[I 10:44:10.991 NotebookApp] Writing notebook server cookie secret to /root/.local/share/jupyter/runtime/notebook_cookie_secret
Traceback (most recent call last):
File "/usr/local/bin/jupyter-notebook", line 10, in <module>
sys.exit(main())
File "/usr/local/lib/python3.5/dist-packages/jupyter_core/application.py", line 266, in launch_instance
return super(JupyterApp, cls).launch_instance(argv=argv, **kwargs)
File "/usr/local/lib/python3.5/dist-packages/traitlets/config/application.py", line 657, in launch_instance
app.initialize(argv)
File "</usr/local/lib/python3.5/dist-packages/decorator.py:decorator-gen-7>", line 2, in initialize
File "/usr/local/lib/python3.5/dist-packages/traitlets/config/application.py", line 87, in catch_config_error
return method(app, *args, **kwargs)
File "/usr/local/lib/python3.5/dist-packages/notebook/notebookapp.py", line 1628, in initialize
self.init_webapp()
File "/usr/local/lib/python3.5/dist-packages/notebook/notebookapp.py", line 1407, in init_webapp
self.http_server.listen(port, self.ip)
File "/usr/local/lib/python3.5/dist-packages/tornado/tcpserver.py", line 143, in listen
sockets = bind_sockets(port, address=address)
File "/usr/local/lib/python3.5/dist-packages/tornado/netutil.py", line 168, in bind_sockets
sock.bind(sockaddr)
OSError: [Errno 99] Cannot assign requested address
What's happening? I mapped the Jupyter notebook port (8888) to a container port (6067), thus I don't understand what's the problem.
Apparently, just modifying the startup.sh script to
/bin/bash -c "jupyter notebook --ip 0.0.0.0 --allow-root --no-browser --NotebookApp.token='sometoken'"
fixed everything.

$'\r': command not found with Docker Compose

I am trying to complete the instructions here: https://docs.docker.com/compose/aspnet-mssql-compose/. I am at the last stage:
$ docker-compose up
I see this error:
DotNetCore$ sudo docker-compose up
Starting dotnetcore_db_1 ... done
Starting dotnetcore_web_1 ... done
Attaching to dotnetcore_db_1, dotnetcore_web_1
web_1 | ./entrypoint.sh: line 2: $'\r': command not found
: invalid optionpoint.sh: line 3: set: -
web_1 | set: usage: set [-abefhkmnptuvxBCHP] [-o option-name] [--] [arg ...]
web_1 | ./entrypoint.sh: line 5: $'\r': command not found
web_1 | ./entrypoint.sh: line 15: syntax error: unexpected end of file
dotnetcore_web_1 exited with code 2
I have spent all day trying to fix this simple error. Here is the entrypoint.sh:
#!/bin/bash
set -e
run_cmd="dotnet run --server.urls http://*:80"
until dotnet ef database update; do
>&2 echo "SQL Server is starting up"
sleep 1
done
>&2 echo "SQL Server is up - executing command"
exec $run_cmd
So far I have tried:
1) Open file using Notepad ++ and select Edit/EOL Conversion. Unix is greyed out. This method is descrbed here: https://askubuntu.com/questions/966488/how-do-i-fix-r-command-not-found-errors-running-bash-scripts-in-wsl
2) sudo dos2unix {filename}. This method is desecribed here: https://askubuntu.com/questions/966488/how-do-i-fix-r-command-not-found-errors-running-bash-scripts-in-wsl
How can I resolve this?
Your entrypoint script has windows linefeeds in it, they aren't valid on a Linux OS and are being parsed as commands to run. Correct that with your editor in the save menu, or use a utility like dos2unix to correct the files.
Once you have removed the linefeeds, you'll need to rebuild your image and then recreate a new container.
You could also set:
git config --global core.autocrlf input
from: https://github.com/docker/toolbox/issues/126

docker-compose up giving error UnicodeDecodeError: 'ascii' codec can't decode byte 0xe2 in position 0: ordinal not in range(128)

I am running my nodejs and blockchain code . And I made a Docker Container with dockerfile and docker compose but i am getting error while running the docker-compose up.
This is the traceback on console :
Traceback (most recent call last):
File "docker-compose", line 6, in <module>
File "compose/cli/main.py", line 68, in main
File "compose/cli/main.py", line 121, in perform_command
File "compose/cli/main.py", line 938, in up
File "compose/project.py", line 430, in up
File "compose/service.py", line 317, in ensure_image_exists
File "compose/service.py", line 918, in build
File "site-packages/docker/api/build.py", line 238, in build
File "site-packages/docker/api/build.py", line 285, in _set_auth_headers
File "site-packages/docker/auth.py", line 97, in resolve_authconfig
File "site-packages/docker/auth.py", line 125, in _resolve_authconfig_credstore
File "site-packages/dockerpycreds/store.py", line 25, in get
File "site-packages/dockerpycreds/store.py", line 57, in _execute
File "subprocess.py", line 711, in __init__
File "subprocess.py", line 1343, in _execute_child
UnicodeDecodeError: 'ascii' codec can't decode byte 0xe2 in position 0: ordinal not in range(128)
Failed to execute script docker-compose
and my docker-compose.yml is :
version: '2'
services:
node-app:
build: app
ports:
- "4000:8080"
volumes:
- ./app/src:/myapp/src
depends_on:
- ethernet_server
environment:
- ETHEREUM_CLIENT_IP=http://192.168.178.22
- ETHEREUM_CLIENT_PORT=8545
ethernet_server:
build: testrpc
I don't know what went wrong.can someone help me to fix it?
Paste the below mentioned lines in your docker file. And it will fix the unicode error.
RUN apt-get update -y
RUN apt-get install --reinstall -y locales
# uncomment chosen locale to enable it's generation
RUN sed -i 's/# pl_PL.UTF-8 UTF-8/pl_PL.UTF-8 UTF-8/' /etc/locale.gen
# generate chosen locale
RUN locale-gen pl_PL.UTF-8
# set system-wide locale settings
ENV LANG pl_PL.UTF-8
ENV LANGUAGE pl_PL
ENV LC_ALL pl_PL.UTF-8
Try building the containers directly with docker:
docker build app
docker build testrpc
then run your docker-compose again.
I was encountering a similar report from docker-compose of UnicodeDecodeError: 'ascii' codec can't decode byte 0xe2 in position 0 and found that using docker to rebuild my containers resolved the problem.
In my case, while saving docker commands i had saved it as a .odt file initially and then changed the file extension in finder app to .yml manually, resulted in above issue.
Used VS code (you can use any other tool or simply select all files as format type while saving) to save the file with .yml extension and then tried executing docker-compose again worked for me.

Resources