Apache PredictionIO - Docker run failed - docker

I have been trying http://predictionio.apache.org/install/install-docker/ this tutorial. I have successfully built Docker image however when I try to run docker run i get the Can't open /etc/predictionio/pio-env.sh error.
docker build -t predictionio/pio pio
docker run -ti predictionio/pio
PS: If I comment out the last line CMD ["sh", "/usr/bin/pio_run"] I can build and run docker image successfully. I can open the file too from docker bash.

I think you need to grant permissions to execute this file. add the following line at the end of your Dockerfile
RUN chmod +x pio_run.sh
also, you might need to change CMD to ENTRYPOINT like following:
ENTRYPOINT ["sh","/usr/bin/pio_run.sh"]

Your output states you are running Windows. Did you use the default command prompt or did you use docker terminal? I had the same error messages in the past on Windows but mysteriously it disappeared after trying the tutorial again. I am not sure what I did different except I might possibly used docker instead of the default command prompt...
Could you also try using docker-compose instead of plain docker commands as described in the tutorial?
Ensure your storage (Postgres, MySQL or ElasticSearch) is running before starting PIO as well.

Just resolved it on my machine.
When you cloned repository on Windows, git converted end of line symbols from Unix-style (\n) to Windows style (\r\n).
You need to open file C:\wherever-you-cloned-pio-repository\predictionio\docker\pio\pio_run and change it back (for e.g. using Visual Studio Code, or Notepad++). Then you need to rebuild the image and it should work.
Also for the future you may want to disable automatic conversion Disable git EOL Conversions

Related

Docker RUN instruction to source bash profile

I run some installation scripts via docker, they change ~/.bashrc but then I need to source it to use installed commands in RUN instructions below.
Tried obvious RUN . ~/.bashrc and got /bin/sh: 13: /root/.bashrc: shopt: not found error.
Tried RUN . ~/.profile and got mesg: ttyname failed: Inappropriate ioctl for device
I do not want to use ENV instructions. The point of having external installation scripts is to use them in non-Docker environments, for example when running unit tests locally. ENV instructions would duplicate environment setup which is already done in installation scripts.
You should not try to set up shell dotfiles in Docker. Many typical paths do not run them at all; for example
# In a Dockerfile
CMD ["some", "command", "here"]
# From the command line
docker run myimage some command here
The Docker environment is, fundamentally, different from a standalone Linux system; in addition to shell dotfiles, "home directory" isn't really a Docker concept, and if you have a multi-part process, on Docker it's standard to run each part in a separate container, but on standalone Linux you could use the init system to keep all of the parts running together. If you're expecting things to work exactly the same with exactly the same installation scripts, a virtual machine would be a better technological match for what you're attempting.
("Inappropriate ioctl for device" also suggests that there are things in the dotfiles that strongly expect to be run from an actual terminal, which you don't necessarily have at docker build time.)
My generic advice here is:
If possible, install things in the "system" directories within the image and avoid needing custom environment variable settings. (Don't use a version manager like nvm or rvm; don't use a Python virtual environment.)
If you do have to set environment variables, ENV is the way to do it.
If you really can't do either of the above, you can set environment variables in an ENTRYPOINT script before launching the main process; but if it's important to you that variables show up in docker inspect or docker exec shells, they won't be set there.
(Also remember that each RUN command launches a new container with a totally new shell environment. You can RUN . .profile; foo, but the environment variable settings won't carry through to the next RUN line.)

Dockerd with Chocolatey

I am using Chocolatey to install Docker.
When I originally run the following command:
choco install docker
and try to run the "docker --version" command, everything goes as expected.
Docker version 17.10.0-ce, build f4ffd25
When I try to run "dockerd" command, it shows as not being part of my path.
'dockerd' is not recognized as an internal or external command,
Looking at the PATH variable, and navigating to where Chocolatey stores the executables, dockerd.exe is not present while docker.exe is. Am I missing something in instructing Chocolatey in adding dockerd?
The reason I need the dockerd executable is so that I can limit the number of concurrent downloads, as shown in the Docker documentation.
This is a decision that the package maintainer(s) for Docker have made. If you have a look here:
https://chocolatey.org/packages/docker#files
You will see that there is a dockerd.exe.ignore file. This file is used to instruct Chocolatey to explicitly not create what is referred to as a shim file, which would make it work from the command line, in the same way as Docker does.
My best suggestion would be to reach out to the maintainers of that package to ask them why this was done, and to perhaps get it changed. You can do this by clicking on the Contact Maintainers link on this page:
https://chocolatey.org/packages/docker
As a workaround, you could add the following path to your Windows PATH environment variable:
C:\ProgramData\chocolatey\lib\docker\tools\docker
Which would allow it to work.

Windows doesn't recognize Docker command

I already installed Docker for windows. when I type docker --version command in Command prompt, it doesn't recognize it at all.
The message will be this:
'docker' is not recognized as an internal or external command,
operable program or batch file.
Did I miss something ?
I had installed Docker 18.06.1-ce version on my Windows 10 machine and faced the similar issue, even though the docker was added the Windows %PATH%.
I moved the docker path to the bottom and that solved my problem.
I restarted the system, it worked, maybe we can say that a restart is required.
You need to start the docker first if you have not and then open powershell.
In the powershell, try to run docker commands.
I installed docker Docker version 19.03.13 build 4484c46d9d, in Windows 10 pro 1903.
I was facing the same issue, then I just renamed 'com.docker.cli' to 'docker' and set the environment variable to 'C:\Program Files\Docker\Docker\resources\bin'
Problem Resolved.
Refer the image:
https://drive.google.com/file/d/1sZwx4udOzJeITV2RDGQKlsOt_TF4Wq2N/view?usp=sharing
https://drive.google.com/file/d/1DpW2DR2n_jCGezwrXuhNtXpSTBWmEDJk/view?usp=sharing
Renaming 'com.docker.cli' to 'docker' helped me finally get windows powershell and cmd terminals to recognize the docker command.
Add docker to PATH variable & refreshenv to keep using the same command prompt
If you've installed using docker toolbox the install path "C:\Program Files\Docker Toolbox"
Manually using Environmental Variable > Path (add docker path here)
Using Command Line
For temporary use set PATH=%PATH%;C:\Program Files\Docker Toolbox
Make sure to take a back up of PATH by echo %PATH% before doing this
For permanent change setx PATH=%PATH%;C:\Program Files\Docker Toolbox
docker: command not found
Windows 7: Just set the path of docker in system variable
Step:1
[Click on path -> edit-> paste the docker location]
Step:2 [Paste the docker location]
In my case C:\Program Files\Docker Toolbox.
now check $ docker version
Make sure the docker.exe path (C:\Program Files\Docker\Docker\resources\bin) is added to the PATH variable.
You can check it as follows:
ECHO %PATH%
The docker path had to be appended at the end of the PATH in my case. After that docker cmd was recognized.
Run as administrator worked for me, both powershell and bash on windows. I did not need to restart.
I checked environment variables and noticed that docker path is as the following path in which "R" in resources uppercase. I fixed the case and everything worked as expected
Path: "C:\Program Files\Docker\Docker\Resources\bin"
Just Restart the system, it is always good practice to restart the system when you install or uninstall any application. Hope this works :)
For those who are facing docker issue on VS Code
I was trying on VS Code Terminal after installation. Restarted and was still facing issue.
Then I tried with fresh terminal of Command Prompt -- it worked!
After that it also working fine on vs code.
In VS Code - close all terminals and open fresh terminal
and try test command:
docker --version
2022 - Windows 11
Add these paths to the PATH variable.
C:\Program Files\Docker\Docker\resources\bin
C:\ProgramData\DockerDesktop\version-bin
Docker PATH variable Windows 11:
You need to restart the system after installation. It worked for me.
Try to install GIT bash and then run this command on it for Windows:
base=https://github.com/docker/machine/releases/download/v0.16.0 &&
mkdir -p "$HOME/bin" &&
curl -L $base/docker-machine-Windows-x86_64.exe > "$HOME/bin/docker-machine.exe" &&
chmod +x "$HOME/bin/docker-machine.exe"
Click Here.
there is some issues with PowerShell or new version of docker due to which I was facing the same issue but then I went to this page and got way to run it.
PS: GIT bash is required.
If you have installed Docker Toolbox in your windows, go and add the environment variable.
PATH = "location of folder that contains docker-machine"
I also face a problem with the installation and running docker. I'm not sure you how did you install docker. I tried this way. I've downloaded the docker toolbox (https://github.com/docker/toolbox/releases) which comes up with docker,docker-machine. Oracle Virtual box which is pretty much enough to start docker locally.Please make sure you have git bash installed in your local. Once the toolbox installation is done click the icon generated on the desktop . Make sure this icon target to your Git bash.exe (you can verify this by right click on icon and find target). and verify docker version
For me, I had to ensure the check box Enable Hyper-V Windows Features is checked as I was installing.
I just wanted to have client installed, without the engine. So earlier, as I was installing, I unchecked that option. And after successful installation, I get the error - windows does not recognize docker.
But now that reinstalled with the checkbox checked, things are working fine now.
I noticed that running docker commands before startup causes this issue, but ensuring that docker is running and executing the commands after, they are then recognised.
I facing the same issue when I try to run docker -v in Vscode Terminal after install it. I try to use cmd and git bash it work fine. Restart your vscode will solve it
Unfortunately, After many tries and restarts, I uninstalled docker and Re-installed it again, and I had to build all things again.
I faced the same problem, them I tried in Powershell which works someHo
Download "docker-machine-Windows-x86_64.exe" from: https://github.com/docker/machine/releases
Rename "docker-machine-Windows-x86_64.exe" to "docker-machine.exe".
Copy "docker-machine.exe" to path C:\Program Files\Docker\Docker\resources\bin.
just add this path in your "Path" inside env variables inside windows
C:\Program Files\Docker\Docker\resources\bin
then restart your gitbash or cmd or ...
everything goes well after that !
*just add in your environmental Path this syntax if you did like (install docker in c drive) me, most probably it would work, for me, this passed well.
(note: if you going to call RefreshEnv.cmd or RefreshEnv
through your CMC with Chocolatey, this path would not be created for you, and you have to do it manualy in Windows 10 operating system).
C:\Program Files\Docker\Docker\resources\bin
In my case, I switched to the windows command prompt instead of the VScode terminal.

How to run Bazel container images on OSX?

According to the documentation at bazelbuild/rules_docker, it should be possible to work with these container images on OSX, and it also claims that it's possible to do so without docker.
These rules do not require / use Docker for pulling, building, or pushing images. This means:
They can be used to develop Docker containers on Windows / OSX without boot2docker or docker-machine installed.
They do not require root access on your workstation.
How do I do that? Here's a simple rule:
go_image(
name = "helloworld_image",
importpath = "github.com/nictuku/helloworld",
library = ":go_default_library",
visibility = ["//visibility:public"],
)
I can build the image with bazel build :helloworld_image. It produces a tar ball in blaze-bin, but it won't run it:
INFO: Running command line: bazel-bin/helloworld_image
Loaded image ID: sha256:08d312b529d30431c68741fd3a31468a02533f27a8c2c29eedc969dae5a39852
Tagging 08d312b529d30431c68741fd3a31468a02533f27a8c2c29eedc969dae5a39852 as bazel:helloworld_image
standard_init_linux.go:185: exec user process caused "exec format error"
ERROR: Non-zero return code '1' from command: Process exited with status 1.
It's trying to run the linux this is OSX, which is silly.
I also tried doing a "docker load" on the .tar content but it doesn't seem to like that format.
$ docker load -i bazel-bin/helloworld_image-layer.tar
open /var/lib/docker/tmp/docker-import-330829602/app/json: no such file or directory
Help? Thanks!
You are building for your host platform by default so you need to build for the container platform if you want to do that.
Since you are using a go binary, you can do cross compilation by specifying --cpu=k8 on the command line. Ideally we would be able to just say that the docker image needs a linux binary (so no need to specify the --cpu command-line flag) but this is still a work in progress in Bazel.

Docker hub automated build fails but locally does not

I have setup an automated build on Docker hub here (the sources are here).
The build goes well locally. I have also tried to rebuild it with --no-cache option:
docker build --no-cache .
And the process completes successfully
Successfully built 68b34a5f493a
However, the automated build fails on Docker hub with this error log:
...
Cloning into 'nerdtree'...
[91mVim: Warning: Output is not to a terminal
[0m
[91mVim: Warning: Input is not from a terminal
[0m
[m[m[0m[H[2J[24;1HError detected while processing command line:
E492: Not an editor command: PluginInstall
E492: Not an editor command: GoInstallBinaries
[91mmv: cannot stat `/go/bin/*': No such file or directory
[0m
This build apparently fails on the following vim command:
vim +PluginInstall +GoInstallBinaries +qall
Note that the warnings Output is not to a terminal and Input is not to a terminal appears also in the local build.
I cannot understand how this can happen. I am using a standard Ubuntu 14.04 system.
I finally figured it out. The issue was related to this one.
I am using Docker 1.0 in my host machine, however a later version is in production in Docker Hub. Without the explicit ENV HOME=... line in the Dockerfile, version 1.0 uses / as home directory, while /root is used by the later version. The result is that vim was not able to find its .vimrc file, since it was copied in / instead of /root. The solution I used is to explicitly define ENV HOME=/root in my Dockerfile, so there are no differences between the two versions.

Resources