Input/output error - docker

I create file and show its contents inside of virtual machine using Docker:
touch file.txt
docker run \
--rm -it \
--volume $(pwd):/app \
--workdir /app \
alpine:3.6 \
sh -c '\
while : ; do \
cat file.txt ;\
sleep 1 ;\
done \
'
Then I change file locally:
date > file.txt
Everything is ok.
But when I change file in PhpStorm, I get an error message for about 20 seconds:
cat: can't open 'file.txt': I/O error
What is possible reason of this behaviour and how can I diagnose it?
(Docker 17.10, VirtualBox 5.1.30, PhpStorm 2017.2.4)

Try disabling "safe write" option in PhpStorm settings: Settings/Preferences | Appearance & Behavior | System Settings --> Use "safe write"...
With that option enabled IDE writes into a temp file first (e.g. file.__temp_jb__) and only then renames it into the actual target (file.txt).
It looks like such manipulation (original file gets deleted and replaced by another one) somehow conflicts with your script.

Related

Firefox crashing inside docker container

Trying to launch firefox from the docker container with X window forwarding. It crashes every time.
Firefox version container - Mozilla Firefox 60.8.0
Host os firefox version - Mozilla Firefox 78.4.0esr
OS - Suse linux (SLES 15 SP2)
When I start the container with option --ipc= host it works fine. I am able to open the browser from the container. But this approach degrades the container isolation and security.
Firefox Crash report trace -
(firefox:29957): Gdk-ERROR **: The program 'firefox' received an X Window System error.
This probably reflects a bug in the program.
The error was 'BadValue (integer parameter out of range for operation)'.
(Details: serial 520 error_code 2 request_code 130 (unknown) minor_code 3)
(Note to programmers: normally, X errors are reported asynchronously;
that is, you will receive the error a while after causing it.
To debug your program, run it with the GDK_SYNCHRONIZE environment
variable to change this behavior. You can then get a meaningful
backtrace from your debugger if you break on the gdk_x_error() function.)
ExceptionHandler::GenerateDump cloned child 30085
ExceptionHandler::WaitForContinueSignal waiting for continue signal...
ExceptionHandler::SendContinueSignalToChild sent continue signal to child
./runbrowser.sh: line 43: 29957 Trace/breakpoint trap (core dumped)
This is my Docker file
FROM hc-us-east-aws-artifactory.cloud.abc.com/docker-prod/gui-apps:0.6
WORKDIR /usr/g/applications//service
COPY ./service/target/rcp-rest-1.8.5.jar ./plat/rcp-rest.jar
COPY ./packageRPM/scripts/common ./scripts
COPY ./packageRPM/scripts/container ./scripts
COPY ./packageRPM/config ./config
RUN chmod -R 777 ./scripts
EXPOSE 8217
ENV DISPLAY=:0.0
USER myuser
ENTRYPOINT ["sh","/usr/g/applications/entrypoint.sh"]
This is the docker startup script
docker run --rm \
--name my-service \
-v /tmp/.X11-unix:/tmp/.X11-unix:rw \
-v /etc/localtime:/etc/localtime \
-e TZ=`ls -la /etc/localtime | cut -d/ -f7-9` \
-e spring_profiles_active=Prod \
--network="host" \
--pid=host \
local/my-service:latest
Please suggest if any other alternative. Thanks in advance.

Need Uri for Activeperl installation in docker

For my use case, I need to create a windows container of Activeperl application. I found a GitHub link which explains the process for a Strawberry Perl.
Code snippet from the link
RUN \
if(!(Test-Path -Path 'C:\Temp')) \
{ \
New-Item \
-Path 'C:\Temp' \
-ItemType Directory \
-Verbose | Out-Null ; \
} ; \
\
Invoke-WebRequest \
-Uri "http://strawberryperl.com/download/$ENV:PERL_VERSION/strawberry-perl-$ENV:PERL_VERSION-64bit.zip" \
-OutFile "C:\\Temp\\strawberry-perl-$ENV:PERL_VERSION-64bit.zip" \
-UseBasicParsing \
-Verbose ; \
\
Expand-Archive \
-Path "C:\\Temp\\strawberry-perl-$ENV:PERL_VERSION-64bit.zip" \
-DestinationPath 'C:\Program Files\Perl' \
-Verbose ; \
\
Set-ItemProperty \
-Path 'HKLM:\\SYSTEM\\CurrentControlSet\\Control\\Session Manager\\Environment' \
-Name 'Path' \
-Value $($ENV:Path + ';C:\Program Files\Perl\perl\bin;C:\Program Files\Perl\perl\site\bin;C:\Program Files\Perl\c\bin') \
-Verbose ;
Strawberryperl Uri - http://strawberryperl.com/download/$ENV:PERL_VERSION/strawberry-perl-$ENV:PERL_VERSION-64bit.zip
For me, I want an equivalent Uri for downloading Activeperl instead of Strawberryperl. I am even ok with a Windows Container Image with activeperl installed already. I just couldn't find any of them.
Since ActivePerl doesn't provide an option for downloading a zipped version of Perl.
This is how I was able to transfer an active Perl to a Windows container
In my host windows machine, I have downloaded the Active Perl from https://activeperl.software.informer.com/download/
The downloaded file is ActivePerl-5.28.1.0000-MSWin32-x64-e90bcbf1.msi
I have installed this at the location C:\Perl64\
Copied this Perl64 folder into a new folder called Perl_root, so that this can be copied into the container via Dockerfile. Create a file called Dockerfile in the same location as Perl_root.
Filesystem_Root
C:\
|__ docker_trial
|___ Perl_root
|___ Dockerfile
Dockerfile:
#pulled a windows container from docker hub
FROM mcr.microsoft.com/windows/servercore:1607-amd64
ADD Perl_root .
Open a command prompt and navigate to the folder where dockerfile is present.
>docker build --tag dockertrail:1.0 .
>docker run -it --name tag1 dockertrail:1.0
Once the terminal inside the docker container opens up, open a PowerShell and create the update the environment variable Path as below
Powershell>[Environment]::SetEnvironmentVariable("Path",$env:Path+"C:\Perl64\site\bin;C:\Perl64\bin","Machine")
exit from the PowerShell and container. Now restart the container tag1, for the environment variable to work.
start the container tag1 again, once it starts, open up a PowerShell and run the command $env:Path you must be able to see the perl path being added to environment variables.
Now check the functioning of Perl using the command perl -v
This should print the perl verion.

Run execlineb when container start failed. Docker for windosw

I'm trying to run simple script inside docker container after start. Initialy previous developer decided to use s6 inside.
#!/usr/bin/execlineb -P
foreground { sleep 2 }
nginx
When i'm trying to start i'm gettings this message
execlineb: usage: execlineb [ -p | -P | -S nmin | -s nmin ] [ -q | -w | -W ] [ -c commandline ] script args
Looks like something wrong with executing this scripts or with execline.
I'm using docker for windows under windows10, however if somebody else trying to build this container in ubuntu(or any othe linux) evething is ok.
Can anybody help with this kind of problem?
DockerImage: simple alpine
According to our research of this "HUGE" problem we found two ways to solve it. Definitely it's a problem with special symbols, like '\r'
Option 1 dostounix:
install dostounix in your container(in docker file)
RUN apk --no-cache add \
dos2unix \
run it againts your sh script.
RUN for file in {PathToYourFiles}; do \
dos2unix $file; \
chmod a+xwr $file; \
done
enjoy your scripts.
Option 2 VsCode(or any textEditor):
Change CRLF 'End Of Line Sequence' to LF
VS Code bottom panel
Line endings options
enjoy your scripts.

Is it possible to add an installer, run it and delete it during one build step in Docker?

I'm trying to create a Docker image from a pretty large installer binary (300+ MB). I want to add the installer to the image, install it, and delete the installer. This doesn't seem to be possible:
COPY huge-installer.bin /tmp
RUN /tmp/huge-installer.bin
RUN rm /tmp/huge-installer.bin # <- has no effect on the image size
Using multiple build stages doesn't seem to solve this, since I need to run the installer in the final image. If I could execute the installer directly from a previous build stage, without copying it, that would solve my problem, but as far as I know that's not possible.
Is there any way to avoid including the full weight of the installer in the final image?
I ended up solving this by using the built-in HTTP server in Python to make the project directory available to the image over HTTP.
Inside the Dockerfile, I can run commands like this, piping scripts directly to bash using curl:
RUN curl "http://127.0.0.1:${SERVER_PORT}/installer-${INSTALLER_VERSION}.bin" | bash
Or save binaries, run them and delete them in one step:
RUN curl -O "http://127.0.0.1:${SERVER_PORT}/binary-${INSTALLER_VERSION}.bin" && \
./binary-${INSTALLER_VERSION}.bin && \
rm binary-${INSTALLER_VERSION}.bin
I use a Makefile to start the server and stop it after the build, but you can use a build script instead.
Here's a Makefile example:
SHELL := bash
IMAGE_NAME := app-test
VERSION := 1.0.0
SERVER_PORT := 8580
.ONESHELL:
.PHONY: build
build:
# Kills the HTTP server when the build is done
function cleanup {
pkill -f "python3 -m http.server.*${SERVER_PORT}"
}
trap cleanup EXIT
# Starts a HTTP server that makes the contents of the project directory
# available to the image
python3 -m http.server -b 127.0.0.1 ${SERVER_PORT} &>/dev/null &
sleep 1
EXTRA_ARGS=""
# Allows skipping the build cache by setting NO_CACHE=1
if [[ -n $$NO_CACHE ]]; then
EXTRA_ARGS="--no-cache"
fi
docker build $$EXTRA_ARGS \
--network host \
--build-arg SERVER_PORT=${SERVER_PORT} \
-t ${IMAGE_NAME}:latest \
.
docker tag ${IMAGE_NAME}:latest ${IMAGE_NAME}:${VERSION}
I think the best way is to download the bin from a website then run it:
RUN wget http://myweb/huge-installer.bin && /tmp/huge-installer.bin && rm /tmp/huge-installer.bin
in this way your image layer will not contain the binary you download
I didn't test it thoroughly, but wouldn't such an approach be viable? (Besides LinPy's answer, which is way easier if you have the possibility to just do it that way.)
Dockerfile:
FROM alpine:latest
COPY entrypoint.sh /tmp/entrypoint.sh
RUN \
echo "I am an image that can run your huge installer binary!" \
&& echo "I will only function when you give it to me as a volume mount."
ENTRYPOINT [ "/tmp/entrypoint.sh" ]
entrypoint.sh:
#!/bin/sh
/tmp/your-installer # install your stuff here
while true; do
echo "installer finished, commit me now!"
sleep 5
done
Then run:
$ docker build -t foo-1
$ docker run --rm --name foo-1 --rm -d -v $(pwd)/your-installer:/tmp/your-installer
$ docker logs -f foo-1
# once it echoes "commit me now!", run the next command
$ docker commit foo-1 foo-2
$ docker stop foo-1
Since the installer was only mounted as a volume, the image foo-2 should not contain it anymore. You could also go and build another Dockerfile based on foo-2 to change the entrypoint, for example.
Cf. docker commit

How do I edit a file after I shell to a Docker container?

I successfully shelled to a Docker container using:
docker exec -i -t 69f1711a205e bash
Now I need to edit file and I don't have any editors inside:
root#69f1711a205e:/# nano
bash: nano: command not found
root#69f1711a205e:/# pico
bash: pico: command not found
root#69f1711a205e:/# vi
bash: vi: command not found
root#69f1711a205e:/# vim
bash: vim: command not found
root#69f1711a205e:/# emacs
bash: emacs: command not found
root#69f1711a205e:/#
How do I edit files?
As in the comments, there's no default editor set - strange - the $EDITOR environment variable is empty. You can log in into a container with:
docker exec -it <container> bash
And run:
apt-get update
apt-get install vim
Or use the following Dockerfile:
FROM confluent/postgres-bw:0.1
RUN ["apt-get", "update"]
RUN ["apt-get", "install", "-y", "vim"]
Docker images are delivered trimmed to the bare minimum - so no editor is installed with the shipped container. That's why there's a need to install it manually.
EDIT
I also encourage you to read my post about the topic.
If you don't want to add an editor just to make a few small changes (e.g., change the Tomcat configuration), you can just use:
docker cp <container>:/path/to/file.ext .
which copies it to your local machine (to your current directory). Then edit the file locally using your favorite editor, and then do a
docker cp file.ext <container>:/path/to/file.ext
to replace the old file.
You can use cat if it's installed, which will most likely be the case if it's not a bare/raw container. It works in a pinch, and ok when copy+pasting to a proper editor locally.
cat > file
# 1. type in your content
# 2. leave a newline at end of file
# 3. ctrl-c / (better: ctrl-d)
cat file
cat will output each line on receiving a newline. Make sure to add a newline for that last line. ctrl-c sends a SIGINT for cat to exit gracefully. From the comments you see that you can also hit ctrl-d to denote end-of-file ("no more input coming").
Another option is something like infilter which injects a process into the container namespace with some ptrace magic: https://github.com/yadutaf/infilter
To keep your Docker images small, don't install unnecessary editors. You can edit the files over SSH from the Docker host to the container:
vim scp://remoteuser#containerip//path/to/document
You can use cat if installed, with the > caracter.
Here is the manipulation :
cat > file_to_edit
#1 Write or Paste you text
#2 don't forget to leave a blank line at the end of file
#3 Ctrl + C to apply configuration
Now you can see the result with the command
cat file
For common edit operations I prefer to install vi (vim-tiny), which uses only 1491 kB or nano which uses 1707 kB.
In other hand vim uses 28.9 MB.
We have to remember that in order for apt-get install to work, we have to do the update the first time, so:
apt-get update
apt-get install vim-tiny
To start the editor in CLI we need to enter vi.
You can open existing file with
cat filename.extension
and copy all the existing text on clipboard.
Then delete old file with
rm filename.extension
or rename old file with
mv old-filename.extension new-filename.extension
Create new file with
cat > new-file.extension
Then paste all text copied on clipboard, press Enter and exit with save by pressing ctrl+z. And voila no need to install any kind of editors.
Sometime you must first run the container with root:
docker exec -ti --user root <container-id> /bin/bash
Then in the container, to install Vim or something else:
apt-get install vim
I use "docker run" (not "docker exec"), and I'm in a restricted zone where we cannot install an editor. But I have an editor on the Docker host.
My workaround is: Bind mount a volume from the Docker host to the container (https://docs.docker.com/engine/reference/run/#/volume-shared-filesystems), and edit the file outside the container. It looks like this:
docker run -v /outside/dir:/container/dir
This is mostly for experimenting, and later I'd change the file when building the image.
After you shelled to the Docker container, just type:
apt-get update
apt-get install nano
You can just edit your file on host and quickly copy it into and run it inside the container. Here is my one-line shortcut to copy and run a Python file:
docker cp main.py my-container:/data/scripts/ ; docker exec -it my-container python /data/scripts/main.py
If you use Windows container and you want change any file, you can get and use Vim in Powershell console easily.
To shelled to the Windows Docker container with PowerShell:
docker exec -it <name> powershell
First install Chocolatey package manager
Invoke-WebRequest https://chocolatey.org/install.ps1 -UseBasicParsing | Invoke-Expression;
Install Vim
choco install vim
Refresh ENVIRONMENTAL VARIABLE
You can just exit and shell back to the container
Go to file location and Vim it vim file.txt
See Stack Overflow question
sed edit file in place
It would be a good option here, if:
To modify a large file, it's impossible to use cat.
Install Vim is not allowed or takes too long.
My situation is using the MySQL 5.7 image when I want to change the my.cnf file, there is no vim, vi, and Vim install takes too long (China Great Firewall). sed is provided in the image, and it's quite simple. My usage is like
sed -i /s/testtobechanged/textwanted/g filename
Use man sed or look for other tutorials for more complex usage.
It is kind of screwy, but in a pinch you can use sed or awk to make small edits or remove text. Be careful with your regex targets of course and be aware that you're likely root on your container and might have to re-adjust permissions.
For example, removing a full line that contains text matching a regex:
awk '!/targetText/' file.txt > temp && mv temp file.txt
(More)
If you can only shell into container with bin/sh (in case bin/bash doesn't work)
and apt or apt-get doesn't work in the container, check whether apk is installed by entering apk in command prompt inside the container.
If yes, you can install nano as follows:
apk add nano
then nano will work as usual
An easy way to edit a few lines would be:
echo "deb http://deb.debian.org/debian stretch main" > sources.list
You can install nano
yum install nano
You can also use a special container which will contain only the command you need: Vim. I chose python-vim. It assumes that the data you want to edit are in a data container built with the following Dockerfile:
FROM debian:jessie
ENV MY_USER_PASS my_user_pass
RUN groupadd --gid 1001 my_user
RUN useradd -ms /bin/bash --home /home/my_user \
-p $(echo "print crypt("${MY_USER_PASS:-password}", "salt")" | perl) \
--uid 1001 --gid 1001 my_user
ADD src /home/my_user/src
RUN chown -R my_user:my_user /home/my_user/src
RUN chmod u+x /home/my_user/src
CMD ["true"]
You will be able to edit your data by mounting a Docker volume (src_volume) which will be shared by your data container (src_data) and the python-vim container.
docker volume create --name src_volume
docker build -t src_data .
docker run -d -v src_volume:/home/my_user/src --name src_data_1 src_data
docker run --rm -it -v src_volume:/src fedeg/python-vim:latest
That way, you do not change your containers. You just use a special container for this work.
First login as root :
docker run -u root -ti bash
Type following commands:
apt-get update &&
apt-get install nano
docker comes up with no editors. so simply install vim, 36MB space don't kill your docker!
Make sure to update the container before trying to install the editor.
apt-get update
apt-get install nano vi

Resources