Do they use environment / config variables to link the persistent storage to the project related docker image ?
So that everytime new VM is assigned, the cloud shell image can be run with those user specific values ?
Not sure to have caught all your questions and concerns. So, Cloud Shell is in 2 parts:
The container that contains all the installed library, language support/sdk, binaries (docker for example). This container is stateless and you can change it (in the setting section of Cloud Shell) if you want to deploy a custom container. For example, it's what is done with Cloud Run Button for deploying a Cloud Run service automatically.
The volume dedicated to the current user that is mounted in the Cloud Shell container.
By the way, you can easily deduce that all you store outside the /home/<user> directory is stateless and not persist. /tmp directory, docker image (pull or created),... all of these are lost when the Cloud Shell start on other VM.
Only the volume dedicated to the user is statefull, and limited to 5Gb. It's linux environment and you can customize the .profile and .bash_rc files as you want. You can store keys in /.ssh/ directory and all the other tricks that you can do on Linux in your /home directory.
Related
I'm using named volumes to persist data on Host machine in the cloud.
I want to take backup of these volumes present in the docker environment so that I can reuse them on critical incidents.
Almost decided to write a python script to compress the specified directory on the host machine and push it to the AWS S3.
But I would like to know if there is any other approaches to this problem?
docker-volume-backup may be helpful. It allows you to back up your Docker volumes to an external location or to a S3 storage.
Why use a Docker container to back up a Docker volume instead of writing your own Python script? Ideally you don't want to make backups while the volume is being used, so having a container on your docker-compose able to properly stop your container before taking backups can effectively copy data without affecting the application performance or backup integrity.
There's also this alternative: volume-backup
I'm configuring a docker container for development purposes with the intent to re-configure it (minimally) for k8s cluster deployment. Immediately I run into the issue of user permissions with volume mounts to my local source directory.
For deployment to the cluster I will bake my source directory into the image, which is really the only change I would want to make for deployment.
I've read many SO articles suggesting running as your local user/group id (1000/1000 in my case).
In docker, writing file to mounted file-system as non-root?
Docker creates files as root in mounted volume
Let non-root user write to linux host in Docker
Understanding user file ownership in docker: how to avoid changing permissions of linked volumes
Is it possible/sane to develop within a container Docker
But all of those questions seem to glance over a seemingly critical detail. When you use --user to alter your user ID within the docker container you lose root, and along with it a lot of functionality, for example whoami doesn't work. It seems to become very cumbersome to test configuration changes in the docker environment, which is common during development.
The options for developing directly into the docker container seem very limited:
Add user/group 1000/1000 to the docker image, which seems to violate the run-anywhere mantra of docker/kubernetes.
chown all your files constantly during development and use root in the container.
Are there other options to this list that is more palatable for developing directly into a docker container?
I am running an OPC UA Server in a Docker container. The OPC UA Server is connecting to a Cloud Service via a ID and a secret that is stored in a config-File. Furthermore the OPC UA Server holds SSH certificates for authentication.
I see a problem when releasing the image to a work group, because everyone would have access to my personal login, and to SSH certificates that were supposed to be unique to the Host that is running the image.
What would be the appropriate way to inject the certificates and the config-Files into a image, without building the whole thing again?
You have two main ways to pass configuration information into a container at runtime:
Use an environment variable for simple string values.
Use a volume. You can use a pure Docker volume, but a bind-mounted volume is often useful for things like key stores. Bind-mounted volumes share a file or directory from the host's filesystem into a specific location on the container filesystem.
Either way, you may need to inject the value into the right place in your config file. Sometimes config files can have variables from the environment - if not, then you can make your container entrypoint run a script to update the configuration file, and then exec your true entry point.
Context:
I have a Java Spring Boot Application which has been deployed to run on a Docker Container. I am using Docker Toolbox to be precise.
The application exposes a few REST API's to upload and download files. The application works fine on Docker i.e. i'm able to upload and download files using API.
Questions:
In the application I have hard coded the path as something like "C:\SomeFolder". What location is this stored on the Docker container?
How do I force the application when running on Docker to use the Host file system instead of Docker's File system?
This is all done by Docker Volumes.
Read more about that in the Docker documentation:
https://docs.docker.com/storage/volumes/
In the application I have hard coded the path as something like "C:\SomeFolder". What location is this stored on the Docker container?
c:\SomeFolder, assuming you have a Windows container. This is the sort of parameter you'd generally set via a command-line option or environment variable, though.
How do I force the application when running on Docker to use the Host file system instead of Docker's File system?
Use the docker run -v option or an equivalent option to mount some directory from the host on that location. Whatever the contents of that directory are on the host will replace what's in the container at startup time, and after that changes in the host should be reflected in the container and vice versa.
If you have an opportunity to rethink this design, there are a number of lurking issues around file ownership and the like. The easiest way to circumvent these issues are to store data somewhere like a database (which may or may not itself be running in Docker) and use network I/O to send data to and from the container, and store as little as possible in the container filesystem. docker run -v is an excellent way to inject configuration files and get log files out in a typical server-oriented use.
I can provide a ubuntu with ssh by docker, and user can setup their env.
For example, he apt-get install something and modify his bashrc, vimrc and so on.
Once I restart this computer, the user still has same env after restart finished.
How can I provide same service by k8s?
Once I restart the node, it will create another pod on other computer.
But the env is based on init image, not the latest env from the user.
The naive way, mount all volume on the shared storage(PV + PVC). Such as /bin /lib /opt /usr /etc /lib64 /root /var /home and so on(Each possible directory may effected by any installation). What is the best practice or other way to do this?
#Saket is Correct.
If a docker container needs to persist its state (in this case the user changing something inside the container), then that state must be saved somewhere... How would you do this with a VM? Answer: save to disk.
In k8s storage is represented as a persistent volume. Something called a PVC (persistent volume claim), is used to maintain the relationship between the POD (your code) and the actual storage volume (whose implementation details you are abstracted from). The latest version of k8s supports the dynamic creation of persistent volumes, so all you have to do is create a unique PVC specific to each user, when deploying their container (I assume here you have a "Deployment" and "Service" for each user as well).
In conclusion... Unusual to run SSH within a container. Have you considered giving each user their own k8s environment instead? For example Openshift is multi-tenanted. Indeed Redhat are integrating Openshift as a backend for Eclipse Che, thereby running the entire IDE on k8s. See:
https://openshift.io/
I would advise you to use ConfigMaps (https://github.com/kubernetes/kubernetes/blob/master/docs/design/configmap.md). This guide should help what you are trying to do: https://kubernetes.io/docs/tasks/configure-pod-container/configure-pod-configmap/#configure-all-key-value-pairs-in-a-configmap-as-pod-environment-variables
Configmaps also allow you to store scripts, so you could have a .bashrc (or a section) stored in a confipmap.