I have created new Mosquitto node in Jelastic as described in documentation.
How can I make persistent data/config volumes for it in order not to loose data when e.g. upgrading docker image?
There were two volumes already added before:
/mosquitto/data and /mosquitto/logs,
one more volume (/mosquitto/config) was added recently,
new updated package was already published to Marketplace.
Cheers!
Related
I'm using confluentinc/cp-server-connect (https://hub.docker.com/r/confluentinc/cp-server-connect) with elasticsearch sink connector (https://www.confluent.io/hub/confluentinc/kafka-connect-elasticsearch) i added trough dockerfile and rebuilding the image and it works just fine. I'm configuring the connector using http requests like it's done in this tutorial https://www.confluent.io/blog/kafka-elasticsearch-connector-tutorial/.
My problem is that I couldn't find a way to keep the connector configuration i set during removing and stopping again the docker container with this image.
I couldn't find any mentions of keeping configuration in docker image's documentation on docker hub or by googling it. I also tried manually searching in the image for where this configuration may be stored but i had no luck. Where should I point with docker volume to save this configuration, or maybe the configuration is kept somewhere else like in a specific topic in kafka?
Yes, the configurations are kept on Kafka topic. The Connect container doesn't store them.
Therefore, don't restart the Kafka (or Zookeeper) container(s), and your configs will be maintained.
I'm definitely new to docker at all, started to use it a while ago and I need to move my stuff from one server to another. I thought that just creating a personal image will solve this issue, but nope :D.
So if I'm right, all data is saved on created volume, right? Like one of the containers is PostgreSQL.
So to move everything i need also backup the volume and export it on a new server?
https://docs.docker.com/storage/volumes/#backup-restore-or-migrate-data-volumes
This is what I found on their docs.
Hope somebody could help me with understanding
Docker, for default, stored images, containers, volumes, and other data, into /var/lib/docker, if not customized by the file /etc/docker/daemons.json as explained here.
In order to move all the graph to a new server you should:
Stop docker service.
Copy data root.
Restart docker service.
Regards.
I have taken some time to create a useful Docker volume for use at work. It has a restored backup of one of our software databases (SQL Server) on it, and I use it for testing/debug by just attaching it to whatever Linux SQL Container I feel like running at the time.
When I make useful Docker images at work, I share them with our team using either the Azure Container Registry or the AWS Elastic Container Registry. If there's a DockerFile I've made as part of a solution, I can store that in our GIT repo for others to access.
But what about volumes? Is there a way to share these with colleagues so they don't need to go through the process I went through to build the volume in the first place? So if I've got this 'databasevolume' is there a way to source control it? Or share it as a file to other users of Docker within my team? I'm just looking to save them the time of creating a volume, downloading the .bak file from its storage location, restoring it etc.
The short answer is that there is no default docker functionality to export the contents of a docker volume and docker export explicitly does not export the contents of the volumes associated with the container. You can backup, restore or migrate data volumes.
Note: if your're backing up a database I'd suggest using the appropriate tools for that database.
I have a container deployed in a pod by Kubernetes running Jenkins. The container is mounted with a persistent storage volume (AWS Elastic File Store) that's currently storing all of the Jenkins instance's user, configuration, job configurations, etc.
I need to update Jenkins. Normally when I do this, the process wipes out the storage, since the whole container gets re-launched. However, I need to figure out how to do this without losing the data.
How do I update Jenkins without losing the info on the storage volume attached to the container?
I ended up finding the answer I needed by first reading this article, which helped me understand the underlying concepts:
http://www.monkeylittle.com/blog/2017/02/08/adding-persistent-volumes-to-jenkins-with-kubernetes-volumes.html
Then I found this article, which shows you how to do it with EFS, specifically:
https://itnext.io/efs-persistent-volumes-on-aws-kubernetes-193e0035bbfb
I couldn't have enough courage to start using docker now I'm feel like came from last century. I want to clear my doubts about docker before get started. My question is mainly for deploying/running docker images on cloud or hosting environment.
Can I build a docker image with any type of server (eg. wildfly, payara) and/or database server (eg. mysql, oracle) and will it work on docker enabled cloud/datacenter?
If it's yes how about persistent datas like database files and static storages (eg. images, uploaded documents, logs) those are stored in docker images or somewhere else? What will happen to those files when I update my application and redeploy new image?
I read posts about what is docker but I couln't find specific answer. Forgive me for not doing enough googling.
I have run docker on AWS and other cloud providers. It is really not that hard if you have some experience with system administration and or devops. Regarding cloud hosters and getting started, most providers have some sort of tutorial on how to get started using docker with their infrastructure:
http://docs.aws.amazon.com/AmazonECS/latest/developerguide/docker-basics.html
https://azure.microsoft.com/en-us/documentation/articles/virtual-machines-linux-dockerextension/
Can I build a docker image with any type of server (eg. wildfly,
payara) and/or database server (eg. mysql, oracle) and will it work on
docker enabled cloud/datacenter?
To get a server up and running, you just need the docker engine installed on the host, there are packages for many distros:
https://docs.docker.com/engine/installation/
After docker engine is installed, you can create dockerfiles for basically any server or service. Hopefully you do not need to, in most cases, since there are countless docker files and pre-configured, vendor maintained images already available on dockerhub (I use wildfly, elk-stack, and mysql for example). Be careful about selecting images are maintained, otherwise you end up with security issues in your images that might never get fixed! Or you have to do it yourself!
Example images:
https://hub.docker.com/r/jboss/wildfly/
https://hub.docker.com/_/mysql/
https://hub.docker.com/_/oraclelinux/
https://hub.docker.com/u/payara/
If it's yes how about persistent datas like database files and static
storages (eg. images, uploaded documents, logs) those are stored in
docker images or somewhere else? What will happen to those files when
I update my application and redeploy new image?
In general, you will want to store persistent data external to the docker image and mount it into the image as a volume:
https://docs.docker.com/engine/tutorials/dockervolumes/
Some cloud based storage providers might be easier to mount or connect to in other ways, but this volume approach is standard, IMO.
For logfiles, I actually push them to an ELK server, so having a volume for the logs is not necessarily required. However, since the ELK server is also a docker image, it does have a volume where the data is persisted.
So you have:
documentation from your cloud hoster (or docker themselves)
a host in your cloud running docker engine
0..n images that you can either grab from dockerhub or build yourself.
storage for persistent data on this host or mounted from elsewhere that you mount into your docker images on startup. this is where e.g. mysql data folders live, or where you can persist logs, etc.
Of course, it can get much more complex from there, e.g. how to transparently scale and update your environment etc., but that is something for e.g. kubernetes or docker swarm or some other solution (I've scripted a bit on my own but do not need the robustness or elastic scalability of large systems).
Regarding cluster management, it should be noted that Swarm is now included in the Docker Core. This has created some controversy in the community and even talks of a fork of the core:
https://technologyconversations.com/2015/11/04/docker-clustering-tools-compared-kubernetes-vs-docker-swarm/
https://jaxenter.com/docker-1-12-is-probably-the-most-important-release-since-1-0-129080.html
http://searchitoperations.techtarget.com/news/450303918/Docker-fork-talk-prompts-container-standardization-brawl
http://www.infoworld.com/article/3118345/cloud-computing/why-kubernetes-is-winning-the-container-war.html
I have experience running docker on Alibaba cloud and AWS as well. I did not see any difference in working with docker on both cloud providers. Docker images can be build same way on all linux platform regardless of the cloud provider. However, persistence of data need to be taken care using docker volumes. However, it is recommended to use managed service such as RDS in Alibaba cloud for databases instead of using docker.
Can I build a docker image with any type of server (eg. wildfly,
payara) and/or database server (eg. mysql, oracle) and will it work on
docker enabled cloud/datacenter?
You can build your own Docker images or use solutions that are already pre-packaged and proven by cloud providers. For example, here is an auto-clustering Docker-based implementation of GlassFish that can be run and managed on Jelastic PaaS.
If it's yes how about persistent datas like database files and static
storages (eg. images, uploaded documents, logs) those are stored in
docker images or somewhere else? What will happen to those files when
I update my application and redeploy new image?
With the above mentioned cluster, all data is kept inside containers and stays without changes after restart. As an option, you can also connect a separate data storage container if you wish to share it across other containers.