I am running a docker container for rethinkdb. I want to load data from the existing rethinkdb database(running on a Linux machine) to the containerized rethinkdb empty clusters. I also want all the clusters to SyncUP and load with existing data.
You can export the existing database and import it into the new cluster.
rethinkdb dump [options]
Since the backup process uses client drivers, it takes advantage of RethinkDB’s concurrency. While it will use some cluster resources, it won’t lock out any clients, and it can be safely run on a live cluster.
For restore
rethinkdb restore filename
The other option can be is to mount the "/data" directory.
storageClassName: local-storage
local:
path: /mnt/disks/ssd1/rethinkdata
If you are in using cloud hosting, you can consider another volume option.
Kubernetes-volumes
rethink-backup
Related
I have recently started learning docker. However when studying swarm mode I see that containers can be scaled up. What I would like to know is once you scale conatiner in replicated mode will the data within the container be replicated too ? or just fresh containers will be spawned ?
For example lets say I created mysql service initially only with 1 copy. I create and update tables in that mysql container. Later I scale it to 3, will newly spawned containers contain same table data ? Also will the data be continuously be replicated across 3 docker instances ?
A replicated service will use fresh container instances per container. Swarm does not take care about replication of persistent data to be stored in volumes.
Dependening on the volume plugin (e.g. local driver /w remote nfs shares) you are limited to read-write-once or read-write-many. Even if your volume allows read-write-many, the service replicas might not support that, for instance mysql will not work if you point n replicas to the same volume. You can leverage swarm service template variables for instance to point your volumes to different target folders of the same nfs share.
Also with swarm, you will want to have storage that needs to be reachable from all nodes, as a container can die and be re-spawned on a different node. So either you will need to use a remote share based on NFS or CIFS (see example usages nfs cifs), a storage cluster like Ceph or GlusterFS or a cloud native storage like Portworx. While you have to take care of HA for remote share solutions, data replication is build in for storage clusters and cloud native storage.
In case a containerized service itself is cluster/replica aware it is usualy better to not use the swarm replica mechanism - unless all instances can be started with the same set of parameters.
I am running Docker Swarm with 3-Masters and 3-Worker nodes.
On this Swarm, I have an Elastic search container which reads data from multiple log files and then writes the data into a directory. Later it reads data from this directory and shows me the logs on a UI.
Now the problem is that I am running only 1 instance of this Elastic Search Container and say for some reason it goes down then docker swarm start this on another machine. Since I have 6 machines I have created the particular directory on all the machines, but whenever I start the docker stack the ES container starts and starts reading/writing directory on the machine where it is running.
Is there a way that we can
Force docker swarm to run a container on a particular machine
or
Map volume to shared/network drive
Both are available.
Force docker swarm to run a container on a particular machine
Add --constraint flag when executing docker service create. Some introduction.
Map volume to shared/network drive
Use docker volume with a driver that supports writing files to an external storage system like NFS or Amazon S3. More introduction.
I need to use Postgres database in Rancher stack (Cattle).
I made environment for my application, that contains api, frontend and database services (in one stack). I want to use multiple hosts for this environment, but if I will add some hosts, database can't properly work, because volume with data will only exists on one host.
I think, I cannot use NFS (or some other network storage) for databases because of IO speed. Am I right? Is there any workflow to use databases in Rancher?
I thought, that I can bind service only to one host of environment, but I didn't find this setting.
Rancher has Storage Services to manage Volumes.
NFS (rancher-nfs) will not work very well for a read/write Postgres database. If you're running on AWS Rancher also supports EBS volumes via convoy
For vanilla Docker local is the only volume type that will work in most places but these are obviously not shared volumes, it's local to the Docker host. If you create an environment scoped local volume for the database then the database container will always be scheduled to the node holding the storage.
Other shared storage volume types require the storage backend and a Docker volume plugin that can manage the movements of external storage. REX-Ray supports a number of storage providers. Flocker also supports a number of storage providers.
I have one serious doubt in docker swarm . I have created the docker-machine in VM
manager1
worker1
worker2
And joined all the worker to manager and create the service like
docker service create --replicas 3 -p 80:80 --name web nginx
I change the index.html in docker service in manager1
When I run the url like http://192.168.99.100 it showing the index.html file that I have changed but the remaining 2 node showing the default nginx page
What is the concept of swarm ? Whether it is used only for the service failure ?
How to make the centralized data storage in docker swarm.
There are a few approaches to ensuring the same app and data are available on all nodes.
Your code (like nginx with your web app) should be built into an image, sent to a registry, and pulled to a Swarm service. That way the same app/code/site is in all the containers of that service.
If you need persistent data, like a database, then you should use a plugin volume driver that lets you store the unique data on shared storage.
CoreOS used with fleet enables one to build services running some docker applications.
But is there any way to run docker services which require its state to be kept between restarts, to be persistent? For instance, databases or services that must store some files to be shared later on.
Because as far as I know, the service can be launched on core-1 machine (for example), and on restart will be launched on another one randomly. So the docker volume can be lost.
The simplest way to maintain a database service is to always schedule the fleet unit to the same machine. You can do this by adding an [X-Fleet] section to the fleet unit file and either assigning the unit to a particular X-ConditionMachineID, or X-ConditionMachineMetadata. See the coreos documentation.
You can then persist data outside your docker containers by mounting a volume from the host machine. The recommended way of doing this is to wrap this data in a separate data container via docker:
docker run --name mongodb-volume -v /home/core/mongodb-data:/data/db busybox
docker run -p 27017 --volumes-from mongodb-volume mongodb:latest
Since /home/core/mongodb-data on a particular machine will store persistent mongodb state and the unit will always be scheduled to that same machine, this will solve your problem.
You can consider to run some distributed filesystem over CoreOS cluster. This way whatever machine your database service container ends on, it will always be able to use database, mounted from DFS.