I'm going to transfer what I worked on the previous EC2 to the ECS.
In traditional EC2, the -v /home/ubuntu:/data option allowed the volume to be set.
First, I added volume through "Volume add in task definition" and proceeded with mounting as before.
However, this did not produce a normal result.
So I have some concerns.
For Ubuntu, it's the /home/ubuntu path, but I'm not sure how the ECS Fargate path is configured.
Secondly, I am wondering if adding :/data at the end of the container path is the right way.
Defined Volume
Volume set to existing EC2 written in JSON
Mount Points in ECS
With Fargate you would need to use an EFS volume for this. You don't have access to host volumes with Fargate.
Related
What I want to accomplish is the equivalent of :
docker run -v /var/run/docker.sock:/var/run/docker sock <image>
EDITED:
I followed this: Does ECS task definition support volume mapping syntax?
But then it is unable to save because this type of bind mount is not available for fargate.
Is there another way to accomplish it for fargate?
You are trying to access the Docker socket from within a container managed by AWS. I guess this is not available. Instead, you need to make API calls to AWS to launch new containers. Maybe something like this? Or probably rethink the whole thing.
My task is to deploy a third-party OSRM service on Amazon ECS Fargate.
For OSRM docker at startup, you need to transfer a file containing geodata.
The problem is that Amazon ECS Fargate does not provide access to the host file system and does not provide the ability to attach files and folders during container deployments.
Therefore, I would like to create an intermediate image that, when building, saved the file with geodata, and when starting the container, it would use it when defining volumes.
Thanks!
As I understand it, Amazon ECS is a plain container orchestrator and does not implement docker swarm, so things like docker configs are off the cards.
However, you should be able to do something like this :-
ID=$(docker create --name my-osrm osrm-base-image)
docker cp ./file.ext $ID:/path/in/container
docker start $ID
The solution turned out to be quite simple.
For this Dockerfile, I created an image on my local machine and hosted it on DockerHub:
FROM osrm/osrm-backend:latest
COPY data /data
ENTRYPOINT ["osrm-routed","--algorithm","mld","/data/andorra-latest.osm.pbf"]
After that, without any settings and volumes, I launched this image in AWS ECS
I want to create some docker images that generates text files. However, since images are pushed to Container Registry in GCP. I am not sure where the files will be generated to when I use kubectl run myImage. If I specify a path in the program, like '/usr/bin/myfiles', would they be downloaded to the VM instance where I am typing "kubectl run myImage"? I think this is probably not the case.. What is the solution?
Ideally, I would like all the files to be in one place.
Thank you
Container Registry and Kubernetes are mostly irrelevant to the issue of where a container will persist files it creates.
Some process running within a container that generates files will persist the files to the container instance's file system. Exceptions to this are stdout and stderr which are both available without further ado.
When you run container images, you can mount volumes into the container instance and this provides possible solutions to your needs. Commonly, when running Docker Engine, it's common to mount the host's file system into the container to share files between the container and the host: docker run ... --volume=[host]:[container] yourimage ....
On Kubernetes, there are many types of volumes. An seemingly obvious solution is to use gcePersistentDisk but this has a limitation in that it these disks may only be mounted for write on one pod at a time. A more powerful solution may be to use an NFS-based solution such as nfs or gluster. These should provide a means for you to consolidate files outside of the container instances.
A good solution but I'm unsure whether it is available, would be to write your files as Google Cloud Storage objects.
A tenet of containers is that they should operate without making assumptions about their environment. Your containers should not make assumptions about running on Kubernetes and should not make assumptions about non-default volumes. By this I mean, that your containers will write files to container's file system. When you run the container, you apply the configuration that e.g. provides an NFS volume mount or GCS bucket mount etc. that actually persists the files beyond the container.
HTH!
I have a Jenkins instance setup using Googles Jenkins on Kubernetes solution. I have not changed any of the settings of the Kubernetes Pod.
When I trigger a new job I am successfully able to get everything up and running until the point of my tests.
My tests use docker-compose. First I make sure to install docker (1.5-1+b1) and docker-compose (1.8.0-2) on the instance (I know I can optimize this by using an image that already includes these, but I am still just in proof-of-concept).
When I run the docker-compose up command everything works and the services start their initialization scripts. However, the mounts are empty. I have verified that the files exist on the Jenkins slave, and the mount is created inside the docker service when I run docker-compose, however they are empty.
Some information:
In order to get around file permissions I am using /tmp as the Jenkins Workspace. I am using SCM to pull my files (successfully) and in the docker-compose file I specify version: '2' and the mount paths with absolute paths. The volume section of the service that fails looks like this:
volumes:
- /tmp/automation:/opt/automation
I changed the command that is run in the service to ls /opt/automation and the result is an empty directory.
What am I missing? I just want to mount a directory into my docker-compose service. This works perfectly from Windows, Ubuntu, and Centos devices. Why won't it work using the Kubernetes instance?
I found the reason it fails here:
A Docker container in a Docker container uses the parent HOST's Docker daemon and hence, any volumes that are mounted in the "docker-in-docker" case is still referenced from the HOST, and not from the Container.
Therefore, the actual path mounted from the Jenkins container "does not exist" in the HOST. Due to this, a new directory is created in the "docker-in-docker" container that is empty. Same thing applies when a directory is mounted to a new Docker container inside a Container.
So it seems like it will be impossible to mount something from the outer docker into the inner docker. And another solution must be found.
I am looking for a kubernetes equivalent of docker -v for mounting the volumes in gcloud.
I am trying to run my container using google-container-engine which uses kubectl to manage clusters. In the kubectl run command I could not fund any provision for mounting the volumes.
kubectl run foo --image=gcr.io/project_id/myimage --port 8080
I checkout their official docs but could not find any clue whatsoever.
As at the moment, It's not possible to mount a persistent Volume in a container by using imperative ways or using generators command (run, expose).Therefore, You could use declarative way to get it done.
Kubernetes provides 2 abstractions for storage in a cluster which are persistent volume claim (PVC) and persistent volume (PV). Moreover, you can use storage class to provide Persistent volume (PV) in a dynamic way.
persistent-volumes.
storage-classes
When you write a manifest file for deployment you need to use a volume claim field to access PVC as well as you will write a PVC to claim PV.