Good day all,
Anyone knows if it's possible to just pull a single container from github? I do have this link https://github.com/aws/sagemaker-pytorch-training-toolkit and I will like to pull the container in this link https://github.com/aws/sagemaker-pytorch-training-toolkit/tree/master/src/sagemaker_pytorch_container.
I did try using build docker build -t https://github.com/abc/sagemaker-pytorch-training-toolkit.git to just build an image of one file but there's an init.py file which i'm not sure if its necessary.
Thanks
You are on a wrong path.
Github does not store docker images, so there is no way you can pull it from there.
AWS Sagemaker provides pre-built images, you just need to select the one you want to use when creating an instance. see https://docs.aws.amazon.com/sagemaker/latest/dg/howitworks-create-ws.html
If you need a docker with pytorch, just run docker pull pytorch/pytorch
Related
I have a custom docker image that run certain security checks on the Github repository and posts the results to a URL for review/analysis. I can see now that Github actions support custom Docker images. My question is will the runner environment variables and the clone repo be automatically mounted into the custom container or I need to pass them individually.
Also my custom container image is around 1G in size. Downloading and running it on every build will slow down the test and build. What is the best option in this case. Is there any way I can cache the image? If not are there any other workaround?
Thanks
My question is will the runner environment variables and the clone repo be automatically mounted into the custom container or I need to pass them individually.
Automatically mounted no, you have to tell him it should use your custom container:
container:
image: ghcr.io/owner/image
# ...
Refs: github docs
Is there any way I can cache the image? If not are there any other workaround?
I think you should check around cache action. I guess you should be able to download your image and cache the path where you download it or something like that.
I'm building a small web app with Vue.js and an Express API, each with their own Dockerfile. I currently am able to build those images and publish them to a private Docker repository, then pull them onto a virtual machine and run them. I want to add Docker Compose, and I've often seen that together with the code for the services, such as
|--..
|__api/
|__client/
|__docker-compose.yml
but that seem then like you can't publish the images to a repository, since Docker Compose builds the images and runs the containers, and so my VM would need to pull all the code, when to my thinking it should just need the images and then know how to run them.
So am I thinking about Docker Compose wrong? I have very little experience with it; I'm just trying to figure out the best way to be able to run the containers and it seems like I should be able to do that on a VM without having to download all the source code to that VM.
You can use docker-compose and still publish the individual images.
I guess that the API and the client have their own Docker files respectively.
So basically you have three options:
Let docker-compose build the images via the build
option.
Just reference the images with the image
option and
make sure they are built before.
Do both so docker-compose will build those images and give them
the name and the tag that you put under the image option.
They are all valid options as far as I am concerned. If you go with
option two I would write a little Makefile or script that makes sure
the images are in place for convenience.
I'm trying to download a tagged docker image
docker pull clkao/postgres-plv8:10-2
and, in a compose file,
postgres:
image: clkao/postgres-plv8:10-2
But receive a manifest not found exception.
Unless I'm mistaken, that tag exists in Docker Hub, however I notice that it doesn't appear on the tags list.
Am I doing something wrong? Or is this perhaps an issue with Docker Hub or the way that repo has been set up?
If it isn't 'my fault', what's a recommendation to move forward? Create my own Dockerfile perhaps?
You might also try
docker pull -a <image>.
The -a will pull all versions of that image, which at least lets you know what is there.
(This is less useful if you really need a specific version, but helped me when I tried to pull an image that for some reason did not have a 'latest' tag.)
Edit: This is actually a really bad idea, since it will pull down the entire history, which for many repositories could be many GB. Better to go look at the repository site and see what tag you want. Note to self: don't post answers when you are tired. :-(
You get the error message because there exist no tag with "10-2".
You can try to figure out why and contact the repository owner or you can try to build your own one.
I just got over this "manifest for / not found: manifest unknown: The named manifest is not known to the registry."
Using
docker login <repo>
Check the docker's image also not only that the tag exists, I was trying to run Flyway version 5.0.1 for an image flyway/flyway which version did not exist, it existed only in version flyway/flyway:latest it seems, whereas 5.0.1 existed and I pulled it but in/from a different repository name, with repository name boxfuse/flyway.
for the error message 'docker manifest unknown'
When you use docker pull, without a tag, it will default to the tag :latest. Make sure that when we are building a image add tag latest or we can access the image by the tag name after image name with colon
I think you are trying to tag your image as v8.10.2. Make sure while tagging image locally you use same tag which you want to pull in future. So steps will be like below:
docker build -t clkao/postgres-pl:v8.10.2 .
docker push clkao/postgres-pl:v8.10.2
docker pull clkao/postgres-pl:v8.10.2
If this is from Git via docker.pkg.github.com then you need to switch to use ghcr.io. The former is deprecated and does not support the manifest endpoint so some docker clients, when they attempt to download various resources, fail with this error message. If you instead publish your image to ghcr (Github Container Repository), the docker image pulling process should complete successfully.
cd <dir with Dockerfile in it>
docker build -f Dockerfile -t ghcr.io/<org_id>/<project_id>:<version> .
docker push ghcr.io/<org_id>/<project_id>:<version>
More info here: https://docs.github.com/en/packages/working-with-a-github-packages-registry/migrating-to-the-container-registry-from-the-docker-registry
Note: The Container registry is currently in public beta and subject
to change. During the beta, storage and bandwidth are free. To use the
Container registry, you must enable the feature preview. For more
information, see "Introduction to GitHub Packages" and "Enabling
improved container support with the Container registry."
I have the following docker images.
$ docker images
REPOSITORY TAG IMAGE ID CREATED SIZE
hello-world latest 48b5124b2768 2 months ago 1.84 kB
docker/whalesay latest 6b362a9f73eb 22 months ago 247 MB
Is there a way I can see the Dockerfile of each docker image on my local system?
The answer at Where to see the Dockerfile for a docker image? does not help me because it does not exactly show the Dockerfile but the commands run to create the image. I want the Dockerfile itself.
As far as I know, no, you can't. Because a Dockerfile is used for building the image, it is not packed with the image itself. That means you should reverse engineer it. You can use docker inspect on an image or container, thus getting some insight and a feel of how it is configured. The layers an image are also visible, since you pull them when you pull a specific image, so that is also no secret.
However, you can usually see the Dockerfile in the repository of the image itself on Dockerhub. I can't say most repositories have Dockerfiles attached, but the most of the repositories I seen do have it.
Different repository maintainers may opt for different ways to document the Dockerfiles. You can see the Dockerfile tab on the repository page if automatic builds are set up. But when multiple parallel versions are available (like for Ubuntu), maintainers usually opt to put links the Dockerfiles for different versions in the description. If you take a look here: https://hub.docker.com/_/ubuntu/, under the "Supported tags" (again, for Ubuntu), you can see there are links to multiple Dockerfiles, for each respective Ubuntu version.
As the images are downloaded directly from the Dockerhub, only the image is pulled from the docker hub into your machine. If you want to see the dockerfile, then you can go to docker hub and type the image name and version name in the tag format (e.g ubuntu:14.04) this will open the image along with Docker file details. Also keep in mind, only if the owner of the image shared their Dockerfile, you can see it. Otherwise not. Most official images will not provide you with Dockerfile.
Hope it helps!
You can also regenerate the dockerfile from an image or use the docker history <image name> command to see what is inside.
check this: Link to answer
TL;DR
So if you have a docker image that was built by a dockerfile, you can recover this information (All except from the original FROM command, which is important, I’ll grant that. But you can often guess it, especially by entering the container and asking “What os are you?”). However, the maker of the image could have manual steps that you’d never know about anyways, plus they COULD just export an image, and re-import it and there would be no intermediate images at that point.
One approach could be to save the image in a image.tar file. Next extract the file and try to explore if you can find Dockerfile in any of the layer directories.
docker image save -o hello.tar hello-world
This will output a hello.tar file.
hello.tar is the compressed output image file and hello-world is the name of the image you are saving.
After that, extract the compressed file and explore the image layer directories. You may find Dockerfile in one of the directories.
However, there is one thing to note, if the image was built while ignoring the Dockerfile in the .dockerignore. Then you will not find the Dockerfile by this approach.
One can easily build docker images through docker build command.
What I'm wondering is the t flag that you can give when building the image. For example:
$ docker build -t ouruser/sinatra:v2 .
According to documentation, the t flag is for tagging and naming purposes. Name is the part before ':', and tag is the part after it. So in our example, the name is ouruser/sinatra, and the tag is v2.
I thought this would be the image name and tag. But apparently, the name is actually some repository name? Why do I think it is? Well, because if you would after this list the images with command:
docker images
You would get a listing like this:
REPOSITORY TAG IMAGE ID CREATED SIZE
ouruser/sinatra latest 5db5f8471261 11 hours ago 446.7 M
Bang! Major shock! You thought you were creating an image with name, and instead, you specified some repository! Related to this, I have some questions:
Where is this repository located?
Can I name the image without creating a repository?
Where and how is this repository used, or could be used?
Where can I find more information about this repository? I only found this, and it doesn't tell much to be honest: docker build docs
Why is it common to use names that consist of two parts like this: somename/someothername?
Thank you for your help!
I believe the confusion here is the word "repository." In Docker, a repository is any group of builds of an image with the same name, and potentially multiple tags. A "registry" server, like hub.docker.com or your own private registry, holds multiple repositories, e.g. the redis repository on the public registry. That one repository has multiple tags for different versions of the build.
So with that background, to answer your questions:
ouruser/sinatra is located on your local Docker host until you do a docker push
No, the repository and the tag is the name of the image.
While local on your system, you can use this image locally. Once you push it up to a registry, you can then pull it down to any other Docker host that has access to that registry. And if you do a docker save you can save that image for a docker load on another host.
I'm sure there is documentation covering this somewhere on docs.docker.com, but I learned from a class.
The username/imagebase format came about to support pushing to your own namespace in hub.docker.com. Without that, whoever makes the first "Redis" image calls it "redis" while the next person makes their own repository called "redis-improved", and we quickly get into a jumble of confusing names where it's not clear who made what and what is a reputable image. That naming isn't required for images you make locally, but is still encouraged since images you pull from hub.docker.com may lack a username if they are maintained by Docker themselves. Without your username, you won't know which images you pulled down and which you built yourself.