I keep getting EOF error when trying to pull images:
docker pull dpage/pgadmin4
Pull complete
942bbf3d7389: Downloading 1.82kB/1.82kB
fbe23c71dc3b: Download complete
7c1be9e99602: Download complete
ccc31a15f27f: Download complete
617b6e01309f: Download complete
e6cfa0ba7132: Download complete
9dd539b143fa: Downloading 3.15kB/3.15kB
6f3ff58d53db: Download complete
a79e40a556fb: Download complete
b05884a10df3: Download complete
3a39531f7518: Download complete
0337d3baf297: Downloading 9.88kB/9.88kB
c7a9de9c5d61: Download complete
unexpected EOF
I've tried to change the network and use my mobile hotspot. Sometimes it works and sometimes it does not.
I've had a couple Cloud Run services live for several months. However, I attempted to make some updates to a service yesterday, and suddenly the scripts I have been using since the beginning are no longer functioning.
gcloud build submit
I've been using the following command to build my node/npm project via the remote docker container:
gcloud builds submit --tag gcr.io/PROJECT_ID/generator
I have a dockerfile and .dockerignore in the same directory as my package.json from where I run this script. However, yesterday I suddenly started getting an error which read that a dockerfile is required when using the --tag parameter and the image would not build.
Tentative Solution
After some research, I tried moving my build cnfig into a gcloudbuild-staging.json, which looks like this:
{
"steps": [
{
"name": "gcr.io/cloud-builders/docker",
"args": [
"build",
"-t",
"gcr.io/PROJECT_ID/generator",
"."
]
}
]
}
And I've chnaged my build script to:
gcloud builds submit --config=./gcloudbuild-staging.json
After doing this, the container will build - or as far as I can tell. The console output looks like this:
------------------------------------------------- REMOTE BUILD OUTPUT --------------------------------------------------
starting build "8ca1af4c-d337-4349-959f-0000577e4528"
FETCHSOURCE
Fetching storage object: gs://PROJECT_ID/source/1650660913.623365-8a689bcf007749b7befa6e21ab9086dd.tgz#1650660991205773
Copying gs://PROJECT_ID/source/1650660913.623365-8a689bcf007749b7befa6e21ab9086dd.tgz#1650660991205773...
/ [0 files][ 0.0 B/ 22.2 MiB]
/ [1 files][ 22.2 MiB/ 22.2 MiB]
-
Operation completed over 1 objects/22.2 MiB.
BUILD
Already have image (with digest): gcr.io/cloud-builders/docker
Sending build context to Docker daemon 785.4kB
Step 1/6 : FROM node:14-slim
14-slim: Pulling from library/node
8bd3f5a20b90: Pulling fs layer
3a665e454db5: Pulling fs layer
11fcaa1377c4: Pulling fs layer
bf0a7233d366: Pulling fs layer
0d4d73621610: Pulling fs layer
bf0a7233d366: Waiting
0d4d73621610: Waiting
3a665e454db5: Verifying Checksum
3a665e454db5: Download complete
bf0a7233d366: Verifying Checksum
bf0a7233d366: Download complete
8bd3f5a20b90: Verifying Checksum
8bd3f5a20b90: Download complete
0d4d73621610: Verifying Checksum
0d4d73621610: Download complete
11fcaa1377c4: Verifying Checksum
11fcaa1377c4: Download complete
8bd3f5a20b90: Pull complete
3a665e454db5: Pull complete
11fcaa1377c4: Pull complete
bf0a7233d366: Pull complete
0d4d73621610: Pull complete
Digest: sha256:9ea3dfdff723469a060d1fa80577a090e14ed28157334d649518ef7ef8ba5b9b
Status: Downloaded newer image for node:14-slim
---> 913d072dc4d9
Step 2/6 : WORKDIR /usr/src/app
---> Running in 96bc104b9501
Removing intermediate container 96bc104b9501
---> 3b1b05ea0470
Step 3/6 : COPY package*.json ./
---> a6eca4a75ddd
Step 4/6 : RUN npm ci --only=production
---> Running in 7e870db13a9b
> protobufjs#6.11.2 postinstall /usr/src/app/node_modules/protobufjs
> node scripts/postinstall
added 237 packages in 7.889s
Removing intermediate container 7e870db13a9b
---> 6a86cc961a09
Step 5/6 : COPY . ./
---> 9e1f0f7a69a9
Step 6/6 : CMD [ "node", "index.js" ]
---> Running in d1b4d054a974
Removing intermediate container d1b4d054a974
---> 672075ef5897
Successfully built 672075ef5897
Successfully tagged gcr.io/PROJECT_ID/generator:latest
PUSH
DONE
------------------------------------------------------------------------------------------------------------------------
ID CREATE_TIME DURATION SOURCE IMAGES STATUS
8ca1af4c-d337-4349-959f-0000577e4528 2022-04-22T20:56:31+00:00 31S gs://PROJECT_ID/source/1650660913.623365-8a689bcf007749b7befa6e21ab9086dd.tgz - SUCCESS
There are no errors in the online logs.
gcloud run deploy
Here is the code I use to deploy the container:
gcloud run deploy generator --image gcr.io/PROJECT_ID/generator --region=us-central1 --set-env-vars ENVIRONMENT=DEV
The console output for this is:
Deploying container to Cloud Run service [generator] in project [PROJECT_ID] region [us-central1]
✓ Deploying... Done.
✓ Creating Revision...
✓ Routing traffic...
Done.
Service [generator] revision [generator-00082-kax] has been deployed and is serving 100 percent of traffic.
Service URL: https://generator-SERVICE_ID-uc.a.run.app
No errors in the run console, either. It shows the deployment as if everything is fine.
The Problem
Nothing is changing. Locally, running this service with the front-end app which accesses it produces successful results. However, my staging version of the app hosted on Firebase is still acting as if the old version of the code is active.
What I've Tried
I've made sure I'm testing and deploying on the same git branch
I've done multiple builds and deployments in case there was some kind of fluke.
I've tried using the gcloud command to update a service's traffic to its latest revision
I've made sure my client app is using the correct service URL. It doesn't appear to have changed but I copy/pasted it anyway just in case
My last successful deployment was on March 19, 2022. Since them, the only thing I've done is update all my WSL linux apps - which would include gcloud. I couldn't tell what version I ws on before, but I'm now on 38.0.0 of the Google Cloud CLI.
I've tried searching for my issue but nothing relevant is coming up. I'm totally stumped as to why all of this has stopped working and I'm receiving no errors whatsoever. Any suggestions or further info I can provide?
gcloud builds submit should (!?) continue to work with --tag as long as there is a Dockerfile in the folder from which you're running the command or you explicitly specify a source folder.
I'm not disputing that you received an error but it would be helpful to see the command you used and the error that resulted. You shouldn't have needed to switch to a build config file. Although that isn't the problem.
Using latest as a tag value is challenging. The term suggests that the latest version of a container image will be used but this is often not what happens. It is particularly challenging when a service like Cloud Run is running an image tagged latest and a developer asks the service to run -- what the developer knows (!) is a different image -- but also tagged latest.
As far as most services are concerned, same tag means same image and so it's possible (!) either that Cloud Run is not finding a different image or you're not providing it with a different image. I'm unclear which alternative is occurring but I'm confident that your use of latest is causing some of your issues.
So.... for starters, please consider using a system in which every time you create a new container, you tag it with a unique identifier. A common way to do this is to use a commit hash (as these change with every commit). Alternatively you can use the container's digest (instead of a tag) to reference an image version. This requires image references of the form {IMG}#sha256:{HASH}.
Lastly, gcloud run now (has always?) supported deployment from source (folder) to running service (it does the Cloud Build process for you and deploys the result to Cloud Run. It may be worth using this flow to reduce your steps and thereby the possibility of error.
See: Deploying from source code
My docker develop site and private registry located on site A. The production site B is connecting by 1 MB bandwidth network.
I have build a large image about 14GB and put it on private docker registry:2.
Because transfer 14GB by 'docker pull' always hang for small network bandwidth. I use docker save then compressed as xz file and transfer to production site. The tar.xz file is about 800MB and suitable for first setup on production site.
Now, I just append one line RUN script on Dockerfile to change the image and push to registry in 1 second. I wish it could pull the samll change on production site for only a few seconds. But, The docker pull command always trying to pull an old large layer about 1.4 GB.
I'm wonder why it is not cached by the first time 'docker load'? Is there any method to improve pull speed.
$ docker pull 192.168.1.2:5000/large
Using default tag: latest
latest: Pulling from large
cbddbc0189a0: Already exists
d79d36371fdd: Already exists
22a28ad6d786: Already exists
8d977d3c35c4: Already exists
82bf6088772f: Already exists
c3721275c963: Already exists
e6094577f1ab: Already exists
fae943dde872: Already exists
a750c973438c: Already exists
eaa46ee01cde: Already exists
c6a587da73bc: Downloading [> ] 1.596MB/1.393GB
1a237f393bcc: Download complete
2f9ea1299990: Download complete
084da7095dce: Download complete
56dc8872f673: Download complete
b564807882b8: Download complete
c6763845a161: Download complete
ab30373e73e1: Download complete
d215b08dc34e: Download complete
b4969cff1817: Download complete
56d6d67ceed8: Downloading [===> ] 463.2kB/7.016MB
eec215ecbaa9: Download complete
5bd3119d46dd: Download complete
dde99cd229a7: Download complete
a9625b7300e1: Download complete
b9bb74922ed7: Download complete
23d0d72df613: Download complete
85bf8e03320b: Download complete
00120183b69f: Download complete
When I download a Docker image, it downloads dependencies, but only displays their hashes. Why does it not display what it is downloading?
For example:
➜ ~ docker run ubuntu:16.04
Unable to find image 'ubuntu:16.04' locally
16.04: Pulling from library/ubuntu
b3e1c725a85f: Downloading 40.63 MB/50.22 MB
4daad8bdde31: Download complete
63fe8c0068a8: Download complete
4a70713c436f: Download complete
bd842a2105a8: Download complete
What's the point in only telling me that it's downloading b3e1c725a85f, etc.?
An image is created on layers of filesystems represented by hashes. After it's creation, the base image tag may point to a completely different set of hashes without affecting any images built off of it. And these layers are based on things like run commands, the tag to call it something like ubuntu:16.04 is only added after the image is made.
So the best that could be done is to say 4a70713c436f is based on adding some directory based on a hash of an input folder itself, or a multi-line run command, neither of which makes for a decent UI. The result may have no tagged name, or it could have multiple tagged names. So the simplest solution is to output what's universal and unchanging for all scenarios, an unchanging hash.
To rephrase that pictorially:
b3e1c725a85f: could be ubuntu:16.04, ubuntu:16, ubuntu:latest, some.other.registry:5000/ubuntu-mirror:16.04
4daad8bdde31: could be completely untagged, just a run command
63fe8c0068a8: could be completely untagged, just a copy file
4a70713c436f: could point to a tagged base image where that tag has since changed
bd842a2105a8: could be created with a docker commit command (eek)
I'd like to pull the images of CentOS, Tomcat, ... using their sha256 code, like in
docker pull myimage#sha256:0ecb2ad60
But I can't find the sha256-code to use anywhere.
I checked the DockerHub repository for any hint of the sha256-code, but couldn't find any. I downloaded the images by their tag
docker pull tomcat:7-jre8
and checked the image with docker inspect to see if there's a sha256 code in the metadata, but there is none (adding the sha256 code of the image would probably change the sha256 code).
Do I have to compute the sha256 code of an image myself and use that?
Latest answer
Edit suggested by OhJeez in the comments.
docker inspect --format='{{index .RepoDigests 0}}' $IMAGE
Original answer
I believe you can also get this using
docker inspect --format='{{.RepoDigests}}' $IMAGE
Works only in Docker 1.9 and if the image was originally pulled by the digest. Details are on the docker issue tracker.
You can get it by docker images --digests
REPOSITORY TAG DIGEST IMAGE ID CREATED SIZE
docker/ucp-agent 2.1.0 sha256:a428de44a9059f31a59237a5881c2d2cffa93757d99026156e4ea544577ab7f3 583407a61900 3 weeks ago 22.3 MB
Simplest and most concise way is:
docker images --no-trunc --quiet $IMAGE
This returns only the sha256:... string and nothing else.
e.g.:
$ docker images --no-trunc --quiet debian:stretch-slim
sha256:220611111e8c9bbe242e9dc1367c0fa89eef83f26203ee3f7c3764046e02b248
Edit:
NOTE: this only works for images that are local. You can docker pull $IMAGE first, if required.
Just saw it:
When I pull an image, the sha256 code is diplayed at the bottom of the output (Digest: sha....):
docker pull tomcat:7-jre8
7-jre8: Pulling from library/tomcat
902b87aaaec9: Already exists
9a61b6b1315e: Already exists
...
4dcef5c50d60: Already exists
Digest: sha256:c34ce3c1fcc0c7431e1392cc3abd0dfe2192ffea1898d5250f199d3ac8d8720f
Status: Image is up to date for tomcat:7-jre8
This sha code
sha256:c34ce3c1fcc0c7431e1392cc3abd0dfe2192ffea1898d5250f199d3ac8d8720f
can be used to pull the image afterwards with
docker pull tomcat#sha256:c34ce3c1fcc0c7431e1392cc3abd0dfe2192ffea1898d5250f199d3ac8d8720f
This way you can be sure that the image is not changed and can be safely used for production.
I found the above methods to not work in some cases. They either:
don't deal well with multiple images with the same hash (in the case of .RepoDigests suggestion - when you want to use a specific registry path)
don't work well when pushing the image to registries
(in the case of .Id where it's a local hash, not the hash in the
registry).
The below method is delicate, but works for extracting the specific full 'name' and hash for a specific pushed container.
Here's the scenario - An image is uploaded separately to 2 different projects in the same repo, so querying RepoDigests returns 2 results.
$ docker inspect --format='{{.RepoDigests}}' gcr.io/alpha/homeapp:latest
[gcr.io/alpha/homeapp#sha256:ce7395d681afeb6afd68e73a8044e4a965ede52cd0799de7f97198cca6ece7ed gcr.io/beta/homeapp#sha256:ce7395d681afeb6afd68e73a8044e4a965ede52cd0799de7f97198cca6ece7ed]
I want to use the alpha result, but I can't predict which index it will be. So I need to manipulate the text output to remove the brackets and get each entry on a separate line. From there I can easily grep the result.
$ docker inspect --format='{{.RepoDigests}}' gcr.io/alpha/homeapp:latest | sed 's:^.\(.*\).$:\1:' | tr " " "\n" | grep alpha
gcr.io/alpha/homeapp#sha256:ce7395d681afeb6afd68e73a8044e4a965ede52cd0799de7f97198cca6ece7ed
In addition to the existing answers, you can use the --digests option while doing docker images to get a list of digests for all the images you have.
docker images --digests
You can add a grep to drill down further
docker images --digests | grep tomcat
You can find it at the time of pulling the image from the respective repository. Below command mentions Digest: sha256 at the time of pulling the docker image.
09:33 AM##~::>docker --version
Docker version 19.03.4, build 9013bf5
Digest: sha256:6e9f67fa63b0323e9a1e587fd71c561ba48a034504fb804fd26fd8800039835d
09:28 AM##~::>docker pull ubuntu
Using default tag: latest
latest: Pulling from library/ubuntu
7ddbc47eeb70: Pull complete
c1bbdc448b72: Pull complete
8c3b70e39044: Pull complete
45d437916d57: Pull complete
**Digest: sha256:6e9f67fa63b0323e9a1e587fd71c561ba48a034504fb804fd26fd8800039835d**
Status: Downloaded newer image for ubuntu:latest
docker.io/library/ubuntu:latest
Once, the image is downloaded, we can do the following
"ubuntu#sha256:6e9f67fa63b0323e9a1e587fd71c561ba48a034504fb804fd26fd8800039835d"
09:36 AM##~::>docker inspect ubuntu | grep -i sha256
"Id": "sha256:775349758637aff77bf85e2ff0597e86e3e859183ef0baba8b3e8fc8d3cba51c",
**"ubuntu#sha256:6e9f67fa63b0323e9a1e587fd71c561ba48a034504fb804fd26fd8800039835d"**
"Image": "sha256:f0caea6f785de71fe8c8b1b276a7094151df6058aa3f22d2902fe6b51f1a7a8f",
"Image": "sha256:f0caea6f785de71fe8c8b1b276a7094151df6058aa3f22d2902fe6b51f1a7a8f",
"sha256:cc967c529ced563b7746b663d98248bc571afdb3c012019d7f54d6c092793b8b",
"sha256:2c6ac8e5063e35e91ab79dfb7330c6154b82f3a7e4724fb1b4475c0a95dfdd33",
"sha256:6c01b5a53aac53c66f02ea711295c7586061cbe083b110d54dafbeb6cf7636bf",
"sha256:e0b3afb09dc386786d49d6443bdfb20bc74d77dcf68e152db7e5bb36b1cca638"
This should have been the Id field, that you could see in the old deprecated Docker Hub API
GET /v1/repositories/foo/bar/images HTTP/1.1
Host: index.docker.io
Accept: application/json
Parameters:
namespace – the namespace for the repo
repo_name – the name for the repo
Example Response:
HTTP/1.1 200
Vary: Accept
Content-Type: application/json
[{"id": "9e89cc6f0bc3c38722009fe6857087b486531f9a779a0c17e3ed29dae8f12c4f",
"checksum": "b486531f9a779a0c17e3ed29dae8f12c4f9e89cc6f0bc3c38722009fe6857087"},
{"id": "ertwetewtwe38722009fe6857087b486531f9a779a0c1dfddgfgsdgdsgds",
"checksum": "34t23f23fc17e3ed29dae8f12c4f9e89cc6f0bsdfgfsdgdsgdsgerwgew"}]
BUT: this is not how it is working now with the new docker distribution.
See issue 628: "Get image ID with tag name"
The /v1/ registry response /repositories/<repo>/tags used to list the image ID along with the tag handle.
/v2/ only seems to give the handle.
It would be useful to get the ID to compare to the ID found locally. The only place I can find the ID is in the v1Compat section of the manifest (which is overkill for the info I want)
The current (mid 2015) answer is:
This property of the V1 API was very computationally expensive for the way images are stored on the backend. Only the tag names are enumerated to avoid a secondary lookup.
In addition, the V2 API does not deal in Image IDs. Rather, it uses digests to identify layers, which can be calculated as property of the layer and are independently verifiable.
As mentioned by #zelphir, using digests is not a good way since it doesn't exist for a local-only image. I assume the image ID sha is the most accurate and consistent across tags/pull/push etc.
docker inspect --format='{{index .Id}}' $IMAGE
Does the trick.
Just issue docker pull tomcat:7-jre8 again and you will get what you want.