Docker Private Registry Image Upload - docker

Question

I'm trying to upload images to my private docker registry using the HTTP API, unfortunately without success.
Apparently I haven't understood the upload process yet and would like to ask if anyone can explain this in detail or push me in the right direction.
So far I have tried the following with curl. To experiment I only use the empty alpine image.
For that i download it to my workstation with docker pull alpine and then I create tar archive with docker save -o alpine.tar alpine
I then unpack this archive into an alpine directory. This is what the content looks like:
ls -R alpine
alpine:
39cb81dcd06e3d4e2b813f56b72da567696fa9a59b652bd477615b31af969239
e7d92cdc71feacf90708cb59182d0df1b911f8ae022d29e8e95d75ca6a99776a.json
manifest.json
alpine/39cb81dcd06e3d4e2b813f56b72da567696fa9a59b652bd477615b31af969239:
json
layer.tar
VERSION
According to the documentation I should first initiate the upload by sending a POST to the /v2//blobs/uploads/ URL, for this I execute the following command:
curl -X POST -L -D headers $DOCKER_HOST/v2/alpine/blobs/uploads
The answer is the following:
cat headers
HTTP/1.1 301 Moved Permanently
Docker distribution api version: registry/2.0
location: /v2/alpine/blobs/uploads/
Date: Tu, 21 Jan 2020 12:42:55 GMT
content length: 0
HTTP/1.1 202 Accepted
content length: 0
Docker distribution api version: registry/2.0
Docker upload guide: ed595b3c-1236-46c8-a759-14187fc60e7d
Location: http://<IPADDRESS>/v2/alpine/blobs/uploads/ed595b3c-1236-46c8-a759-14187fc60e7d?_state=GjzU_y-YDQherf4xXO57KyEonSSSwNEM8FiF8rmNfuN7Ik5hbWUiOiJhbHBpbmUiLCJVVUlEIjoiZWQ1OTViM2MtMTIzNi00NmM4LWE3NTktMTQxODdmYzYwZTdkIiwiT2Zmc2V0IjowLCJTdGFydGVkQXQiOiIyMDIwLTAxLTIxVDEyOjQyOjU1LjMxNDYzNTg0NVoifQ%3D%3D
Range: 0-0
X Content Type Options: nosniff
Date: Tu, 21 Jan 2020 12:42:55 GMT
now i would like to do a monolithic upload, as described in the docker-registry documentation.
For this I make the following curl request:
url=http://<IPADDRESS>/v2/alpine/blobs/uploads/ed595b3c-1236-46c8-a759-14187fc60e7d?_state=GjzU_y-YDQherf4xXO57KyEonSSSwNEM8FiF8rmNfuN7Ik5hbWUiOiJhbHBpbmUiLCJVVUlEIjoiZWQ1OTViM2MtMTIzNi00NmM4LWE3NTktMTQxODdmYzYwZTdkIiwiT2Zmc2V0IjowLCJTdGFydGVkQXQiOiIyMDIwLTAxLTIxVDEyOjQyOjU1LjMxNDYzNTg0NVoifQ%3D%3D?digest=sha256:39cb81dcd06e3d4e2b813f56b72da567696fa9a59b652bd477615b31af969239
layerpath=/home/user/alpine/39cb81dcd06e3d4e2b813f56b72da567696fa9a59b652bd477615b31af969239/layer.tar
curl -X PUT -H "Content-Type=application/octet-stream" --data-binary #"$layerpath" $url
For this I receive an answer that means nothing to me:
{"errors":[{"code": "BLOB_UPLOAD_INVALID", "message": "blob upload invalid", "detail":212}]}
I don't know what I'm doing wrong. For any help I am very grateful.

Answer 1

Okay, I figured it out.
For all the others who are also dealing with this question here is a little script which should clarify the handling of the HTTP-Api:
#!/bin/bash
# Global Variables
MANIFEST="./Manifest.json"
DOCKER_HOST=$1
REPOSITORY=$2
LAYERPATH=$3
CONFIGPATH=$4
SIZE=
DIGEST=
LOCATION=
CONFIGSIZE=
CONFIGDIGEST=
LAYERSIZE=
LAYERDIGEST=
# Functions
function initiateUpload(){
LOCATION=$(curl -X POST -siL -v -H "Connection: close" $DOCKER_HOST/v2/$REPOSITORY/blobs/uploads | grep Location | sed '2q;d' | cut -d: -f2- | tr -d ' ' | tr -d '\r')
}
function patchLayer(){
layersize=$(stat -c%s "$1")
LOCATION=$(curl -X PATCH -v -H "Content-Type: application/octet-stream" \
-H "Content-Length: $layersize" -H "Connection: close" --data-binary #"$1" \
$LOCATION 2>&1 | grep 'Location' | cut -d: -f2- | tr -d ' ' | tr -d '\r')
SIZE=$layersize
}
function putLayer(){
DIGEST=sha256:$(sha256sum $1 | cut -d ' ' -f1)
url="$LOCATION&digest=$DIGEST"
curl -X PUT -v -H "Content-Length: 0" -H "Connection: close" $url
}
function uploadManifest(){
((size=$(stat -c%s "$MANIFEST")-1))
curl -X PUT -vvv -H "Content-Type: application/vnd.docker.distribution.manifest.v2+json"\
-H "Content-Length: $size" -H "Connection: close" \
-d "$(cat "$MANIFEST")" $DOCKER_HOST/v2/$REPOSITORY/manifests/latest
}
# Check Parameters
if [ $# -lt 4 ]
then
echo "Error: No arguments supplied."
echo "Usage: upload.sh <DOCKER_HOST> <REPOSITORY> <LAYER> <CONFIG>"
exit 1
fi
#upload Layer
initiateUpload
patchLayer $LAYERPATH
LAYERSIZE=$SIZE
putLayer $LAYERPATH
LAYERDIGEST=$DIGEST
#upload Config
initiateUpload
patchLayer $CONFIGPATH
CONFIGSIZE=$SIZE
putLayer $CONFIGPATH
CONFIGDIGEST=$DIGEST
cat > $MANIFEST << EOF
{
"schemaVersion": 2,
"mediaType": "application/vnd.docker.distribution.manifest.v2+json",
"config": {
"mediaType": "application/vnd.docker.container.image.v1+json",
"size": $CONFIGSIZE,
"digest": "$CONFIGDIGEST"
},
"layers": [
{
"mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
"size":$LAYERSIZE,
"digest": "$LAYERDIGEST"
}
]
}
EOF
uploadManifest
It's not pretty, but it works for that.
If you need more information, I will gladly extend this answer