Download manifest of private repository from Docker Registry API - docker

Normally, e.g., for the alpine image, we obtain an auth token via:
curl -i "https://auth.docker.io/token?service=registry.docker.io&scope=repository:library/alpine:pull"
Then we can use it to obtain the manifest from the registry:
curl -i -H "Authorization: Bearer $TOKEN" -H "Accept: application/vnd.docker.distribution.manifest.list.v2+json" https://registry-1.docker.io/v2/library/alpine/manifests/latest
When we replace library/alpine with a private repository of ours (ourcompany/ourrepo) obtaining a token still works, however, downloading the manifest results in:
HTTP/1.1 401 Unauthorized
Content-Type: application/json
Docker-Distribution-Api-Version: registry/2.0
Www-Authenticate: Bearer realm="https://auth.docker.io/token",service="registry.docker.io",scope="repository:ourcompany/ourrepo:pull",error="insufficient_scope"
Date: Tue, 26 May 2020 10:32:56 GMT
Content-Length: 168
Strict-Transport-Security: max-age=31536000
{"errors":[{"code":"UNAUTHORIZED","message":"authentication required","detail":[{"Type":"repository","Class":"","Name":"ourcompany/ourrepo","Action":"pull"}]}]}
How to circumvent this 401 error?
Do we need to obtain additional tokens? Send authentication credentials in addition? Do something completely differently?

You need to authenticate the call to /token using Basic Authentication
https://docs.docker.com/registry/spec/auth/jwt/#getting-a-bearer-token
Then from that you receive a Bearer token which you use as you did with a public repository. The /token endpoint only supports basic authentication.
curl -i -H "Authorization: Basic $BASIC_AUTH" "https://auth.docker.io/token?service=registry.docker.io&scope=repository:myprivaterepo/myprivateimage:pull"
curl -i -H "Authorization: Bearer $TOKEN" -H "Accept: application/vnd.docker.distribution.manifest.list.v2+json" https://registry-1.docker.io/v2/myprivaterepo/myprivateimage/manifests/latest

Related

How to start an interactive shell with docker container using restful API?

I'm trying to learn about the Docker Engine restful API, v1.41, but that's a problem. I managed to use cURL successfully to send all HTTP requests concerning containers, images, volumes and network, but it's impossible to me to use them for exec instances.
I read in the official documentation that when I use the docker exec command, he behaves the same way as calling POST /containers/:id/exec and then POST /exec/:id/start.
Now, in order to execute docker exec -it my_container bash that's what I did and got:
[claudio#gulliver ~]$ curl -X POST --unix-socket /run/docker.sock\
http/containers/18d7dee7470d/exec\
-H "Content-Type: application/json"\
-d '{
"AttachStdin":true,
"AttachStdout": true,
"AttachStderr": true,
"Tty": true,
"Cmd":["bash"]
}'
{"Id": "853938b2621606f85c04409ec7e345b884d46b95985a4fca0e8ddf28e20e1f79"}
[claudio#gulliver ~]$ curl -X POST --unix-socket /run/docker.sock\
http/exec/853938b2621606f85c04409ec7e345b884d46b95985a4fca0e8ddf28e20e1f79/start\
-H "Content-Type: application/json"\
-d '{
"Detach":false,
"Tty":true
}' --verbose
Note: Unnecessary use of -X or --request, POST is already inferred.
* Trying /run/docker.sock:0...
* Connected to http () port 80 (#0)
> POST /exec/853938b2621606f85c04409ec7e345b884d46b95985a4fca0e8ddf28e20e1f79/start HTTP/1.1
> Host: http
> User-Agent: curl/7.85.0
> Accept: */*
> Content-Type: application/json
> Content-Length: 28
>
* Mark bundle as not supporting multiuse
< HTTP/1.1 200 OK
< Content-Type: application/vnd.docker.raw-stream
< Api-Version: 1.41
< Docker-Experimental: false
< Ostype: linux
< Server: Docker/20.10.21 (linux)
* no chunk, no close, no size. Assume close to signal end
<
And from now I cannot do anything, just interrupt the process.
Is there a way I don't know to interact with this?

Docker API push to private registry error

I can't manage to push an image to a private registry using the docker API. I have read everything I found everywhere and tried everything with no luck...
I tried :
curl -X POST -H "X-Registry-Auth:XXXXXXXXXXXXXXX" http://dockerapiurl:2375/images/registryurl/python/push?tag=6
OR
curl -X POST -H 'X-Registry-Auth:{"username": "xxxxxx","password": "xxxxx", "serveraddress": "xxxx.url.net", "auth": ""}' http://dockerapiurl:2375/images/registryurl/python/push?tag=6
I always get the same error :
{"errorDetail":{"message":"errors:\ndenied: requested access to the resource is denied\nunauthorized: authentication required\n"},"error":"errors:\ndenied: requested access to the resource is denied\nunauthorized: authentication required\n"}
If I use docker push in CLI mode everything works, what am I doing wrong?
Thanks!!
it needs to be encoded in base 64, try this
XRA=`echo "{ \"username\": \"xxxxxx\", \"password\": \"xxxxxx\", \"email\": \"youmail#example.org\", \"serveraddress\": \"xxxxxx\" }" | base64 --wrap=0`
curl -X POST -d "" -H "X-Registry-Auth: $XRA" http://dockerapiurl:2375/images/registryurl/python/push?tag=6
end result should look like this
curl -X POST -d "" -H "X-Registry-Auth: eyAidXNlcm5hbWUiOiAieHh4eHh4IiwgInBhc3N3b3JkIjogInh4eHh4eCIsICJlbWFpbCI6ICJ5b3VtYWlsQGV4YW1wbGUub3JnIiB9Cg==" http://dockerapiurl:2375/images/registryurl/python/push?tag=6

not able to create Kafka topic with Kafka Rest Proxy - HTTP 415 Unsupported Media Type javax.ws.rs.NotSupportedException

I am following this documentation. I am able to get cluster's information like this:
curl -sk -X GET "https://xx.xx.xx.xx:8443/v3/clusters/"
the previous request works fine. However, when I try to create a topic I get HTTP 415 Unsupported Media Type javax.ws.rs.NotSupportedException error
command:
curl -sk -X POST \
-H "Content-Type: application/json" \
-d "{\"topic_name\":\"test1\",\"partitions_count\":6,\"replication_factor\":3,\"configs\":[]}" \
"https://xx.xx.xx.xx:8443/v3/clusters/xxxxxxx/topics"
does anyone have an idea on how to solve this issue?
I had the same issue, this worked for me.
curl -X POST -H "Content-Type: application/vnd.kafka.json.v2+json"
--data '{"records":[{"value":{"foo":"bar"}}]}' "https://localhost:8443/topics/test"
The example from this blog post worked for me:
POST http://localhost:8443/v3/clusters/<CLUSTER_ID>/topics
Content-Type: application/vnd.api+json
{
"data": {
"attributes": {
"topic_name": "topic",
"partitions_count": 1,
"replication_factor": 2,
"configs": []
}
}
}
So changing the Content-Type to application/vnd.api+json and using a different request body

KeyCloak bearer-only client shouldn't be able to invoke a secured endpoint when its credentials are wrong, but he can, why?

I have a Spring Boot application with this configuration:
server:
port: 9292
keycloak:
auth-server-url: http://localhost:8180/auth
realm: SampleRealm
resource: non-existing
public-client: false
principal-attribute: preferred_username
credentials:
secret: wrong-secret
bearer-only: true
I get an access token using another valid client (cli1, secret1):
curl -X POST \
-H "Authorization: Basic c2ItYXBwOmEyY2ViZmI2LTBjMzgtNDNiNS1hMDAwLThhYmUzYjU5YjJiMQ==" \
-H "Content-Type: application/x-www-form-urlencoded" \
-d 'username=someuser&password=somepassword&grant_type=password' \
"http://localhost:8180/auth/realms/SampleRealm/protocol/openid-connect/token"
Now I use that bearer token to invoke my Spring Boot Service:
curl -X GET \
http://localhost:9292/me \
-H 'Authorization: Bearer eyJhbGciOiJSUzI1NiIsInR5cCIgOiAiSldUIiwia2lkIiA6ICJPVE5xWF9jYWRXbEc1dGZYRmJVdEJ2V25hb2NTTGhuSm9LWndpOGxkYjZZIn0.eyJqdGkiOiI5ZTdlMjRmZC1lZDZmLTQzZTItYTFjZC1iMjlkMWRkN2I5ZWUiLCJleHAiOjE1MTk2NjQwODMsIm5iZiI6MCwiaWF0IjoxNTE5NjYzNzgzLCJpc3MiOiJodHRwOi8vbG9jYWxob3N0OjgxODAvYXV0aC9yZWFsbXMvU2FtcGxlUmVhbG0iLCJhdWQiOiJzYi1hcHAiLCJzdWIiOiIyNTFlZjNhNS1iNTRkLTQ4MmMtYTAzZS0wN2MzN2M0OGE5ZWIiLCJ0eXAiOiJCZWFyZXIiLCJhenAiOiJzYi1hcHAiLCJhdXRoX3RpbWUiOjAsInNlc3Npb25fc3RhdGUiOiJlNjdjMDBiYy0zODUxLTQ4ZjYtYTIxZi1hNDVhOGI0NGQyOGMiLCJhY3IiOiIxIiwiYWxsb3dlZC1vcmlnaW5zIjpbXSwicmVhbG1fYWNjZXNzIjp7InJvbGVzIjpbInVtYV9hdXRob3JpemF0aW9uIiwidXNlciJdfSwicmVzb3VyY2VfYWNjZXNzIjp7ImFjY291bnQiOnsicm9sZXMiOlsibWFuYWdlLWFjY291bnQiLCJtYW5hZ2UtYWNjb3VudC1saW5rcyIsInZpZXctcHJvZmlsZSJdfX0sIm5hbWUiOiJKb3NlIiwicHJlZmVycmVkX3VzZXJuYW1lIjoiamFpbmlnbyIsImdpdmVuX25hbWUiOiJKb3NlIiwiZW1haWwiOiJqYWluaWdvQHByb2ZpbGUuZXMifQ.bkMPSEvUHnVr5QoCsldKcFjKw3E_3Rhdu_SJ6LbgUehysAsLuG6pyjAQ4uqShTKphuXjOUf3E1eFMlttKSxZstCqP7iRU-OyHueGZ-_zGNx1ycvDBWSxCSmQufu9cx_dmnYW4NR9u5sSsZ052eDX0T0VgCvxeTtLJCsoH741SmJIVUvzrkPagKF_M_INVBQ3qaOds74o088qJy4GVJ8eZGqgsW9YOW6nNLV6kERwLAD9WZJoEARCdTBuGARTVJZuJ0lYVI0-jI0wN88T1G3vX3DZS0HIAROmgIait89PZ5wyfOu9u6ohTyFsi3uHV6uSJcN7x7t51snnBpr9KSSMMQ' \
-H 'Cache-Control: no-cache'
The Spring Boot App is correctly invoking the secured endpoint but it shouldn't be allowed to because the resource (non-existing) and secret (wrong-secret) don't actually exist, they haven't even been configured in KeyCloak!!! Why is this working? Shouldn't the client have its client-id client-secret validated?
o.k.a.BearerTokenRequestAuthenticator : Verifying access_token
o.k.a.BearerTokenRequestAuthenticator : access_token: xxxxxxxxxx.signature
o.k.a.rotation.JWKPublicKeyLocator : Going to send request to retrieve new set of realm public keys for client non-existing
o.k.a.rotation.JWKPublicKeyLocator : Realm public keys successfully retrieved for client non-existing. New kids: [OTNqX_cadWlG5tfXFbUtBvWnaocSLhnJoKZwi8ldb6Y]
o.k.a.BearerTokenRequestAuthenticator : successful authorized
Realm public keys successfully retrieved for client non-existing What??? non-existing client doesn't exist!!

cURL request with custom X-Auth-Token header Redirects to login page

I am using Alvaro's greach2014 repo to experiment the behavior of Spring Security Rest plugin. I can login to the application and generate authentication token. However, when I try to send a get request, I am being redirected to login page. Here is my cURL response
curl -X GET -i -H "X-Auth-Token: Bearer 6mp70e1h702ig5lp5l4j2dlbdbh5aiip" -H "Accept: application/json" http://localhost:8080/restful-grails-springsecurity-greach2014/categories
HTTP/1.1 302 Found
Server: Apache-Coyote/1.1
Set-Cookie: JSESSIONID=7D36AB88B14828CDB7085BBB8722A35F; Path=/restful-grails-springsecurity-greach2014/; HttpOnly
Location: http://localhost:8080/restful-grails-springsecurity-greach2014/login/auth
Content-Length: 0
Date: Tue, 07 Jul 2015 05:59:16 GMT
I have also tried with Bearer flag too but the result is same
curl -X GET -i -H "X-Auth-Token: Bearer 6mp70e1h702ig5lp5l4j2dlbdbh5aiip" -H "Accept: application/json" http://localhost:8080/restful-grails-springsecurity-greach2014/categories
Similarly, I have tried Authorization Bearer header too but it resulted in timeout exception:
```
curl -X GET -i -H "Authorization: Bearer 6mp70e1h702ig5lp5l4j2dlbdbh5aiip" -H "Accept: application/json" http://localhost:8080/restful-grails-springsecurity-greach2014/categories
HTTP/1.1 500 Internal Server Error
Server: Apache-Coyote/1.1
Content-Type: text/html;charset=utf-8
Content-Language: en
Content-Length: 4472
Date: Tue, 07 Jul 2015 05:59:46 GMT
Connection: close
Apache Tomcat/7.0.52 - Error report HTTP Status 500 - Timeout waiting for valuetype Exception reportmessage Timeout waiting for valuedescription The server encountered an internal error that prevented it from fulfilling this request.exception net.spy.memcached.OperationTimeoutException: Timeout waiting for value
net.spy.memcached.MemcachedClient.getAndTouch(MemcachedClient.java:1179)
net.spy.memcached.MemcachedClient.getAndTouch(MemcachedClient.java:1196)
grails.plugin.springsecurity.rest.token.storage.MemcachedTokenStorageService.findExistingUserDetails(MemcachedTokenStorageService.groovy:64)
grails.plugin.springsecurity.rest.token.storage.MemcachedTokenStorageService.loadUserByToken(MemcachedTokenStorageService.groovy:37)
grails.plugin.springsecurity.rest.RestAuthenticationProvider.authenticate(RestAuthenticationProvider.groovy:55)
grails.plugin.springsecurity.rest.RestTokenValidationFilter.doFilter(RestTokenValidationFilter.groovy:75)
grails.plugin.springsecurity.web.filter.GrailsAnonymousAuthenticationFilter.doFilter(GrailsAnonymousAuthenticationFilter.java:53)
grails.plugin.springsecurity.rest.RestAuthenticationFilter.doFilter(RestAuthenticationFilter.groovy:139)
grails.plugin.springsecurity.web.authentication.RequestHolderAuthenticationFilter.doFilter(RequestHolderAuthenticationFilter.java:53)
grails.plugin.springsecurity.web.authentication.logout.MutableLogoutFilter.doFilter(MutableLogoutFilter.java:62)
grails.plugin.springsecurity.rest.RestLogoutFilter.doFilter(RestLogoutFilter.groovy:80)
com.brandseye.cors.CorsFilter.doFilter(CorsFilter.java:82)
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
java.lang.Thread.run(Thread.java:745)
root cause net.spy.memcached.internal.CheckedOperationTimeoutException: Timed out waiting for operation - failing node: localhost/127.0.0.1:11211
net.spy.memcached.internal.OperationFuture.get(OperationFuture.java:167)
net.spy.memcached.MemcachedClient.getAndTouch(MemcachedClient.java:1168)
net.spy.memcached.MemcachedClient.getAndTouch(MemcachedClient.java:1196)
grails.plugin.springsecurity.rest.token.storage.MemcachedTokenStorageService.findExistingUserDetails(MemcachedTokenStorageService.groovy:64)
grails.plugin.springsecurity.rest.token.storage.MemcachedTokenStorageService.loadUserByToken(MemcachedTokenStorageService.groovy:37)
grails.plugin.springsecurity.rest.RestAuthenticationProvider.authenticate(RestAuthenticationProvider.groovy:55)
grails.plugin.springsecurity.rest.RestTokenValidationFilter.doFilter(RestTokenValidationFilter.groovy:75)
grails.plugin.springsecurity.web.filter.GrailsAnonymousAuthenticationFilter.doFilter(GrailsAnonymousAuthenticationFilter.java:53)
grails.plugin.springsecurity.rest.RestAuthenticationFilter.doFilter(RestAuthenticationFilter.groovy:139)
grails.plugin.springsecurity.web.authentication.RequestHolderAuthenticationFilter.doFilter(RequestHolderAuthenticationFilter.java:53)
grails.plugin.springsecurity.web.authentication.logout.MutableLogoutFilter.doFilter(MutableLogoutFilter.java:62)
grails.plugin.springsecurity.rest.RestLogoutFilter.doFilter(RestLogoutFilter.groovy:80)
com.brandseye.cors.CorsFilter.doFilter(CorsFilter.java:82)
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
java.lang.Thread.run(Thread.java:745)
note The full stack trace of the root cause is available in the Apache Tomcat/7.0.52 logs.Apache Tomcat/7.0.52
The correct way is to use
curl -X GET -i -H "Authorization: Bearer 6mp70e1h702ig5lp5l4j2dlbdbh5aiip" -H "Accept: application/json" http://localhost:8080/restful-grails-springsecurity-greach2014/categories
The exception was thrown because I was not running memcached locally. Fortunately, ubuntu had one in its rep and I simply had to use apt-get install memcached and sudo service memcached start. Everything worked fine.

Resources