Getting the GitLab runner registration token from the command line - docker

Context
To automate adding local GitLab runners to a local GitLab server instance running on docker, I wrote a boilerplate code that downloads and installs a Selenium browser that logs into GitLab and navigates to the GitLab runner section within the admin options, then clicks on "Register an instance runner" and searches for the "unhide" button, and then reads the runner registration token from the source code. This is a somewhat elaborate process, and currently not stable.
The GitLab server runs in docker (sudo docker ps -a):
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
51asd8ed1a44 gitlab/gitlab-ce:latest "/assets/wrapper" 49 minutes ago Up 49 minutes (healthy) 0.0.0.0:80->80/tcp, :::80->80/tcp, 0.0.0.0:443->443/tcp, :::443->443/tcp, 0.0.0.0:23->22/tcp, :::23->22/tcp gitlab
Question
How can one retrieve the GitLab runner registration token from the CLI from GitLab directly?

You can get a specific Project's runner registration token from the Get Single Project API operation. In the response, it will have an attribute called runners_token which matches the runner registration token for the queried project.
curl --location --request GET 'https://gitlab.example.com/api/v4/projects/1' --header 'Authorization: Bearer xxxTOKENxxx'
At the moment, there doesn't appear to be a way to retrieve the Instance-wide registration token you can see from the Admin area, however unless the token is frequently reset, you could store it securely somewhere and use it when you register new runners until it changes.
For example, you can access a project's CI Variables via the API, so you could store the Admin registration token in a protected project, then access that via the Get Single Variable API:
curl --header "PRIVATE-TOKEN: <your_access_token>" "https://gitlab.example.com/api/v4/projects/1/variables/TEST_VARIABLE_1"
This returns the following information about the variable:
{
"key": "TEST_VARIABLE_1",
"variable_type": "env_var",
"value": "TEST_1",
"protected": false,
"masked": true,
"environment_scope": "*"
}

Related

How do I obtain an HTTP access token from a bitbucket repository on bitbucket cloud

I need to create an HTTP access token for a repository which allows me to pull modules from it while building a nodeJS application in another repository.
This was done in the past by using a personal access token from one of the employees and I want to change that.
I refered to this article " https://confluence.atlassian.com/bitbucketserver/personal-access-tokens-939515499.html " in which the steps are stated as follows:
Create HTTP access tokens for projects or repositories
HTTP access tokens can be created for teams to grant permissions at the project or repository level rather than for specific users.
To create an HTTP access token for a project or repository (requires project or repository admin permissions):
From either the Project or Repository settings, select HTTP access tokens.
Select Create token.
Set the token name, permissions, and expiry.
The problem is in my repository settings, I can't find "HTTP access tokens".
I'm using Bitbucket cloud whereas the article refers to bitbucket Server, does that make a problem? If so, this option isn't available in bitbucket cloud?
Atlassian has vast documentation, but I have a problem with it and still don't understand how to get an access token to be able simply download archives from private repositories.
So here is my step by step tutorial
Insert your workspace name instead of {workspace_name} and go to the following link in order to create an OAuth consumer
https://bitbucket.org/{workspace_name}/workspace/settings/api
set callback URL to http://localhost:8976 (doesn't need to be a real server there)
select permissions: repository -> read
use consumer's Key as a {client_id} and open the following URL in the browser
https://bitbucket.org/site/oauth2/authorize?client_id={client_id}&response_type=code
after you press "Grant access" in the browser it will redirect you to
http://localhost:8976?code=<CODE>
Note: you can spin your local server to automate this step
use the code from the previous step and consumer's Key as a {client_id}, and consumer's Secret as {client_secret}:
curl -X POST -u "{client_id}:{client_secret}" \
https://bitbucket.org/site/oauth2/access_token \
-d grant_type=authorization_code \
-d code={code} \
you should receive similar json back
{
"access_token": <access_token>,
"scopes": "repository",
"token_type": "bearer",
"expires_in": 7200,
"state": "authorization_code",
"refresh_token": <refresh_token>
}
use the access token in the following manner
curl https://api.bitbucket.org/2.0/repositories/{workspace_name} \
--header "Authorization: Bearer {access_token}
Whilst your question is about Bitbucket Cloud, the article you linked is for Atlassian's self-hosted source control tool Bitbucket Server. They have different functionality for different use cases, which is why they don't look the same.
Depending on your use case you can use App passwords or OAuth instead.
Full disclosure: I work for Atlassian
Easiest way to do it is:
Create an OAuth consumer in your Bitbucket settings (also provide dummy redirect like localhost:3000, copy KEY and SECRET.
Use curl -X POST -u "KEY:SECRET" https://bitbucket.org/site/oauth2/access_token -d grant_type=client_credentials to get JSON data with access-token.

Cloud Scheduler has Permission Denied when attempting to run a Cloud Run job

I have created a simple Cloud Run job. I am able to trigger this code via a curl command:
curl -H "Authorization: Bearer $(gcloud auth print-identity-token)" https://sync-<magic>.a.run.app
(Obviously <magic> is actually something else)
Cloud Run is configured for Ingress to Allow All Traffic and with Authentication to be required.
I followed this documentation: https://cloud.google.com/run/docs/triggering/using-scheduler
And created a service account, granted it the Cloud Run Invoker Role and then setup an HTTP scheduled job to GET the same URL I tested with CURL. I have Add OIDC Token selected, and I provide the service account created above and the Audience which is the same URL I used with curl.
When I attempt to trigger this job (or when it triggers based of the native cron) it fails with:
{ "status": "PERMISSION_DENIED", "#type": "type.googleapis.com/google.cloud.scheduler.logging.AttemptFinished", "targetType": "HTTP", "jobName": "projects/<project>/locations/<region>/jobs/sync", "url": "https://sync-<magic>.a.run.app/" }
Again <project>, <region> and <magic> have real values.
I tried using service-YOUR_PROJECT_NUMBER#gcp-sa-cloudscheduler.iam.gserviceaccount.com with YOUR_PROJECT_NUMBER updated appropriately as the service account that runs the scheduled job. It has the same error.
Any advice on how to debug this would be greatly appreciated!
Here is what i did which solved the issue altogether and now I get the success flag when running a secure Cloud Run service via a Cloud Scheduler job -
Create your service on Cloud run - let's call it "hello" and make it secured by removing "allUsers" permission from the list of Permissions PRINCIPALS - you should get an error when going to the endpoint as such - Error: Forbidden
Your client does not have permission to get URL / from this server.
Create an IAM service account for cloud scheduler - let's call it "cloud-scheduler" you will get this: cloud-scheduler#project-ID.iam.gserviceaccount.com now comes the important part :
Give your SA the ability to run Scheduler Jobs by adding the -
Cloud Run Invoker & Cloud Scheduler Job Runner permissions
Create your Cloud scheduler job and add the new SA to it according to google procedure :
Auth header: Add OIDC token
Service account: cloud-scheduler#project-id.iam.gserviceaccount.com
Audience : https://Service.url.from.cloud.run.service/
Add to your cloud run service an additional principal that will let your SA access to cloud run invoker
Run your scheduler and voila - all green !
Enjoy
I have tried to create a new service account, gave it Cloud run invoker role. Disable the Cloud Scheduler API and re-enable it.
The only thing that work for me is changing Auth header from Add OIDC token to None.
For some reason Cloud Scheduler change None back to Add OIDC token and Trigger cloud run normally

how to use\connect to sonatype nexus docker registry v2 api in a web application?

I have a private sonatype nexus repository manager OSS 3.25.1-04 container running on a vm (with nginx routing from docker.io to repo manager url) that contains a few repositories, one of them is a docker registry.
I want to use the docker registry v2 api from a react app to get a listing for the docker images in the repository and maybe some more metrics about the repo and its contents.
I tried calling the api directly: https://nexus3:8083/v2/_catalog but got 401 UnAuthorized in the response when checking the devtools network tab
Then to login to the api I tried using https://auth.docker.io/token?service=registry.docker.io&scope=repository:samalba/my-app:pull,push when substituting samalba/my-app with my own registry and example docker image. I know this link is to get token for only this image couldn't find one for the entire api (it didn't work anyway)
Could use some help on how to connect to the api\get jwt token and using it or how to use the api with http instead
A few things may be going on. First, try just using basic authentication and seeing if that works. Additionally, you may need to set some additional headers to connect to nexus / sonatype. Here is an example with curl:
curl -H "Accept: application/vnd.docker.distribution.manifest.v2+json" -H "Content-Type: application/json" -H "User-Agent: docker/20.10.14" -u username:password -i https://nexus3:8083/v2/_catalog
Note the user agent field -- i've run into issues where the authentication layer is filtering out for the docker user agent.
If that still doesn't work, then the next thing you can look for is to see if the registry response with the header www-authenticate. This means you will need to first authenticate with that service to retrieve a Bearer token, and then you can pass that back to the registry using the Authorization header instead of basic auth.
Hope that helps.

How to invoke a Cloud Run app without having to add the Authorization Token

I have a cloud run app deployed that is for internal use only.
Therefore only users of our cluster should have access to it.
I added the permission for allAuthenticated members giving them the role Cloud Run Invoker.
The problem is that those users (including me) now have to add authorization bearer header everytime I want to access that app.
This is what Cloud Run suggests to do (somehow useless when u wanna simply visit a frontend app)
curl -H \
"Authorization: Bearer $(gcloud auth print-identity-token)" \
https://importer-controlroom-frontend-xl23p3zuiq-ew.a.run.app
I wonder why it is not possible to be realized as authorized member like the GCP figures out. I can access the cluster but have to add the authorization header to access the cloud run app as authorized member? I find this very inconvenient.
Is there any way to make it way more fun to access the deployed cloud run app?
PS: I do not want to place the app in our cluser - so only fully managed is an option here
You currently can't do that without the Authorization header on Cloud Run.
allAuthenticated subject means any Google user (or service account), so you need to add the identity-token to prove you're one.
If you want to make your application public, read this doc.
But this is a timely request! I am currently running an experiment that lets you make requests to http://hello and automatically get routed to the full domain + automatically get the Authorization header injected! (This is for communication between Cloud Run applications.)
GCP now offers a proxy tool for making this easier, although it's in beta as of writing this.
It's part of the gcloud suite, you can run:
gcloud beta run services proxy $servicename --project $project --region $region
It will launch a webserver on localhost:8080, that forwards all requests to the targeted service, injecting the user's GCP token into all requests.
Of course this can only be used locally.

401 error when trying GET request to Hawkbit Server with Gateway Security Token

Q1:
I'm running a Hawkbit server on localhost in a docker container and activated the option "Allow a gateway to authenticate and manage multiple targets through a gateway security token" in the settings of the web UI that I access via http://localhost:8080/.
Now I'm using Postman to send a GET request to http://localhost:8080/default/controller/v1/25 with the header
key: GatewayToken, value: <The gateway token shown in the Hawkbit web UI>
Using this header, I'm supposed to be able to authenticate my Postman client against the Hawkbit server (compare e.g. https://www.eclipse.org/hawkbit/concepts/authentication/), however I'm always getting a "401 Unauthorized" response.
Even if I enable "Allow targets to download artifacts without security credentials" which should enable any client to get a ressource even without authentification, I get a 401.
What am I doing wrong?
Q2:
The Hawkbit server is running in Docker started via "docker-compse up -d" as described here: https://www.eclipse.org/hawkbit/gettingstarted/
In order to solve the problem of Q1, I wanted to check the output of Hawkbit inside the container, but I'm not too familiar with docker and couldn't find out how. I was able to get inside the conainer using
docker exec -it docker_hawkbit_1 /bin/sh
which bring me into the container's file system at /opt/hawkbit. But that's not what I was looking for. How can I see the log/output of the Hawkbit/Spring Boot application running inside the container?
Q1:
The key of the request should not be GatewayToken, but Authorization. The header of the request will then look as follows:
key: Authorization, value: GatewayToken <token>
Q2:
Try the following command to see the logs:
docker logs -f docker_hawkbit_1

Resources