How to create queue in Rabbitmq - docker

I am creating a new image taking base as rabbitmq and trying to create queue,exchange which will be reflected on localhost url once the server is up. I am able to manually create queue within rabbitmq container. But I want to achieve this either through dockerfile or entrypoint.sh. I want the exchange, queue to be available as soon as rabbitmq server is up. Please suggest any way to achieve it. Any sample example will be helpful.

Rabbitmq has a Management HTTP API. You can use this api to interact with rabbitmq.
You can create an exchange by doing a PUT request to http://localhost:15672/api/exchanges/${vhost}/${name}. Similarly, you can create a queue by
doing a PUT to http://localhost:15672/api/queues/${vhost}/${name}.
You can call these using curl in the entrypoint script.

You can use HareDu 2 like so:
var result = _container.Resolve<IBrokerObjectFactory>()
.Object<Queue>()
.Create(x =>
{
x.Queue("fake_queue");
x.Configure(c =>
{
c.IsDurable();
c.AutoDeleteWhenNotInUse();
c.HasArguments(arg =>
{
arg.SetQueueExpiration(1000);
arg.SetPerQueuedMessageExpiration(2000);
});
});
x.Targeting(t =>
{
t.VirtualHost("HareDu");
t.Node("Node1");
});
});

Here is a practical example with curl and the REST HTTP API which was already mentioned.
First of all, the HTTP REST API is a separate plugin and if not installed, you have to install it with the command:
rabbitmq-plugins enable rabbitmq_management
if you want to install it in your Docker image, you can do:
RUN rabbitmq-plugins enable rabbitmq_management
Once you plugin is installed, you can just call the API with a curl command:
curl --location --request PUT 'http://localhost:15671/api/queues/%2F/TEST_QUEUE' \
--header 'Content-Type: application/json' \
--header 'Authorization: Basic Z3Vlc3Q6Z3Vlc3Q=' \
--data-raw '{
"auto_delete": false,
"durable": true,
"arguments": {}
}'
The Basic auth provided is guest/guest as the default user in the docker image.
The trick is for the default vhost, you have to put %2F in your url (equivalent to /) this can make you lose a lot of time if you try other variations.
Here are some references:
Management plugin documentation
HTTP API documentation
If you need a practical docker example, you can have a look to my solace integration project where I set up a rabbitMQ for testing purpose here.

Related

Configure IoTEdge module to receive messages port 53000

I'm loosely following along this article to develop and debug modules for IoTEdge
https://learn.microsoft.com/en-us/azure/iot-edge/how-to-visual-studio-develop-module?view=iotedge-2020-11
The article leverages the iotedgehubdev which is where, presumably, the configuration exists to expose port 53000.
My question is, without using the simulator or iotedgehubdev tool, how do I configure the port to allow messages to be sent using this type of syntax
curl --header "Content-Type: application/json" --request POST --data '{"inputName": "input1","data":"hello world"}' http://localhost:53000/api/v1/messages
// Register callback to be called when a message is received by the module
await ioTHubModuleClient.SetInputMessageHandlerAsync("input1", PipeMessage, ioTHubModuleClient);
static async Task<MessageResponse> PipeMessage(Message message, object userContext)
{
....
}
Target environment: Ubuntu, IoTEdge 1.1.4, published via IoTHub pulled from ACR
Development: Windows 11, Visual Studio 2022, debug via SSH to docker module on Ubuntu
Once the module is up and running, I want to send a post request to the module from the Ubuntu machine hosting the module. The module is being published from IoTHub
I've looked across many articles for clues on how port 53000 is setup and listening but haven't found anything that helps so far.
Appreciate the help.
Sending a message is now easy, once your code is running on Simulator, you can send messages by issuing a CURL request to the endpoint you received when starting the Simulator. Please follow below Reference in which we have detailed information about:
( curl --header "Content-Type: application/json" --request POST --data '{"inputName": "input1","data":"hello world"}' http://localhost:53000/api/v1/messages)
Even I looked across many articles for clues. How to Set up port 53000 without using the simulator or iotedgehubdev tool. if you want to work without using the simulator or iotedgehubdev.
you can reach out to Azure Support or Can raise a GitHub Issue.
You can refer this article( Azure IoT Edge Simulator — Easily run and test your IoT Edge application | by Xavier Geerinck | Medium ) by Xavier Geerinck
We have to build a custom API module which will listen to the port just what the iotedgedev utility is doing, in which ever language you are writing it in.
Create a Rest API.
Use Azure Devices Module Client module with IOTEdge enable in project file.
Create an output in your custom API module and send the message using module client.
Create the route config in you deployment file and provide the output of this module output to the input of another module in the routes section.
Edit: Do not forget to mention the createOptions Portbindings and the Exposed Ports section, like for e.g
"createOptions": {
"ExposedPorts": {
"9000/tcp": {}
},
"HostConfig": {
"PortBindings": {
"9000/tcp": [
{
"HostPort": "9000"
}
]
}
}
}

Create repo on Bitbucket programmatically

I used to do
curl -k -X POST --user john#outlook.com:doe13 "https://api.bitbucket.org/1.0/repositories" -d "name=logoApp"
and success.
now I got : error
{"type": "error", "error": {"message": "Resource removed", "detail": "This API is no longer supported.\n\nFor information about its removal, please refer to the deprecation notice at: https://developer.atlassian.com/cloud/bitbucket/deprecation-notice-v1-apis/"}}
Does anyone know a know way to do this ?
There's a difference between a success from curl (OK:200) and an error from the service you're trying to use. The error, however, mentions that you're trying to use the Cloud Rest API version 1, which is deprecated effective 30 June 2018.
Read this for more information.
I don't use Bitbucket Server (a local option), and I think that has more features for this sort of thing.
For the public Bitbucket, you can still do it but it isn't documented.
The v1.0 API has been removed, and the new v2.0 API doesn't document a POST to a /repositories. Instead, you have to hit an endpoint that includes the repo that doesn't yet exist: /repositories/workspace/repo_slug
The JSON payload needs to know the project for the repo: look in the slug for a project that already exists. Fill in the user/team and repo name in the URL. And, you can make an application password so you aren't using your account password. This app password can limit the scope of what that access can do.
% curl -X POST --user 'user:app_pass' \
-H "Content-type: application/json" \
-d '{"project":{"key":"PROJ"}}' \
"https://api.bitbucket.org/2.0/repositories/USER/REPO"

How to get Openshift session token using rest api calls

As part of an automated tests suite I have to use OpenShift's REST APIs to send commands and get OpenShift's status. To authenticate these API calls I need to embed an authorization token in every call.
Currently, I get this token by executing the following commands with ssh on the machine where OpenShift is installed:
oc login --username=<uname> --password=<password>
oc whoami --show-token
I would like to stop using the oc tool completely and get this token using HTTP calls to the APIs but am not really able to find a document that explains how to use it. If I use the option --loglevel=10 when calling oc commands I can see the HTTP calls made by oc when logging in but it is quite difficult for me to reverse-engineer the process from these logs.
Theoretically this is not something specific to OpenShift but rather to the OAuth protocol, I have found some documentation like the one posted here but I still find it difficult to implement without specific examples.
If that helps, I am developing this tool using ruby (not rails).
P.S. I know that normally for this type of job one should use Service Account Tokens but since this is a testing environment the OpenShift installation gets removed and reinstalled fairly often. This would force me to re-create the service account every time with the oc command line tool and again prevent me from automatizing the process.
I have found the answer in this GitHub issue.
Surprisingly, one curl command is enough to get the token:
curl -u joe:password -kv -H "X-CSRF-Token: xxx" 'https://master.cluster.local:8443/oauth/authorize?client_id=openshift-challenging-client&response_type=token'
The response is going to be an HTTP 302 trying to redirect to another URL. The redirection URL will contain the token, for example:
Location: https://master.cluster.local:8443/oauth/token/display#access_token=VO4dAgNGLnX5MGYu_wXau8au2Rw0QAqnwq8AtrLkMfU&expires_in=86400&token_type=bearer
You can use token or combination user/password.
To use username:password in header, you can use Authorizartion: Basic. The oc client commands are doing simple authentication with your user and password in header. Like this
curl -H "Authorization: Basic <SOMEHASH>"
where the hash is exactly base64 encoded username:password. (try it with echo -n "username:password" | base64).
To use token, you can obtain the token here with curl:
curl -H Authorization: Basic $(echo -n username:password | base64)" https://openshift.example.com:8443/oauth/authorize\?response_type\=token\&client_id\=openshift-challenging-client
But the token is replied in the ugly format format. You can try to grep it
... | grep -oP "access_token=\K[ˆ&]*"
You need to use the correct url for your oauth server. In my case, I use openshift 4.7 and this is the url:
https://oauth-openshift.apps.<clustername><domain>/oauth/authorize\?response_type\=token\&client_id\=openshift-challenging-client
oc get route oauth-openshift -n openshift-authentication -o json | jq .spec.host
In case you are using OpenShift CRC:
Then the URL is: https://oauth-openshift.apps-crc.testing/oauth/authorize
Command to get the Token:
curl -v --insecure --user developer:developer --header "X-CSRF-Token: xxx" --url "https://oauth-openshift.apps-crc.testing/oauth/authorize?response_type=token&client_id=openshift-challenging-client" 2>&1 | grep -oP "access_token=\K[^&]*"
Note:
2>&1 is required, because curl writes to standard error
--insecure: because I have not set up TLS certificate
Adjust the user and password developer as needed (crc developer/developer is standard user in crc, therefore good for testing.)
Token is per default 24h vaild
Export the Token to an environment Variable
export TOKEN=$(curl -v --insecure --user developer:developer --header "X-CSRF-Token: xxx" --url "https://oauth-openshift.apps-crc.testing/oauth/authorize?response_type=token&client_id=openshift-challenging-client" 2>&1 | grep -oP "access_token=\K[^&]*")
And Use the token then in, e.g., oc login:
oc login --token=$TOKEN --server=https://api.crc.testing:6443

How to list Docker images in a Codefresh private registry?

I'm currently using Codefresh's free private registry to store my images. I am using CircleCI for my Docker builds, so from there I use docker login so I can push and pull to the Codefresh registry. This works fine. However, Codefresh only lists images in its web app for which it did the building, so I cannot easily see a catalogue of them. I suspect this is by design, so that users stick with Codefresh for CI. However, if possible, I would like to list the images I have in the registry.
I assume that this registry is a basic v2 version of the standard Docker Registry. So, I have had a hunt around, and found this console utility to manage images. However, it does not seem to work for me. I don't know whether this is because registry management tools are still immature generally (web searches indicate they were added much later, and there are lot of folks for whom this simple task has become a substantial undertaking) or whether there is something unusual with Codefresh.
Here is what I am trying:
reg -d -r r.cfcr.io -u myusername -p cfaccesstoken
(The -d is for debug).
This results in:
2017/10/18 11:24:43 registry.ping url=https://r.cfcr.io/v2/
2017/10/18 11:24:44 registry.catalog url=https://r.cfcr.io/v2/_catalog
2017/10/18 11:25:53 registry.catalog url=https://r.cfcr.iohttps://r.cfcr.io/v2/_catalog?n=1000&last=davigsantos/davigsantos/codeflix
FATA[0075] Get https://r.cfcr.iohttps//r.cfcr.io/v2/_catalog?n=1000&last=davigsantos/davigsantos/codeflix: dial tcp: lookup r.cfcr.iohttps on 127.0.1.1:53: no such host
The penultimate line contains a container name I don't recognise - I hope it is a public one, and not one I should not be seeing!
The last line indicates some sort of fatal error, with all sorts of URL fragments mashed together in ways that definitely won't work.
I have also discovered that dropping the access token makes no difference; the output is exactly the same:
reg -d -r r.cfcr.io -u myusername
What else can I try? I am running Mint Linux and would be fine with swapping to another utility if necessary. Is there something wrong about the way I am issuing this command, or is Codefresh running a non-standard registry that is incompatible with standard API calls?
Update
It looks like Codefresh also has their own API, though it does not seem to be documented as far as I can tell. Running the get operation produces this error:
Failed to authenticate request because no token was provided
That's encouraging, so I will try to find out how to provide a token in a curl call; the Swagger web interface does not seem to permit it. However I am conscious that, if I can get the API working, it may not list my Docker images anyway, since they were not built by Codefresh.
Update 2
I have found some articles on the Codefresh blog that hint how to authenticate on the API, the format is a header thus:
--header "x-access-token: (token)"
However I get this error:
{"status":401,"code":"2401","name":"UNAUTHORIZED_ERROR","message":"Failed to authenticate request because token is not valid","context":{}}
The token I am using is the same one as I use for docker login, which works. I notice I was not specifying my username, so I am also adding this curl option:
-u (user):(token)
As you can see, I am close to trying random things now, as there does not seem to be official help online.
Update 3
Prompted by a comment below, it seems that Docker maintains an access token after login is used, in ~/.docker/config.json.
I therefore tried this:
reg -d -r r.cfcr.io -u myusername -p dockeraccesstoken
(Note how cfaccesstoken has been changed to dockeraccesstoken).
This returns much more quickly now (as opposed to appearing to hang), but returns nothing:
2017/10/24 10:56:16 registry.ping url=https://r.cfcr.io/v2/
2017/10/24 10:56:18 registry.catalog url=https://r.cfcr.io/v2/_catalog
Repositories for r.cfcr.io
You need to first generate a token on CodeFresh User Settings
Next I will show you how to login from terminal
curl -H 'Host: r.cfcr.io' -H 'User-Agent: ItsMe' -H 'Authorization: Basic <AUTH>' --compressed 'https://r.cfcr.io/v2/token?account=xxxx&scope=repository%3A<user>/<name>%3Apush%2Cpull&service=r.cfcr.io'
You can get the <AUTH> by running
echo <account>:<token> | base64
This will return you a big token
{"expires_in":43200,"issued_at":"2017-10-24T03:34:54.118390368-07:00","token":"APMm...P91"}%
Now you can use that token to make a docker API call
$ curl -IH 'Host: r.cfcr.io' -H 'User-Agent: ItsMe' -H 'Authorization: Bearer APMm+...aRL' -X HEAD 'https://r.cfcr.io/v2/<user>/<user>/testci/blobs/sha256:c7fefcc4c54c63f32b5422ede095793eb5ff0f45ad7a22861974ec9b61e7749b'
HTTP/1.1 200 OK
Docker-Distribution-API-Version: registry/2.0
Content-Length: 22488057
Date: Tue, 24 Oct 2017 10:42:23 GMT
Content-Type: text/html
Server: Docker Registry
X-XSS-Protection: 1; mode=block
X-Frame-Options: SAMEORIGIN
For registry search access below should have worked
curl -H 'Host: r.cfcr.io' -H 'User-Agent: ItsMe' -H 'Authorization: Basic ' --compressed 'https://r.cfcr.io/v2/token?account=xxxx&scope=registry%3Acatalog%3Asearch&service=r.cfcr.io'
But it doesn't and neither does registry:catalog:* for scopes. This is like hitting a target with blind folds and not even knowing in which direction we stand. You best bet is to get them to disclose some information to you
Edit-1: Getting the catalog
So I finally managed to crack it, but the result is a bit unfortunate. I ended up getting catalog/repositories of every single user. I checked, you still can't pull those repo. So their docker images are safe as such. These look like so:
Edit-2: Fetching all repos
After we notified Codefresh of this situation, here is how to do the fetch. One needs to first generate a token using below:
curl -H 'Host: r.cfcr.io' -H 'User-Agent: MyAgent' -H 'Authorization: Basic .....' --compressed 'https://r.cfcr.io/v2/token?account=<account>&service=r.cfcr.io'
And then using the same we can query the complete catalog:
curl -H "User-Agent: ItsMe" -H 'Authorization: Bearer <TOKEN>' 'https://r.cfcr.io/v2/_catalog?n=10' --compressed
I have a partial answer, and I think it is useful enough on its own for others approaching the same difficulty. I got some kind assistance via the chat widget on Codefresh's support page.
Proprietary API
Regarding the Codefresh API, I had not spotted that the domain g.codefresh.io is the same as their control panel. So, to authenticate, all I had to do was log into the control panel - oops! That reveals the call to https://g.codefresh.io/api/images and a much more complicated access token than I've been using - perhaps it is OAuth. It looks like this:
curl \
-X GET \
--header "Accept: application/json" \
--header "x-access-token: (36chars).(143chars).(22chars)-(4chars)-(15chars)" \
"https://g.codefresh.io/api/images"
As I considered in the question, it does look like the /api/images endpoint is for Codefresh builds only. So, this is out for me - I want everything in the registry.
Open API
So, turning to Docker's open standard for accessing registry, my support contact said this:
if you want to connect to cfcr.io through docker CLI, you can.
The username is your username at Codefresh. The password is a token you can generate at user settings -> "GENERATE" button at the Codefresh registry section.
That's what I've been doing so far, and it works for login, push and pull. It does not seem to work for the reg utility in ls mode though. Either I am still doing something wrong, or there are restrictions at Codefresh as to how the private registry may be used.
Unfortunately, it's not possible to use search and catalog commands for Codefresh Registry. The Codefresh registry (r.cfcr.io) is based on Google Container Registry (aka GCR) and Google Registry does not implement v1 Docker registry API and _catalog function.
Once they do, things will work for Codefresh Registry too.

Using REST API to deploy Liberty Docker containers

Has anyone been able to use the WebSphere Liberty REST API to deploy remote Docker containers? The docs describe all the steps, but I am not able to repeat the results. I get an error when calling REST deploy function (details are posted on the other forum).
If anyone was able to do it - please let me know and would be great if you share how you were able to do it.
Not long after posting this question I tried using the curl command with the JSON header spec and now it works. Here is the curl command I am using that works:
curl --verbose --insecure -X POST --data #docker_deploy.json --header "Content-Type: application/json" -u ${ADMIN_USER}:${ADMIN_PASSWORD} https://${CONTROLLER_HOST_NAME}:${CONTROLLER_PORT}/ibm/api/collective/v1/deployment/deploy

Resources