I'm flowing this link, all is OK before step 3.4. But when I use this mosquitto_pub command to publish the message to the Bosch IOT hub.
mosquitto_pub -h mqtt.bosch-iot-hub.com -p 8883 -u 4712#t65a6a75c6b4f429981ba16d1b3d5f414 -P secret -t telemetry/t65a6a75c6b4f429981ba16d1b3d5f414/4712 -m '"103.3223","3.2322"' --cafile iothub.crt
But the output log didn't have any message like the tutorial
Why? What's the problem?
It seems that you have not started the normaliser middleware with any outbound handler plugin. So in your case, in order to send any incoming telemetry data to an AMQP topic, you need to start the middleware with the AMQP plugin
Once done so, you should be able to see the logs in the middleware dashboard, under monitoring, appearing as outbound messages.
Sorry for the confusion, I have changed the tutorial in order to make it clearer, see https://github.com/eclipse/vorto/blob/master/docs/tutorials/create_mapping_pipeline.md#4-setting-up-eclipse-vorto-payload-normalization-middleware-with-vorto-amqp-plugin
Related
I am running mosquitto broker on ubuntu on aws ec2, using TLS over port 8883. For example :
mosquitto_sub -h domain.com -t topic --cafile /etc/mosquitto/certs/ca.crt -p 8883
The client is esp8266 using TLS configuration.
Is it possible for someone to still intercept or read the outgoing or incoming mqtt message?
All communication is on port 8883 using a certificate.
(1) The most trivial answer with MQTT context is:
Use another MQTT client and subscribe to '#' to see all messages your broker receives from anywhere.
If you know the topics your esp8266 client is publishing or subscribed to,
subscribe to all these with the other client (e.g. with 'mosquitto_sub').
The broker provides messages on the topic "topic" mentioned above to all subscribed clients, independent from their communication channel security.
This way you get the messages for the esp8266 with any other client.
For example, if the unencrypted channel is not deactivated: mosquitto_sub -h domain.com -t topic -p 1883
TLS ensures encryption for the communication channel only, and not for the actual payload before putting it onto the channel between client and server.
(2) If (1) is not the expected answer and you rather want to wiretap the encrypted communication:
Short answer is: no, not with trivial tooling; that's the purpose of TLS.
A good article for MQTT security fundamentals: https://www.hivemq.com/blog/mqtt-security-fundamentals-tls-ssl/
Long answer is:
[Disclaimer: no assumptions about advanced abilities of 3rd parties]
The TLS communication is properly encrypted, and to decrypt the MQTT communication requires the session secrets client and server use after
their successful TLS handshake.
A solutions requires you as the operator of the broker.
There is a short article about how to do it:
https://adrianalin.gitlab.io/popsblog.me/posts/decrypt-mqtt-tls-traffic/
In essence:
You can use Wireshark to eavesdrop the traffic, if you configure the session secrets there.
To get the secrets you adjust and re-build mosquitto according the article, so that your broker print the session secrets.
(Whether you are able to setup this with root privileges on your EC2 instance in AWS, is another question).
My mosquitto.conf is (This is the entire conf file since everything else is just commented out)
log_dest file /mosquitto/log/mosquitto.log
log_type all
connection_messages true
log_timestamp true
log_timestamp_format [%H:%M:%S]
persistence true
persistence_location /mosquitto/data/
listener 1883
listener 9001
protocol websockets
Then i run
docker exec -it ecs-Eclipse_mosquitto_MQTT-11-Eclipse-mosquitto-MQTT-eafxxxxxxx mosquitto_pub -t presence -m hellothere -q qos=2
(NOTE I dont have any active subscribers yet online)
Navigated to /mosquitto/data and there was indeed a file called mosquitto.db
So i could deduce that my message hellothere is stored in mosquitto.db
Then i ran the command
docker exec -it ecs-Eclipse_mosquitto_MQTT-11-Eclipse-mosquitto-MQTT-eafxxxxxxx mosquitto_sub -t presence -q qos=2
What i expect
For the MQTT broker to send the stored message since subscriber is now online
What actually happens
indefinite waiting for the message
Maybe i am missing something ?
I think the relevant bit of the mqtt spec is:
When a Server takes ownership of an incoming Application Message it
MUST add it to the Session state of those clients that have matching
Subscriptions.
In your case when the message is published there are no clients with matching subscriptions (because you have not run mosquitto_sub at that point) so it does nothing further.
If you want messages to be stored for a client you must first connect, as that client, and subscribe to the relevant topic (with QOS>0 and CleanSession=0). After doing that any messages received whilst the client is offline will be stored and delivered when it reconnects (as long as CleanSession=0).
Note: To do the above with mosquitto_sub you will need to use the --id parameter to set the client id (messages will only be queued for the specific clients that have subscribed to the relevant topics). You will also need the -c flag to disable the 'clean session' flag.
So the steps will be as follows (I have removed the extra docker bits - you can add these back in):
mosquitto_sub -c --id subscriber1 -t presence -q 2
This will subscribe and then wait for any messages; you can safely close it (it's the subscription bit that needs to happen). Next publish your messgae:
mosquitto_pub -t presence -m hellothere -q 2
Now you can start mosquitto_sub again to retrieve the stored messages:
mosquitto_sub -c --id subscriber1 -t presence -q 2
Warning: When you subscribe like this the broker will store any messages received whilst you are offline. This can become an issue if you are testing a broker that processes a lot of messages (because storing all of the messages takes space). Probably not an issue here as you are only testing but something to bear in mind... Running mosquitto_sub without the -c will clear any subscriptions - there are also options in the mosquitto config that allow you to limit the number of messages held and how long they are held.
MQTT doesn't work that way.
Messages are only queued for clients that have previously been connected, it is a pub/sub system not a message queuing system.
A client needs to have been connected in the past and subscribed to the topic the message in question. It then needs to reconnect after the message has been published with the cleanSession flag set to false and using the same client id.
You can read more about this on the HiveMQ blog here
I am integrating Jenkins in Slack to facilitate build from Slack.
I configured the Build Trigger section of my job
Then I added Slash Commands to slack and configured it as below.
Now, I went to slack and typed /dev-backend, and it is throwing the error,
/dev-backend failed with the error "invalid_url"
I tried hitting the URL directly from postman which is giving the Authentication required response.
How can I add the authentication so that I can trigger build from slack? I thought the token provided in the build triggers section of Jenkins was enough.
I know it's kind of outdated question but I was also struggling with this error, so maybe it will help someone in the future. If you are getting /command failed with the error "invalid_url" and you are able to hit your application endpoint via eg. Bash shell:
curl -X POST -H 'Authorization: Bearer your-bearer-token' -H 'Content-type: application/json;charset=utf-8' --data '{"channel":"#test","text":"Hello, Slack!"}' http://127.0.0.1:8000/your/application/endpoint
it means that Slack can't access/see the URL that you passed in Slack API Applications Settings.
In my case the problem was that I was passing http://127.0.0.1:8000/some/endpoint in Slash Commands configuration which was visible only by my computer. The easiest way to make it visible for Slack is to use ngrok.
So if you are getting this error the steps are as follows:
Install ngrok
Run your app on localhost, eg. http://127.0.0.1:8000
In Bash shell type ngrok http 8000
Copy url returned by ngrok, in my case http://c609-91-234-49-229.ngrok.io
Paste copied URL to Slack Slash Commands configuration instead of localhost.
I'm having trouble in starting MQTT Client in Eclipse Hono.
I'm using The following command to start the client
java -jar hono-example-0.6-exec.jar --hono.client.host=hono.eclipse.org --hono.client.port=15672 --hono.client.username=consumer#HONO --hono.client.password=verysecret --spring.profiles.active=receiver --tenant.id=bob
which starts the client accepting telemetry data produced by the device, but didn't catch the data published through MQTT.
What may be wrong with this approach?
The command you are using does not start an MQTT client but starts the receiver for consuming (AMQP 1.0) messages from devices belonging to tenant bob. In order to see something happening, you need to have a device that belongs to tenant bob publish some data. If you want to use MQTT for that purpose you may want to use the mosquitto_pub command line client as described in the Getting Started guide. However, make sure that you use the correct username and password. From what I can see in the device registry on hono.eclipse.org you have registered a device with id 1112 and auth-id sensor1. So the command to publish should look something like:
mosquitto_pub -h hono.eclipse.org -u sensor1#bob -Pthepasswordyouregistered -t telemetry -m "hello"
Again, make sure to replace thepasswordyouregistered with the real password that you have registered for device 1112.
On Tika's website it says (concerning tika-app-1.2.jar) it can be used in server mode. Does anyone know how to send documents and receive parsed text from this server once it is running?
Tika supports two "server" modes. The simpler and original is the --server flag of Tika-App. The more functional, but also more recent is the JAX-RS JSR-311 server component, which is an additional jar.
The Tika-App Network Server is very simple to use. Simply start Tika-App with the --server flag, and a --port ### flag telling it what port to listen on. Then, connect to that port and send it a single file. You'll get back the html version. NetCat works well for this, something like java -jar tika-app.jar --server --port 12345 followed by nc 127.0.0.1 12345 < MyFileToExtract will get you back the html
The JAX-RS JSR-311 server component supports a few different urls, for things like metadata, plain text etc. You start the server with java -jar tika-server.jar, then do HTTP put calls to the appropriate url with your input document and you'll get the resource back. There are loads of details and examples (including using curl for testing) on the wiki page
The Tika App Network Server is fairly simple, only supports one mode (extract to HTML), and is generally used for testing / demos / prototyping / etc. The Tika JAXRS Server is a fully RESTful service which talks HTTP, and exposes a wide range of Tika's modes. It's the generally recommended way these days to interface with Tika over the network, and/or from non-Java stacks.
Just adding to #Gagravarr's great answer.
When talking about Tika in server mode, it is important to differentiate between two versions which can otherwise cause confusion:
tika-app.jar has the --server --port 9998 options to start a simple server
tika-server.jar is a separate component using JAX-RS
The first option only provides text extraction and returns the content as HTML. Most likely, what you really want is the second option, which is a RESTful service exposing many more of Tika's features.
You can simply download the tika-server.jar from the Tika project site. Start the server using
java -jar tika-server-x.x.jar -h 0.0.0.0
The -h 0.0.0.0 (host) option makes the server listen for any incoming requests, otherwise without it it would only listen for requests from localhost. You can also add the -p option to change the port, otherwise it defaults to 9998.
Then, once the server has started you can simply access it using your browser. It will list all available endpoints.
Finally to extract meta data from a file you can use cURL like this:
curl -T testWORD.doc http://example.com:9998/meta
Returns the meta data as key/value pairs one per line. You can also have Tika return the results as JSON by adding the proper accept header:
curl -H "Accept: application/json" -T testWORD.doc http://example.com:9998/meta
[Update 2015-01-19] Previously the comment said that tika-server.jar is not available as download. Fixed that since it actually does exist as a binary download.
To enhance Gagravarr perfect answer:
If your document is got from a WEB server => curl -u
"http://myserver-domain/*path-to-doc*/doc-name.extension" | nc
127.0.0.1 12345
And it is even better if the document is protected by password => curl -u
login:*password*
"http://myserver-domain/*path-to-doc*/doc-name.extension" | nc
127.0.0.1 12345