Evening all,
Setting up a fairly straightforward Mosquitto -> Node-Red -> Mariadb deployment, from docker-compose. Compose file as below:
version: '3.8'
services:
mqtt:
container_name: mosquitto
image: eclipse-mosquitto:latest
restart: always
ports:
- "1883:1883"
volumes:
- ./mosquitto.conf:/mosquitto/config/mosquitto.conf
- /mosquitto/data
- /mosquitto/log
nodered:
container_name: node-red
image: nodered/node-red:latest
restart: always
ports:
- "1880:1880"
volumes:
- node_red_user_data:/data
links:
- "mariadb:mariadb"
mariadb:
container_name: mariadb
image: mariadb:latest
restart: always
command: --default-authentication-plugin=mysql_native_password
ports:
- "3306:3306"
- "33060:33060"
environment:
- MYSQL_RANDOM_ROOT_PASSWORD=yes
- MYSQL_DATABASE=test
- MYSQL_USER=testuser
- MYSQL_PASSWORD=password
volumes:
- mariadb_data_container:/var/lib/mysql
volumes:
mariadb_data_container:
node_red_user_data:
mosquitto_persistence:
networks:
default:
name: primary
Mosquitto to Node-Red is working well enough, and I've set up the database and table in mariadb, but I'm not having any luck getting Node-Red to talk to Maria. Getting this error thrown back at me.
Error: connect ECONNREFUSED 127.0.0.1:3306
The Node-red node in question is as follows:
[{"id":"d93d7d2b.ee27f","type":"mysql","z":"ec0540ab.8b4e2","mydb":"68416de0.8f91a4","name":"XDK Environmental Data","x":750,"y":260,"wires":[["d0d7439f.9b88d"]]},{"id":"68416de0.8f91a4","type":"MySQLdatabase","z":"","name":"Write to mariadb","host":"localhost","port":"3306","db":"XDK_FEM","tz":""}
Full Node-red flow here, in case that's in some way useful.
[{"id":"ec0540ab.8b4e2","type":"tab","label":"MQTT_MYSQL_write","disabled":false,"info":""},{"id":"772011e7.51dd4","type":"mqtt in","z":"ec0540ab.8b4e2","name":"XDK1_Output","topic":"BCDS/XDK110/example/out","qos":"2","datatype":"utf8","broker":"ac9b691.6c35998","x":90,"y":260,"wires":[["f7d34d6b.63919","8336338b.e648c"]]},{"id":"d0d7439f.9b88d","type":"debug","z":"ec0540ab.8b4e2","name":"","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"true","targetType":"full","statusVal":"","statusType":"auto","x":990,"y":260,"wires":[]},{"id":"d79d4ae0.fcf958","type":"function","z":"ec0540ab.8b4e2","name":"Create query in topic","func":"var out = \"INSERT INTO XDK1_raw (timestamp,message)\"\nout = out + \"VALUES ('\" + new Date().toISOString() + \"','\" \nout = out + msg.payload + \"');\"\nmsg.topic=out;\n\nreturn msg;","outputs":1,"noerr":0,"initialize":"","finalize":"","x":500,"y":260,"wires":[["d93d7d2b.ee27f"]]},{"id":"9b43a338.781","type":"comment","z":"ec0540ab.8b4e2","name":"Log everything","info":"","x":100,"y":200,"wires":[]},{"id":"d93d7d2b.ee27f","type":"mysql","z":"ec0540ab.8b4e2","mydb":"68416de0.8f91a4","name":"XDK Environmental Data","x":750,"y":260,"wires":[["d0d7439f.9b88d"]]},{"id":"f7d34d6b.63919","type":"debug","z":"ec0540ab.8b4e2","name":"","active":false,"tosidebar":true,"console":false,"tostatus":false,"complete":"payload","targetType":"msg","statusVal":"","statusType":"auto","x":310,"y":180,"wires":[]},{"id":"8336338b.e648c","type":"json","z":"ec0540ab.8b4e2","name":"","property":"payload","action":"str","pretty":true,"x":270,"y":300,"wires":[["cbee55ed.b7a668","d79d4ae0.fcf958"]]},{"id":"cbee55ed.b7a668","type":"debug","z":"ec0540ab.8b4e2","name":"","active":false,"tosidebar":true,"console":false,"tostatus":false,"complete":"false","statusVal":"","statusType":"auto","x":480,"y":380,"wires":[]},{"id":"ac9b691.6c35998","type":"mqtt-broker","z":"","name":"XDK_Mosquitto","broker":"192.168.1.115","port":"1883","clientid":"","usetls":false,"compatmode":false,"keepalive":"60","cleansession":true,"birthTopic":"","birthQos":"0","birthPayload":"","closeTopic":"","closeQos":"0","closePayload":"","willTopic":"","willQos":"0","willPayload":""},{"id":"68416de0.8f91a4","type":"MySQLdatabase","z":"","name":"Write to mariadb","host":"localhost","port":"3306","db":"XDK_FEM","tz":""}]
I've tried various combinations of ports and configs, and after digging myself deeper into a hole I've just reset everything to start from scratch.
Any insight gratefully received!
EDIT
I realise this isn't a very helpful update for anyone facing the same image, but in the end I deleted the container and started again from the docker-compose and it worked. Lord only knows. Working theory is that before hitting on the right answer (using 'mariadb' instead of 'localhost') I broke something trying a wrong answer.
The important thing to remember is that each container has it's own loopback device (lo, 127.0.0.1) and the host for the containers has it's own as well and they are all totally separate.
So you can not reference the MariaDB container as 127.0.0.1 from the Node-RED container as that address points to the Node-RED container.
You need to use the hostname mariadb not 127.0.0.1 when entering the details or the mariadb config node.
I want to setup my neo4j database and my .NetCore Web Api with Docker. I created the following docker-compose file :
version: '3.4'
services:
server:
image: ${DOCKER_REGISTRY-}server
network_mode: "bridge"
build:
context: .
dockerfile: Server/Dockerfile
stdin_open: true
tty: true
environment:
- CHOKIDAR_USEPOLLING=true
depends_on:
- neo4j
neo4j:
image: "neo4j:latest"
network_mode: "bridge"
ports:
- "7474:7474"
- "7687:7687"
volumes:
- $HOME/neo4j/data:/data
- $HOME/neo4j/logs:/logs
- $HOME/neo4j/import:/var/lib/neo4j/import
- $HOME/neo4j/plugins:/plugins
environment:
- NEO4J_AUTH=neo4j/admin
Within my .NetCore Server I check whether I can reach the neo4j address (172.17.0.3:7474), which works. Connecting to the Neo4J Database does not work with the following code :
_client = new GraphClient(new Uri("http://172.17.0.3:7474/db/data/"), "neo4j", "admin");
_client.Connect();
The error message is :
System.Exception: 'Received an unexpected HTTP status when executing the request.
The response status was: 404 Not Found
The library doesn't fully support Neo4j 4.x yet -still in development-.
You can either use an older image of Neo4j (using Neo4j:3.5.19 connects successfully), or you can use a different driver.
Using Spring cloud Stream 2.1.4 with Spring Boot 2.1.10, I'm trying to target a local instance of Kafka.
This is an extract of my projetc configuation so far:
spring.kafka.bootstrap-servers=PLAINTEXT://localhost:9092
spring.kafka.streams.bootstrap-servers=PLAINTEXT://localhost:9092
spring.cloud.stream.kafka.binder.brokers=PLAINTEXT://localhost:9092
spring.cloud.stream.kafka.binder.zkNodes=localhost:2181
spring.cloud.stream.kafka.streams.binder.brokers=PLAINTEXT://localhost:9092
spring.cloud.stream.kafka.streams.binder.zkNodes=localhost:2181
But the binder keeps on calling a wrong target :
java.io.IOException: Can't resolve address: kafka.example.com:9092
How can can I specify the target if those properties won't do he trick?
More, I deploy the Kafka instance through a Docker Bitnami image and I'd prefer not to use SSL configuration (see PLAINTEXT protocol) but I'm don't find properties for basic credentials login. Does anyone know if this is hopeless?
This is my docker-compose.yml
version: '3'
services:
zookeeper:
image: bitnami/zookeeper:latest
container_name: zookeeper
environment:
- ZOO_ENABLE_AUTH=yes
- ZOO_SERVER_USERS=kafka
- ZOO_SERVER_PASSWORDS=kafka_password
networks:
- kafka-net
kafka:
image: bitnami/kafka:latest
container_name: kafka
hostname: kafka.example.com
depends_on:
- zookeeper
ports:
- 9092:9092
environment:
- ALLOW_PLAINTEXT_LISTENER=yes
- KAFKA_CFG_LISTENERS=PLAINTEXT://:9092
- KAFKA_CFG_ADVERTISED_LISTENERS=PLAINTEXT://:9092
- KAFKA_CFG_ZOOKEEPER_CONNECT=zookeeper:2181
- KAFKA_ZOOKEEPER_USER=kafka
- KAFKA_ZOOKEEPER_PASSWORD=kafka_password
networks:
- kafka-net
networks:
kafka-net:
driver: bridge
Thanks in advance
The hostname isn't the issue, rahter the advertised listeners protocol//:port mapping that causes the hostname to be advertised, by default. You should change that, rather than the hostname.
kafka:
image: bitnami/kafka:latest
container_name: kafka
hostname: kafka.example.com # <--- Here's what you are getting in the request
...
environment:
- ALLOW_PLAINTEXT_LISTENER=yes
- KAFKA_CFG_LISTENERS=PLAINTEXT://:9092
- KAFKA_CFG_ADVERTISED_LISTENERS=PLAINTEXT://:9092 # <--- This returns the hostname to the clients
If you plan on running your code outside of another container, you should advertise localhost in addition to, or instead of the container hostname.
One year later, my comment still is not been merged into the bitnami README, where I was able to get it working with the following vars (changed to match your deployment)
KAFKA_CFG_LISTENER_SECURITY_PROTOCOL_MAP=PLAINTEXT:PLAINTEXT,PLAINTEXT_HOST:PLAINTEXT
KAFKA_CFG_LISTENERS=PLAINTEXT://:29092,PLAINTEXT_HOST://:9092
KAFKA_CFG_ADVERTISED_LISTENERS=PLAINTEXT://kafka.example.com:29092,PLAINTEXT_HOST://localhost:9092
All right: got this to work by looking twice to the "dockerfile" (thx to cricket_007):
kafka:
...
hostname: localhost
For the record: I could get rid of all properties above, default being for Kafka localhost:9092
I am trying to integrate my own ABCI-application with the localnet. The docker-compose looks as
version: '3'
services:
node0:
container_name: node0
image: "tendermint/localnode"
ports:
- "26656-26657:26656-26657"
environment:
- ID=0
- LOG=${LOG:-tendermint.log}
volumes:
- ./build:/tendermint:Z
command: node --proxy_app=tcp://abci0:26658
networks:
localnet:
ipv4_address: 192.167.10.2
abci0:
container_name: abci0
image: "abci-image"
volumes:
- $GOPATH/src/samplePOC:/go/src/samplePOC
ports:
- "26658:26658"
build:
context: .
dockerfile: $GOPATH/src/samplePOC/Dockerfile
command: /go/src/samplePOC/samplePOC
networks:
localnet:
ipv4_address: 192.167.10.6
Both the nodes and the abci- containers are built successfully. The ABCI server is started successfully and the nodes are trying to make connections. However the main problem is that the I see the two are not able to communicate with each other.
I get the following error:
node0 |E[2019-10-29|15:14:28.525] abci.socketClient failed to connect
to tcp://abci0:26658. Retrying... module=abci-client connection=query
err="dial tcp 192.167.10.6:26658: connect: connection refused"
Can someone please help me here?
My first thought is that you may need to add a depends_on: ["abci0"] to node0, as the ABCI application must be listening before Tendermint will try to connect.
Of course, TM should continue to retry so this may not be the issue.
Another thing you can try, is to run tendermint on your host machine, and attempt to connect to the exposed port of ABCI port on abci0 (26658) to isolate the problem to the docker configuration.
If you're not able to run tendermint node --proxy_app=tcp://localhost:26658 the problem likely lies in your ABCI application.
I assume you've initialized a directory in the volume you mount into node0?
I got this working with the kvstore example from Tendermint.
version: "3.4"
services:
kvstore-app:
image: alpine
expose:
- "26658"
volumes:
- ./kvstore-example:/home/dev/kvstore-example
command: "/home/dev/kvstore-example --socket-addr tcp://kvstore-app:26658"
tendermint-node:
image: tendermint/tendermint
depends_on:
- kvstore-app
ports:
- "26657:26657"
environment:
- TMHOME=/tmp/tendermint
volumes:
- ./tmp/tendermint:/tmp/tendermint
command: node --proxy_app=tcp://kvstore-app:26658
I'm not exactly sure why your docker-compose.yml isn't working, but it's likely that you are not binding the socket of your abci application in a way that is accessible to the node. I'm explicitly telling the abci application to do so with the argument --socket-addr tcp://kvstore-app:26658". Additionally, I'm just exposing the port of the abci application on the docker network, but I think mapping the port should do this implicitly.
Also I would get rid of all the network stuff. Personally, I use the network configuration only if I have some very specific network goals in mind.
Good Morning,
I'm currently unable to get any of the listeners working to display data in the Hygieia dashboard. Right now I'm just trying to get sonar results displayed. The sonar instance is up and running at localhost:9000 with scan results.
My compose file looks like the following:
mongodb:
image: mongo:latest
container_name: mongodb
command: mongod --smallfiles
ports:
- "27017:27017"
volumes:
- ./mongo:/var/lib/mongo/:rw
volume_driver: local
hygieia-api:
image: hygieia-api:latest
container_name: hygieia-api
ports:
- "8080:8080"
volumes:
- ./logs:/hygieia/logs
links:
- mongodb:mongo
hygieia-ui:
image: hygieia-ui:latest
container_name: hygieia-ui
ports:
- "8088:80"
links:
- hygieia-api
hygieia-sonar-codequality-collector:
image: hygieia-sonar-codequality-collector:latest
container_name: hygieia-sonar-codequality
volumes:
- ./logs:/hygieia/logs
links:
- mongodb:mongo
- hygieia-api
My override file:
hygieia-sonar-codequality-collector:
environment:
- SONAR_CRON=*/1 * * * *
- SONAR_URL=http://localhost:9000
- SONAR_API=http://localhost:9000/api/projects/index/
My application.properties file:
dbname=dashboard
dbhost=localhost
dbport=27017
dbusername=dashboarduser
dbpassword=dbpassword
sonar.cron=0 0/5 * * * *
sonar.servers[0]=http://localhost:9000
sonar.metrics=ncloc,line_coverage,violations,critical_violations,major_violations,blocker_violations,sqale_index,test_success_density,test_failrues,test_errors,tests
I've had the same issue with JIRA and Jenkins as well and have not been able to get any test results into the dashboard. Thank you for any help.
EDIT:
2017-03-08 17:34:32,007 ERROR c.c.d.collector.DefaultSonarClient - org.springframework.web.client.ResourceAccessException: I/O error on GET request for "http://0.0.0.0:9000/api/resources?format=json":Connect to 0.0.0.0:9000 [/0.0.0.0] failed: Connection refused (Connection refused); nested exception is org.apache.http.conn.HttpHostConnectException: Connect to 0.0.0.0:9000 [/0.0.0.0] failed: Connection refused (Connection refused)
This seems to be my issue, however I am able to connect to the above url successfully..
Fixed my issue by pointing to the ip address of the container instead of localhost.