FIWARE - TTN IoT - Agent - docker

I am trying to get the ORION-Broker running with IoTA-LoRa. Unfortunately I get an Error-Message while trying to add a device to the Agent.
I use the following HowTo to use TheThingsNetwork: https://fiware-lorawan.readthedocs.io/en/latest/users_manual/index.html#the-things-network-ttn
Does anyone got it running? I always get the error:
Cannot read property 'findOne' of undefined.
Any Ideas of solving it?
Payload and Error-Message
Unfortunately a restart of IoTA does not work.
This is my Docker-compose file:
Unfortunately it is not working for me. Following I Post my Docker-compose file:
version: "3.1
services:
mongodb:
image: mongo:latest
hostname: mongo-db
container_name: db-mongo
ports:
- "27017:27017"
volumes:
- /mnt/docker/mongodb:/data
orion:
image: fiware/orion
hostname: orion
container_name: fiware-orion
depends_on:
- mongodb
expose:
- "1026"
ports:
- "1026:1026"
command: -dbhost mongodb
iot-agent:
image: ioeari/iotagent-lora
hostname: iot-agent
container_name: fiware-iot-agent
depends_on:
- mongodb
expose:
- "4041"
ports:
- "4041:4041"
environment:
- "IOTA_CB_HOST=orion"
- "IOTA_CB_PORT=1026"
- "IOTA_NORTH_PORT=4041"
- "IOTA_REGISTRY_TYPE=mongodb"
- "IOTA_MONGO_HOST=mongo-db"
- "IOTA_MONGO_PORT=27017"
- "IOTA_MONGO_DB=iotagent-lorawan"
- "IOTA_PROVIDER_URL=http://iot-agent:4041"
I try to add a Device like it is described in the FIWARE-HowTo in the above link.
The IoTA in general is running. The call: http://myhost:4041/iot/about works fine. But operating devices with it always fails.
And here is the full error message:
HTTP/1.1 500 Internal Server Error
X-Powered-By: Express
Fiware-Correlator: c430da57-9b39-4ef1-a7e7-f27f2e6b2113
Content-Type: application/json; charset=utf-8
Content-Length: 76
ETag: W/"4c-Mk5iR3/t76SWs53TC2DG+2Oquj0"
Date: Tue, 17 Sep 2019 08:14:03 GMT
Connection: close
{"name":"TypeError","message":"Cannot read property 'findOne' of undefined"}

The error message indicates a Mongo-DB problem, but I think this maybe Docker related, specifically an image versioning issue.
mongodb:
image: mongo:latest
orion:
image: fiware/orion
iot-agent:
image: ioeari/iotagent-lora
In each case you are explicitly or implicitly using the bleeding-edge latest Docker image, for example in the case of mongo-db this is currently mongo:4.2.0, maybe it was something else yesterday, maybe there is a breaking change since the LoRaWAN IoT Agent was last updated.
It would be better to fix to using the common set of builds used at the time of the FIWARE_7.7.1 release:
fiware/orion:2.2.0
mongo:3.6
fiware/iotagent-lorawan:1.2.3
And try again to see Mongo-DB is better behaved.
Note: It looks like version tags are missing on ioeari/iotagent-lora, but latest corresponds to bff2c15d which was release 1.2.1 which has not yet been duplicated in the FIWARE account. For certainty, it could be better to build your own from sources.
Second point - your "IOTA_MONGO_HOST=mongo-db" and Orion connection command command: -dbhost mongodb do not match try the following:
version: "3.1"
services:
iot-agent:
image: fiware/iotagent-lorawan:1.2.2
hostname: iot-agent
container_name: fiware-iot-agent
depends_on:
- mongo-db
expose:
- "4041"
ports:
- "4041:4041"
environment:
- "IOTA_CB_HOST=orion"
- "IOTA_CB_PORT=1026"
- "IOTA_NORTH_PORT=4041"
- "IOTA_REGISTRY_TYPE=mongodb"
- "IOTA_MONGO_HOST=mongo-db"
- "IOTA_MONGO_PORT=27017"
- "IOTA_MONGO_DB=iotagent-lorawan"
- "IOTA_PROVIDER_URL=http://iot-agent:4041"
- IOTA_LOG_LEVEL=DEBUG # The log level of the IoT Agent
mongo-db:
image: mongo:3.6
hostname: mongo-db
container_name: db-mongo
expose:
- "27017"
ports:
- "27017:27017"
command: --bind_ip_all --smallfiles
volumes:
- mongo-db:/data
orion:
image: fiware/orion
hostname: orion
container_name: fiware-orion
depends_on:
- mongo-db
expose:
- "1026"
ports:
- "1026:1026"
command: -dbhost mongo-db
volumes:
mongo-db: ~
One or other of your mongo-db connections is failing, this will cause mongoose to have a null pointer.

The error Cannot read property 'findOne' of undefined generally in FIWARE environment setup arises because of synchronization problem between IoT Agent and MongoDB.
In our case we also had this problem multiple times, the idea is to start the MongoDB first and then start IoT Agent, and if MongoDB is restarted, sometimes we have to restart the IoT Agent again after MongoDB.

Related

Traefik 2 network between 2 containers results in Gateway Timeout errors

I'm trying to set up 2 docker containers with docker-compose, 1 is a Traefik proxy and the other is a Vikunja kanban board container.
They both have their own docker-compose file. I can start the containers and the Traefik dashboard doesn't show any issues but when I open the URL in a browser I only get a Gateway Timeout error.
I have been looking at similar questions on here and different platforms and in nearly all other cases the issue was that they were placed on 2 different networks. However, I added a networks directive to the Traefik docker-compose.yml and still have this problem, unless I'm using them wrong.
The docker-compose file for the Vikunja container
(adapted from https://vikunja.io/docs/full-docker-example/)
version: '3'
services:
api:
image: vikunja/api
environment:
VIKUNJA_DATABASE_HOST: db
VIKUNJA_DATABASE_PASSWORD: REDACTED
VIKUNJA_DATABASE_TYPE: mysql
VIKUNJA_DATABASE_USER: vikunja
VIKUNJA_DATABASE_DATABASE: vikunja
VIKUNJA_SERVICE_JWTSECRET: REDACTED
VIKUNJA_SERVICE_FRONTENDURL: REDACTED
volumes:
- ./files:/app/vikunja/files
networks:
- web
- default
depends_on:
- db
restart: unless-stopped
labels:
- "traefik.enable=true"
- "traefik.http.routers.vikunja-api.rule=Host(`subdomain.domain.de`) && PathPrefix(`/api/v1`, `/dav/`, `/.well-known/`)"
- "traefik.http.routers.vikunja-api.entrypoints=websecure"
- "traefik.http.routers.vikunja-api.tls.certResolver=myresolver"
frontend:
image: vikunja/frontend
labels:
- "traefik.enable=true"
- "traefik.http.routers.vikunja-frontend.rule=Host(`subdomain.domain.de`)"
- "traefik.http.routers.vikunja-frontend.entrypoints=websecure"
- "traefik.http.routers.vikunja-frontend.tls.certResolver=myresolver"
networks:
- web
- default
restart: unless-stopped
db:
image: mariadb:10
command: --character-set-server=utf8mb4 --collation-server=utf8mb4_unicode_ci
environment:
MYSQL_ROOT_PASSWORD: REDACTED
MYSQL_USER: vikunja
MYSQL_PASSWORD: REDACTED
MYSQL_DATABASE: vikunja
volumes:
- ./db:/var/lib/mysql
restart: unless-stopped
command: --max-connections=1000
networks:
- web
networks:
web:
external: true
The network directives for the api and frontend services in the Vikunja docker-compose.yml were present in the template (I added one for the db service for testing but it didn't have any effect).
networks:
- web
After getting a docker error about the network not being found I created it via docker network create web
The docker-compose file for the Traefik container
version: '3'
services:
traefik:
image: traefik:v2.8
ports:
- "80:80"
- "443:443"
- "8080:8080" # dashboard
volumes:
- /var/run/docker.sock:/var/run/docker.sock
- ./letsencrypt:/letsencrypt
- ./traefik.http.yml:/etc/traefik/traefik.yml
networks:
- web
networks:
web:
external: true
I've tried adding the Traefik service to the Vikunja docker-compose.yml in one file but that didn't have any effect either.
I'm thankful for any pointers.
For debugging you could try to configure all container to use the host network to enusre they are realy on the same netwok.
i had a similar issue trying to run two different dockers and getting a
"Gateway Timeout". My issue was solved after changing the mapping in the second docker for traefik and accessing the site with :84 at the end (http://sitename:84)
traefik:
image: traefik:v2.0
container_name: "${PROJECT_NAME}_traefik"
command: --api.insecure=true --providers.docker
ports:
- '84:80'
- '8084:8080'

HTTPS for docker containers

I am developing a workflow service as a training project. Abstracting from the details, everything you need to know for this question is in the image. For deployment, I rented a server and ran docker-compose on it. Everything works well, but what I'm worried about is that ports 8000 and 5432 are open.
The first question is, is it worth worrying? And if so, how to get rid of it?
Docker-compose file content below
version: "3"
services:
db:
container_name: 'emkk-db'
image: postgres
volumes:
- ./backend/data:/var/lib/postgresql/data
env_file:
- ./backend/db.env
ports:
- "5432:5432"
backend:
container_name: 'emkk-backend'
image: emkk_backend
build: ./backend
volumes:
- ./backend:/emkk/backend
env_file:
- ./backend/.env
ports:
- "8000:8000"
depends_on:
- db
frontend:
container_name: 'emkk-frontend'
image: emkk_frontend
build: ./frontend
command: npm run start
env_file:
- ./frontend/.env
volumes:
- /emkk/frontend/node_modules
- ./frontend:/emkk/frontend
ports:
- "80:80"
depends_on:
- backend
I also want to configure HTTPS protocol. I tried installing nginx and putting a certificate on it using a certbot, and then proxying requests to containers. I sat with this for several hours and I still did not manage to achieve anything better than a HTTPS for the nginx start page.
Maybe I'm doing completely wrong things, but I'm new to this, I haven't had to deal with deployments before. I would be grateful for your answers, which will contain an idea or an example of how you can do this.
If you don't have a connection to 8000 (probably WAS) or 5432 (database) from an external server, you can change docker-compose.yml to:
you have to expose only necessary ports for external clients.
when you connect to backend from web, you should use service name like backend:8000
when you connect to db from backend, you should use service name like db:5432
version: "3"
services:
db:
container_name: 'emkk-db'
image: postgres
volumes:
- ./backend/data:/var/lib/postgresql/data
env_file:
- ./backend/db.env
backend:
container_name: 'emkk-backend'
image: emkk_backend
build: ./backend
volumes:
- ./backend:/emkk/backend
env_file:
- ./backend/.env
depends_on:
- db
frontend:
container_name: 'emkk-frontend'
image: emkk_frontend
build: ./frontend
command: npm run start
env_file:
- ./frontend/.env
volumes:
- /emkk/frontend/node_modules
- ./frontend:/emkk/frontend
ports:
- "80:80"
depends_on:
- backend
And, you can use nginx proxy manager to service with HTTPS and a certificate from the certbot.

Issue with Collabora CODE, Nextcloud & Nginx using docker-compose

I am trying to get a docker-compose.yaml together to run Nextcloud and Collabora CODE via Nginx Proxy Manager.
So far I have Nginx and Nextcloud working really nicely with persistent volumes so that my configs survive kill & rm. My issue is that I cannot get my Collabora CODE instance to link to Nextcloud. There are multiple bits that I might have got wrong so I'll dump as much info as I can here.
I have the following subdomains all pointing at my server:
collabora.domain.tld nextcloud.domain.tld nginx.domain.tld
... and set up as proxy hosts:
As I mentioned, the Nginx and Nextcloud setups are great. When I point my browser at collabora.domain.tld I see the OK message. I can also access the admin page at collabora.domain.tld/loleaflet/dist/admin/admin.html
The NPM entry for collabora.domain.tld is below:
My docker-compose.yaml has gone through several iterations in an attempt to get this working, but my current attempt is below:
version: '3'
volumes:
nextcloud-data:
nextcloud-db:
npm-data:
npm-ssl:
npm-db:
networks:
frontend:
backend:
services:
code:
image: collabora/code:latest
restart: always
environment:
- password=${COLLABORA_PASSWORD:?Not defined!}
- username=${COLLABORA_USERNAME:?Not defined!}
- domain=${COLLABORA_DOMAIN:?Not defined!}
expose:
- "9980"
networks:
- frontend
- backend
nextcloud-app:
image: nextcloud:stable
restart: always
volumes:
- nextcloud-data:/var/www/html
environment:
- MYSQL_PASSWORD=${NC_MYSQL_PASSWORD:?Not defined!}
- MYSQL_DATABASE=nextcloud
- MYSQL_USER=nextcloud-user
- MYSQL_HOST=nextcloud-db
networks:
- frontend
- backend
nextcloud-db:
image: mariadb
restart: always
command: --transaction-isolation=READ-COMMITTED --binlog-format=ROW --innodb-file-per-table=1 --skip-innodb-read-only-compressed
volumes:
- nextcloud-db:/var/lib/mysql
environment:
- MYSQL_ROOT_PASSWORD=${NC_MYSQL_ROOT_PASSWORD:?Not defined!}
- MYSQL_PASSWORD=${NC_MYSQL_PASSWORD:?Not defined!}
- MYSQL_DATABASE=nextcloud
- MYSQL_USER=nextcloud-user
networks:
- backend
npm-app:
image: jc21/nginx-proxy-manager:latest
restart: always
ports:
- "80:80"
- "81:81"
- "443:443"
environment:
- DB_MYSQL_HOST=npm-db
- DB_MYSQL_PORT=3306
- DB_MYSQL_USER=npm-user
- DB_MYSQL_PASSWORD=${NPM_MYSQL_PASSWORD:?Not defined!}
- DB_MYSQL_NAME=npm
volumes:
- npm-data:/data
- npm-ssl:/etc/letsencrypt
networks:
- frontend
- backend
npm-db:
image: jc21/mariadb-aria:latest
restart: always
environment:
- MYSQL_ROOT_PASSWORD=${NPM_MYSQL_ROOT_PASSWORD:?Not defined!}
- MYSQL_DATABASE=npm
- MYSQL_USER=npm-user
- MYSQL_PASSWORD=${NPM_MYSQL_PASSWORD:?Not defined!}
volumes:
- npm-db:/var/lib/mysql
networks:
- backend
$COLLABORA_DOMAIN is set to nextcloud.domain.tld.
Any ideas what I have done wrong, and how to get my Nextcloud connected to CODE?
What do your custom locations look like? See, e.g. https://www.collaboraoffice.com/code/nginx-reverse-proxy/
I have a very similar setup, except the collabora instance is not in docker compose, as that never worked for me.
Make sure you have specified your domain environmental variable correctly (dot escaping etc.)

HIVE Docker container to use with existing Hadoop instances

I have 2 working containers for Hadoop HDFS (I have built a Dockerfile for them starting from the one here) and need to add a HIVE instance to the set.
I cannot find a good working image to use (it would be nice to have version 2.3.4).
Is there anything you suggest, easy to add, that I could use?
Thank you!
Edit:
Here is a try I did:
hive-server:
container_name: hive-server
image: bde2020/hive:2.3.2-postgresql-metastore
depends_on:
- hadoop-namenode
env_file:
- ./hive_build/hadoop-hive.env
environment:
HIVE_CORE_CONF_javax_jdo_option_ConnectionURL: "jdbc:postgresql://hive-metastore/metastore"
SERVICE_PRECONDITION: "hive-metastore:9083"
ports:
- "10000:10000"
restart: unless-stopped
hive-metastore:
container_name: hive-metastore
image: bde2020/hive:2.3.2-postgresql-metastore
depends_on:
- hive-server
- hive-metastore-postgresql
env_file:
- ./hive_build/hadoop-hive.env
command: /opt/hive/bin/hive --service metastore
environment:
SERVICE_PRECONDITION: "hadoop-namenode:50070 hadoop-datanode1:50075 hive-metastore-postgresql:5432"
ports:
- "9083:9083"
restart: unless-stopped
hive-metastore-postgresql:
container_name: hive-metastore-postgresql
image: bde2020/hive-metastore-postgresql:2.3.0
ports:
- "5433:5432"
restart: unless-stopped
but when I enter it and try to connect I get an error:
docker exec -it hive-server bash
/opt/hive/bin/beeline -u jdbc:hive2://localhost:10000
error:
19/05/03 09:13:46 [main]: WARN jdbc.HiveConnection: Failed to connect to localhost:10000
Could not open connection to the HS2 server. Please check the server URI and if the URI is correct, then ask the administrator to check the server status.
Error: Could not open client transport with JDBC Uri: jdbc:hive2://localhost:10000: java.net.ConnectException: Connection refused (Connection refused) (state=08S01,code=0)

Error : Unable to use host type networking with links

I am new to the Docker World.I have got my db and server connected. Now i want to connect my frontend with server. I an unable to use network mode with link. Without using network mode, i am unable to view my frontend site on localhost:8080.
How can i connect my front and server?
version: '2.2'
services:
front:
build: ../Devops/operations-portal
ports:
- "127.0.0.1:8080:8080"
network_mode: "host"
volumes:
- ../Devops/operations-portal
links:
- server
server:
build: ../Devops/data-mining-backend
ports:
- "5000:5000"
volumes:
- ../Devops/data-mining-backend/:/app
links:
- db
db:
image: mongo:3.6.5
ports:
- "27017:27017"
volumes:
- '../database:/data/db'
command: mongod --smallfiles --logpath=/dev/null # --quiet

Resources