I am using this :- https://hub.docker.com/r/apache/nifi/;
running the below for secure instance :-
docker run --name nifi \
-v /User/hddev/certs/localhost:/opt/certs \
-p 8443:8443 \
-e AUTH=tls \
-e NIFI_WEB_PROXY_HOST=ec2-hostname.aws.com:8443 \
-e KEYSTORE_PATH=/opt/certs/keystore.jks \
-e KEYSTORE_TYPE=JKS \
-e KEYSTORE_PASSWORD=securepass \
-e TRUSTSTORE_PATH=/opt/certs/truststore.jks \
-e TRUSTSTORE_PASSWORD=securepass \
-e TRUSTSTORE_TYPE=JKS \
-e INITIAL_ADMIN_IDENTITY='CN=admin, OU=NIFI' \
-d \
apache/nifi:latest
WebUI works fine I am able to access but with s2s I get this at the RPG,
java.net.UnknownHostException for docker-container-id:8443
o.a.n.r.util.SiteToSiteRestApiClient Failed to create transaction for
https://docker-container-id:8443/nifi-api/data-transfer/input-ports/someportaddress/transactions
When I see the property for ,it takes the container id :- 2eftfdf
nifi.web.https.host=2eftfdf
nifi.remote.input.host=2eftfdf
due to below code.
https://github.com/apache/nifi/blob/master/nifi-docker/dockerhub/sh/secure.sh#L61
I was able to get it work by by changing the environment variable $HOSTNAME to public dns of ec2.
Is there setting in to achieve this without chaging. env
Do we need to update nifi.remote.route.{protocol}.{name}.when related properties? If not then how?
Related
This question already has answers here:
Why docker container exits immediately
(16 answers)
Closed 9 months ago.
I am new to this. I have installed docker on my Raspi. I am trying to install transmission on the docker. I use the following;
docker run --cap-add=NET_ADMIN -d \
--name=transmission \
-v /mnt/extDrive1:/data \
-v /etc/localtime:/etc/localtime:ro \
-e CREATE_TUN_DEVICE=true \
-e OPENVPN_PROVIDER=EXPRESSVPN \
-e OPENVPN_CONFIG=my_expressvpn_uk_-_london_udp \
-e OPENVPN_USERNAME=XXX\
-e OPENVPN_PASSWORD=XXX \
-e WEBPROXY_ENABLED=false \
-e LOCAL_NETWORK=192.168.0.0 \
--log-driver json-file \
--log-opt max-size=10m \
-p 9091:9091 \
haugene/transmission-openvpn
I go through the debug on https://haugene.github.io/docker-transmission-openvpn/debug/
All is fine until I get to the section 'Checking if Transmission is running'.
When I run docker ps, there are no containers in the list.
What have I done wrong? Ultimately, I am trying to access transmission through localhost:9091.
Edit: So I have made some progress, but still having issues;
docker start transmission temporarily. populates the container ID
docker exec -it <container-id> bash comes up with the following error:
Error response from daemon: Container XXXX is not running
It seems that container is exiting out as you are not running it in the detached mode. Try this:
docker run -itd --cap-add=NET_ADMIN -d \
--name=transmission \
-v /mnt/extDrive1:/data \
-v /etc/localtime:/etc/localtime:ro \
-e CREATE_TUN_DEVICE=true \
-e OPENVPN_PROVIDER=EXPRESSVPN \
-e OPENVPN_CONFIG=my_expressvpn_uk_-_london_udp \
-e OPENVPN_USERNAME=XXX\
-e OPENVPN_PASSWORD=XXX \
-e WEBPROXY_ENABLED=false \
-e LOCAL_NETWORK=192.168.0.0 \
--log-driver json-file \
--log-opt max-size=10m \
-p 9091:9091 \
haugene/transmission-openvpn
My goal is simple: Load a hello-world external plugin for Kong. I've decided to do this via mounted volume, but am running into significant problems that stop me from being able to run my simple hello-world external plugin.
I've isolated my problem down to the following commands, from scratch:
docker network create kong-net
docker run -d --name kong-database \
--network=kong-net \
-p 5432:5432 \
-e "POSTGRES_USER=kong" \
-e "POSTGRES_DB=kong" \
-e "POSTGRES_PASSWORD=kong" \
postgres:9.6
docker run --rm \
--network=kong-net \
-e "KONG_DATABASE=postgres" \
-e "KONG_PG_HOST=kong-database" \
-e "KONG_PG_PASSWORD=kong" \
-e "KONG_CASSANDRA_CONTACT_POINTS=kong-database" \
kong:latest kong migrations bootstrap
docker run -it --name kong \
--network=kong-net \
-e "KONG_DATABASE=postgres" \
-e "KONG_PG_HOST=kong-database" \
-e "KONG_PG_PASSWORD=kong" \
-e "KONG_CASSANDRA_CONTACT_POINTS=kong-database" \
-e "KONG_LUA_PACKAGE_PATH=/plugins/?.lua" \
-e "KONG_CUSTOM_PLUGINS=helloworld" \
-e "KONG_PROXY_ACCESS_LOG=/dev/stdout" \
-e "KONG_ADMIN_ACCESS_LOG=/dev/stdout" \
-e "KONG_PROXY_ERROR_LOG=/dev/stderr" \
-e "KONG_ADMIN_ERROR_LOG=/dev/stderr" \
-e "KONG_ADMIN_LISTEN=0.0.0.0:8001, 0.0.0.0:8444 ssl" \
-v "kong/plugins/helloworld:/plugins" \
-p 8000:8000 \
-p 8443:8443 \
-p 8001:8001 \
-p 8444:8444 \
kong:latest
In my local directory, I have the plugin source code sitting in ./kong/plugins/helloworld/
According to the Kong docs, I'm adjusting my Kong config file by adding "KONG_" prior to the item I want adjusted. This means I have KONG_CUMSTOM_PLUGINS=helloworld and KONG_LUA_PACKAGE_PATH=/plugins/?.lua
Docker and Kong doesn't like this, and when running the above commands I am greeted with the following:
docker: invalid reference format.
See 'docker run --help'.
Eliminating things one-by-one, the problem surrounds both the -e "KONG_CUSTON_PLUGINS=helloworld" and -e "KONG_LUA_PACKAGE_PATH=/plugins/?.lua" \ bits.
What's going wrong here?
I checked many resources such as:
https://github.com/dylanmei/docker-zeppelin
http://bigdatums.net/2017/02/26/running-apache-zeppelin-on-docker/
But non has a clear way to set the driver memory, I tried to set variables like:
-e SPARK_SUBMIT_OPTIONS="--driver-memory 6G" and
-e ZEPPELIN_JAVA_OPTS="-Dspark.driver.memory=6g"
without any success.
Here is my full command:
docker run -p 8080:8080 \
-p 4040:4040 \
-v $PWD/logs:/logs \
-v $PWD/notebook:/notebook \
-v $PWD/jars:/jars \
-v $PWD/conf/interpreter.json:/zeppelin/conf/interpreter.js \
-v $PWD/data:/data \
-e ZEPPELIN_LOG_DIR='/logs' \
-e ZEPPELIN_NOTEBOOK_DIR='/notebook' \
-e SPARK_SUBMIT_OPTIONS="--driver-memory 6G" \
apache/zeppelin:0.8.1
OK, after some investigations I had to add -e ZEPPELIN_INTP_MEM="-Xmx6g" to the options.
I am setting up ksql-cli with confluent version 3.3.0 in following way
#zookeper
docker run -d -it \
--net=host \
--name=zookeeper \
-e ZOOKEEPER_CLIENT_PORT=32181 \
confluentinc/cp-zookeeper:3.3.0
#kafka
docker run -d \
--net=host \
--name=kafka \
-e KAFKA_ZOOKEEPER_CONNECT=localhost:32181 \
-e KAFKA_ADVERTISED_LISTENERS=PLAINTEXT://localhost:29092 \
-e KAFKA_OFFSETS_TOPIC_REPLICATION_FACTOR=1 \
confluentinc/cp-kafka:3.3.0
#schema-registry
docker run -d \
--net=host \
--name=schema-registry \
-e SCHEMA_REGISTRY_KAFKASTORE_CONNECTION_URL=localhost:32181 \
-e SCHEMA_REGISTRY_HOST_NAME=localhost \
-e SCHEMA_REGISTRY_LISTENERS=http://localhost:8081 \
confluentinc/cp-schema-registry:3.3.0
i am running ksql-cli docker image in following manner
docker run -it \
--net=host \
--name=ksql-cli \
-e KSQL_CONFIG_DIR="/etc/ksql" \
-e KSQL_LOG4J_OPTS="-Dlog4j.configuration=file:/etc/ksql/log4j-rolling.properties" \
-e STREAMS_BOOTSTRAP_SERVERS=localhost:29092 \
-e STREAMS_SCHEMA_REGISTRY_HOST=localhost \
-e STREAMS_SCHEMA_REGISTRY_PORT=8081 \
confluentinc/ksql-cli:0.5
when i am running ksql-cli by going in bash of container in folowing way
docker exec -it ksql-cli bash
and running ksql-cli in following way:
./usr/bin/ksql-cli local
It is giving me following error:
Initializing KSQL...
Could not fetch broker information. KSQL cannot initialize AdminCLient.
By default, the ksql-cli attempts to connect to the Kafka brokers on localhost:9092. It looks like your setup is using a different port, so you'll need to provide this on the command line, e.g.
./usr/bin/ksql-cli local --bootstrap-server localhost:32181
You'll probably also need to specify the schema registry port, so you may want to use a properties file, e.g. :
./usr/bin/ksql-cli local --properties-file ./ksql.properties
Where ksql.properties has:
bootstrap.servers=localhost:29092
schema.registry.url=localhost:8081
Or provide both on the command line:
./usr/bin/ksql-cli local \
--bootstrap-server localhost:29092 \
--schema.registry.url http://localhost:8081
Note, from KSQL version 4.1 onwards the commands and properties change name. ksql-cli becomes just ksql. The local mode disappears - you'll need to run a ksql-server node or two explicitly. --property-file becomes --config-file and schema.registry.url becomes ksql.schema.registry.url.
I retrieved the most recent image using docker pull boomi/atom:2.3.0
I then run the following script (using placeholders for USERNAME, PASSWORD and ACCOUNT_ID):
#!/bin/bash
atom_name=boomidemo01
docker stop $atom_name
docker rm $atom_name
docker run -p 9090:9090 -h boomidemo01 -e URL="platform.boomi.com" \
-e BOOMI_USERNAME=<USERNAME> -e BOOMI_PASSWORD=<PASSWORD> \
-e BOOMI_ATOMNAME=$atom_name \
-e BOOMI_CONTAINERNAME=$atom_name \
-e BOOMI_ACCOUNTID=<ACCOUNT_ID> \
--name $atom_name \
-d -t boomi/atom:2.3.0
But the atom fails to start (not able to connect on port 9090 via a browser on http://127.0.0.1:9090). Did anyone managed to use docker for running a Boomi atom?
I eventually figured it out... the following script works
#!/bin/bash
atom_name=boomidemo01
host_dir=/home/user/Boomi
docker stop $atom_name
docker rm $atom_name
docker run -p 9090:9090 -h $atom_name \
-v $host_dir:/home/boomi/host_dir \
-e URL=https://platform.boomi.com \
-e BOOMI_USERNAME=<USERNAME> \
-e BOOMI_PASSWORD=<PASSWORD> \
-e BOOMI_ATOMNAME=$atom_name \
-e BOOMI_CONTAINERNAME=$atom_name \
-e BOOMI_ACCOUNTID=<ACCOUNT_ID> \
-e PROXY_HOST= \
-e PROXY_USERNAME= \
-e PROXY_PASSWORD= \
-e PROXY_PORT= \
-e DOCKERUID= \
-e SYMLINKS_DIR= \
-e ATOM_LOCALHOSTID=$atom_name \
-e INSTALL_TOKEN= \
--name $atom_name \
-d -t boomi/atom:2.3.0
Download the docker install script from within the UI.
Generate the token.
Run the script with the name, and token.
The port does not matter because Atomsphere is never pinged. It fetches processes for you.