I am trying to start a docker image (https://hub.docker.com/r/parrotstream/hbase/)
on Windows 10 with
docker-compose -p parrot up
but I get this error:
ERROR:
Can't find a suitable configuration file in this directory or any
parent. Are you in the right directory?
Supported filenames: docker-compose.yml, docker-compose.yaml
Executing the command in the directory with the docker image in it does not work either.
I am new to using Docker and I am unsure how to start the container. According to the Docker Hub page of the image, this is all I have to do. Am I missing something?
Thanks
Edit:
As pointed out by the replies, I've downloaded the folder from github, including the docker-compose.yml. I am currently getting an error because of my permission.
ERROR: for hbase Cannot start service hbase: driver failed programming external connectivity on endpoint hbase (5fb66c3b2b0d3092edce09f03cc803cc3ea447c07a1a2135271238de626458c6): Error starting userland proxy: Bind for 0.0.0.0:8080: unexpected error Permission denied
ERROR: for hbase Cannot start service hbase: driver failed programming external connectivity on endpoint hbase (5fb66c3b2b0d3092edce09f03cc803cc3ea447c07a1a2135271238de626458c6): Error starting userland proxy: Bind for 0.0.0.0:8080: unexpected error Permission denied
ERROR: Encountered errors while bringing up the project.
Do I have a wrong configuration in docker?
The actual docker-compose.yml that you are looking for may be the one hosted in their github repo found here.
version: '3'
services:
hbase:
container_name: hbase
build:
context: .
dockerfile: Dockerfile
image: parrotstream/hbase:latest
external_links:
- hadoop
- zookeeper
ports:
- 8080:8080
- 8085:8085
- 9090:9090
- 9095:9095
- 60000:60000
- 60010:60010
- 60020:60020
- 60030:60030
networks:
default:
external:
name: parrot_default
By default, docker-compose tries to read the configuration from a file named docker-compose.yml within you current working directory. You could override this behavior with docker-compose -f <anotherfile.yml>.
Options:
-f, --file FILE Specify an alternate compose file
(default: docker-compose.yml)
Yes, command needs a compose file and the readme assumes that you have a docker-compose.yml in the directory where you execute the command.
You can find one in the linked repository from DockerHub parrot-stream/docker-hbase
You need to create a docker-compose file as follows
# docker-compose.yml
version: '2'
services:
parrot:
image: parrotstream/hbase
then you can create a build and run is using
docker-compose build parrot # build image
docker-compose up parrot # run
Related
I'm new using docker and spark.
My docker-compose.yml file is
volumes:
shared-workspace:
services:
notebook:
image: docker.io/jupyter/all-spark-notebook:latest
build:
context: .
dockerfile: Dockerfile-jupyter-jars
ports:
- 8888:8888
volumes:
- shared-workspace:/opt/workspace
And the Dockerfile-jupyter-jars is:
FROM docker.io/jupyter/all-spark-notebook:latest
USER root
RUN wget https://repo1.maven.org/maven2/mysql/mysql-connector-java/8.0.28/mysql-connector-java-8.0.28.jar
RUN mv mysql-connector-java-8.0.28.jar /usr/local/spark/jars/
USER jovyan
To it start up a run
docker-compose up --build
The server is up and running and I'm interested to use spark-sql, but it is throwing and error trying to connect to mysql server:
com.mysql.cj.jdbc.exceptions.CommunicationsException: Communications link failure
I can see the mysql-connector-java-8.0.28.jar in the "jars" folder, and I have used same sql instruction in apache spark non docker version and it works.
Mysql db server is also reachable from the same server I'm running the Docker.
Do I need to enable something to reach external connections? Any idea?
Reference: https://hub.docker.com/r/jupyter/all-spark-notebook
The docker-compose.yml and Dockerfile-jupyter-jars files were correct, since I was using mysql-connector-java-8.0.28.jar it requires a SSL or to disable explicitly.
jdbc:mysql://user:password#xx.xx.xx.xx:3306/inventory?useSSL=FALSE&nullCatalogMeansCurrent=true
I'm going to left this example for: Docker - all-spark-notebook with MySQL dataset
I am new to docker. I have been assigned with a task that uses Docker container for development. I followed the tutorial for installing the Docker and the containers on Windows 10, but I have the following error: network remaxmdcrm_remaxmd-network declared as external, but could not be found
The steps I've done so far are:
Cloned the repository from GitHub.
Installed Docker on my laptop.
Once I installed Docker, I went in the root of my project and ran the following command. docker-compose build -d -t docker-compose.yml - docker-compose.yml being the file in the root dir.
I opened Docker app and I ran the images created.
I ran the command docker-compose up. When I ran this command, the error I specified at the beginning appears. network remaxmdcrm_remaxmd-network declared as external, but could not be found
docker-compose.yml
services:
ui:
build:
context: .
dockerfile: Dockerfile.development
volumes:
- .:/app
ports:
- "5000:5000"
restart: unless-stopped
networks:
- remaxmdcrm_remaxmd-network
redis:
image: 'redis:alpine'
networks:
- remaxmdcrm_remaxmd-network
networks:
remaxmdcrm_remaxmd-network:
external: true
Ran: docker ps -a
ID IMAGE
5e6cf997487c remaxmd-site_ui:latest
451009e0a2a6 redis:alpine
85e7cde67d05 docmer-compose.yml:latest
I might do something wrong here. Can somebody help me? I much appreciate your time!
I solved the issue, finally. The issue came from the fact that I had in docker-compose.yml remaxmdcrm_remaxmd-network declared as external. The external network was not created during installation, thus I needed to create a bridging network.
I ran the command docker network create "name_of_network"
For further details, here is the full documentation this
You can see docker network ls and uses bridge network
You shouldn't have to run a command to create the network prior to running docker compose, Docker should create the network if it doesn't exist. The reason you're getting this error is because you're declaring the network as external, which means that Docker expects it to already exist. If you need a new one, remove external: true
Good morning,
I am currently try to figure out how I create File shares on Ubuntu as the host OS for docker. On Windows and OSX you can set up Filesharing as below:
I require access to the File share in my docker-compose as an example see below:
version: '3.9'
services:
node_gauc:
image: node-g:v1
ports:
- "444:444" # https test port
volumes:
- ./NodeServer/cert/https.crt:/usr/share/node/cert/https.crt
- ./NodeServer/cert/key.pem:/usr/share/node/cert/key.pem
build:
context: .
dockerfile: ./NodeServer/dockerfile
restart: unless-stopped
container_name: node-g
If I don't have access when I build and start the container I get the following issues:
ERROR: for node-g Cannot start service node_g: error while creating mount source path '/usr/share/t/work/6b37be0079afed03/NodeServer/cert/https.crt': mkdir /usr/share/t: read-only file system
ERROR: for node_g Cannot start service node_g: error while creating mount source path '/usr/share/t/work/6b37be0079afed03/NodeServer/cert/https.crt': mkdir /usr/share/t: read-only file system
ERROR: Encountered errors while bringing up the project.
I am still unsure why its trying to create a directory but I suppose that is another matter.
Is it possible to create File share on a Ubuntu host server similar to what you can on OSX(Mac) or Windows OS?
Many thanks for your help
I have a simple docker-compose.yml file defined this way:
version: '3'
services:
db:
image: postgres:10.5-alpine
ports:
- 5432:5432
volumes:
- ./tmp/postgres_data:/var/lib/postgresql/data
web:
build:
context: .
dockerfile: Dockerfile
command: /bin/bash -c "rm -f /tmp/server.pid && bundle exec rails server -b 0.0.0.0 -P /tmp/server.pid"
ports:
- 3000:3000
depends_on:
- db
volumes:
- .:/app
I'm using [Kompose] (https://kubernetes.io/docs/tasks/configure-pod-container/translate-compose-kubernetes/#kompose-up) for converting my docker-compose.yml to Kubernetes
When I do
kompose convert, everything looks fine.
This is the output:
✗ kompose convert
INFO Kubernetes file "db-service.yaml" created
INFO Kubernetes file "web-service.yaml" created
INFO Kubernetes file "db-deployment.yaml" created
INFO Kubernetes file "db-claim0-persistentvolumeclaim.yaml" created
INFO Kubernetes file "web-deployment.yaml" created
INFO Kubernetes file "web-claim0-persistentvolumeclaim.yaml" created
👇🏽My issue is when I do kompose up I get the following errors👇🏽
✗ kompose up
WARN Volume mount on the host "/Users/salo/Desktop/ibm-watson-ruby/tmp/postgres_data" isn't supported - ignoring path on the host
INFO Build key detected. Attempting to build image 'web'
INFO Building image 'web' from directory 'ibm-watson-ruby'
INFO Image 'web' from directory 'ibm-watson-ruby' built successfully
INFO Push image enabled. Attempting to push image 'web'
INFO Pushing image 'library/web:latest' to registry 'docker.io'
WARN Unable to retrieve .docker/config.json authentication details. Check that 'docker login' works successfully on the command line.: Failed to read authentication from dockercfg
INFO Authentication credentials are not detected. Will try push without authentication.
INFO Attempting authentication credentials 'docker.io
ERRO Unable to push image 'library/web:latest' to registry 'docker.io'. Error: denied: requested access to the resource is denied
FATA Error while deploying application: k.Transform failed: Unable to push Docker image for service web: unable to push docker image(s). Check that `docker login` works successfully on the command line
As a note, I'm currently logged in to my Docker Hub account. I did docker login
Thanks in advance! :)
Kubernetes basically requires a Docker registry to be able to work; it cannot build local images. You mention you have a Docker Hub account, so you need to add a reference to that as the container's image: in the docker-compose.yml file.
version: '3'
services:
web:
build: .
image: myname/web # <-- add this line
ports:
- 3000:3000
depends_on:
- db
# command: is in the image
# volumes: overwrite code in the image and don't work in k8s
When Kompose tries to push the image, it will use the image: name. You should be able to separately attempt to docker-compose push which will do the same thing.
Note that I deleted the volumes: that bind-mounts your application code into your container. This setup does not work in Kubernetes: it doesn't have access to your local system and you can't predict which node will actually be running your application. It's worth double-checking in plain Docker that your built image works the way you expect, without overwriting its code. (Debugging this in Docker will be easier than debugging it in Kubernetes.)
I would like to run two docker containers via docker-compose.
The following project structure is present:
|- service
|- docker-compose
|- DockerFile
|- config.yaml
|- client
|- docker-compose
|- DockerFile
I try to run two containers via the following command:
$ docker-compose -f ./client/docker-compose.yml -f ./service/docker-compose.yml up
Everything seems to be working just fine, except for one error message:
{"action":"startup","error":"config file 'config.yaml' not found.","level":"error","msg":"could not load config","time":"2020-03-17T15:17:19Z"}
But when I navigate into each directory and run them separately everything works just fine. So it seems that the config file which is stated in the volume is somehow not found.
The docker-compose file is:
./service/docker-compose.yml
services:
thing:
command:
- --host
- 0.0.0.0
- --port
- '8080'
- --scheme
- http
- --config-file
- config.yaml
ports:
- 8080:8080
volumes:
- ./config.yaml:/config.yaml
web:
build: .
environment:
- web_host=http://my-site.com:8080
depends_on:
- anotherThing
links: thing:thing.com
version: '3.4'
The configuration file itself has some arbitrary info:
---
authentication:
my_arbitrary_key:
enabled: true
Any idea how to make sure the config file is found when running docker-compose from a sub-directory? Or am I misusing the docker-compose command...
Update
Interestingly enough if I swap the files in the docker-compose command I get it doesn't run.
So when I use:
$ docker-compose -f ./service/docker-compose.yml -f ./client/docker-compose.yml up
Docker doesn't run and the error is:
Cannot create container for service thing: invalid volume specification: '/Users/user/Site/service/config.yaml:config.yaml:rw': invalid mount config for type "bind": invalid mount path: 'config.yaml' mount path must be absolute
It's looking for config file in outer directory, you could use absolute path or put it in some .env
Best practice would be to use single docker-compose file which could have relative path of other folder
Brief background : Docker-compose is utility to run multiple docker containers - called as services, - Compose is a tool for defining and running multi-container Docker applications. With Compose, you use a YAML file to configure your application’s services. Then, with a single command, you create and start all the services from your configuration. To learn more about all the features of Compose, see the list of features.
Hence it's not necessary to have two docker-compose files over here , we should have one which will start both services also you can user docker-compose to run one service if required
Find more details at - https://docs.docker.com/compose/