I'm running superset on MacOS in docker and I'm trying to get OAuth working.
I’ve edited the config file /docker/pythonpath_dev/superset_config.py and added the OAuth configuration.
One of the lines I added was
AUTH_TYPE = AUTH_OAUTH
This required me to import the auth types as below:
from flask_appbuilder.security.manager import (
AUTH_OID,
AUTH_REMOTE_USER,
AUTH_DB,
AUTH_LDAP,
AUTH_OAUTH,
)
When I try to start up superset with the following command: docker-compose -f docker-compose-non-dev.yml up
I get the following error:
File "/usr/local/lib/python3.7/site-packages/flask_appbuilder/security/manager.py", line 250, in __init__
from authlib.integrations.flask_client import OAuth
ModuleNotFoundError: No module named 'authlib'
I'm fairly new to docker itself. How do I go about resolving this?
In case anybody else comes across this, the solution was to add the Authlib module to the python env on the docker image.
The process for adding a new python module to the docker image is documented here: https://github.com/apache/superset/blob/master/docker/README.md#local-packages
Quoted below in case that file changes:
If you want to add python packages in order to test things like DBs locally, you can simply add a local requirements.txt (./docker/requirements-local.txt) and rebuild your docker stack.
Steps:
1. Create ./docker/requirements-local.txt
2. Add your new packages
3. Rebuild docker-compose
a. docker-compose down -v
b. docker-compose up
Important was running docker-compose up and not docker-compose -f docker-compose-non-dev.yml up. The latter does not seem to rebuild the docker image.
Related
I am trying to push a docker image on Google Cloud Platform container registry to define a custom training job directly inside a notebook.
After having prepared the correct Dockerfile and the URI where to push the image that contains my train.py script, I try to push the image directly in a notebook cell.
The exact command I try to execute is: !docker build ./ -t $IMAGE_URI, where IMAGE_URI is the environmental variable previously defined. However I try to run this command I get the error: /bin/bash: docker: command not found. I also tried to execute it with the magic cell %%bash, importing the subprocess library and also execute the command stored in a .sh file.
Unfortunately none of the above solutions work, they all return the same command not found error with code 127.
If instead I run the command from a bash present in the Jupyterlab it works fine as expected.
Is there any workaround to make the push execute inside the jupyter notebook? I was trying to keep the whole custom training process inside the same notebook.
If you follow this guide to create a user-managed notebook from Vertex AI workbench and select Python 3, then it comes with Docker available.
So you will be able to use Docker commands such as ! docker build . inside the user-managed notebook.
Example:
I am taking a beginner course and not able to create an image on terminal.
Here is the course I am taking(Sorry, the course is in Japanese course but just for the reference). I am at 1:01:39 where he proceeds to create docker image.
First, the error message contained Failed to solve with frontend docker file.v0:failed to create LLB definition:the Dockerfile cannot be empty
I ran below command and managed to get rid of Failed to solve with frontend docker file.v0:failed to create LLB definition (Failed To Resolve With FrontEnd DockerFIle.v0):
export DOCKER_BUILDKIT=0
export COMPOSE_DOCKER_CLI_BUILD=0
Now I'm left with the Dockerfile cannot be empty
I saw in some page that people fail to locate because of not capitalized dockerfile but that's not my case.
I have also tried -f on the command to direct file
I saw that you, don't save your dockerfile, try save changes and run docker build again.
Also saw same issue on github, it could be helpful
Try to change COPY to ADD
https://github.com/docker/compose/issues/5170
Firstly every step is done seemingly successfully (not any error reported). But per what I understand about how to check if the new config is applied OK, it seems to be failed updating the config.
Suppose I have a config file with a simple content like this:
Well done
I created a config (the first version) like this:
echo 'Well done' | docker config create my-config -
Now I have a local file named my-config.txt (on the host machine) with content as described above, it's used as a template (source) to clone over the target on the Docker container. On the Docker container, there is already a config file with the same content (originally). Now I change the content of the file my-config.txt (on the host machine) to something like this:
Well done !!!
And next I update the current docker service (created before) by using docker service update to apply the new config file, like this:
//firstly create another version of config
docker config create my-config-2 /home/my_user/my-config.txt
docker service update \
--config-add source=my-config-2,target=my-config.txt \
--config-rm my-config \
my-service
As I said, it seems to execute successfully. But when I try opening the my-config.txt file on the Docker container, its content is kept unchanged, like this:
docker exec [container_id] cat my-config.txt
It still shows Well done whereas the expected content should be Well done !!!. Isn't that what it should be? Am I doing something wrong here? Or could you suggest something to diagnose this issue or something different from what I've done without having to trying to solve this issue?
I have been trying http://predictionio.apache.org/install/install-docker/ this tutorial. I have successfully built Docker image however when I try to run docker run i get the Can't open /etc/predictionio/pio-env.sh error.
docker build -t predictionio/pio pio
docker run -ti predictionio/pio
PS: If I comment out the last line CMD ["sh", "/usr/bin/pio_run"] I can build and run docker image successfully. I can open the file too from docker bash.
I think you need to grant permissions to execute this file. add the following line at the end of your Dockerfile
RUN chmod +x pio_run.sh
also, you might need to change CMD to ENTRYPOINT like following:
ENTRYPOINT ["sh","/usr/bin/pio_run.sh"]
Your output states you are running Windows. Did you use the default command prompt or did you use docker terminal? I had the same error messages in the past on Windows but mysteriously it disappeared after trying the tutorial again. I am not sure what I did different except I might possibly used docker instead of the default command prompt...
Could you also try using docker-compose instead of plain docker commands as described in the tutorial?
Ensure your storage (Postgres, MySQL or ElasticSearch) is running before starting PIO as well.
Just resolved it on my machine.
When you cloned repository on Windows, git converted end of line symbols from Unix-style (\n) to Windows style (\r\n).
You need to open file C:\wherever-you-cloned-pio-repository\predictionio\docker\pio\pio_run and change it back (for e.g. using Visual Studio Code, or Notepad++). Then you need to rebuild the image and it should work.
Also for the future you may want to disable automatic conversion Disable git EOL Conversions
I am trying to customize tuna-app chaincode of the tuna-app example. I want to use cid package inside my chaincode to make ABAC decisions about who is allowed to run the chaincode. When I try to install chaincode, I get the following error:
Error: Error getting chaincode code chaincode:
Error getting chaincode package bytes: Error obtaining dependencies for github.com/hyperledger/fabric/core/chaincode/lib/cid:
<go, [list -f {{ join .Deps "\n"}} github.com/hyperledger/fabric/core/chaincode/lib/cid]>: failed with error: "exit status 1"
cannot load package: package github.com/hyperledger/fabric/core/chaincode/lib/cid: cannot find package "github.com/hyperledger/fabric/core/chaincode/lib/cid" in any of:
/opt/go/src/github.com/hyperledger/fabric/core/chaincode/lib/cid (from $GOROOT)
/opt/gopath/src/github.com/hyperledger/fabric/core/chaincode/lib/cid (from $GOPATH)
I am usind Docker to run peer, orderer, ca, and cli containers. The Docker image which is used to build chaincode is hyperledger/fabric-ccenv. This image is created using Dockerfile; the interesting line I found was:
ADD payload/goshim.tar.bz2 $GOPATH/src/
which adds the tar.bz2 inside the $GOPATH/src folder (I believe). The .tar.bz2 file contains all Go packages used by chaincode. I tried to insert the cid package and to create a new .tar.bz2 file with the package inside. Then I rebuilt the image. The image now contains the cid package, but I still get the same error.
Why is it still missing the package?
In the startFabric.sh from your tuna-app, you launch the cli container using:
docker-compose -f ./docker-compose.yml up -d cli
Have a look at the mounting declaration of the persistent volumes in your compose yaml file. You should see something like this because the tuna-app is based on fabcar from the fabric-samples:
./../chaincode/:/opt/gopath/src/github.com/
If you see this declaration, copy in your local machine the folder /hyperledger/fabric/core/chaincode/lib/cid into your chaincode folder. You should find it in chaincode/abac if you are using the last version of fabric samples (https://github.com/hyperledger/fabric-samples).
I think you should not create a new goshim.tar.bz2. If you think it is easier make sure cid is in the correct path within the archive, e.g. github.com/hyperledger/fabric/core/chaincode/lib/cid
To test this you can make a debug output:
ADD payload/goshim.tar.bz2 $GOPATH/src/
RUN ls $GOPATH/src/github.com/hyperledger/fabric/core/chaincode/lib/cid
I would recommend to download cid within the Dockerfile:
RUN go get -d github.com/hyperledger/fabric/core/chaincode/lib/cid