Plotly shows blank graphs in AWS Sagemaker JupyterLab - machine-learning

Background: I am new to the Python world and am using Plotly for creating basic graphs in Python. I am using AWS Sagemaker's JupyterLab for creating the python scripts.
Issue: I have been trying to run the basic codes mentioned on Plotly's website however even those are returning blank graphs.
Issue Resolution Tried by myself:
pip installed plotly version 4.6.0
Steps mentioned on https://plotly.com/python/getting-started/ for JupyterLab support have already been executed
Code Example:
import plotly.graph_objects as go
fig = go.Figure(data=go.Bar(y=[2, 3, 1]))
fig.show()

I recently had the same issue. Simple change suggested here helped me. I know this is a temporary workaround until a proper fix is found.
// fig = go.Figure()
fig = go.FigureWidget() // replace with this
// fig.show()
fig // remove .show()

Sagemaker notebook instances are using (As of Jan 2022), for some reason, jupyterlab==1.2.21. You can verify that by running pip freeze | grep lab from the terminal or !pip freeze | grep lab from a notebook.
According to the documentation, you'll need to install the following jupyterlab extensions (which are not needed if sagemaker was running jupyterlab 3):
jupyterlab-plotly
jupyter-widgets/jupyterlab-manager
You can install those on a up-and-running instance by running
jupyter labextension install jupyterlab-plotly#5.5.0 #jupyter-widgets/jupyterlab-manager in the terminal or notebook (using ! if you are running on the notebook ofcourse). Notice that the jupyterlab-plotly extension version (here 5.5.0) should match the plotly version you are installing. Mismatches my cause issues. In this case by plotly version is 5.5.0 and thus that's also the jupyterlab-plotly version I've installed.
If you need, like I did, to have it ready upon spinning up a notebook instance, you'll need to:
Create a lifecycle script
To it, add:
PATH=$PATH:/home/ec2-user/anaconda3/envs/JupyterSystemEnv/bin - To ensure nodejs path which is needed for the extension installation
pip install plotly==5.5.0 - To ensure a specific version
jupyter labextension install jupyterlab-plotly#5.5.0 #jupyter-widgets/jupyterlab-manager - To ensure same version
of coures, you can change the version according to the most up to date.

I think that documentation is not on par. You now need to install jupyterlab-plotly extension.
jupyter labextension install jupyterlab-plotly
UPDATE
I followed a mix of instructions here and here.
First Enable Extention manager from jupyter-lab
then from terminal
conda install -c conda-forge "nbformat" "ipywidgets>=7.5" -y
jupyter labextension install jupyterlab-plotly
jupyter labextension install #jupyter-widgets/jupyterlab-manager plotlywidget
And within your environment
conda install nbformat

Related

Drake Mathematical Program Tutorial

I am running Drake on Ubuntu 20.04 using WSL2.
I use python3.8.10 and Drake1.2.0.
I tried running the "Mathematical Program Tutorial" obtained from deepnote on my PC, but the behavior of the ipopt solver is unnatural and does not give the expected results.
The 1st error is occurred in the section using ipopt solver.
All components of the solution is printed as "nan"
The 2nd error is below about "get_solver_details().status"
RuntimeError: The solver_details has not been set yet.
I can see both errors in "Demo on manually choosing a solver" in the tutorial.
The result is following
SolutionResult.kUnknownError
x* = [nan nan]
Solver is IPOPT
---------------------------------------------------------------------------
RuntimeError Traceback (most recent call last)
<ipython-input-12-2d1b3835c54a> in <module>
25 print("x* = ", result.GetSolution(x))
26 print("Solver is ", result.get_solver_id().name())
---> 27 print("Ipopt solver status: ", result.get_solver_details().status,
28 ", meaning ", result.get_solver_details().ConvertStatusToString())
RuntimeError: The solver_details has not been set yet.
Thank you in advance.
P.S.
I installed pydrake for venv by pip commands
python3 -m venv env
env/bin/pip install --upgrade pip
env/bin/pip install drake
sudo apt-get install --no-install-recommends \
libpython3.8 libx11-6 libsm6 libxt6 libglib2.0-0
source env/bin/activate
I just download the folder "Tutorial" from deepnote and put it under env.
Then, I run it by Jupyter Notebook as
jupyter notebook
and open env/Tutorials/mathematical_program.ipynb
It turns out that the pip drake == 1.2.0 version has a bug in the IpoptSolver compilation.
As a work-around, you can use SnoptSolver instead, or else use the https://drake.mit.edu/from_binary.html release (unpacking a zipped binary, instead of using pip).
It's possible that the pydrake.solvers.ipopt.IpoptSolver class (which is a wrapper around the https://coin-or.github.io/Ipopt/ library) does not run correctly under WSL2, due to using some odd libc API which doesn't work on Windows. We will need more information to reproduce the problem and try to debug.
Can you state exactly how you installed pydrake (i.e., show us the command lines you used). Was it via pip (https://drake.mit.edu/pip.html) or just via binary (https://drake.mit.edu/from_binary.html)?
Can you state exactly how you ran Jupyter (the command line) to launch the notebook? Was it python3 -m pydrake.tutorials or something else?
Looks like this may not be tied to WSL, but instead pip build (or just binary build). Ran into this on Ubuntu 20.04 (no WSL). Per Drake Slack, filed issue:
https://github.com/RobotLocomotion/drake/issues/17162

Mamba installing a package into wrong environment

The background is, I'm responsible for maintaining a fancy Docker image that is used by our team for analytics. It uses a Jupyter notebook image as the base, and then adds various customisations, extra packages, etc.
One of the team members recently wanted to run Tensorflow. No problem, I'll just run mamba install and add it to the image. However, this created an issue: Tensorflow 2.4.3 (the latest version) is somehow incompatible with R 4.1.1 (also the latest version) or something else in the ecosystem, causing R to to be downgraded to 3.6.3. So I created a new environment and installed TF into that:
FROM hongooi/jupytermodelrisk:1.0.0
RUN mamba create -n tensorflow --clone base
# Make RUN commands use the new environment
RUN echo "conda activate tensorflow" >> ~/.bashrc
SHELL ["/bin/bash", "--login", "-c"]
RUN mamba install -y 'tensorflow=2.4.3'
But when I rebuilt the image, I found that while the tensorflow env had been created, the Tensorflow package had been installed into the base env, not the tensorflow env. Has anyone else encountered this? I can verify, if I login to the container, that the tensorflow env has been created: it just doesn't contain the Tensorflow package.
I don't get this problem if run the create, activate and install commands from inside the container. It's only when I try to do it in the Dockerfile.
I use mamba instead of conda because the latter takes forever to run, given the number of packages installed. In fact, trying to run conda install tensorflow crashes after ~5 hours.
Not an expert on dockerfiles, but in general you could just use the -n flag to the install command to specify the target environment for the installation like so:
mamba install -n tensorflow -y tensorflow=2.4.3

How to install VS Code extensions in a Dockerfile?

Is there a way to install VS Code extensions in a Dockerfile?
Apparently, while most browser-based VS Code forks (including openvscode-server) do not permit headless installation of VS Code extensions (as seen from my other answer), it is possible to do such automated installs using docker build in one of them: Code Server (code-server), like in this sample Dockerfile:
FROM ubuntu:22.04
RUN apt update && apt install -y curl
# install VS Code (code-server)
RUN curl -fsSL https://code-server.dev/install.sh | sh
# install VS Code extensions
RUN code-server --install-extension redhat.vscode-yaml \
--install-extension ms-python.python
Relevant fragment of the docker build log:
[..]
---> Running in 59eea050a2db
[2022-11-13T10:13:58.762Z] info Wrote default config file to ~/.config/code-server/config.yaml
Installing extensions...
Installing extension 'redhat.vscode-yaml'...
Installing extension 'ms-python.python'...
Extension 'redhat.vscode-yaml' v1.10.1 was successfully installed.
Extension 'ms-python.python' v2022.16.1 was successfully installed.
[..]
Per the VS Code Documentation, the extensions property is specific to VS Code and can only be configured using .devcontainer.
The best you can do is if the extension has a CLI, you can install that. For example,
RUN npm install prettier -D --save-exact
Then use npx:
npx prettier --check .
This is sadly disallowed by design, as confirmed by this error message you will see in your docker build log when you attempt to run code --install-extension or openvscode-server --install-extension:
Command is only available in WSL or inside a Visual Studio Code terminal.
This is also confirmed (and tagged) as being as designed by one of VS Code Remote devs in this GitHub issue:
This is correct the 'vs code server CLI' is only available from the integrated terminal.
So VS Code Remote or even openvscode-server make it impossible to automate installs of first- or third-party extensions for this popular Microsoft IDE, unless you run their custom terminal inside their closed-source IDE, which normally entails buying a license for their GUI-based closed-source operating system as well;)

install %%R cell magic in docker from jupyter docker stack

I tried installing the datascience jupyter docker image (tag 45b8529a6bfc, last update Feb 14, 2019) from docker stacks. My entire dockerfile:
FROM jupyter/datascience-notebook:45b8529a6bfc
USER $NB_UID
When I open a new Jupyter notebook with an R kernel, the notebook works fine. When I try a %%R cell magic in an ipython notebook, it doesn't work:
%%R
3+4
UsageError: Cell magic `%%R` not found.
I wandered around various stackoverflow answers and internet searches, tried installing rpy2 (it was already installed). Didn't work.
Suggestions?
Load the jupyter extension before you try to use it:
%load_ext rpy2.ipython
I tried %load_ext rpy2.ipython as suggested by #lgautier, and got the error message No module named 'simplegeneric'. Once I pip installed simplegeneric, everything works and I don't need the load_ext statement.
Not sure why the dockerfile doesn't install simplegeneric, but there you have it.

How to install YugaByte on Docker for Windows

The instructions at https://docs.yugabyte.com/latest/quick-start/docker/install/ state that Docker for Windows is supported, however the yb-docker-ctl utility in the step that follows seems to be a *nix app and does not run on Windows 10 Pro. How do I install a 3-node local YugaByte cluster on Docker for Windows? (by the way StackOverflow would not let me add a YugaByte tag to the question, I could only add Docker)
The yb-docker-ctl utility is actually a Python2 script that will run on Windows 10 Pro if you have Python2 installed. I prefer to use Chocolately (https://chocolatey.org) to manage my package installations, so you could install python2 (not python -- as that will default to python3) using choco install python2 from PowerShell or CMD. You can also install wget in the same manner.
You will then need to a couple of changes to yb-docker-ctl. The script utilizes os.path.join which will utilize the Windows default of \\ for path separator. Add the line import posixpath after line 10 of yb-docker-ctl and substitute posixpath.join for os.path.join at lines 227 and 377.
After you have made those modifications, you can run python yb-docker-ctl create to create your 3 node cluster.

Resources