How do I get Polynote running with Kubernetes? - docker

Question

I'm hoping to run Polynote and in particular against my Kubernetes cluster. Unfortunately I'm not having any luck, the error messages are not particularly helpful, and as far as I can tell it's new enough that there isn't already a reference Kubernetes configuration I can use to make this work.
With the YAML file below I'm getting it to boot up successfully. When I port forward and try to access the pod, though, it crashes the pod, which then restarts and unfortunately the error message I get is literally Killed, which isn't super instructive. I started with the bare Docker image, then added the configuration they suggested in the Docker notes in their repository.
---
apiVersion: v1
kind: ConfigMap
metadata:
name: polynote-config
namespace: dev
labels:
app: polynote
data:
config.yml: |-
listen:
host: 0.0.0.0
storage:
dir: /opt/notebooks
mounts:
examples:
dir: examples
---
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
name: polynote
namespace: dev
spec:
replicas: 1
template:
metadata:
labels:
app: polynote
spec:
containers:
- name: polynote
image: polynote/polynote:latest
resources:
limits:
memory: "100Mi"
requests:
memory: "100Mi"
ports:
- containerPort: 8192
volumeMounts:
- name: config
mountPath: /opt/config/config.yml
readOnly: true
subPath: config.yml
volumes:
- name: config
configMap:
defaultMode: 0600
name: polynote-config
Edit: For clarity, here is the entirety of the logging from the pod:
[INFO] Loading configuration from config.yml
[INFO] Loaded configuration: PolynoteConfig(Listen(8192,127.0.0.1),Storage(tmp,notebooks,Map()),List(),List(),Map(),Map(),Behavior(true,Always,List()),Security(None),UI(/))
[WARN] Polynote allows arbitrary remote code execution, which is necessary for a notebook tool to function.
While we'll try to improve safety by adding security measures, it will never be completely safe to
run Polynote on your personal computer. For example:
- It's possible that other websites you visit could use Polynote as an attack vector. Browsing the web
while running Polynote is unsafe.
- It's possible that remote attackers could use Polynote as an attack vector. Running Polynote on a
computer that's accessible from the internet is unsafe.
- Even running Polynote inside a container doesn't guarantee safety, as there will always be
privilege escalation and container escape vulnerabilities which an attacker could leverage.
Please be diligent about checking for new releases, as they could contain fixes for critical security
flaws.
Please be mindful of the security issues that Polynote causes; consult your company's security team
before running Polynote. You are solely responsible for any breach, loss, or damage caused by running
this software insecurely.
[zio-default-async-1-1076496284] INFO org.http4s.blaze.channel.nio1.NIO1SocketServerGroup - Service bound to address /127.0.0.1:8192
[zio-default-async-1-1076496284] INFO org.http4s.server.blaze.BlazeServerBuilder -
_____ _ _
| __ \ | | | |
| |__) |__ | |_ _ _ __ ___ | |_ ___
| ___/ _ \| | | | | '_ \ / _ \| __/ _ \
| | | (_) | | |_| | | | | (_) | || __/
|_| \___/|_|\__, |_| |_|\___/ \__\___|
__/ |
|___/
Server running at http://127.0.0.1:8192
[zio-default-async-1-1076496284] INFO org.http4s.server.blaze.BlazeServerBuilder - http4s v0.20.6 on blaze v0.14.6 started at http://127.0.0.1:8192/
Killed

Answer 1

The problem turned out to be a couple of things. First, the memory limit that I set was indeed too low. It needs something in the neighborhood of 2 GB of memory to boot up successfully. Second, It turns out that I hadn't mounted any storage for the notebook files.
Here's the manifest that I came up with that does work. I'm aware that the way I'm mounting storage for the notebooks is perhaps not optimal, but now that I know it's working I feel comfortable tweaking it.
---
apiVersion: v1
kind: ConfigMap
metadata:
name: polynote-config
namespace: dev
labels:
app: polynote
data:
config.yml: |-
listen:
host: 0.0.0.0
storage:
dir: /opt/notebooks
mounts:
examples:
dir: examples
---
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
name: polynote
namespace: dev
spec:
replicas: 1
template:
metadata:
labels:
app: polynote
spec:
containers:
- name: polynote
image: polynote/polynote:latest
resources:
limits:
memory: "2000Mi"
ephemeral-storage: "100Mi"
requests:
memory: "2000Mi"
ephemeral-storage: "100Mi"
ports:
- containerPort: 8192
volumeMounts:
- name: config
mountPath: /opt/config/config.yml
readOnly: true
subPath: config.yml
- name: data
mountPath: /opt/notebooks/
volumes:
- name: config
configMap:
defaultMode: 0600
name: polynote-config
- name: data
emptyDir: {}