JupyterHub Remote Spawn

In some use-cases users might want to execute their Jupyter notebooks outside of the JupyterHub cluster but still maintain the authentication, single point of entry and other features provided JupyterHub.

This repo implements a proxy container which allows users to spawn their Jupyter server on any machine which can connect on TCP port 443 to the JupyterHub cluster.

Idea

• Spawn Jupyter sessions on a machine outside of the JupyterHub cluster
• Access and share a Jupyter sessions via public accessible URL of the JupyterHub cluster
• Execute the Jupyter session on a machine your data resides (e.g. HPC frontend node)
• Mount your JupyterHub home on that machine to exchange data between your machine and the JupyterHub cluster

Involved Components

• SSH client

• (Remote) Machine

  • Python scripts:
    • rwth-jupyter
    • jupyterhub-singleuser
  • Chisel tunnel client
  • Fuse SSHFS mount

• JupyterHub Proxy Container

  • Chisel tunnel server
  • sftp-server

• JupyterHub Token Reflector Service

Flow

User

1. Get a remote shell on the system where the Jupyter session should run
2. Installs rwth-juptyer script via Pythons pip command.
3. Execute rwth-jupyter script and opens link as instructed
4. Authenticate at JupyterHub
5. Get redirected to Jupyter notebook session

Internal

sequenceDiagram
	autonumber
	participant Shell
	User->>Shell: logs in
	note over Shell: install & run<br>rwth-jupyter
	Shell->>User: present link to<br>Token Reflector
	User->>Token Reflector: visits
	note over User,JupyterHub: OAuth2 flow
	Shell->>Token Reflector: get token
	note over Shell: store token
	Shell->>JupyterHub: request start of Proxy Container
	JupyterHub->>Proxy Container: spawns
	note over Proxy Container: start Chisel server
	note over Proxy Container: start SFTP server
	Shell->>Proxy Container: establishes tunnel connection
	note over Shell: start Juptyer server
	note over Shell: mount SSHFS
	Shell->>User: present link to Proxy Container
	User->>Proxy Container: visit Jupyter notebook via public URL

1. The user gets login shell on the remote system where the Jupyter session should run.
   • The user then installs and executes the rwth-jupyter Python script
2. The script presents a link to the Token Reflector service to the user. This link includes a random nonce which allows the script to fetch a JupyterHub API token as soon as the user completed authentication
3. The user visits the link and gets redirected to the Token Reflector service
   • The Token Reflector service then performs a normal OAuth2 flow of the user against the JupyterHub
4. After completion of the authentication, the rwth-jupyter script retrieves a new API token from the service using the random nonce.
   • This API token is stored on the system for later use
   • This relieves the user from repeating the authentication flow
5. The rwth-jupyter script requests the spawn of a new Proxy Container via the JupyterHub API.
6. The JupyterHub spawns the requested conainter for the user
   • A Chisel tunnel server is started as part of the container
   • A SFTP server is started as part of the container
7. The rwth-jupyter scripts use the Chisel client to establish a tunnel to the newly spawned Chisel server running inside the Proxy Container.
   • After the successful tunnel establishment, the rwth-jupyter script
   • Starts a Jupyter server session on the remote machine
   • Mounts the JupyterHub home directory via SSHFS through the tunnel
8. The rwth-jupyter scripts presents a link for accessing the Juptyer server on the remote machine.
   • This link points to the public JupyterHub cluster
   • The JupyterHub cluster directs the requests to the ProxyContainer
   • Which in-turn proxies the requests via the tunnel to the Jupyter server running on the remote system

Contents

• image: The Dockerfile for building the Proxy Container
• python: The code of the rwth-jupyter script

Links

• https://github.com/libfuse/sshfs
• https://github.com/jpillora/chisel
• https://man7.org/linux/man-pages/man8/sftp-server.8.html