How to work on the RWTH Cluster

Note

Check out the Video tutorial. This is especially useful for all Windows users!

Prerequisites

Helpful pointers

Way 1: Using your web-browser

Important

You need to be in the network of RWTH. Use VPN if necessary.

Login

In your browser, open

https://login23-x-1.hpc.itc.rwth-aachen.de:3300/auth/ssh/

and log into the cluster.

  1. Create a session, e.g. MATE (us)

  1. (optional) You may want to download the Desktop Clients instead of using your browser.

Installation

  1. Open a terminal
  2. Download the Container (Apptainer) of our course, e.g.
mkdir CMM
cd CMM
apptainer pull oras://registry.git.rwth-aachen.de/mbd/courses/containers/dolfinx:latest
ls

Working with Jupyter notebooks

  1. Open a terminal and navigate to your CMM folder.
  2. Open firefox
  3. Start a Jupyter server with
apptainer run dolfinx_latest.sif notebook
  1. Click the link or copy it to your browser

Working with Python scripts

  1. Open a terminal and navigate to your CMM folder.

  2. Run your python script

    apptainer run dolfinx_latest.sif test.py

Submitting batch jobs to the cluster

T.b.d. (Currently not supported)

Visualization with Paraview

  1. Open a terminal and navigate to your CMM folder.

  2. Load the required module to start Paraview …

    module purge; 
    module load GCC/12.3.0; 
    module load OpenMPI/4.1.5; 
    module load ParaView/5.11.2

    … and open Paraview

    paraview
  3. Open output .pvd output file that you created with FenicsX.

Way 2: Using VS-Code

  1. Install the ‘Remote - SSH’ Extension

  1. Open a remove window (bottom left green icon)
  2. Select ‘Connect to Host’

  1. Login to a login node, e.g.

    login23-3.hpc.itc.rwth-aachen.de

  1. Check that you are connected to the cluster

  2. Download the Apptainer in the VS-Code terminal:

    mkdir CMM
    cd CMM
    apptainer pull oras://registry.git.rwth-aachen.de/mbd/courses/containers/dolfinx:latest
    ls
  3. Select the CMM folder as your working directory

  1. In the terminal, start the Jupyter server and copy the link to the Jupyter server
apptainer run dolfinx_latest.sif notebook
  1. Click ‘Select Kernel’

  2. Click ‘Existing Jupyter Server’, paste the link to the Jupyter server, change the display name (e.g. Dolfinx)

  1. Select the ‘Python 3 (ipykernel)’ /opt/conda/envs/fenicsx-env/bin/python.

Done.

Way 3: Local installation

If you want do develop locally, feel free to do so. Working on a local VSCode instance can provide extra convenience, such as access to better linting / python autocompletion and other VSCode extensions.

There are however two things to keep in mind:

1. You are responsible to set things up

  1. Your submission needs run run on the environment we provide.
Important

Make sure that your solution works with the official container on the cluster before you submit your homework!

Using a development container

Tested only on Ubuntu / Linux and Windows 11, it probably also work on Mac but no guarantee.

We provide an “unoffical” VSCode development container to make it as easy as possible to get started with your local VSCode installation.

To try it out, follow these instructions:

  1. Set up docker and VSCode for use with the development container following the official guide

  2. Create a working directory for your VSCode project, for example cmm_notebooks. Then, download the development container configuration .devcontainer and extract it into this folder.

mkdir cmm_notebooks
cd cmm_notebooks
wget https://mbd.pages.rwth-aachen.de/courses/cmm/content/exercises/notebooks/intro.out.ipynb
wget https://mbd.pages.rwth-aachen.de/courses/cmm/devcontainer.tar.gz
tar -xvzf devcontainer.tar.gz
rm devcontainer.tar.gz
  1. Open the folder cmm_notebooks in VSCode. A notification should pop up asking you to open the folder in the development container: vs code dev container popup Alternatively, press CTRL + Shift + P and enter Dev Containers: Rebuild and Reopen in Container.

  2. Download and copy the example notebook in your cmm_notebooks folder. Open it in VSCode. The first time you open it you might have to select the Python kernel (top right). Select the kernel from /dolfinx-env/bin/python (first in the list with a star icon)

selecting dolfinx kernel in jupyter notebook
  1. Click run all and it should work. Sometimes, the first “Run all” for some reason randomly skips the first cell so just try at least twice.

Install Dolfinx

We generally would recommend to try with the development container first(see previous section). If you really want to build all ingredients from scratch, you can find the source at:

https://fenicsproject.org/download/

Run your first example program

After you run this section, your folder structure should look like this:

  1. Download our example program in your CMM folder
wget https://mbd.pages.rwth-aachen.de/courses/cmm/content/exercises/notebooks/intro.out.ipynb
  1. Download necessary libraries in to the CMM folder
wget https://mbd.pages.rwth-aachen.de/courses/cmm/content/exercises/notebooks/library.tar.gz
tar -xvzf library.tar.gz
  1. Start a Jupyter server with FastX or VS-Code
  2. In the notebook, select run all.
  3. Use Paraview to visualize the output_poisson/simulation.vtu file.

If all this works without issues, you are set up for this course.

Data transfer from your local machine to the cluster and back

FastX/General

This is what the RWTH IT Center recommends: FastX

In addition to that, we personally recommend using filezilla.

VS-Code

VS-Code allows to drag and drop files.