Update Installation and Testing authored by Beck, Lauritz's avatar Beck, Lauritz
[[_TOC_]]
* [Introduction](#introduction)
* [Installation](#installation)
* [Windows](#windows)
* [Linux](#linux)
* [Mac OS X](#mac-os-x)
* [Testing the Installation](#testing-the-installation)
* [Installing Jupyter](#installing-jupyter)
* [Notes on Third-party libraries](#notes-on-third-party-libraries)
## Introduction
**Note that you also need a working installation if you want to compile BoSSS
from the C# sources yourself.
BoSSS requires a set of prerequisites
(e.g. Microsoft .NET or Mono, MPI, certain _native libraries_)
on a machine, which are set up by the installation
procedure outlined below.**
We briefly outline the steps required to get started with using and/or developing BoSSS applications.
**In any case**, the [installation](#installation) of the pre-compiled version of BoSSS should be the first step.
Upon finishing this step, you can either
use [BoSSSpad](./BoSSSpad) to run pre-compiled applications as well as -- or --
you can start [working with the source code](./Working-with-BoSSS-as-a-Developer).
**Note that you also need a working installation if you want to compile BoSSS from the C# sources yourself. BoSSS requires a set of prerequisites (e.g. Microsoft .NET or Mono, MPI, certain _native libraries_) on a machine, which are set up by the installation procedure outlined below.**
We briefly outline the steps required to get started with using and/or developing BoSSS applications. **In any case**, the [installation](#installation) of the pre-compiled version of BoSSS should be the first step. Upon finishing this step, you can either use [BoSSSpad](/bosss1/experimental/-/wikis/Getting-Started/BoSSSpad) to run pre-compiled applications as well as -- or -- you can start [working with the source code](/bosss1/experimental/-/wikis/Getting-Started/Working-with-BoSSS-as-a-Developer).
## Installation
The advantage of C#, resp. .NET is that _the executable is platform-independent_.
This implies e.g. that an executable compiled on a Windows/Mac laptop
can be copied to a Linux/Unix HPC system and executed there without re-compilation.
As discussed in the [Core Concepts](./Core-concepts#c-Sharp,-.net-and-mono),
this obviously requires some _middleware_ and some libraries to be present on the target computer. In the following, we will thus summarize some best practices for the installation of the [pre-compiled BoSSS binaries](#notes-on-third-party-libraries) on different operating systems.
The advantage of C#, resp. .NET is that _the executable is platform-independent_. This implies e.g. that an executable compiled on a Windows/Mac laptop can be copied to a Linux/Unix HPC system and executed there without re-compilation. As discussed in the [Core Concepts](/bosss1/experimental/-/wikis/Getting-Started/Core-concepts#c-Sharp,-.net-and-mono), this obviously requires some _middleware_ and some libraries to be present on the target computer. In the following, we will thus summarize some best practices for the installation of the [pre-compiled BoSSS binaries](#notes-on-third-party-libraries) on different operating systems.
### Windows
To install on Windows, just o execute the BoSSS Windows installer.
Afterwards, you can [test the installation](#testing-the-installation).
To install on Windows, just execute the BoSSS Windows installer. Afterwards, you can [test the installation](#testing-the-installation).
The BoSSS installer already includes all required native libraries and prerequisites. These are:
1. Microsoft Visual C++ Redistributable (x64)
2. [Microsoft MPI](https://msdn.microsoft.com/en-us/library/bb524831(v=vs.85).aspx). Check the MPI installation by executing the command `mpiexec` in a console window
3. <del>The [.NET framework](https://www.microsoft.com/net/download/framework) is installed
The BoSSS Installer checks whether these components are in place and installs them, if required.
In the case of some problem, one can also download and install these items manually.</del>
3. ~~The~~ [~~.NET framework~~](https://www.microsoft.com/net/download/framework) ~~is installed The BoSSS Installer checks whether these components are in place and installs them, if required. In the case of some problem, one can also download and install these items manually.~~
### Linux
The BoSSS group maintains basic packages of the
[native libraries](#notes-on-third-party-libraries)
listed in for Linux.
These are compiled against recent versions of the Open MPI and the Intel MKL library and should provide compatibility and decent performance on a variety of systems.
However, compatibility this can not guaranteed for any Linux distribution -- binary platform independence in Linux not possible in general; since we use Ubuntu, all Debian derivatives might most likely work. Note further, high-performance computers typically provide specifically optimized versions of BLAS and LAPACK, so for the ultimate performance on a certain system, you&#39;ll need to build your own version of the BoSSS native libraries.
The BoSSS group maintains basic packages of the [native libraries](#notes-on-third-party-libraries) listed in for Linux. These are compiled against recent versions of the Open MPI and the Intel MKL library and should provide compatibility and decent performance on a variety of systems. However, compatibility this can not guaranteed for any Linux distribution -- binary platform independence in Linux not possible in general; since we use Ubuntu, all Debian derivatives might most likely work. Note further, high-performance computers typically provide specifically optimized versions of BLAS and LAPACK, so for the ultimate performance on a certain system, you'll need to build your own version of the BoSSS native libraries.
The steps for installing BoSSS on Linux are:
1. Extensive notes on the installation of .NET [here](./NET5-Upgrade#installation-of-net-5-in-linux);
In short:
Download and install [.NET5 SDK](https://dotnet.microsoft.com/download/dotnet/5.0),
see also https://docs.microsoft.com/de-de/dotnet/core/install/linux for the
most recent versions and best practices for your system.
<del> Obtain [Mono](https://fdy-tuda.openproject.com/www.mono-project.com) (version 6.0 or higher), either by installing a [pre-built package](https://fdy-tuda.openproject.com/www.mono-project.com/download/) or by [compiling](https://fdy-tuda.openproject.com/www.mono-project.com/docs/compiling-mono/linux/) it from a [tarball](https://download.mono-project.com/sources/mono/). </del>
2. Obtain/install/load OpenMPI
3. Obtain/install/load native libraries `libBoSSSnative_seq.so`, `libBoSSSnative_omp.so` and `libBoSSSnative_mpi.so`
* Pre-compiled versions of these shared libraries for Linux can be found in the Windows installation, in the directory `$BOSSS_INSTALL/bin/native/linux/amd64-openmpi`. (There should also be a `zip`\-File that you can unpack on any system.)
* If these pre-build libraries are not compatible with your system
(we do our best, but binary compatibility across distributions is quite difficult on Linux),
you will need to compile them yourself,
the source code can be found at https://github.com/FDYdarmstadt/BoSSS-native, resp. https://git.rwth-aachen.de/kummer/BoSSS-native
1. Extensive notes on the installation of .NET [here](/bosss1/experimental/-/wikis/Getting-Started/NET5-Upgrade#installation-of-net-5-in-linux); In short: Download and install [.NET5 SDK](https://dotnet.microsoft.com/download/dotnet/5.0), see also <https://docs.microsoft.com/de-de/dotnet/core/install/linux> for the most recent versions and best practices for your system. ~~Obtain~~ [~~Mono~~](https://fdy-tuda.openproject.com/www.mono-project.com) ~~(version 6.0 or higher), either by installing a~~ [~~pre-built package~~](https://fdy-tuda.openproject.com/www.mono-project.com/download/) ~~or by~~ [~~compiling~~](https://fdy-tuda.openproject.com/www.mono-project.com/docs/compiling-mono/linux/) ~~it from a~~ [~~tarball~~](https://download.mono-project.com/sources/mono/)~~.~~
2. Obtain/install/load OpenMPI
3. Obtain/install/load native libraries `libBoSSSnative_seq.so`, `libBoSSSnative_omp.so` and `libBoSSSnative_mpi.so` \* Pre-compiled versions of these shared libraries for Linux can be found in the Windows installation, in the directory `$BOSSS_INSTALL/bin/native/linux/amd64-openmpi`. (There should also be a `zip`-File that you can unpack on any system.) \* If these pre-build libraries are not compatible with your system (we do our best, but binary compatibility across distributions is quite difficult on Linux), you will need to compile them yourself, the source code can be found at <https://github.com/FDYdarmstadt/BoSSS-native>, resp. <https://git.rwth-aachen.de/kummer/BoSSS-native>
### Mac OS X
Since Mac OS X is a Unix system, most things should work similar to Linux.
There has been, however, no interest so far for running BoSSS on Apple Computers,
therefore we do not provide official guidelines.
Since Mac OS X is a Unix system, most things should work similar to Linux. There has been, however, no interest so far for running BoSSS on Apple Computers, therefore we do not provide official guidelines.
The first step would be to compile the [Native Libraries](#notes-on-third-party-libraries).
For this, we recommend that users should try either Mac Ports (https://www.macports.org/) or Homebrew (https://brew.sh/) to have a system which close to Linux (i.e. has `gcc`, `gfortran`, etc.). Using only Mac OS tools might be even harder.
After you have compiled the library, you can proceed analogously to the procedure under Linux.
The first step would be to compile the [Native Libraries](#notes-on-third-party-libraries). For this, we recommend that users should try either Mac Ports (<https://www.macports.org/>) or Homebrew (<https://brew.sh/>) to have a system which close to Linux (i.e. has `gcc`, `gfortran`, etc.). Using only Mac OS tools might be even harder. After you have compiled the library, you can proceed analogously to the procedure under Linux.
## Testing the Installation
Execute, after the installation the command
```code
```plaintext
dotnet BoSSSpad.dll --check
```
or
```code
```plaintext
BoSSSpad.exe --check
```
to verify that the installation works and the native libraries are
in place.
Furthermore, check that the environment variable `$BOSSS_INSTALL`
is defined.
The native libraries should be in the following location
to verify that the installation works and the native libraries are in place. Furthermore, check that the environment variable `$BOSSS_INSTALL` is defined. The native libraries should be in the following location
* On Windows: `$BOSSS_INSTALL/bin/native/win/amd64`
* On Linux: `$BOSSS_INSTALL/bin/native/linux/amd64-openmpi`
## Installing Jupyter
In order to use BoSSS properly,
a working installation of [**Jupyter**](https://jupyter.org/)
(aka. Jupiterlab, Jupyter-Notebkook, etc.)
**with support for [dotnet-interactive](https://github.com/dotnet/interactive)**
is required.
Prior to the migration to .NET5, BoSSS used its own custom notebook format (`bws`, for BoSSS work sheet) and featured a notebook/worksheet editor embedded in [BoSSSpad](./BoSSSpad).
Since Mid-2020, [dotnet-interactive](https://github.com/dotnet/interactive)
is supported by Jupyter and
reached a level
of maturity so that it can replace the custom BoSSSpad solution.
Although a dependency on an additional software component is a drawback
(e.g. the user has to install and configure it)
the advantages of a widely-used of-the-shelf solution with a large user-basis
such as Jupyter
outweigh the disadvantages.
In order to use BoSSS properly, a working installation of [**Jupyter**](https://jupyter.org/) (aka. Jupiterlab, Jupyter-Notebkook, etc.) **with support for** [**dotnet-interactive**](https://github.com/dotnet/interactive) is required. Prior to the migration to .NET5, BoSSS used its own custom notebook format (`bws`, for BoSSS work sheet) and featured a notebook/worksheet editor embedded in [BoSSSpad](/bosss1/experimental/-/wikis/Getting-Started/BoSSSpad). Since Mid-2020, [dotnet-interactive](https://github.com/dotnet/interactive) is supported by Jupyter and reached a level of maturity so that it can replace the custom BoSSSpad solution. Although a dependency on an additional software component is a drawback (e.g. the user has to install and configure it) the advantages of a widely-used of-the-shelf solution with a large user-basis such as Jupyter outweigh the disadvantages.
Provided that Python is already installed, one can install Jupyter by
```code
```plaintext
pip install jupyterlab
```
resp.
```code
```plaintext
pip3 install jupyterlab
```
probably best as `root` or with administrator rights.
This works in similar for Linux and Windows.
probably best as `root` or with administrator rights. This works in similar for Linux and Windows.
* Further details can be found in the [Original Documentation](https://jupyter.org/install).
* An alternative to the manual installation is the use of the
semi-commercial [Anaconda Python distribution](https://www.anaconda.com/products/individual)
Once Jupyter is installed, it can be started/tested by launching a web browser (e.g. Firefox)
and running either `jupyter-lab` or `jupyter-notebook` from a terminal.
The GUI should then open in the browser.
For a first install, only Python notebooks are available in Jupyter. The Support for C#,
resp. `dotnet-interactive` must be added.
(See also in the [Original Documentation](https://github.com/dotnet/interactive/blob/main/docs/NotebooksLocalExperience.md).)
1. The available kernels can be listed by
`jupyter kernelspec list` in a python-aware terminal, e.g. Anaconda Prompt:
```code
* An alternative to the manual installation is the use of the semi-commercial [Anaconda Python distribution](https://www.anaconda.com/products/individual)
Once Jupyter is installed, it can be started/tested by launching a web browser (e.g. Firefox) and running either `jupyter-lab` or `jupyter-notebook` from a terminal. The GUI should then open in the browser.
For a first install, only Python notebooks are available in Jupyter. The Support for C#, resp. `dotnet-interactive` must be added. (See also in the [Original Documentation](https://github.com/dotnet/interactive/blob/main/docs/NotebooksLocalExperience.md).)
1. The available kernels can be listed by `jupyter kernelspec list` in a python-aware terminal, e.g. Anaconda Prompt:
```plaintext
flori@Stormbreaker:~$ jupyter kernelspec list
Available kernels:
python3 /usr/local/share/jupyter/kernels/python3
```
2. Next, `dotnet interactive` must be installed.
Prefeably (?) the [latest release](https://www.nuget.org/packages/Microsoft.dotnet-interactive/) via:
```
2. Next, `dotnet interactive` must be installed. Prefeably (?) the [latest release](https://www.nuget.org/packages/Microsoft.dotnet-interactive/) via:
```plaintext
dotnet tool install --global Microsoft.dotnet-interactive
```
or, from the development channel:
```code
```plaintext
dotnet tool install -g --add-source "https://pkgs.dev.azure.com/dnceng/public/_packaging/dotnet-tools/nuget/v3/index.json" Microsoft.dotnet-interactive
```
Especially if you installed `dotnet` in your local user account, e.g.
if you are not using root/superuser priviliges,
you might have to modify the path set the DOTNET_ROOT` correctly in your
in your `.bashrc`, where `xxxxxxx` is the name of your home directory:
```
Especially if you installed `dotnet` in your local user account, e.g. if you are not using root/superuser priviliges, you might have to modify the path set the DOTNET_ROOT`correctly in your in your`.bashrc`, where `xxxxxxx\` is the name of your home directory:
```plaintext
export PATH=$PATH:/home/xxxxxxx/.dotnet/tools
export DOTNET_ROOT=/home/xxxxxxx/dotnet-sdk-5.0.301/
```
3. Then install the Jupyter kernel:
```code
```plaintext
dotnet interactive jupyter install
```
4. Verify that Jupyter supports C#:
```code
```plaintext
flori@Stormbreaker:~$ jupyter kernelspec list
Available kernels:
.net-csharp /home/flori/.local/share/jupyter/kernels/.net-csharp
......@@ -167,11 +127,13 @@ resp. `dotnet-interactive` must be added.
.net-powershell /home/flori/.local/share/jupyter/kernels/.net-powershell
python3 /usr/local/share/jupyter/kernels/python3
```
Really easy, isn't it?
## Notes on Third-party libraries
The pre-compiled native library package already contains the following components:
* required BLAS and LAPACK functionality (from Intel MKL)
* Metis graph partitioning
* Direct sparse solver PARDISO (from Intel MKL)
......@@ -180,9 +142,4 @@ The pre-compiled native library package already contains the following component
* Tecplot IO
* etc. (This list might not be complete and also might change)
Of course, you can compile the third-party libraries yourself,
e.g. to replace the Intel MKL by some other BLAS/LAPACK distribution.
But be aware that it is a tedious process!
The source of those native libraries which are open source can be found at
https://github.com/FDYdarmstadt/BoSSS-native resp. https://git.rwth-aachen.de/kummer/BoSSS-native.
If you want to use other versions of certain libraries from their original maintainers, be aware that sometimes the API of those might change, which usually yields to a segmentation fault.
\ No newline at end of file
Of course, you can compile the third-party libraries yourself, e.g. to replace the Intel MKL by some other BLAS/LAPACK distribution. But be aware that it is a tedious process! The source of those native libraries which are open source can be found at <https://github.com/FDYdarmstadt/BoSSS-native> resp. <https://git.rwth-aachen.de/kummer/BoSSS-native>. If you want to use other versions of certain libraries from their original maintainers, be aware that sometimes the API of those might change, which usually yields to a segmentation fault.
\ No newline at end of file