Update Installation and Testing authored by Beck, Lauritz's avatar Beck, Lauritz
[[_TOC_]] * [Introduction](#introduction)
* [Installation](#installation)
* [Windows](#windows)
* [Linux](#linux)
* [Mac OS X](#mac-os-x)
* [Testing the Installation](#testing-the-installation)
* [Installing Jupyter](#installing-jupyter)
* [Notes on Third-party libraries](#notes-on-third-party-libraries)
## Introduction ## Introduction
**Note that you also need a working installation if you want to compile BoSSS **Note that you also need a working installation if you want to compile BoSSS from the C# sources yourself. BoSSS requires a set of prerequisites (e.g. Microsoft .NET or Mono, MPI, certain _native libraries_) on a machine, which are set up by the installation procedure outlined below.**
from the C# sources yourself.
BoSSS requires a set of prerequisites
(e.g. Microsoft .NET or Mono, MPI, certain _native libraries_)
on a machine, which are set up by the installation
procedure outlined below.**
We briefly outline the steps required to get started with using and/or developing BoSSS applications.
**In any case**, the [installation](#installation) of the pre-compiled version of BoSSS should be the first step.
Upon finishing this step, you can either
use [BoSSSpad](./BoSSSpad) to run pre-compiled applications as well as -- or --
you can start [working with the source code](./Working-with-BoSSS-as-a-Developer).
We briefly outline the steps required to get started with using and/or developing BoSSS applications. **In any case**, the [installation](#installation) of the pre-compiled version of BoSSS should be the first step. Upon finishing this step, you can either use [BoSSSpad](/bosss1/experimental/-/wikis/Getting-Started/BoSSSpad) to run pre-compiled applications as well as -- or -- you can start [working with the source code](/bosss1/experimental/-/wikis/Getting-Started/Working-with-BoSSS-as-a-Developer).
## Installation ## Installation
The advantage of C#, resp. .NET is that _the executable is platform-independent_. The advantage of C#, resp. .NET is that _the executable is platform-independent_. This implies e.g. that an executable compiled on a Windows/Mac laptop can be copied to a Linux/Unix HPC system and executed there without re-compilation. As discussed in the [Core Concepts](/bosss1/experimental/-/wikis/Getting-Started/Core-concepts#c-Sharp,-.net-and-mono), this obviously requires some _middleware_ and some libraries to be present on the target computer. In the following, we will thus summarize some best practices for the installation of the [pre-compiled BoSSS binaries](#notes-on-third-party-libraries) on different operating systems.
This implies e.g. that an executable compiled on a Windows/Mac laptop
can be copied to a Linux/Unix HPC system and executed there without re-compilation.
As discussed in the [Core Concepts](./Core-concepts#c-Sharp,-.net-and-mono),
this obviously requires some _middleware_ and some libraries to be present on the target computer. In the following, we will thus summarize some best practices for the installation of the [pre-compiled BoSSS binaries](#notes-on-third-party-libraries) on different operating systems.
### Windows ### Windows
To install on Windows, just o execute the BoSSS Windows installer. To install on Windows, just execute the BoSSS Windows installer. Afterwards, you can [test the installation](#testing-the-installation).
Afterwards, you can [test the installation](#testing-the-installation).
The BoSSS installer already includes all required native libraries and prerequisites. These are: The BoSSS installer already includes all required native libraries and prerequisites. These are:
1. Microsoft Visual C++ Redistributable (x64) 1. Microsoft Visual C++ Redistributable (x64)
2. [Microsoft MPI](https://msdn.microsoft.com/en-us/library/bb524831(v=vs.85).aspx). Check the MPI installation by executing the command `mpiexec` in a console window 2. [Microsoft MPI](https://msdn.microsoft.com/en-us/library/bb524831(v=vs.85).aspx). Check the MPI installation by executing the command `mpiexec` in a console window
3. <del>The [.NET framework](https://www.microsoft.com/net/download/framework) is installed 3. ~~The~~ [~~.NET framework~~](https://www.microsoft.com/net/download/framework) ~~is installed The BoSSS Installer checks whether these components are in place and installs them, if required. In the case of some problem, one can also download and install these items manually.~~
The BoSSS Installer checks whether these components are in place and installs them, if required.
In the case of some problem, one can also download and install these items manually.</del>
### Linux ### Linux
The BoSSS group maintains basic packages of the The BoSSS group maintains basic packages of the [native libraries](#notes-on-third-party-libraries) listed in for Linux. These are compiled against recent versions of the Open MPI and the Intel MKL library and should provide compatibility and decent performance on a variety of systems. However, compatibility this can not guaranteed for any Linux distribution -- binary platform independence in Linux not possible in general; since we use Ubuntu, all Debian derivatives might most likely work. Note further, high-performance computers typically provide specifically optimized versions of BLAS and LAPACK, so for the ultimate performance on a certain system, you'll need to build your own version of the BoSSS native libraries.
[native libraries](#notes-on-third-party-libraries)
listed in for Linux.
These are compiled against recent versions of the Open MPI and the Intel MKL library and should provide compatibility and decent performance on a variety of systems.
However, compatibility this can not guaranteed for any Linux distribution -- binary platform independence in Linux not possible in general; since we use Ubuntu, all Debian derivatives might most likely work. Note further, high-performance computers typically provide specifically optimized versions of BLAS and LAPACK, so for the ultimate performance on a certain system, you&#39;ll need to build your own version of the BoSSS native libraries.
The steps for installing BoSSS on Linux are: The steps for installing BoSSS on Linux are:
1. Extensive notes on the installation of .NET [here](./NET5-Upgrade#installation-of-net-5-in-linux);
In short:
Download and install [.NET5 SDK](https://dotnet.microsoft.com/download/dotnet/5.0),
see also https://docs.microsoft.com/de-de/dotnet/core/install/linux for the
most recent versions and best practices for your system.
<del> Obtain [Mono](https://fdy-tuda.openproject.com/www.mono-project.com) (version 6.0 or higher), either by installing a [pre-built package](https://fdy-tuda.openproject.com/www.mono-project.com/download/) or by [compiling](https://fdy-tuda.openproject.com/www.mono-project.com/docs/compiling-mono/linux/) it from a [tarball](https://download.mono-project.com/sources/mono/). </del>
2. Obtain/install/load OpenMPI
3. Obtain/install/load native libraries `libBoSSSnative_seq.so`, `libBoSSSnative_omp.so` and `libBoSSSnative_mpi.so`
* Pre-compiled versions of these shared libraries for Linux can be found in the Windows installation, in the directory `$BOSSS_INSTALL/bin/native/linux/amd64-openmpi`. (There should also be a `zip`\-File that you can unpack on any system.)
* If these pre-build libraries are not compatible with your system
(we do our best, but binary compatibility across distributions is quite difficult on Linux),
you will need to compile them yourself,
the source code can be found at https://github.com/FDYdarmstadt/BoSSS-native, resp. https://git.rwth-aachen.de/kummer/BoSSS-native
1. Extensive notes on the installation of .NET [here](/bosss1/experimental/-/wikis/Getting-Started/NET5-Upgrade#installation-of-net-5-in-linux); In short: Download and install [.NET5 SDK](https://dotnet.microsoft.com/download/dotnet/5.0), see also <https://docs.microsoft.com/de-de/dotnet/core/install/linux> for the most recent versions and best practices for your system. ~~Obtain~~ [~~Mono~~](https://fdy-tuda.openproject.com/www.mono-project.com) ~~(version 6.0 or higher), either by installing a~~ [~~pre-built package~~](https://fdy-tuda.openproject.com/www.mono-project.com/download/) ~~or by~~ [~~compiling~~](https://fdy-tuda.openproject.com/www.mono-project.com/docs/compiling-mono/linux/) ~~it from a~~ [~~tarball~~](https://download.mono-project.com/sources/mono/)~~.~~
2. Obtain/install/load OpenMPI
3. Obtain/install/load native libraries `libBoSSSnative_seq.so`, `libBoSSSnative_omp.so` and `libBoSSSnative_mpi.so` \* Pre-compiled versions of these shared libraries for Linux can be found in the Windows installation, in the directory `$BOSSS_INSTALL/bin/native/linux/amd64-openmpi`. (There should also be a `zip`-File that you can unpack on any system.) \* If these pre-build libraries are not compatible with your system (we do our best, but binary compatibility across distributions is quite difficult on Linux), you will need to compile them yourself, the source code can be found at <https://github.com/FDYdarmstadt/BoSSS-native>, resp. <https://git.rwth-aachen.de/kummer/BoSSS-native>
### Mac OS X ### Mac OS X
Since Mac OS X is a Unix system, most things should work similar to Linux. Since Mac OS X is a Unix system, most things should work similar to Linux. There has been, however, no interest so far for running BoSSS on Apple Computers, therefore we do not provide official guidelines.
There has been, however, no interest so far for running BoSSS on Apple Computers,
therefore we do not provide official guidelines.
The first step would be to compile the [Native Libraries](#notes-on-third-party-libraries). The first step would be to compile the [Native Libraries](#notes-on-third-party-libraries). For this, we recommend that users should try either Mac Ports (<https://www.macports.org/>) or Homebrew (<https://brew.sh/>) to have a system which close to Linux (i.e. has `gcc`, `gfortran`, etc.). Using only Mac OS tools might be even harder. After you have compiled the library, you can proceed analogously to the procedure under Linux.
For this, we recommend that users should try either Mac Ports (https://www.macports.org/) or Homebrew (https://brew.sh/) to have a system which close to Linux (i.e. has `gcc`, `gfortran`, etc.). Using only Mac OS tools might be even harder.
After you have compiled the library, you can proceed analogously to the procedure under Linux.
## Testing the Installation ## Testing the Installation
Execute, after the installation the command Execute, after the installation the command
```code
```plaintext
dotnet BoSSSpad.dll --check dotnet BoSSSpad.dll --check
``` ```
or or
```code
```plaintext
BoSSSpad.exe --check BoSSSpad.exe --check
``` ```
to verify that the installation works and the native libraries are
in place. to verify that the installation works and the native libraries are in place. Furthermore, check that the environment variable `$BOSSS_INSTALL` is defined. The native libraries should be in the following location
Furthermore, check that the environment variable `$BOSSS_INSTALL`
is defined.
The native libraries should be in the following location
* On Windows: `$BOSSS_INSTALL/bin/native/win/amd64` * On Windows: `$BOSSS_INSTALL/bin/native/win/amd64`
* On Linux: `$BOSSS_INSTALL/bin/native/linux/amd64-openmpi` * On Linux: `$BOSSS_INSTALL/bin/native/linux/amd64-openmpi`
## Installing Jupyter ## Installing Jupyter
In order to use BoSSS properly, In order to use BoSSS properly, a working installation of [**Jupyter**](https://jupyter.org/) (aka. Jupiterlab, Jupyter-Notebkook, etc.) **with support for** [**dotnet-interactive**](https://github.com/dotnet/interactive) is required. Prior to the migration to .NET5, BoSSS used its own custom notebook format (`bws`, for BoSSS work sheet) and featured a notebook/worksheet editor embedded in [BoSSSpad](/bosss1/experimental/-/wikis/Getting-Started/BoSSSpad). Since Mid-2020, [dotnet-interactive](https://github.com/dotnet/interactive) is supported by Jupyter and reached a level of maturity so that it can replace the custom BoSSSpad solution. Although a dependency on an additional software component is a drawback (e.g. the user has to install and configure it) the advantages of a widely-used of-the-shelf solution with a large user-basis such as Jupyter outweigh the disadvantages.
a working installation of [**Jupyter**](https://jupyter.org/)
(aka. Jupiterlab, Jupyter-Notebkook, etc.)
**with support for [dotnet-interactive](https://github.com/dotnet/interactive)**
is required.
Prior to the migration to .NET5, BoSSS used its own custom notebook format (`bws`, for BoSSS work sheet) and featured a notebook/worksheet editor embedded in [BoSSSpad](./BoSSSpad).
Since Mid-2020, [dotnet-interactive](https://github.com/dotnet/interactive)
is supported by Jupyter and
reached a level
of maturity so that it can replace the custom BoSSSpad solution.
Although a dependency on an additional software component is a drawback
(e.g. the user has to install and configure it)
the advantages of a widely-used of-the-shelf solution with a large user-basis
such as Jupyter
outweigh the disadvantages.
Provided that Python is already installed, one can install Jupyter by Provided that Python is already installed, one can install Jupyter by
```code
```plaintext
pip install jupyterlab pip install jupyterlab
``` ```
resp. resp.
```code
```plaintext
pip3 install jupyterlab pip3 install jupyterlab
``` ```
probably best as `root` or with administrator rights.
This works in similar for Linux and Windows. probably best as `root` or with administrator rights. This works in similar for Linux and Windows.
* Further details can be found in the [Original Documentation](https://jupyter.org/install). * Further details can be found in the [Original Documentation](https://jupyter.org/install).
* An alternative to the manual installation is the use of the * An alternative to the manual installation is the use of the semi-commercial [Anaconda Python distribution](https://www.anaconda.com/products/individual)
semi-commercial [Anaconda Python distribution](https://www.anaconda.com/products/individual)
Once Jupyter is installed, it can be started/tested by launching a web browser (e.g. Firefox) and running either `jupyter-lab` or `jupyter-notebook` from a terminal. The GUI should then open in the browser.
Once Jupyter is installed, it can be started/tested by launching a web browser (e.g. Firefox)
and running either `jupyter-lab` or `jupyter-notebook` from a terminal. For a first install, only Python notebooks are available in Jupyter. The Support for C#, resp. `dotnet-interactive` must be added. (See also in the [Original Documentation](https://github.com/dotnet/interactive/blob/main/docs/NotebooksLocalExperience.md).)
The GUI should then open in the browser.
1. The available kernels can be listed by `jupyter kernelspec list` in a python-aware terminal, e.g. Anaconda Prompt:
For a first install, only Python notebooks are available in Jupyter. The Support for C#,
resp. `dotnet-interactive` must be added. ```plaintext
(See also in the [Original Documentation](https://github.com/dotnet/interactive/blob/main/docs/NotebooksLocalExperience.md).)
1. The available kernels can be listed by
`jupyter kernelspec list` in a python-aware terminal, e.g. Anaconda Prompt:
```code
flori@Stormbreaker:~$ jupyter kernelspec list flori@Stormbreaker:~$ jupyter kernelspec list
Available kernels: Available kernels:
python3 /usr/local/share/jupyter/kernels/python3 python3 /usr/local/share/jupyter/kernels/python3
``` ```
2. Next, `dotnet interactive` must be installed. 2. Next, `dotnet interactive` must be installed. Prefeably (?) the [latest release](https://www.nuget.org/packages/Microsoft.dotnet-interactive/) via:
Prefeably (?) the [latest release](https://www.nuget.org/packages/Microsoft.dotnet-interactive/) via:
``` ```plaintext
dotnet tool install --global Microsoft.dotnet-interactive dotnet tool install --global Microsoft.dotnet-interactive
``` ```
or, from the development channel: or, from the development channel:
```code
```plaintext
dotnet tool install -g --add-source "https://pkgs.dev.azure.com/dnceng/public/_packaging/dotnet-tools/nuget/v3/index.json" Microsoft.dotnet-interactive dotnet tool install -g --add-source "https://pkgs.dev.azure.com/dnceng/public/_packaging/dotnet-tools/nuget/v3/index.json" Microsoft.dotnet-interactive
``` ```
Especially if you installed `dotnet` in your local user account, e.g.
if you are not using root/superuser priviliges, Especially if you installed `dotnet` in your local user account, e.g. if you are not using root/superuser priviliges, you might have to modify the path set the DOTNET_ROOT`correctly in your in your`.bashrc`, where `xxxxxxx\` is the name of your home directory:
you might have to modify the path set the DOTNET_ROOT` correctly in your
in your `.bashrc`, where `xxxxxxx` is the name of your home directory: ```plaintext
```
export PATH=$PATH:/home/xxxxxxx/.dotnet/tools export PATH=$PATH:/home/xxxxxxx/.dotnet/tools
export DOTNET_ROOT=/home/xxxxxxx/dotnet-sdk-5.0.301/ export DOTNET_ROOT=/home/xxxxxxx/dotnet-sdk-5.0.301/
``` ```
3. Then install the Jupyter kernel: 3. Then install the Jupyter kernel:
```code
```plaintext
dotnet interactive jupyter install dotnet interactive jupyter install
``` ```
4. Verify that Jupyter supports C#: 4. Verify that Jupyter supports C#:
```code
```plaintext
flori@Stormbreaker:~$ jupyter kernelspec list flori@Stormbreaker:~$ jupyter kernelspec list
Available kernels: Available kernels:
.net-csharp /home/flori/.local/share/jupyter/kernels/.net-csharp .net-csharp /home/flori/.local/share/jupyter/kernels/.net-csharp
...@@ -167,11 +127,13 @@ resp. `dotnet-interactive` must be added. ...@@ -167,11 +127,13 @@ resp. `dotnet-interactive` must be added.
.net-powershell /home/flori/.local/share/jupyter/kernels/.net-powershell .net-powershell /home/flori/.local/share/jupyter/kernels/.net-powershell
python3 /usr/local/share/jupyter/kernels/python3 python3 /usr/local/share/jupyter/kernels/python3
``` ```
Really easy, isn't it? Really easy, isn't it?
## Notes on Third-party libraries ## Notes on Third-party libraries
The pre-compiled native library package already contains the following components: The pre-compiled native library package already contains the following components:
* required BLAS and LAPACK functionality (from Intel MKL) * required BLAS and LAPACK functionality (from Intel MKL)
* Metis graph partitioning * Metis graph partitioning
* Direct sparse solver PARDISO (from Intel MKL) * Direct sparse solver PARDISO (from Intel MKL)
...@@ -180,9 +142,4 @@ The pre-compiled native library package already contains the following component ...@@ -180,9 +142,4 @@ The pre-compiled native library package already contains the following component
* Tecplot IO * Tecplot IO
* etc. (This list might not be complete and also might change) * etc. (This list might not be complete and also might change)
Of course, you can compile the third-party libraries yourself, Of course, you can compile the third-party libraries yourself, e.g. to replace the Intel MKL by some other BLAS/LAPACK distribution. But be aware that it is a tedious process! The source of those native libraries which are open source can be found at <https://github.com/FDYdarmstadt/BoSSS-native> resp. <https://git.rwth-aachen.de/kummer/BoSSS-native>. If you want to use other versions of certain libraries from their original maintainers, be aware that sometimes the API of those might change, which usually yields to a segmentation fault.
e.g. to replace the Intel MKL by some other BLAS/LAPACK distribution. \ No newline at end of file
But be aware that it is a tedious process!
The source of those native libraries which are open source can be found at
https://github.com/FDYdarmstadt/BoSSS-native resp. https://git.rwth-aachen.de/kummer/BoSSS-native.
If you want to use other versions of certain libraries from their original maintainers, be aware that sometimes the API of those might change, which usually yields to a segmentation fault.
\ No newline at end of file