MeLOn
Gaussian process

Training

MeLOn supports two options for training Gaussian processes:

  • Using python with the
  • Using Matlab with the provided training framework

It is recommended that you rely on the provided example scripts for training your model. The reason for this is that MeLOn doesn't support all functionality included in gpytorch and by using the examples you can assure to generate models which are compatible with the MeLOn functionalities.

Python

Requirements

  • Python (>=3.6)
  • gpytorch (>=1.3.1)
  • scikit-learn (>=0.23.2)
  • numpy (>=1.19.4)
  • pyDOE (>=0.3.8)

Using the example scripts

There are two python example scripts provided that show how to train a gaussian process using gpytorch. The difference between the scripts is how the trained model is handled.

  • The script example_training_gp_pymelon.py creates a MeLOn GPData object and directly starts to solve an optimization problem which includes the model by making use of the MAiNGO and MeLOn python interfaces (pymaingo and pymelon).
  • The script example_training_gp.py creates a json file that can be parsed by the C++ version of MeLOn (e.g. required for the parallelized solution of an optimization problem as using MPI is not yet supported by pymaingo).

The training data used for the examples is generated within the scripts themselves (see sections "SET PARAMETERS" and "GENERATE DATA"), but you can also load your data from files using a preferred method. In addition to the parameters for generating/loading your data you have to set the following parameters according to your needs:

  • Kernel parameter MATERN: Value of the matern kernel parameter nu
    • 1: nu = 1/2
    • 3: nu = 3/2
    • 5: nu = 5/2
  • Number of training iterations N_TRAINING_ITER

For generating a GPData object that can be used with the pymelon and pymaingo interfaces use the function generate_melon_gp_object. For saving the model to a json file that can be used by the MeLOn C++ version use the function save_model_to_json.

Matlab

The training script gives you an example of how to train a Gaussian process in Matlab. In the provided example the training data is generated within the script, but you can also load the data using your preferred method. In addition to the training data you also have to provide the lower and upper bounds of the input variables you use for training, these are used for a min-max feature scaling. Further, you can specify the following training options:

  • Kernel parameter (Opt.GP(1).matern): Value of the matern kernel parameter nu
    • 1: nu = 1/2
    • 3: nu = 3/2
    • 5: nu = 5/2
    • inf: nu = infinity
  • Stopping criterium for internal solver (Opt.GP(1).fun_eval): Number of function evaluations for each hyperparameter.

To train and save the model you have to use the functions Train_GP_and_return_hyperparameters and Write_GP_to_json respectively.

Usage

To include the trained model in your MAiNGO problem formulation you can either use the MeLOn C++ or the pymaingo and pymelon python interface. An example of how to include a Gaussian process in your optimization problem can be found in the MAiNGO folder.

Constructors

Prediction and variance

Internal variables

Information on training data