Surrogate models: Improving the exploitation of your sophisticated models

Submitted by COCOP on 2019, July 2 - 12:59

By Norbert Link and Norbert Holzknecht from BFI (www.bfi.de)

Surrogate models are a technology to extract knowledge made by sophisticated software models and to accelerate the achievement of results made by such models.

In industry, often models are applied which need a lot of time and computer power to calculate the results. This can become a problem if there is the need to vary some of the model's parameters and to recalculate the model for all the occurring variations. The number of variations may rise extremely high, so getting the results needs to much time or is even impossible.

Another case is the use of black box models, which are more and more implemented in industrial processes because of the increasing power of modern technologies like Machine Learning Algorithms. But often it is impossible to extract knowledge from such models, so the improvement of the processes is not possible because the relations between inputs and the output cannot be expressed by interpretable equations or rules.

For both cases, a solution can be found in the development of so-called Surrogate Models.

Surrogate models are representatives or mimics of the original model (or function). For instance, many design problems or physical processes are simulated by application of finite element models. Such "expensive" computer models can be mimicked by another model which needs a smaller number of evaluations and works faster than the original one.

        

Left: original model needing a high number of parameter variations. Right: surrogate model with less variations

For the other case, the interpretation of a not interpretable (black box) model, like a conventional neural net or deep learning model, can be done by developing a surrogate model basing on interpretable technologies like linear regression.

In literature there are many different methods described for their use as surrogates, so here only some examples are listed:

  • Focus on interpretable surrogate models: 
    • Linear/logistic regression, Kriging
    • Naïve Bayes Classifier
    • Decision trees
  • Focus on accelerating modelling by surrogates: 
    • Polynomial response surfaces 
    • Artificial neural networks and Deep Learning 
    • Random forest

For surrogate modelling, different packages support the user by offering a collection of surrogating modelling methods and benchmarking functions:

  • Surrogate Modeling Toolbox for Python
  • SUMO Toolbox for Matlab
  • MATSuMoTo for Matlab
  • SURROGATES toolbox for Octave/Matlab
  • Surrogate package for R (clinical trials)

The following references give a first overview about the application of surrogate modelling:

Although the pilot cases implemented in the COCOP project are based on mathematical and data-driven models, the surrogate models are compatible with the COCOP concept/solution to enable plant-wide monitoring and control by using the model-based, predictive, coordinating optimisation concept.

Follow the discussion in the COCOP Debate Group of Linkedin