Machine learning with nonlinear state space models

Contemporary automation systems require accurate models for analysis, design, and control. Oftentimes, it is not possible to derive satisfactory models by first principles. This is often the case in process engineering or in mechatronic systems, where the overall process is just too complex, or the development of those first-principle models would just be too expensive. Thus, there is a strong demand for data-driven modeling approaches. Additionally, there are enormous successes in machine learning concerning the generation of models solely by the use of data.
Driven by the need for accurate models as well as the successes and advances in machine learning, a novel class of model structures and associated training algorithms for building data-driven nonlinear dynamic models is developed. The new identification procedure and the resulting model is called local model state space network (LMSSN). It fuses nonlinear state space models with local model networks (LMNs). The LMSSN is designed as a user-friendly and deterministic approach with warm-start initialization (with a linear model), incrementally growing complexity, favorable and robust extrapolation behavior, easy interpretation, and straightforward incorporation possibilities for prior knowledge.
Furthermore, recurrent neural networks (RNNs) and their similarities to nonlinear state space models are elaborated on. Popular RNNs like the long short-term memory (LSTM) and gated recurrent unit (GRU) models are translated into controls perspective and a comprehensive study comparing different RNN structures is carried out.
The overall outstanding performance of the LMSSN is demonstrated on various applications. It is shown that the LMSSN can accurately model a wide variety of processes by consistently computing expressive yet compact models. The LMSSN is benchmarked against state-of-the-art nonlinear system identification algorithms and achieves similar or superior results. The practical usefulness and applicability of LMSSN are strikingly demonstrated on two real-world processes.