Impact of Visual Range Optical Blur Aberrations on Deep Learning based Models for Image Classification and Object Detection

Deep Neural Networks (DNNs) have proven to be successful in various computer vision applications such that vision models even infer in safety-critical situations. Therefore, the models need to be robust to disturbances such as image noise or blur. While seminal benchmarks exist to evaluate model robustness to diverse image corruptions, blur is often approximated in an overly simplistic way to model e.g. defocus, while ignoring the different blur kernel shapes that result from optical systems.
This thesis proposes several test methods and datasets of optical blur corruptions, referred to as OpticsBench and LensCorruptions. OpticsBench examines primary aberrations such as coma, defocus, and astigmatism, i.e. aberrations that can be represented by varying a single parameter of Zernike polynomials and are easy to interpret. The OpticsBench method ensures comparability of the optical blur corruptions to in-size or in-accuracy matched references to investigate relative distribution shifts. To go beyond the principled but synthetic setting of primary aberrations, LensCorruptions samples linear combinations in the vector space spanned by Zernike polynomials, corresponding to real lenses. The evaluation on differently blurred ImageNet-1k images shows that the results for models confronted to the OpticsBench blur and to a disk-shaped reference blur, vary up to 5 % and the class-wise accuracy varies up to 30 % for optical image corruptions against a disk-shaped reference. We therefore conclude that the colour-dependent kernel shape must be taken into account when testing the model robustness as the accuracy cannot be explained sufficiently with the reference blur accuracy. To this end, we provide a large test dataset of optical blur distribution shifts, using the complementary LensCorruptions method, which simplifies future robustness research. The evaluation shows that some models favour certain blur types when confronted with a large number of lens blurs. Since increased image blur is generally detrimental to performance, we also show that the performance loss caused by optical aberrations can be significantly compensated for using the OpticsAugment data augmentation method and demonstrate superior performance compared to a strong baseline on both the proposed optical blur test datasets and 2D and 3D corruptions. Finally, since lens blur generally depends on the image location, a space-variant image blur corruption and corresponding test dataset are discussed on the Berkeley Deep Drive automotive dataset for several object detection models. To analyse how this spatial variance may affect the local performance of object detection models, the local evaluation method SRIA is proposed. In summary, the thesis provides several concepts and methods to test for optical blur corruptions and to improve the model robustness. The combination of these efficient evaluation methods and improvements in model robustness increases the safety of future computer vision systems.