SMILE III – Safety analysis and verification/validation of ML based systems

Future autonomous vehicles will rely on Deep machine learning algorithms (DML), whose correct behaviour cannot be guaranteed by traditional software engineering approaches. This uncertainty is a big obstacle to ensure the quality of systems using DML, especially in automotive safety critical applications.
The SMILE program addresses this challenge via developing method(s) that allow DML-based functions to be included into safety critical vehicular applications with quality control requirements from industrial standards.

SMILE program consists of several consecutive research projects:

• SMILE I project studied state-of-art within Verification and Validation for DML systems and mapping of the challenges faced by the automotive industry. SMILE I concluded that we should focus on a Safety Cage Concept within the follow-up project(s).
• SMILE II project investigated the latest vehicle safety standards: (i) ISO 26262 - functional safety, and lately complemented by (ii) PAS specification of ISO 21448 - safety of the intended functionalities; implemented and tested a number of safety cage architectures in simulators, developed a framework for supervisor comparison; and developed future research tool set including generated datasets and end-to-end vehicle controller. SMILE II observed that ISO 26262 alone cannot accommodate ML-based systems, the complement with the most recently published SOTIF (2019) will be more applicable. However, in its PAS edition, SOTIF only focused on what should be covered during systems engineering, leaves out how to achieve these goals.

With the success and promising results from SMILE I/II, SMILE III project will further develop the safety cage concept into a reference system architecture and prototype(s), together with data management approaches, facilitating compliance with the evolving safety standards (including the insufficient prescriptive ISO/PAS 21448 edition).