Towards simulation-based verification of autonomous navigation systems

Autonomous ships are expected to change water-based transport of both cargo and people, and large investments are being made internationally. There are many reasons for such transformation and interest, including shifting transport of goods from road to sea, reducing ship manning costs, reduced dangerous exposure for crew, and reduced environmental impact. Situational awareness (SA) systems and Autonomous navigation systems (ANS) are key elements of autonomous ships. Safe deployment of ANS will not be feasible based on real-life testing only, but will require large-scale, systematic simulation-based testing in addition to assurance of the development process. DNV GL proposes to use a digital twin, meaning a digital representation of key elements of the autonomous ship as a key tool for simulation-based testing. The digital twin contains comprehensive mathematical models of the ship and its equipment, including all sensors and actuators. The complete simulation-based test system complementing the digital twin should consist of a virtual world to simulate environmental conditions, geographical information and interaction with other maritime traffic. Finally, the test system must include a test management system that controls simulation of the digital twin and the virtual world, generates test scenarios as well as evaluates the test scenario results. An automatic scenario generation tool should search for low ANS performance, and ultimately establish sufficient coverage of the possible scenario space. The test scenario evaluation should automatically consider safety, conformance to collision regulations at sea (COLREG), and possibly also the efficiency of the ship navigation. This paper presents a comprehensive prototype of a test system for ANS. Key topics are simulation-based testing, interfacing the simulator and ANS, cooperation with ANS manufacturers, dynamic test scenario generation, automatic assessment towards COLREG and experiences from the cooperation with The Norwegian Defense Establishment (FFI).