Date of Award
Spring 2023
Access Type
Dissertation - Open Access
Degree Name
Doctor of Philosophy in Mechanical Engineering
Department
Mechanical Engineering
Committee Chair
Eric Coyle
First Committee Member
Brian Butka
Second Committee Member
Marc Compere
Third Committee Member
Patrick Currier
Fourth Committee Member
Jianhua Liu
College Dean
James Gregory
Abstract
Autonomous surface vessels (ASV) can potentially improve the safety of vessels traditionally operated by humans. Despite advancements in autonomous on-road vehicles, many of these advancements have yet to be realized for ASVs. This is primarily due to lacking ASV sensing platforms and public datasets for ASV-based perception research. To that end, this dissertation demonstrates the design of a synchronized multi-modal sensing platform for ASVs utilizing GPS/INS, LiDAR, LWIR cameras, HDR camera, and high-resolution cameras. The sensing platform is designed to maximize the overlap of sensors for multi-modal research and provides accurate intrinsic and extrinsic calibration between each sensor. Furthermore, the GPS provides precise time stamp capabilities for each sensor. This sensing platform is used for data collection to create a novel multi-modal marine dataset with LiDAR ground truth annotations. Finally, the dataset is used to demonstrate the feasibility of deep learning for semantic segmentation in a marine environment. This is done with three separate models: a LiDAR-only model, a camera-only model, and a deep fusion model. The LiDAR model demonstrates the feasibility of using LiDAR image projections as a valuable data representation. The camera model demonstrates the feasibility of cross-modality learning with a camera input and LiDAR ground truth. Finally, the fusion network demonstrates the advantages of combining features between multiple modalities to improve model performance. While each model showcases its strengths, future work may explore improved model optimization, integrating transformers and using temporal features to detect moving objects better.
Scholarly Commons Citation
Thompson, David J., "Neural Network Fusion of Multi-Modal Sensor Data For Autonomous Surface Vessels" (2023). Doctoral Dissertations and Master's Theses. 741.
https://commons.erau.edu/edt/741
GS9
