This project is intended to upgrade the latest Mixed-Reality headsets to an on-road mixed reality driving simulator 

• Build a digital twin framework with a real vehicle driving on the testing track and a corresponding digital vehicle in the online traffic microsimulation to interact with the virtual traffic fleet

• Develop an efficient real-time virtual hologram pose correction algorithm using a mixed-reality camera (ZED 2i)

• Use a multi-step kalman filter to enhance virtual pose update frequency and pose transition smoothness

• Implementation in different kinds of traffic scenarios, such as simple car following, multi-agent highway traffic, and intersection with pedestrians, etc

This project uses our Lab's patented simulation platform to conduct Eco-connected vehicle research

• Reconstruct HoloLens 2 localization algorithm to enable stable and accurate hologram projection from the driver's view inside a moving vehicle.

• Establish cross-platform communication between the VisSim simulator and the HoloLens 2 using socket programming + Google Protobuf

• Build virtual traffic scenes in Unity 3D