Hardware-Software Co-design for Light Transport Acquisition and Adaptive Non-Line-of-Sight Imaging
Description
In the rapidly evolving field of computer vision, propelled by advancements in deeplearning, the integration of hardware-software co-design has become crucial to overcome
the limitations of traditional imaging systems. This dissertation explores the integration
of hardware-software co-design in computational imaging, particularly in light transport
acquisition and Non-Line-of-Sight (NLOS) imaging. By leveraging projector-camera
systems and computational techniques, this thesis address critical challenges in imaging
complex environments, such as adverse weather conditions, low-light scenarios, and the
imaging of reflective or transparent objects.
The first contribution in this thesis is the theory, design, and implementation of a slope
disparity gating system, which is a vertically aligned configuration of a synchronized
raster scanning projector and rolling-shutter camera, facilitating selective imaging through
disparity-based triangulation. This system introduces a novel, hardware-oriented approach
to selective imaging, circumventing the limitations of post-capture processing.
The second contribution of this thesis is the realization of two innovative approaches
for spotlight optimization to improve localization and tracking for NLOS imaging. The
first approach utilizes radiosity-based optimization to improve 3D localization and object
identification for small-scale laboratory settings. The second approach introduces a learningbased
illumination network along with a differentiable renderer and NLOS estimation
network to optimize human 2D localization and activity recognition. This approach is
validated on a large, room-scale scene with complex line-of-sight geometries and occluders.
The third contribution of this thesis is an attention-based neural network for passive
NLOS settings where there is no controllable illumination. The thesis demonstrates realtime,
dynamic NLOS human tracking where the camera is moving on a mobile robotic platform. In addition, this thesis contains an appendix featuring temporally consistent
relighting for portrait videos with applications in computer graphics and vision.
Date Created
The date the item was original created (prior to any relationship with the ASU Digital Repositories.)
2024
Agent
- Author (aut): Chandran, Sreenithy
- Thesis advisor (ths): Jayasuriya, Suren
- Committee member: Turaga, Pavan
- Committee member: Dasarathy, Gautam
- Committee member: Kubo, Hiroyuki
- Publisher (pbl): Arizona State University