A feedback methodology for task-driven fine-grained pixel control in smart cameras

Thumbnail Image
Mudassar, Burhan Ahmad
Mukhopadhyay, Saibal
Associated Organization(s)
Supplementary to
Camera systems of today capture signals at the highest quality to produce a faithful approximation of what they observe. However, the available bandwidth limits the amount of data the camera can transmit. Advances in camera technologies will only compound the problem further as advent of digital pixel technologies and 3D integration promises unprecedented gains in resolution and frame rates of cameras. As cameras are being increasingly used to drive many mission-critical autonomous applications ranging from traffic monitoring to disaster recovery to defense, a uni-directional processing pipeline misses the opportunity to create a 'true' smart camera. In such applications ‘useful information’ depends on the tasks and is defined using complex features, rather than only changes in captured signal. In this research, we tackle this problem by proposing a smart imager that applies high level task-driven feedback at the input space. Specifically, this camera system only captures useful information pertaining to an end-user defined task and at the highest quality. The feedback system applies the task feedback at the encoder and sensor layer. The proposed camera enhances the performance of the task while being bandwidth efficient. Uncertainty information is incorporated into the feedback path to improve performance in challenging scenarios with many false positives and false negatives. Lastly, we address visual challenges that impact the feedback control such as small object detection and moving camera action detection.
Date Issued
Resource Type
Resource Subtype
Rights Statement
Rights URI