Our research discovers how the rolling shutter and movable lens structures
widely found in smartphone cameras modulate structure-borne sounds onto camera
images, creating a point-of-view (POV) optical-acoustic side channel for
acoustic eavesdropping. The movement of smartphone camera hardware leaks
acoustic information because images unwittingly modulate ambient sound as
imperceptible distortions. Our experiments find that the side channel is
further amplified by intrinsic behaviors of Complementary
metal-oxide-semiconductor (CMOS) rolling shutters and movable lenses such as in
Optical Image Stabilization (OIS) and Auto Focus (AF). Our paper characterizes
the limits of acoustic information leakage caused by structure-borne sound that
perturbs the POV of smartphone cameras. In contrast with traditional
optical-acoustic eavesdropping on vibrating objects, this side channel requires
no line of sight and no object within the camera’s field of view (images of a
ceiling suffice). Our experiments test the limits of this side channel with a
novel signal processing pipeline that extracts and recognizes the leaked
acoustic information. Our evaluation with 10 smartphones on a spoken digit
dataset reports 80.66%, 91.28%, and 99.67% accuracies on recognizing 10 spoken
digits, 20 speakers, and 2 genders respectively. We further systematically
discuss the possible defense strategies and implementations. By modeling,
measuring, and demonstrating the limits of acoustic eavesdropping from
smartphone camera image streams, our contributions explain the physics-based
causality and possible ways to reduce the threat on current and future devices.

By admin