Numerous methods, such as lightfield acquisition
and geometric reconstruction, use cameras to
acquire visual samples of the environment and
construct a model. Unfortunately, efficiently
obtaining models of complex real-world locations is
a challenging task. In particular, the aforementioned
important environments are often actively in use,
containing moving objects, such as people entering
and leaving the scene. The methods listed have
difficulty in capturing the color and structure of the
environment while in the presence of moving and
temporary occluders. An efficient acquisition
process must be able to proceed even in such
situations.
Our key idea is to generalize the concept of a
camera to take samples over space and time. Such a
camera can easily and interactively detect and
sample moving objects while continuously moving
through the environment. Furthermore, the camera is
able to acquire images of the (hidden) scene
regardless of the occlusions produced by moving
obstacles. This reduces capture time by avoiding
having to revisit temporarily occluded surfaces. For
example, acquisition can use a hand-held version of
our camera design to acquire a lightfield of a large
statue even if people are walking in the field-of-
view of the camera and between the camera and the
statue (as would often occur for an important
structure on display). Our camera system can be
mounted on a car and used to acquire images of the
architectural structures of an urban neighborhood
even in the presence of other moving cars and
people. Our camera can be used to simultaneously
move within and capture images of a large museum
even during normal operation hours when the
museum is full of visitors. Since acquisition of a
large environment requires significant time, it is
unrealistic to ask site supervisors to close-down a
location for a lengthy capture session. However,
visually stunning, important, and thus actively-used
sites are precisely the ones we are interested in
capturing.
In this paper, we introduce a class of cameras
called lag cameras. The main concept is to have a
small cluster of cameras where at least one camera
follows (or “lags”) behind a lead camera and to
interactively acquire space-time samples of the
environment. In particular, follow cameras capture
the scene from approximately the same viewpoints
as lead cameras but at later instances in time. A lag
camera supports and assists various space-time
processing methods including space-time stereo,
foreground object processing, environment
reconstruction, and lumigraphs and lightfields.
Figure 1a illustrates a space-time sampling of an
environment using a general n-view lag camera. The
horizontal axis corresponds to samples taken over
3D space. The vertical axis corresponds to samples
taken over time. As the cameras move, the lead
camera (rightmost camera in Figure 1a) moves to a
new viewpoint and all other cameras follow and
capture views of the environment from previously
visited viewpoints but at later instances in time.
Figure 1b shows a picture of our first
implementation of a lag camera using only two
cameras. Figure 1c shows example images captured
by our lag camera. These images are acquired from
approximately the same viewpoint and at nearby
instances in time. Hence, a lag camera can move
through the environment, efficiently capturing
image samples, yet easily and interactively detecting
moving foreground objects. Moreover, since both
the occluder and the lag camera are moving, the
system can both omit foreground objects and obtain
samples of surfaces behind the occluder. Figure 1e
shows an example application of lag cameras to
produce novel views using a modified unstructured
lumigraph. Figure 1d shows a naïve reconstruction
from the same viewpoint with a moving occluder
partially appearing.
Our main contributions are as follows:
• We describe lag cameras that can move through
an environment while acquiring space-time
samples of the scene.
• We develop a new motion detection algorithm
using a lag camera to create motion masks. This
technique allows us to easily and interactively
detect moving objects in the scene while the
camera itself is undergoing motion. The
detected moving objects can then be extracted,
reconstructed or removed from the capture.
• We provide a method for acquiring samples of a
static background scene even in the presence of
moving occluders in between the camera and
the scene. This allows us to acquire images of
in-use environments even if people enter and
leave the field of view of the camera.
2 RELATED WORK
Our lag camera design borrows ideas from multiple
areas of research. Previous methods have used
camera clusters to increase field of view or construct
LAG CAMERA: A MOVING MULTI-CAMERA ARRAY FOR SCENE ACQUISITION
99