The ability to perceive and interpret the geometric shape and semantic meaning of materials and objects is essential for an intelligent visual system. My research is particularly focused on the problems of shape reconstruction, material recognition as well as geometrically inspired object and scene reconstruction and categorization.

A number of extensively studied cues, notably stereoscopic disparity, texture gradient, motion parallax, contours and shading, have been shown to carry valuable information on object surface shape. Unfortunately, many objects of interest and most man-made surfaces, such as a silver plate, an industrial structure or a clean automobile, are smooth and shiny, violating the hypotheses that underlie the analysis of those cues. For specular objects, however, one important but traditionally overlooked cue is the reflection of the environment: a deformed picture of the surrounding scene can be seen on the surface of the specular object, and the degree and type of deformation depend upon its shape.

In this talk I introduce a geometrical and algebraic characterization of how a patch of the scene is mapped into an image by a mirror surface of given shape. I will then develop solutions to the inverse problem of deriving surface shape from mirror reflections in a single image and demonstrate that local information about the geometry of the surface can be fully estimated up to third order. Our theoretical results are validated by both numerical simulations and experiments with real surfaces.

Then I will show that the information from reflected scenes can be also used for recognizing specular materials. This intuition stems from the observation that the surrounding scene is highly distorted when reflected off regions of high curvature or occluding contours. We call these features static specular flow (SSF). We show how to characterize SSF based on low level image analysis, and use SSF for learning statistical models describing reflective surfaces. We evaluate our results on a dataset of 120 images containing specular surfaces and show that our algorithm can achieve good classification accuracy.

Finally, if time permits, I will highlight some of my recent and ongoing works on object and scene categorization by exploiting both 2D and 3D geometric constraints.