Irregular lattices for complex shape grammar facade parsing

High-quality urban reconstruction requires more than multi-view reconstruction and local optimization. The structure of facades depends on the general layout, which has to be optimized globally. Shape grammars are an established method to express hierarchical spatial relationships, and are therefore suited as representing constraints for semantic facade interpretation. Usually inference uses numerical approximations, or hard-coded grammar schemes. Existing methods inspired by classical grammar parsing are not applicable on real-world images due to their prohibitively high complexity. This work provides feasible generic facade reconstruction by combining low-level classifiers with mid-level object detectors to infer an irregular lattice. The irregular lattice preserves the logical structure of the facade while reducing the search space to a manageable size. We introduce a novel method for handling symmetry and repetition within the generic grammar. We show competitive results on two datasets, namely the ECP Paris2010 and the ICG Graz50. The former includes only Hausmannian, while the latter includes Classicism, Biedermeier, Historicism, Art Nouveau and post-modern architectural styles.

Overview

Irregular lattices for complex shape grammar facade parsing
H. Riemenschneider, U. Krispel, W. Thaller, M. Donoser, S. Havemann, D. Fellner, H. Bischof, CVPR 2012 (PDF, scholar, ICG Graz50)

Results

Here are some precision/recall curves, and overview figure of the method.

First, we semantically classify the facades and create our novel irregular lattices which are a rectangular splitting of the image into tiles depending on the terminal symbols (windows, doors, etc).




Due to our novel semantic and object detection cues, the lattice confidence (red curve) achieves a higher precision/ recall (+35% at EER, top: ICG Graz50, bottom: ECP Paris2010) compared to standard split proposals based on gradients.

ICG Graz50
ECP Paris2010

After parsing the lattices with our detected symmetry and repetition, we can render the parse tree as a transformed 3D model of the buildings.




And of course you can texture it and make entire buildings and have something like this:

Downloads

ICG Graz50 - Facade Layout Analaysis
For evaluation we proposed a new dataset called ICG Graz50 containing 50 facade images and semantic annotation. It is especially suited for analysis of layout structure (e.g. symmetry and repetition as in our paper). Yet it also is useful for semantic segmentation, detection and architectural analysis as it includes more architectural styles like Classicism, Biedermeier, Historicism, Art Nouveau and post-modern styles. See readme.txt for details.

This page has been edited by Hayko Riemenschneider