Responsive Open Space

A performative spatial environment integrating audio-visual compositions responsive to the engagement of participants among themselves.

Funding

FFG Innovationsscheck

Project Parntners

CGV: Hyosun Kim, Christoph Schinko
ORTLOS Space Engineering: Andrea Redi, Ivan Redi

Introduction

Multiple small, low-cost, high resolution projectors are capable of building scalable large display environments, but need complicated calibration procedures to correct geometric misalignment within and across the different projectors. The goal of the project is the preparation and distribution of an open source software solution to adapt existing interactive multimedia installations to multi-projection environments. A particular challenge here is to adapt to non-standard configurations dealing with arbitrary projection tiles, different lighting conditions, corners of buildings and so on. Our solution provides a robust and efficient way for the geometric calibration procedure and in addition it is easy to use even for non-technical users.

Problem Statement

A projection setup consisting of multiple projectors and a non-planar projection surface as well as a camera covering the whole projection area is used by creative artists to display their artwork. Using the unaligned, overlapping projectors on the non-planar projection surface without compensating the deformations would lead to various artifacts as can be seen in the following figure.
Artifacts emerging from overlapping projections, non-planar projection surfaces and unaligned projectors need to be compensated to create a visually appealing projection.
Therefore, a uniform display area needs to be created that compensates all emerging problems. The only source of input available for measuring is the camera behind the projection surface.

Processing Pipeline

The framework consists of two tools dealing with the following problems:

• measuring the deformations of the projection surface using a camera and computer vision algorithms
• compensating the deformations using the measurements of the first stage and apply soft-edge blending

The first step is to project a checkerboard pattern using each projector of the setup. Then, pictures of the pattern are taken using the camera. A corner detection algorithm deals with the deformed checkerboard patterns on nonplanar surfaces and also with the natural complex illumination condition. Once corner points are detected, graph theory is applied to generate a quad mesh. Three classes of corners with different numbers of neighbors are detected (2-neighbors, 3-neighbors, 4-neighbors). The detected corner points are handed over to a Processing script responsible for rendering. This is done by integrating a client sketch (the one to be rendered) into the rendering routines of the script. It renders the client sketch to an offscreen buffer which is then used as a texture. A set of texture coordinates is calculated out of the detected corner points to compensate the deformations of the projection setup. It is used for applying the client texture to a quad mesh of the same dimensions as used for the previously mentioned checkerboard. An additional soft-edge blending texture is applied as a last step.

Results

The results of a test scenario with four projectors displaying different images (before and after compensation with our tools) can be seen in the following figure. Displaying images on four overlapping projections using a non-planar projection surface leads to various visual problems like nonlinear transformations and overlapping artifacts (left). The corrected results (right) of our processing pipeline show none of these problems at all, with the exception of variations in brightness because of overlapping projections.

Software

The checkerboard recognition software can be downloaded as a zipped file (12.5MB). The Processing script is licensed under the GNU General Public License, version 3.0 (GPLv3) and can be downloaded as a zipped file (958KB).