Pinocchio

"Probability-based Inference of Composition and Conducting behaviour of humans in interactive organizations"

Introduction

The goal of the Pinocchio project is to enable children to experience and develop an understanding for classical music in a playful manner by conducting a virtual orchestra. We are developing Pinocchio as a game to be usable in a living room environment as well as in a museum setting. The user is able to configure the orchestra, which is modeled as a set of individually controllable musicians. Each musician is associated with audio and video streams and can be freely located in 3D space. The orchestra adjusts to the tempo and volume according to the conductor?s gestures. Pinocchio works with several types of batons, differing in tracking method and in algorithms for gesture recognition. The long-term goal is the creation of a multi-modal, device independent framework for gesture-based applications which require motor skills or the control and operation of a complex set of sensors in intelligent house or car driver assistance systems.

---

Presentations

• Pinocchio Presentation for Apple Visit (Keynote-zip)
• Pinocchio Presentation for Apple Visit (PDF Version)

---

Videos

• VSO Introduction Movie - A short, commented introduction to VSO (Pinocchio's predecessor), its goals, technology and vision
• Pinocchio Demo: visual tracking of the baton, gesture recognition with feed-forward neural network (QuickTime Movie, 4,6MB)
• Wii Demo: conducting pop music with the Nintendo Wii controller (QuickTime Movie, 5,3MB)
• Open CV Face Tracking Demo on OS X (QuickTime Movie, 6.2MB)

---

Team

	Christoph Teschner	christophteschner (at) gmail.com
	Simon Bierbaum	bierbaum (at) in.tum.de
	Dominik Schmidt	dominik.schmidt (at) der-weihnacht.de
	Eva Fenzl	fenzl (at) fs.tum.de
	Peter Lachenmaier	lachenma (at) in.tum.de

mail: ls1 dot pinocchio (at) in dot tum d0t de