Sonic participation in the evolving audio feedback system

ISEA 2016 Artist Talks

Link to the introduction of the original audio feedback system


First scenario: tension between the system’s feedback sounds and audience sound (with strong echo effect)

Second scenario: evolution of the structures processing the recorded audience sounds


Intentional Control of an Audio Feedback by Evolutionary Control of Second-order Structures

EvoMUSART (Part of Evostar 2015) paper


Aiming for high-level intentional control of audio feedback, though microphones, loudspeakers and digital signal processing, we present a system adapting toward certain sonic features. Users control the system by selecting and changing feature objectives in real-time. The system has a second-order structure in which the internal signal processing algorithms are developed according to an evolutionary process. Genotypes develop signal-processing algorithms, and fitness is measured by analysis of the incoming audio feedback. A prototype is evaluated experimentally to measure changes of audio feedback depending on the target conditions. By enhancing interactivity of an audio feedback through the intentional control, we expect that feedback systems could be utilized more effectively in the fields of musical interaction through being balanced between nonlinearity and interactivity.

Demo sounds

  • Low spectral flatness (pure tone)
  • High spectral flatness (noise signal)
  • Low spectral centroid (dark sound)
  • High spectral centroid (bright sound)
  • Low tremolo (low amplitude fluctuation)
  • High tremolo (high amplitude fluctuation)
  • Low vibrato (low frequency fluctuation)
  • High vibration (high frequency fluctuation)
  • Low pitch
  • High pitch

TAPIR: theoretically audible, but practically inaudible range

The “theoretically audible, but practically inaudible range” (TAPIR) is the highest area of human hearing bandwidth, which is barely perceptible for most people but can be transmitted and received by stereotypical transducers. We believe TAPIR has a strong potential as a new media for music, sonic arts, and mobile media. The research discussed includes the development of TAPIR systems for acoustic location, gesture/motion detection, and proximity detection/user identification.


  • The TAPIR Acoustic Locater (TAL)
    TAL is a prototype of TAPIR-powered location system based on the time-of-arrival (ToA)
  • IAMHear (TAL-based tabletop interface)

IAMHear is a novel tabletop interface for music performance and sound making, in which smart mobile devices are used as on-table objects for interaction. The system allows for acoustic location mechanism using virtually inaudible sound without any special sensors, making itself simpler in structure and easier to implement. As a music sequencer, IAMHear enables the user to make music by placing objects on table. Pitch and timbre of sounds are determined by the location/orientation of tabletop objects as well as ambient noise.

  • Data transmission via TAPIR signal

Website address text is transmitted to a mobile device via TAPIR signal (sounds at high frequency bands).

Electronic pipe instrument using audio feedback

오디오 피드백을 활용한 전자 파이프 악기 디자인

[demo site]
[SMC 2011 paper]

This work presents an electronic pipe instrument with audio feedback loops consisting of speakers, microphones, low-pass filters, and amplifiers.

Resonant sounds are generated by the loops without air blowing, and they seems to be similar to the acoustic resonance of a closed tube.

In order to play the instrument, control parameters such as gain factor and cutoff frequency that change the resonances are mapped with MIDI signals using a binary and expressive mapping.


Sound synthesis of the geomungo using digital waveguide modeling

디지털 도파관 이론에 기반한 거문고 소리 합성에 관한 연구

[M.S. Thesis]
[AES Journal paper]

[demo site]

This paper presents a sound synthesis method for the geomungo, a Korean traditional plucked-string instrument, based on physical modeling. Commuted waveguide synthesis method is used as a basic synthesis algorithm in this work. Recorded geomungo tones are analyzed by estimating frequency responses and fundamental frequency curves.

A synthesis model proposed here consists of a fractional delay allpass filter, a loss filter which combines a ripple filter and an one-pole filter, and a digital delay line. Calibration of parameters in the model is done by minimizing the difference between the magnitude response of the designed filter and estimated losses of harmonic partials.

A time-varying model which has Lagrange interpolation FIR filter as a fractional delay filter, time-varying loss filter and delay line, and a gain factor to preserve energy is proposed and evaluated to generate geomungo tones which have fluctuating pitches. Gain control using a sinusoidal function in the time-varying loss filter is also discussed.

In addition, a hybrid-modal synthesis model which synthesizes the signal with the resonators at low frequencies and with the digital waveguide model at high frequencies is proposed. Finally, real-time sound synthesis application is implemented based on The Synthesis toolkit(STK). The former shows that physical modeling based on digital waveguide theory is an e.ective method to synthesize geomungo sounds.

Musical control of pipe based on acoustic resonance

[NIME 2011 paper]

[demo page]


We introduce an interactive pipe interface that recognizes touches on tone holes of the pipe by measuring, low-pass filtering, and amplifying the resonances in the pipe without touch sensors. Because this work is based on acoustic principles of woodwind instruments without complex electronics, a simple and durable interface is developed. The measured signals are analyzed to show the resonant sound changes as covering tone holes differently. The audible resonance signal in the pipe interface can be used as a sonic event for musical expression in itself, or also as an input for mapping to generate different sounds.


광주에서 걸려온 전화 Calls from Gwangju

Interactive telephone booth installation with Sung Jun Yoo and Sungbo Shim
인터랙티브 전화부스 인스톨레이션

09/2011 ~ 10/2011
Gwangju Design Biennale
Gwangju Biennale Hall, Gwangju

광주에서 결려온 전화-2


arduino + max/msp + infrared distance sensors + speaker units & amps
아두이노, Max/MSP, 적외선 센서, 스피커로 구성
– 관객이 전화부스에 다가가면 전화기에서 벨소리가 울리고, 수화기를 들면 듣고 싶은 음성 메시지를 선택하라는 안내 문구가 들린다. 이에 따라 관객이 번호를 누르면 해당 음성 메시지를 들려주는 시스템 (음성 콘텐츠는 광주에 관련된 여러가지 이야기들로 구성됨)

스크린샷 2013-10-20 오전 12.40.32