Intentional Control of an Audio Feedback by Evolutionary Control of Second-order Structures

EvoMUSART (Part of Evostar 2015) paper

figure_installation

Aiming for high-level intentional control of audio feedback, though microphones, loudspeakers and digital signal processing, we present a system adapting toward certain sonic features. Users control the system by selecting and changing feature objectives in real-time. The system has a second-order structure in which the internal signal processing algorithms are developed according to an evolutionary process. Genotypes develop signal-processing algorithms, and fitness is measured by analysis of the incoming audio feedback. A prototype is evaluated experimentally to measure changes of audio feedback depending on the target conditions. By enhancing interactivity of an audio feedback through the intentional control, we expect that feedback systems could be utilized more effectively in the fields of musical interaction through being balanced between nonlinearity and interactivity.

Demo sounds

  • Low spectral flatness (pure tone)
  • High spectral flatness (noise signal)
  • Low spectral centroid (dark sound)
  • High spectral centroid (bright sound)
  • Low tremolo (low amplitude fluctuation)
  • High tremolo (high amplitude fluctuation)
  • Low vibrato (low frequency fluctuation)
  • High vibration (high frequency fluctuation)
  • Low pitch
  • High pitch
Advertisements

TAPIR: theoretically audible, but practically inaudible range

The “theoretically audible, but practically inaudible range” (TAPIR) is the highest area of human hearing bandwidth, which is barely perceptible for most people but can be transmitted and received by stereotypical transducers. We believe TAPIR has a strong potential as a new media for music, sonic arts, and mobile media. The research discussed includes the development of TAPIR systems for acoustic location, gesture/motion detection, and proximity detection/user identification.

TAPIR_spectrum

  • The TAPIR Acoustic Locater (TAL)
    TAL is a prototype of TAPIR-powered location system based on the time-of-arrival (ToA)
  • IAMHear (TAL-based tabletop interface)

IAMHear is a novel tabletop interface for music performance and sound making, in which smart mobile devices are used as on-table objects for interaction. The system allows for acoustic location mechanism using virtually inaudible sound without any special sensors, making itself simpler in structure and easier to implement. As a music sequencer, IAMHear enables the user to make music by placing objects on table. Pitch and timbre of sounds are determined by the location/orientation of tabletop objects as well as ambient noise.

  • Data transmission via TAPIR signal

Website address text is transmitted to a mobile device via TAPIR signal (sounds at high frequency bands).

Electronic pipe instrument using audio feedback

오디오 피드백을 활용한 전자 파이프 악기 디자인

[demo site]
[SMC 2011 paper]
Pipe_implementation

This work presents an electronic pipe instrument with audio feedback loops consisting of speakers, microphones, low-pass filters, and amplifiers.

Resonant sounds are generated by the loops without air blowing, and they seems to be similar to the acoustic resonance of a closed tube.

In order to play the instrument, control parameters such as gain factor and cutoff frequency that change the resonances are mapped with MIDI signals using a binary and expressive mapping.

Pipe_diagram_2

Sound synthesis of the geomungo using digital waveguide modeling

디지털 도파관 이론에 기반한 거문고 소리 합성에 관한 연구

[M.S. Thesis]
[AES Journal paper]

[demo site]

This paper presents a sound synthesis method for the geomungo, a Korean traditional plucked-string instrument, based on physical modeling. Commuted waveguide synthesis method is used as a basic synthesis algorithm in this work. Recorded geomungo tones are analyzed by estimating frequency responses and fundamental frequency curves.

A synthesis model proposed here consists of a fractional delay allpass filter, a loss filter which combines a ripple filter and an one-pole filter, and a digital delay line. Calibration of parameters in the model is done by minimizing the difference between the magnitude response of the designed filter and estimated losses of harmonic partials.

A time-varying model which has Lagrange interpolation FIR filter as a fractional delay filter, time-varying loss filter and delay line, and a gain factor to preserve energy is proposed and evaluated to generate geomungo tones which have fluctuating pitches. Gain control using a sinusoidal function in the time-varying loss filter is also discussed.

In addition, a hybrid-modal synthesis model which synthesizes the signal with the resonators at low frequencies and with the digital waveguide model at high frequencies is proposed. Finally, real-time sound synthesis application is implemented based on The Synthesis toolkit(STK). The former shows that physical modeling based on digital waveguide theory is an e.ective method to synthesize geomungo sounds.

Musical control of pipe based on acoustic resonance

[NIME 2011 paper]

[demo page]

pipe

We introduce an interactive pipe interface that recognizes touches on tone holes of the pipe by measuring, low-pass filtering, and amplifying the resonances in the pipe without touch sensors. Because this work is based on acoustic principles of woodwind instruments without complex electronics, a simple and durable interface is developed. The measured signals are analyzed to show the resonant sound changes as covering tone holes differently. The audible resonance signal in the pipe interface can be used as a sonic event for musical expression in itself, or also as an input for mapping to generate different sounds.

pipe_design

빨랫줄 합창단 Airer Choir

Interactive airer installation
(former ‘Dadara-sun’)
– clothesline as a metaphor for a musical interface

10/2010
Studio SEMI
KT&G SangSangMadang, Seoul
상상마당 Studio SEMI전 (v.4 전시)

09/2010
Incheon Digital Art Festival (INDAF)
Tommorow City, Incheon
인천디지털아트페스티벌 (v.3 전시)

[NIME 2011 paper]

3_1

In this paper, we discuss the use of the clothesline as a metaphor for designing a musical interface called Airer Choir. This interactive installation is based on the function of an ordinary object that is not a traditional instrument, and hanging articles of clothing is literally the gesture to use the interface. Based on this metaphor, a musical interface with high transparency was designed. Using the metaphor, we explored the possibilities for recognizing of input gestures and creating sonic events by mapping data to sound. Thus, four different types of Airer Choir were developed. By classifying the interfaces, we concluded that various musical expressions are possible by using the same metaphor.

1_1
v.1 for a demonstration in a short indoor perofrmance
(optical sensors)

2_2
v.2 is a large-scale interactive installation that can be installed both indoors and outdoors
(webcams)

3_2
v.3 for Incheon Digital Art Festival (INDAF)
(acoustic resonance in pipes)

4_2
v.4 for Studio SEMI (Simple and Easy-to-use Musical Interfaces) exhibition
(piezo sensors)