Web Based Multi-sensor Instruments For Everyone

This online installation is a set of musical instruments created with Web Technologies, available online at https://codepen.io/collection/nxPrgG. The instruments are built for mobile phones using the touch screen and gyroscope as input sensors to control pitch, timbre, rhythm and other musical qualities. The instruments are built using new XML-based format called WebAudioXML (accepted paper for Sound And Music Computing Conference 2020). This technology makes it possible for everyone with basic knowledge of HTML and XML to build advanced web audio applications with complex audio node structures and sensor data mapping to audio parameters. The syntax relies on XML only and no programming language is required. WebAudioXML has been tested in 20+ student projects during spring 2020 at the Royal College of Music in Stockholm and has enabled the students to create innovative interactive digital instruments using their mobile phone and an online XML editor. This was extra valuable in times of the Covid-19 lock-down when the students didn’t have access to any hardware or facilities at the campus.

Here is one example of an instrument built with WebAudioXML. It is a three note, polyphonic synthesizer for three fingers and a touch screen. The position of the fingers controls the pitch of oscillators and the cutoff frequencies of biquad filters. The pitches can be sliding freely or snapped to any scale and tuning.

See the Pen MultiTouch 3-voice Synth and Delay by Hans Lindetorp (@hanslindetorp) on CodePen.dark

Introduction and presentation

For a video introduction and guide for how to manipulate the code, please see the presentation video below:

Explore:

Please use the explorative area on Codepen for checking out the different instruments and trying out making your own using WebAudioXML:

https://codepen.io/collection/nxPrgG

There is also a Github page with source code available for users that wants do dive deeper into building advanced instruments using WebAudioXML:

https://github.com/hanslindetorp/WebAudioXML

Tutorial to learn the basics of the WebAudioXML syntax:

WebAudioXML documentation:

https://github.com/hanslindetorp/WebAudioXML/wiki

Participate:

If you create an interesting instrument using this technology, please don’t hesitation to participate in the online sharing of Web Audio Instruments by posting a comment including a URL to your instrument. Also, feel free to contact me regarding ideas for further development of WebAudioXML and how it can contribute to the NIME community.

5,589 thoughts on “Web Based Multi-sensor Instruments For Everyone”