Root repository of the BCHJam project.
BCHJam that won the 1st place in the 2024 br41n.io for the category BCI Programming & Arts.
BCHJam is a BCMI for Live Music Performance in Shared Mixed Reality Environments.
Thanks to its distributed OSC components it allows networked music performance according to the Internet of Musical Things paradigm.
BCHJam features a distributed architecture that communicates over the network using the Open Sound Control (OSC) protocol.
The musician is represented on the left, with both their instrument and their BCI device. The signals from the BCI are streamed via Bluetooth and processed by a computer running the BCI-console, where active and passive signal processing is performed. Signals from the active BCI selection of targets are sent via the OSC protocol over Wi-Fi to a DAW running a number of virtual audio effects. The audio signal from the musician’s instrument is also sent to the DAW for processing. Passive BCI signals (alpha and beta waves) are sent to both the DAW and one or more XR headsets, worn by audience members. Finally, the audio reaches the audience member either through acoustic propagation from a set of speakers (in the case of a co-located performance) or through an audio transmission system over the network (for remote performances).
https://www.youtube.com/live/Gvx00c3obz0?si=OuJIG-r6ZcV2c6XE&t=15624
https://www.youtube.com/live/Gvx00c3obz0?si=2BEH6Bxqbo8EHCO-&t=21693
This repository contains the control interface developed in Unity.
The project depends on the Unicorn Unity Interface to build the Brain-Computer Interface commands and data stream.
Finally, we used Open Sound Control (OSC) to communicate between the BCI and the audio effects. The OSC property senders can be customized and redirected to multiple receivers.
The project is tested on Unity 2022.3.xx
CIMIL Underground Team, 2024
Alberto Boem
Gregorio Andrea Giudici
Michele Romani
Domenico Stefani
Devis Zanoni
Please remember to cite this work if you use it in your project.
@INPROCEEDINGS{10704087,
author={Romani, Michele and Giudici, Gregorio Andrea and Stefani, Domenico and Zanoni, Devis and Boem, Alberto and Turchet, Luca},
booktitle={2024 IEEE 5th International Symposium on the Internet of Sounds (IS2)},
title={BCHJam: a Brain-Computer Music Interface for Live Music Performance in Shared Mixed Reality Environments},
year={2024},
volume={},
number={},
pages={1-9},
keywords={Headphones;Instruments;Ecosystems;Music;Mixed reality;Visual effects;Brain-computer interfaces;Internet;Wireless fidelity;Tuning;Brain-computer interfaces;Mixed Reality;Performance Ecosystem;Internet of Musical Things},
doi={10.1109/IS262782.2024.10704087}}