Vision
The scientific vision for the Internet of Sounds research field has been originally proposed in the following journal articles, but it constantly evolves as the field progresses:
- Turchet, L., Lagrange, M., Rottondi, C., Fazekas, G., Peters, N., Østergaard, J., Font, F., Bäkström, T., & Fischione, C. (2023). The Internet of Sounds: Convergent Trends, Insights and Future Directions. IEEE Internet of Things Journal. PDF
- Turchet, L., Fischione, C., Essl, G., Keller, D., & Barthet, M. (2018). Internet of Musical Things: Vision and challenges. IEEE Access, 6, 61994-62017. PDF
- Turchet, L., Fazekas, G., Lagrange, M., Ghadikolaei, H. S., & Fischione, C. (2020). The Internet of Audio Things: State of the art, vision, and challenges. IEEE Internet of Things Journal, 7(10), 10233-10249. DOI
The IoS field addresses musical and non-musical domains in networked contexts. We see the IoS as the union of the two paradigms of the Internet of Musical Things (IoMusT) and the Internet of Audio Things (IoAuT). Both fields have been envisioned as sub-fields of the general IoT field. Therefore, the IoS is a specialization of the IoT, where one of the prime objectives is to enable processing and transmission of musical and non-musical data and information related to sound and sound-based communication and interaction.
In the proposed vision, the IoS enables the connection of digital and physical domains by means of appropriate information and communication technologies, fostering novel applications and services based on musical and non-musical information. The IoS enables the integration and cooperation among heterogeneous devices with different sensing, computational, and communication capabilities and resources, in musical and non-musical contexts as well as in co-located and remote settings.
In March 2023, the IEEE, based on a proposal of the IoS community, has launched an IEEE Emerging Technology Initiative on the Internet of Sounds.
Research agenda
The following points indicate a roadmap for guiding a research agenda towards the solution of the most relevant open challenges for the IoS that will allow to progress the field:
1) To progress the design of embedded platforms specific for the IoS, including the integration of methods for low-latency processing, and the support to the latest generation of cellular networks;
2) To progress the design of reliable, autonomous sensing devices to better monitor and understand our environment through the audio modality;
3) To progress the design of Sound Things, with new solutions for the analysis of sound-related information based on the edge computing paradigm and the most advanced machine listening approaches;
4) To advance the current connectivity infrastructure, with the implementation of novel interoperable protocols for the exchange of sound-related information;
5) To define standards (e.g., for protocols, shared ontologies, formats, APIs) that will allow one to reduce fragmentation and facilitate interoperability among Sound Things as well as the services they offer including audio processing and storage; such endeavors could entail the creation of dedicated Web of Sound Things architectures;
6) To create advanced IoS ecosystems, in both the musical and audio domain, which include enabling technologies and communities of users interacting with them;
7) To investigate ethical concerns and define appropriate measures to address them; this includes the definition of principles for an Ethical IoS that can inform design, development, and evaluation of IoS ecosystems, their hardware and software components, and the interactions of stakeholders; moreover, this entails tackling the challenges of i) privacy and security of personal data, with a “privacy by design” approach; ii) sustainability aspects at all levels, from design and manufacturing to usage and long term adoption; iii) inclusiveness and accessibility, conceiving systems for various categories of people;
8) To explore the integration of haptic feedback and motion tracking mechanisms in those IoS scenarios where gestural data play a fundamental role (e.g., in the context of remote music teaching) or when enhanced immersion in artificial soundscapes is required;
9) To devise methods capable of minimizing the need for the user to configure the Sound Things: the Sound Things should self-configure and automatically adapt to changing environments and use cases.
10) To ease the adoption by the industry of IoS related technologies by providing end-to-end systems that are easy to deploy and to adapt to new needs;
11) To investigate novel network communication protocols for the IoS that will be compatible with the network slicing concept introduced in 5G;
12) To establish distributed machine learning tasks (training and inference) that are robust to the data missing and data imbalance problems and that can run over optimized or ad-hoc communication protocols so that the computations will be reliable and with limited latencies;
13) To develop cognitive processing methods to fuse acoustic signals from ad-hoc microphone arrays into a single high-quality audio stream;
14) To design processing frameworks that allows dynamic distribution of audio processes within the network with service guarantees e.g., to take advantage of both low-latency edge computing and scalable cloud processing;
15) To devise new temporally accurate datasets and methods for the real-time analysis and classification of musical and non-musical sonic content on embedded systems;
16) To integrate the blockchain into the IoS, and create effective and efficient applications based on such integration such as secure audio distribution and traceability.