Activities

The Sound Music Movement Interaction team (previously known as the Real-Time Musical Interactions team) carries out research and development on interactive systems dedicated to music and performances.

Our work relates to all aspects of the interactive process, including the capture and multimodal analysis of the gestures and sounds created by musicians, tools for the synchronization and management of interaction, as well as techniques for real-time synthesis and sound processing. These research projects and their associated computer developments are generally carried out within the framework of interdisciplinary projects that include scientists, artists, teachers, and designers and find applications in creative projects, music education, or in digital audio industrial fields.

Resources of the team

Interest Areas

Modeling and Analysis of Sounds and Gestures. This theme covers the theoretical developments concerning the analysis of the sound and gesture flow, or more generally, multi-modal temporal morphologies. This research concerns diverse techniques for audio analysis, the study of the gestures of performing musicians or dancers.

Technologies for Multimodal Interaction. This theme concerns our tools for analysis and multimodal recogntion of movements and sound; tools for synchronization (gesture following, for example) and visualization.

Interactive Sound Synthesis and Processing. This focuses essentially on synthesis and sound processing methods based on recorded sounds or large sound bodies.

Systems for Gesture Capture and Augmented Instruments. This theme focuses on the developments the team has made in terms of gestural interfaces and augmented instruments for music and performances.

National and European Research Projects

  • MIM : Enhancing Motion Interaction through Music Performance
  • Rapid-Mix : Real-time Adaptive Prototyping for Industrial Design of Multimodal Interactive eXpressive technology
  • Cosima : Collaborative Situated Media
  • EFFICAC(e) : Extended Frameworks For 'In-Time' Computer-Aided Composition
  • SKAT-VG : Sketching Audio Technologies using Vocalizations and Gestures
  • MusicBricks : Musical Building Blocks for Digital Makers and Content Creators
  • Gemme : Musical Gesture: Models and Experiments
  • Physis : Modeling, Transformation, and Synthesis of Sounds for Interactive Virtual Worlds
  • Legos : Sensori-motor learning in gesture-based interactive sound systems
  • Wave : Web Audio: Editing/Visualization
  • Voice4Games : Voice Processing for Video Game Production
  • Sample Orchestrator 2
  • Topophonie
  • Same : Sound And Music for Everyone Everyday Everywhere Every way
  • Interlude : New Digital Paradigms for Exploration and Interaction of Expressive Movement with Music.
  • Sample Orchestrator
  • SemanticHIFI : Browsing, listening, interacting, performing, sharing on future HIFI systems

Software (design and development)

Specialist Areas

Interactivity, real-time computer science, human-computer interaction, signal processing, motion capture, modeling sound and gesture, statistical modeling and automatic learning, real-time sound analysis and synthesis.

Participants

Head Researcher: Frédéric Bevilacqua
Researchers/developers: Éric Boyer, Riccardo Borghesi, Gabriel Meseguer Brocal, Gaël Dubus, Benjamin Matuszewski, Jean-Philippe Lambert, Norbert Schnell, Diemo Schwarz
Doctoral Students: Marco Antonio Suarez-Cifuentes, Emmanuel Fléty, Kevin Sanlaville

The Sound Music Movement Interaction team is a part of the Science and Technologies of Music and Sound Laboratory (Unité Mixte de Recherche UMR9912).

                  

Collaborations

Goldsmiths University of London (United Kingdom), Phonotonic (Paris), McGill University (Canada), CNMAT Berkeley (USA), Genoa University (Italy), ENSCI (Paris), LIMSI (Université Paris-Sud), ENSAD (Paris), GRAME (Lyon), ICK Amsterdam, Atelier des Feuillantines (Paris), Association Puce Muse (Rungis), Univers Sons/Ultimate Sound Bank (Paris), Cycling’74 (USA), Da Fact (Paris), Voxler (Paris), No Design (Paris), HKU (Hilversum), UserStudio (Paris), BEK (Norway), IEM (Austria), STEIM (Amsterdam), PSA (France), University Pompeu Fabra (Spain), Nokia Research Center (Finland), TKK (Finland), KTH (Sweden), UCL (United Kingdom), UCSD (USA)

Publications

Journal articles

    Refereed journals

    • B. Caramiaux, F. Bevilacqua, T. Bianco, N. Schnell, O. Houix, P. Susini « The Role of Sound Source Perception in Gestural Sound Description », ACM Transaction on Applied Perception, vol. 11, n° 1, 2014
    • B. Caramiaux, N. Montecchio, A. Tanaka, F. Bevilacqua « Adaptive Gesture Recognition with Variations Estimation for Interactive Systems », ACM Transactions on Interactive Intelligent Systems, 2014
    • B. Caramiaux, J. Françoise, N. Schnell, F. Bevilacqua « Mapping Through Listening », Computer Music Journal, 2014
    • O. Houix, N. Misdariis, P. Susini, F. Bevilacqua, F. Gutierrez « Sonically Augmented Artifacts: Design Methodology Through Participatory Workshops », Sound, Music, and Motion. Lecture Notes in Computer Science, Décembre, 2014
    • S. Manitsaris, A. Glushkova, F. Bevilacqua, F. Moutarde « Capture, modeling and recognition of expert technical gestures in wheel-throwing art of pottery », ACM Journal on Computing and Cultural Heritage, 2014
    • D. Schwarz « Interacting with a Corpus of Sounds », eContact!, vol. 2, n° 16, Août, 2014
    • B. Zamborlin, F. Bevilacqua, M. Gillies, M. d'Inverno « Fluid gesture interaction design: applications of continuous recognition for the design of modern gestural interfaces », ACM Transactions on Interactive Intelligent Systems,, vol. 3, n° 4, 2014
    • E. Boyer, B. Babayan, F. Bevilacqua, M. Noisternig, A. Roby-Brami, S. Hanneton « From ear to hand: the role of the auditory-motor loop in pointing to an auditory source », Frontiers In Computational Nuroscience, vol. 26, n° 7, Avril, 2013
    • S. Fdili Alaoui, F. Bevilacqua, B. Bermudez, C. Jacquemin « Dance Interaction with physical model visualization based on movement qualities », International Journal of Arts and Technologies, vol. 4, n° 6, 2013
    • B. Hackbarth, N. Schnell, P. Esling, D. Schwarz « Composing Morphology: Concatenative Synthesis as an Intuitive Medium for Prescribing Sound in Time », Contemporary Music Review, vol. 32, n° 1, Avril, 2013
    • F. Bevilacqua, F. Baschet, S. Lemouton « The Augmented String Quartet: Experiments and Gesture Following », Journal of New Music Research, vol. 1, n° 41, 2012
    • T. Bianco, V. Freour, I. Cossette, F. Bevilacqua, R. Caussé « Measures of facial muscle activation, intraoral pressure and mouthpiece force in trumpet playing », Journal of New Music Research, 2012
    • B. Caramiaux, M. Wanderley, F. Bevilacqua « Segmenting and Parsing Instrumentalists' Gestures », Journal of New Music Research, vol. 1, n° 41, Avril, 2012
    • N. Rasamimanana « Towards a Conceptual Framework for Exploring and Modelling Expressive Musical Gestures », Journal of New Music Research,, vol. 1, n° 41, 2012
    • B. Bermudez, S. DeLahunta, M. Hoogenboom, C. Ziegler, F. Bevilacqua, S. Fdili Alaoui « The Double Skin/Double Mind Interactive Installation », The Journal for Artistic Research (JAR), Janvier, 2011
    • A. Bonardi, F. Rousseaux, D. Schwarz, B. Roadley « La collection numérique comme modèle pour la synthèse sonore, la composition et l'exploration multimédia interactives », Revue Musimédiane, n° 6, Juin, 2011
    • J. Bloit, N. Rasamimanana, F. Bevilacqua « Modeling and segmentation of audio descriptor profiles with Segmental Models », Pattern Recognition Letters, 2010
    • A. Bonardi, F. Rousseaux « Interactions médiatisées sur scène », TECHNIQUE ET SCIENCE INFORMATIQUES, vol. 29, n° 8, Décembre, 2010
    • A. Cont « A coupled duration-focused architecture for realtime music to score alignment », IEEE Transaction on Pattern Analysis and Machine Intelligence, vol. 32, n° 6, Juin, 2010
    • N. Rasamimanana, F. Kaiser, F. Bevilacqua « Perspectives on gesture-sound relationships informed from acoustic instrument studies », Organised Sound, vol. 14, n° 2, 2009
    • N. Rasamimanana, F. Bevilacqua « Effort-based Analysis of Bowing Movements: Evidence of Anticipation Effects », Journal of New Music Research, vol. 37, n° 4, 2008
    • D. Schwarz, E. Brunet « theconcatenator CD Contributors' Notes », Leonardo Music Journal, n° 18, Novembre, 2008
    • S. DeLahunta, F. Bevilacqua « Sharing Descriptions of Movement », International Journal of Performance and Digital Media, vol. 1, n° 3, 2007
    • D. Schwarz « Corpus-Based Concatenative Synthesis : Assembling sounds by content-based selection of units from large sound databases », IEEE Signal Processing, vol. 24, n° 2, Mars, 2007
    • D. Schwarz « Concatenative Sound Synthesis: The Early Years », Journal of New Music Research, vol. 35, n° 1, Mars, 2006

    Non-refereed journals

    • E. Boyer, F. Bevilacqua, F. Phal, S. Hanneton « Low-cost motion sensing of table tennis players for real time feedback », International Journal of Table Tennis Sciences, vol. 8, Décembre, 2013
    • F. Guédy « L'expérimentation musicale et pédagogique sur le geste de l'Atelier des Feuillantines avec l'équipe Interaction Musicale Temps Réel de l'Ircam », L'Education Musicale, 2011

    Book sections

    • E. Boyer, Q. Pyanet, S. Hanneton, F. Bevilacqua « Learning Movement Kinematics with a Targeted Sound », Sound, Music, and Motion. 10th International Symposium, CMMR 2013, Marseille, France, October 15-18, 2013. Revised Selected Papers, (Spinger), 2014
    • D. Schwarz, B. Caramiaux « Interactive Sound Texture Synthesis Through Semi-Automatic User Annotations », Sound, Music, and Motion. Lecture Notes in Computer Science, Vol. 8905, ed. Aramaki, M., Derrien, O., Kronland-Martinet, R., Ystad, S. (Springer International Publishing, Marseille), 2014
    • A. Dessein, A. Cont, G. Lemaitre « Real-time detection of overlapping sound events with non-negative matrix factorization », Matrix Information Geometry, ed. Nielsen, Frank and Bhatia, Rajendra (Springer, Marseille), 2013
    • A. Dessein, A. Cont, G. Lemaitre « Real-time detection of overlapping sound events with non-negative matrix factorization », Matrix Information Geometry, ed. Nielsen, Frank and Bhatia, Rajendra (Springer, Marseille), 2012
    • F. Bevilacqua, N. Schnell, S. Fdili Alaoui « Gesture capture: Paradigms in interactive music/dance systems », Emerging Bodies, ed. G. Klein and S. Noeth (transcript Verlag, Marseille), 2011
    • F. Bevilacqua, N. Schnell, N. Rasamimanana, B. Zamborlin, F. Guédy « Online Gesture Analysis and Control of Audio Processing », Musical Robots and Interactive Multimodal Systems. Springer Tracts in Advanced Robotics Vol 74, ed. Jorge Solis and Kia C. Ng (Springer Verlag, Marseille), 2011
    • A. Bonardi « Les Petites Absences. Du théâtre à la musique en passant par l'informatique temps réel », Les interactions entre musique et théâtre, ed. Guy Freixe et Bertrand Porot (Editions l'Entretemps, Montpellier), 2011
    • N. Schnell, F. Bevilacqua, F. Guedy, N. Rasamimanana « Playing and Replaying ?? Sound, Gesture and Music Analysis and Re-Synthesis for the Interactive Control and Re-Embodiment of Recorded Music », Klang und Begriff. Gemessene Interpretation - Computergestützte Aufführungsanalyse im Kreuzverhör der Disziplinen, ed. Heinz von Loesch, Stefan Weinzierl (Schott Verlag, Mainz), 2011
    • F. Bevilacqua, B. Zamborlin, A. Sypniewski, N. Schnell, F. Guédy, N. Rasamimanana « Continuous realtime gesture following and recognition », Gesture in Embodied Communication and Human-Computer Interaction. Lecture Notes in Computer Science (LNCS) volume 5934, (Springer Verlag, Mainz), 2010
    • T. Bianco, V. Freour, N. Rasamimanana, F. Bevilacqua, R. Caussé « On gestural variation and coarticulation effects in sound control », Gesture in Embodied Communication and Human-Computer Interaction. Lecture Notes in Computer Science (LNCS) volume 5934, (Springer Verlag, Mainz), 2010
    • B. Caramiaux, F. Bevilacqua, N. Schnell « Towards a Gesture-Sound Cross-Modal Analysis », Gesture in Embodied Communication and Human-Computer Interaction. LNAI 5934, (Springer Verlag, Mainz), 2010
    • S. Dahl, F. Bevilacqua, N. Rasamimanana, R. Bresin, M. Clayton, L. Leante « Gesture in Performance », Musical Gestures. Sound, Movement, and Meaning, (R. I Godoy and M. Leman, Mainz), 2009
    • R. Cahen, C. Jacquemin, D. Schwarz « Navigation et composition dans un univers graphique 3D de grains sonores », Information & Visualisation: Enjeux, recherches et applications, ed. Sophie Chauvin (Cépaduès Editions, Toulouse), 2008
    • F. Bevilacqua « Momentary notes on capturing gestures », capturing intentions, ed. Scott deLahunta (Emio Greco/PC and the Amsterdam School fo the Arts, Toulouse), 2007
    • A. Cont, S. Dubnov, G. Assayag « Anticipatory Model of Musical Style Imitation using Collaborative and Competitive Reinforcement Learning », Anticipatory Behavior in Adaptive Learning Systems, (Martin Butz and Olivier Sigaud and Gianluca Baldassarre, Berlin), 2007
    • N. Rasamimanana, D. Bernardin, M. Wanderley, F. Bevilacqua « String Bowing Gestures at Varying Bow Stroke Frequencies: A Case Study », Advances in Gesture-Based Human-Computer Interaction and Simulation, (Springer Verlag, Berlin), 2007
    • F. Bevilacqua, N. Rasamimanana, N. Schnell « Interfaces gestuelles, captation du mouvement et création artistique », L'inouï #2. Revue de l'Ircam, (Editions Léo Scheer, Berlin), 2006
    • N. Rasamimanana, E. Fléty, F. Bevilacqua « Gesture Analysis of Violin Bow Strokes », Lecture Notes in Computer Science, Vol. 3881. Gesture in Human-Computer Interaction and Simulation: 6th International Gesture Workshop, GW 2005, Berder Island, France, May 18-20, 2005, Revised Selected Papers, ed. Sylvie Gibet, Nicolas Courty, Jean-François Kamp (Springer Berlin / Heidelberg, Berlin), 2006

    Conference proceedings articles

    Phd Thesis

    Training course reports

    Reports

    Edited books / Monographs

    • F. Bevilacqua, S. Fdili Aloui, F. Jules, P. Pasquier, T. Schiphorst « Proceedings of the 2014 International Workshop on Movement and Computing (MOCO 14) », (ACM, New York), 2014
    • A. Bonardi « Représentations opérationnelles : L'opéra entre Berio et Turing », ed. Francis Rousseaux (Editions Delatour France, Sampzon), 2008
    S M T W T F S
            1 2 3
    4 5 6 7 8 9 10
    11 12 13 14 15 16 17
    18 19 20 21 22 23 24
    25 26 27 28 29 30 31
                 
    Previous monthPrevious dayNext dayNext month

    The french Gateway to Contemporary Music ResourcesThe french Gateway to Comtemporary Music Resourcesclose

    Veuillez installer Flash pour afficher ce lecteur.