Whether it’s jazz recordings, concert footage, photographs or interviews, EPFL researchers are applying machine learning and digital museology techniques to archives of cultural data to achieve new artistic and creative experiences.
Though the word ‘archive’ often connotes endless shelves of undisturbed books or film canisters, modern digitization techniques have made cultural archives more accessible as data sources driving research and education. But members of the Living Archive Research Group, which includes EPFL’s Cultural Heritage and Innovation Center (CHC), see even greater potential in cultural archives: they want to harness the power of these data in the name of creativity.
"The Living Archive Research Group is interested in using archives to create new performances and artistic compositions, and to study the impact of such creations on the public,” explains CHC Operational Director Alain Dufaux. “The CHC’s role is to provide archives to valorize them, not only through technology or the social and human sciences, but through artistic creation itself.”
Together with colleagues Christophe Fellay of the Valais School of Art (EDHEA); Didier Grandjean and Carole Varone of the University of Geneva’s Swiss Center for Affective Sciences (CISA); and Irene Hediger from Zurich University of the Arts (ZHdk); Dufaux is working to develop novel creative and artistic experiences driven by the priceless data contained in cultural archives – from recordings of decades of performances at the Montreux Jazz and Verbier Festivals, to the personal audiovisual collections of Swiss astronaut Claude Nicollier.
The research group recently organized a workshop entitled “Archives, A Material for Emotions?” presenting the work of the Living Archive Research Group at this year’s Montreux Jazz Festival. Participants were immersed in a high-resolution acoustic environment while they listened to some of the 5,000 festival recordings preserved as part of the Montreux Jazz Digital Project; a collection that is listed in UNESCO’s Memory of the World Register and supported by the Claude Nobs Foundation as well as the CHC. Participants’ emotions and feelings were recorded as they listened, and the results will be presented and discussed by members of the research group. The workshop follows Montreux Sonic Choreography, which the research group organized at the festival’s 2021 edition.
For a deeper investigation into the relationship between music and emotions, the group is launching a larger study in collaboration with University of Lausanne professor Davide Picca. The researchers aim to evaluate emotions perceived by Montreux Jazz Festival audience members following a live concert, and then compare them to emotions experienced while discovering concert recordings on different types of platforms, including immersive installations.
A new role for archives
The researchers believe that with the development of digital methods in artificial intelligence, virtual reality, and data science, academics and artists alike have a unique opportunity to rethink and reinvent the role of archives as spaces for public engagement with culture, as well as data sources for new forms of artistic creation.
For example, the CHC is currently working with the College of Humanities Laboratory for Experimental Museology (eM+), led by Sarah Kenderdine, as part of a new Sinergia-funded research project that aims to create an immersive platform for displaying archives to the public. This platform will facilitate interactive participation by audience members as well as the integration of musical improvisations by live musicians. The eM+ will also integrate Nicollier’s archives, digitized by the CHC, into the upcoming Cosmos Archaeology exhibition. To be held in September at EPFL Pavilions, the exhibition will use virtual reality and exabyte-sized astronomical datasets to “reveal the Cosmos as a moving living structure”.
Fellay, Grandjean and Dufaux also foresee future projects using new compositions and artistic creations to fill gaps in collections like the Montreux Jazz archive; creating pieces using artificial intelligence; creating performances allowing audiences to virtually participate by adding their own musical elements; and analyzing the emotions of audiences and musicians resulting from such experiences.
Dufaux notes that a paradigm shift is anticipated in the music industry as well as in study of cultural archives, especially in the wake of the pandemic and in the context of evolving technologies like NFTs (non-fungible tokens) and the metaverse. “One of our main goals is to reflect on the form that the production of musical archives should take in the future, based on the needs of artists and the public,” he says.