Writer: Bodil Malmström

Faculty of Arts at Lund University has together with LTH, School of Economics and Faculty of Law, as well as a number of external partners such as AXIS and Amazonas, a grant of three million in three years has been granted to develop new methods of producing and making culture available. The thematic collaboration initiative is led by Sanimir Resić, dean at K-fak and coordinated by Jesper Larsson, with a solid background in the cultural sector from previous directorships of Malmö Live and Kulturhuset Stadsteatern in Stockholm.

“The rapid digitisation of the performing arts has opened up new worlds for how audiences can take part in culture where we need to think again so that consumers become active instead of passive.

This is the opinion of Jesper Larsson who won guides around the concert hall at Malmo Live which has world-class acoustics and is the home arena of the Malmö Symphony Orchestra. In the project, Malmó Live concert hall will serve as an experimental environment in the form of a living lab.

In the future streaming service, you should be able to choose which camera angles or clipping you prefer, whether you want to zoom in on any instrument, and also choose whether you want any instrument to sound louder. If you are curious about someone else's version of the concert, you can also enjoy it. Through facial recognition, you will receive information about the people in the picture and you will also chat with other visitors and ask questions to the artists before the concert and during the break.

The digital format also opens up new visual possibilities.

“When we move the concert experience out of the geographical location into a digital universe, we are not tied visually to what it looks like in the concert hall. We could experience the concert as a hike in the Alps or a psychedelic AI-generated landscape to lose oneself in or why not from inside a clarinet?

The aim of this thematic collaboration initiative is to reach out to new groups that do not usually take place in the concert hall at Malmö Live.

“We could connect schools and libraries and provide a good platform for children and young people. We also want to reach out to nursing homes and prisons. At the same time, the project has a huge upside — we are not going to ruin or change the format for those who already love this.

During the pandemic, Malmö Live used cameras from AXIS Communication, a leading network camera company, to stream its concerts to audiences. A positive experience that has now grown bigger. The ultimate solution for a personalized live broadcast places great demands on new digital solutions, including artificial intelligence, the development of new business models and legal issues around copyright and personal privacy, among other things. In order to achieve the goal that the digital audience at home in the sofa will have an experience at least as amazing as a physical audience in the concert hall, the breadth of Lund University will be required.

“The basic idea is that I, as a listener, choose a seat in the concert hall. Today we have the technology in place and many solutions on how to zoom image digitally. But new technology is required for individually selected digital zoom of the sound, says Jesper Larsson.

Researchers at LTH want to investigate a more automated way of managing production so that there are not so many technicians, image producers and sound engineers required. The answer is to plug in AI.

“We want the AI to read the notes and also know where the musicians are sitting and where the cameras are standing. Let's say I want the trombone to sound stronger at a piece -- how do we zoom sound? We see a streaming service where the computer should be able to image zoom the harp in relation to the trumpet based on how the score is written and also be able to zoom the sound, says Jesper Larsson.

The technical challenges are numerous and therefore both researchers and teachers are involved at LTH.

“This project gives me a “We choose to go to the moon” feeling, says Erik Larsson, a professor at Integrated Electronics Systems who is already starting to involve students studying for a master's degree in computer engineering and electrical engineering with automation technology.

“In the autumn, the students will work on different project courses and I hope this can lead to exciting degree projects for the university engineers in the spring and that we can also make other faculties realise the potential of this exciting project.

Digitalisation poses many new requirements, not least when it comes to copyright. How should legislation keep up with the technological developments that our society is undergoing when at the same time all music is protected by copyright? Anyone who makes a living off their music needs to have an opportunity to get paid.

“It's a huge issue, especially if we're thinking of a global market. In this landscape, there will be a great deal of conflict, discussion and concern. But imagine how Spotify has revolutionized the music life. Here we connect researchers from the Faculty of Law who will delve deeper into this complexity.

When every concert event is streamed and when the cameras are more and more close than previous TV productions, it creates more pressure on the musicians. How does it affect the musicians if the cameras can constantly zoom in, not just when playing.

“Over the past 15 years, we have had great individualized opportunities to view and film athletes' achievements. Even the crowd in the stadium has been exposed. This whole thing with integrity is a field of research in itself. When it comes to our musicians, I see that sensitive situations can be recorded and disseminated and that can add to the stage fright. But we have to deal with it, if you choose the path of musicians, you cannot avoid being exposed. Unfortunately, it can be an invasion of personal privacy, says Jesper Larsson.

The initiative also wants to take a closer look at a new type of payment model and is considering a cryptocurrency where more money ends up in the pockets of the creators.

“Ordinary payment systems leak so much money to the banking system. We want more money to end up with us streaming the concert. The economy could be kept tighter, says Jesper Larsson.

Jesper Larsson and researchers from the School of Economics envision a global system that connects members around the world, where consumers will be able to buy a streamed concert from the Philharmonic in Berlin, Los Angeles or Malmö Symphony Orchestra through a point system. The range of experiences has no limitations and can become worldwide.