How Synthetic Media Helps Museums Create More Engaging Content

Apart from tourism, it is hard to find an industry that has suffered more from the pandemic than museums.

According to the Network of European Museum Organization (NEMO), 75% of museums reported income losses between 1,000 to 30,000 Euro per week in 2020.

There is no doubt that these figures are probably similar for countries in North America. As a result of these income losses, the most common change initiated is discontinuing programs and projects (55%), followed by reducing visitor capacity (49%), and finally, reallocating staff (42%). 

But those are not the most alarming numbers. Over 90% of museums responding to the NEMO survey reported a drop in visitation ranging from 10% to over 75% even after reopening.

What does this mean? Even if we take into account the continued long-term softening of quarantine restrictions, museum visitation has almost certainly lost the interest of the public.

In this case, retaining returning visitors should become a long-term strategy for museums. One possible tool that can help is synthetic media. In this article, we'll take a closer look at this new trend, the technologies behind it, and their applications.

The current state of museum exhibitions

Typically, museums can be divided into the following categories according to their specialization: 

• general
• natural history and natural science
• science and technology
• history
• art

Museums often host both permanent and temporary exhibitions and unite them by theme, time, or author. Except for exhibitions of rare artifacts, almost all of them lend themselves to digitalization. And this is not about creating a website with a 3D photographic model of space but designing a completely different digital experience.

Well-known exhibition halls use virtual or augmented reality technologies to revolutionize museums and expand the experience and accessibility of exhibitions for their visitors. Let's briefly go through a couple of examples.

Cairo's Egyptian Museum uses mixed reality techniques, including HoloLens holograms, to create a life-like visual representation of ancient war chariots and ancient Egyptian soldiers.

Visitors to the exposition can observe the chariots recreated with great accuracy and life-size soldiers right in the center of the museum's space. Holograms of King Tutankhamun and his consort Queen Ankhesenamun conduct a museum tour instead of a traditional guide.

The rulers of ancient Egypt explain how they used to live and demonstrate how they would use some of the thousands of tools on display.

The same kind of technology has been used in a couple of other prominent projects, including: 

• The National Holocaust Centre and Museum in the Witnessing the Kindertransport exhibition
• Intrepid, Air & Sea Museum in its Defying Gravity: Women in Space installation
• National Geographic Museum with the Becoming Jane chimpanzee story

Most of the technologies using synthetic media were used in museums even before the pandemic. But due to the size and scope of the pandemic, it became clear that museum experiences needed to adapt to the digital space to survive. 

Holograms, augmented and virtual reality

Following in the footsteps of the media market, digital holograms were the first technology to be widely used by museums.

Imagine being able to present an Egyptian pharaoh or young French emperor before the very eyes of every museum visitor in all their glory. These holograms met the requirements of museums and provided a sufficient level of immersion and innovation.

With the advent of the pandemic, everything changed. Without visitors physically attending a museum’s space, the technology tied to a specific location is rendered useless.

It became clear that museums needed to develop methods for delivering a completely remote experience for visitors. This meant that management teams would have to come up with ways to recreate both the museum space and its historical figures using interactive 3D elements. And most importantly, identify the types of technology required to complete the task.

VR goggles, now widely available, deliver the ultimate immersive experience in a virtual museum. If a museum isn’t interested in developing a space to utilize this technology, they can create online 3D spaces and make them accessible on their website. In both cases, each method expands a museum’s reach to a wider audience.

In earlier posts, we’ve covered the technical aspect of creating virtual characters in detail — VTubers and holograms. If you’re not familiar with these types of technology, take a moment to browse these posts so that you are more familiar with the tasks you need to tackle before creating unique characters for a museum.

In short, you need to create and dub 3D characters. For a museum, these can be both historical or fictional characters of a particular cultural and historical period.

In the end, it doesn't really matter what you settle on. Building a mixed reality experience using tools like Microsoft HoloLens or making your museum accessible as a remote 3D experience demands that your characters be memorable. 

How synthetic media voice cloning helps museums appeal to younger audiences

The vast majority of today's youth consume digital content. Many of them are very young people, but in 10-15 years they will make up the majority of the mainstream audience.

Modern museums are grappling with a difficult task: how, while maintaining the unique spirit of the museum space, can we attract an audience that is used to experiencing art and history in an untraditional way?

To understand what today’s technology is capable of, take a look at this Respeecher project that involved collaborating with the MIT Center for Advanced Virtuality.

In the "In Event of Moon Disaster," we re-created Nixon's hypothetical speech in the potentially unfortunate event of the Apollo 11 mission ending in disaster. On another occasion, we helped recreate a digital Vince Lombardi, the famous Green Bay Packers coach, for the 2020 Super Bowl finals.

Just imagine that you are able to accurately recreate almost any historical figure as if they lived not so long ago.

For a younger audience, the opportunity to witness a modern and lifelike Babe Ruth, Patton, or John F. Kennedy on their monitors or via a museum's augmented reality technology would be an unforgettable experience.

With the help of Respeecher, holographic history now has a unique voice. Using Respeecher's voice cloning technology, we can recreate any voice of a historical figure and synthesize unlimited audio content based using that vocal recreation.

The only condition is an hour's worth of the original voice for the intended person. Discover more about the voice cloning process by reading our voice synthesis FAQ  and downloading this whitepaper on increasing audio resolution with Respeecher to learn how audio super resolution works and how you can benefit from it.

Here are just a few of the most common use cases for synthesized voice in museums:

• A behind-the-scenes narration on behalf of a historical figure. These characters are able to comment on the events of an era firsthand.
• Creating unique voices for digital museum characters.
• Dubbing in dozens of languages (in the historical figure’s original voice as if they are a native speaker), which exponentially expands the potential audience for a digital museum.

To understand how our voice cloning technology works - drop us a line and we will walk you through the entire process. Respeecher is the only speech synthesis software that works with Hollywood Studios, leading game developers, and educational organizations.