Arbolitos felices: How Cinedigm is using AI to dub Bob Ross

The Big Story

AI dubbing is helping Bob Ross go global

The late Bob Ross has been a double-edged sword for Cinedigm : When the video distribution company launched a 24/7 Bob Ross streaming channel in April 2020, it quickly became one of Cinedigm's most popular channels.

However, when Cinedigm executives were looking for ways to bring the channel to international audiences, they quickly realized that it was prohibitively expensive to hire dubbing professionals to localize the show's 30-some seasons. At the same, Cinedgm couldn't afford not to take Ross abroad.

"It's kind of a no-win situation for us," Cinedigm Chief Product and Technology Officer Tony Huidor told me this week.

Now, Cinedigm is using AI to dub Bob Ross for Spanish-speaking audiences in Latin America. The company has teamed up with U.K.-based AI translation specialist Papercup to dub the show, and hopes to tackle additional languages in the near future.

• Papercup's tech is automatically transcribing the videos, then translating the text and ultimately adding a synthetic voice as a dubbing track.
• The process still involves some manual quality control, but Papercup is able to turn around a whole season of the show in one to two weeks.
• That's not only faster than traditional dubbing, but also a lot cheaper. "It's costing us 20% of what it would cost us If we were to send this out to get localized and dubbed in a traditional dubbing process," Huidor said.

Key to making AI work for the show was to pick a suitable voice that doesn't aim to sound like Ross, but can convey some of the original's calming qualities and positive vibe ("happy little accidents!").

• Papercup has been developing its own custom AI voices, CEO Jesse Shemen told me. "We've created our own voices that are specifically suited for video," he said.
• Using standard text-to-speech voices developed for call centers or voice assistants for videos like these would lead to "a very jarring experience," he said.
• AI dubbing works especially well for content like the "The Joy of Painting," since it features just one speaker and not too much action or range of emotions.
• A show like Netflix's "Squid Game," on the other hand, wouldn't work just yet. "Far too much emotional complexity," Shemen noted.

Sooner or later, a lot more content will be AI-dubbed, despite those shortcomings.

• Cinedigm is already considering using the technology for additional projects, including documentaries and reality shows. The company also wants to bring Ross to new markets in Europe, Asia and elsewhere.
• Papercup is also talking to a lot of clients outside of Hollywood, including multinational corporations that are regularly producing massive amounts of internal training videos. "They have to spend a ridiculous amount to localize and dub," Shemen said.
• AI-dubbed content may even come to U.S. audiences soon: Huidor told me that he is already thinking about dubbing foreign content in English this way. "We're very much interested in bidirectional use of the technology," he said.

You can check out an example of Papercup's synthetic dubbing of the Bob Ross show here.

Overheard

"The thing about Hollywood is, you can behave badly, you can be rude, you can make duds, but the thing you cannot do is fuck with people's money. You just don't do that and hide behind technology as the reason why." — An anonymous producer venting to the Hollywood Reporter about Disney's decision to move some of its films from theaters to streaming.

"They never pivoted. Ever. Over nine years." — Media analyst Andrew Rosen, pointing out another reason why Ozy failed.

A MESSAGE FROM ZOOM

Zoom — the communications platform that has become synonymous with streaming video calls — experienced an even greater increase in that same time period. In June 2020, close to 3 million consults took place through Zoom's 100 top EPIC integrations. With its simple, reliable interface that patients — and health care workers — already knew, Zoom became the go-to for health care offices everywhere.

Learn more

Watch Out

Light Field Lab gets ready to ship holographic displays

Holographic display startup Light Field Lab is getting ready to commercialize its technology: The company will begin signing up customers for its SolidLight platform, and expects to ramp up production of SolidLight panels next year.

Light Field Lab is looking beyond producing yet another display. It's been working on holographic display technology since 2016, and wants to build an end-to-end platform that includes the hardware necessary to render holograms in real time as well as authoring software to create holographic content.

• I was able to check out an early demo of Light Field Lab's tech three years ago, and it was impressive: The demo included videos of butterflies and fish moving through thin air as well as 3D dioramas.
• Those early demos were based on a 4-by-6-inch display; to commercialize its technology, Light Field Lab is now building 28-inch display modules consisting of multiple of these panels, which themselves can then be combined to make TV- or even video wall-sized holographic displays.
• The company didn't release pricing information for these SolidLight displays, but said that customers should expect to pay about as much as a premium fine pitch video wall would cost them.
• Light Field Lab has already sold a couple of its pre-production units to customers, and expects them to show up in the field as early as next year.

Holographic video walls could be a big boon for location-based entertainment as well as corporate use cases. However, there's a reason the company's funders include telcos like Verizon, Comcast and Liberty Global: Ultimately, Light Field Lab wants to produce holographic displays for the consumer market, and supplying those displays with 3D content will require massive broadband pipes.

Fast Forward

• On Protocol: Netflix tops App Store charts, thanks to 'Squid Game.' It's the first time Netflix's app has held that spot since at least 2014.
• Netflix has brought its "play something" feature to Android. The shuffle button for indecisive binge viewers was previously only available on smart TVs and streaming devices.
• NBCUniversal and Google strike a YouTube TV deal. NBC stations and other networks will remain on YouTube TV, and Google apparently doesn't plan a price increase — at least for now.
• On Protocol: Unity strikes a deal with UFC to bring real-time 3D to sports. The company's new Metacast tech promises to give sports fans video-game-like superpowers.
• Pay TV providers will lose $33.6 billion to cord cutting over the next five years. S&P Kagan's latest outlook is bad news for Comcast, DIRECTV & Co.
• IGTV is sorta kinda dead. Instagram's ambitious longform video offshoot is being folded into the main IG app.

Auf Wiedersehen

Dear readers, I need your help! Halloween is just around the corner, and I once again struggle to come up with a good costume idea. My general preference are costumes that are very low effort: One year, I dressed up as a DVD pirate (eyepatch + DVD "necklace"). Last time, I was simply wearing a beak and a sign that read "Toucan do it!" Suffice to say, my kids were not impressed. So how can I wow them without spending hours on a proper getup? Any and all suggestions welcome — and I promise I'll post a picture if I end up using one of them.