A Curious Mind with Dominic Monaghan | Cream VR | Presented by Hulu & Microsoft

A Curious Mind is a Hulu VR exclusive, presented by Microsoft. It is a pop-science, cross-platform VR series that takes users on unforgettable immersive journeys, illuminating the most mind-blowing concepts science and nature have to offer. Led by Dominic Monaghan (star of Lord of the Rings, and Lost) and supported by top shelf talent such as Phil Plait (science writer to Bill Nye and part of the Hubble Space Telescope team), with music by Electric Youth, (soundtrack to the feature film Drive). The series is sponsored by Microsoft and distributed by Hulu.

A Curious Mind was created by Toronto-based Virtual Reality content producers CreamVR, makers of projects like the “Agnus Die” VR Short-film, and “Letter From the Trench” (Discovery VR)  in collaboration with animation and visual effects studio Thought Café  and spatialized audio by Grayson Matthews. The SkyBox 360/VR plugins, (now part of Adobe CC 2018) were used for a lot of the post-production of A Curious Mind. The plugins allowed for editing directly inside Adobe Premiere Pro and After Effects, providing a robust and simple way to edit 360° footage.

To get more insights on the project and on how the Skybox plugins were used. We had the chance to speak with Andrew MacDonald & Tristan Cezair, both Creative Directors at CreamVR and the two forces behind the creation of A Curious Mind.

How did Cream VR get started on A Curious Mind?

Andrew (A): The project started a long time ago, even before 360° was as big a thing as it is now with Dave Brady (CEO at Cream VR) who for a long time, wanted to do a series in the style of Cosmos: A Space Odyssey, which you can say was the inspiration for this project.

Tell us more about Dominic Monaghan: He’s a really big deal, having played in Lord of the Rings. How did he get involved in this?

A: We knew Dominic from having worked with him in the past on Wild Things , a series for Travel channel where he explores nature and wildlife in exotic and remote areas,

A Curious Mind is also a lot about Dom. He is a curious mind and is really interested in learning new things. Not only that, he also enjoys sharing that knowledge with an audience. He’s really altruistic in that way.

How was it working with him for this project?

A: He was fantastic. Dom really is a professional and his skill was essential while filming this. It was a lot of script to deliver on camera, we couldn’t use standard teleprompters because of the 3D camera. You always fear having someone freezing when they  miss a line, but Dom was a true professional, always able to deliver a fantastic performance. We couldn’t have asked for a better host.

What role played the Mettle tools in Post?

Tristan (T): The Mettle tools were essential to our  post-production, We used them whenever we were editing in Adobe After Effects and Premiere. The bulk of the editing and post was done in After Effects, with the Mettle plugins and the final project was assembled in Premiere Pro.

The Skybox Extractor and Composer tools were very useful during pre-production planning and post-production workflows to undistort, align, and re-wrap our stereoscopic live action footage.

The Mettle tools were used to generate all of our skyboxes (Infinity backgrounds). We also needed a fast and efficient solution for the end credits and Intro which were  all done in After Effects, using Skybox studio. For instance, The 180° stereo for the credits were generated via Skybox. We rendered two instances of the video, where the camera was offset in specific ways to generate a stereo pair of video for each “eye” of the intro and end credits. Then it was all composited in After Effects and assembled in Premiere Pro.

The Mettle Effects were also used during our final finishing stages as an easy way to add additional equirectangular stereoscopic effects and filters to our final composites.

Wow! You’ve really used the Mettle tools a lot!

T: Mettle Skybox built into Adobe CC opens up a simple, robust and effective way to create and edit 360°  video, using a familiar workflow and our “go to” tool sets inside After Effects and Premiere Pro. Not only is it effective and robust but it isn’t as costly as the other alternative workflows. Creating and rendering our content in After Effects allowed us to achieve our end results in a fraction of the time and a fraction of the cost which was essential in such a demanding project.

Adobe After effects and Premiere were essential in compositing our rendered stereo CGI from Cinema4D and the tracking data that was generated using Canvas360 and PFTrack.

For the sound design, Hulu had specific requirements for the ambisonic audio delivery. For this, we had the help of a creative audio agency in Toronto whom were able to spatialize the amazing music composed by Electric Youth and add subtle complimentary sound design to take the whole thing home.

What do you think is important to keep in mind in a 360° Production like A Curious Mind?

A: Up until now, we had worked on narrative projects, where it’s tricky to have story drive a Point of Interest. You have to design a scene a certain way, so you have some sense of control of where the viewer’s gaze is directed.  This helps with smooth editing in 360, where when you transition to another scene, that transition would make sense to the story.

In a project like this, it was much easier, because we had a host, Dom, that helps to guide the gaze of the viewer by pointing and gesturing . From that perspective, a project like A Curious Mind was much easier, especially with an actor as talented as Dominic.

What camera did you use? 

A: Everything was filmed with a pair of stereo cameras in front of a green screen. We added 360° set extensions and assets in Post..

We originally tested using  a pair of Black Magic Micro-studio cameras with 2 x 220 iZugar lenses, but those lenses proved to be too wide.

T: Whilst that was great for tracking information (and depth information) we needed to capture our host at a higher resolution/pixel density across a much larger percentage of the camera’s sensor. So we opted to capture Dom specifically with a pair of  Lumix 8mm lenses on the same camera bodies and that resulted in a much more useful picture..

A: That solved one problem and created another. Reducing the FOV meant that in order to get close enough to Dom and yet capture his whole body from a natural eye level,we had to shoot from slightly above and tilt the camera downward, and that ended up creating a bit of a hurdle  for us in Post.

What sort of hurdle?

T: Because we had to tilt the Camera downwards we ran into issues when trying to solve and composite the moving camera shots into our CGI footage out of Cinema4D. Specifically the Stereo Equirectangular renderer in Cinema4D, at the time, did not support rendering a camera view that was not perfectly straight and horizontal. Since we shot pointing downwards our footage was a bit incompatible or at least it wasn’t a straightforward process to solve and composite back into our final assembly. Thankfully the fine folks at Maxon allowed us to access beta features of their software that allowed us to solve and render footage that wasn’t perfectly horizontal.

 What was the issue with the rendering?

T : We were rendering about 7 1/2 minutes of 8k footage, with lots of layered heavy effects applied to it. We needed a solution that allowed us to get back our footage within a few weeks.

A: At first we looked at SIRTnet ( Sheridan’s Screen Industry Reseach and Training centre), which was a government-funded initiative in Toronto providing fiber optic networks that connected Post production houses in the city to a cloud rendering solution.

Since our initial engagement SIRT had restructured a little bit internally to become SIRTnet. They had sent 3 racks , full of blade servers to our studios by truck, each weighing about 300-pounds. We had to install that in a room that wasn’t your typical cooled, soundproofed server room running power from the dryer and stove plug. The whole thing pulled 70 amps and sounded like a jet engine!

So how long did the rendering take?

A: We had 48 nodes in the Render Farm and it took about 14 days (day and night)  to render the whole thing.

Tell us about Hulu and Microsoft: How are they involved in the project?

A: About a year ago, Dave met with Hulu and pitched the idea. Hulu had started investing in 360° and they were commissioning new content. Microsoft came in as a sponsor and the project was released exclusively on their new Microsoft Mixed Reality headset for the first month. Followed by a wider release on HuluVR in November.  

Tell us more about the VR and AR experiences:

A: The AR version of Dom’s show is the fun, engaging, shareable experience. It’s being developed in AR and will be available across all mobile platforms. This AR experience puts Dominic in your living room, where your coffee table becomes his lab and you can engage in all sorts of volumetric interactive experiences. Using our proprietary motion capture techniques we are striving to produce the most efficient highest quality human avatars capable of running on mobile phones.

The  VR is designed to meld together the best of both the documentary/factual-entertainment space and the gaming world. The user can physically participate in the story-telling and shape the experience. The user will be able to conduct experiments with our host Dominic as he explores mind-bending scientific concepts and the “big ideas” of the universe. From manipulating time and space to pulling apart the atom, the experiences to explore are limitless and truly unique.

Many thanks to both of you! We look forward to watching the Series and seeing what you do next!