Media Lab Intern Spotlight: Laura Chen's Virtual Reality Tour of the Met

From Don Undeen, Senior Manager of Media Lab:

Panoramas such as John Vanderlyn's View of the Palace and Gardens of Versailles were a popular form of art and entertainment for much of the nineteenth century. Patented in 1787 by Robert Barker, panoramas were large circular rooms with a continuous scene (usually a landscape or event) painted all the way around to create an illusion of depth and perspective. The sense of "being there" would then be enhanced with careful lighting and, sometimes, moving elements. A lot of the new technology of the Industrial Revolution was brought to bear, bringing the virtual-reality experience to a public thirsty for novelty. (Learn more about John Vanderlyn's work on MetPublications.)



With this in mind, it's easy to see the latest surge in virtual-reality hardware—Oculus Rift, Project Morpheus, Google Cardboard, et al.—as part of a long tradition of creative people using technology to transport audiences outside of their everyday lives. One of the things I love about Laura's Oculus Rift project is that she didn't try to create a "virtual Met," but rather used the Museum's architecture as a starting point for a fantastical world—one populated with butterflies, disembodied voices, and artworks that defy the laws of physics.

«For my Media Lab internship during the Spring 2014 semester, I decided to pair 3D models of the Met's structure I received from the Buildings Department with official Audio Guide content. The aim was to construct an immersive virtual-reality tour of the Museum, complete with 3D-scanned models of art pieces on view in the galleries of Greek and Roman Art. This environment, created with the game engine Unity, can be experienced either on a computer screen or through the virtual-reality head-mounted display Oculus Rift.»

[@portabletext/react] Unknown block type "__block", specify a component for it in the `components.types` prop

Screen shot showing the play mode in Unity, divided into two screens for Oculus Rift

With Oculus Rift, users can wander around the Museum, listen to the Audio Guide, admire artworks from the collection, walk up stairs, watch butterflies, and even enter surreal mash-up models.

The viewer, as seen inside the virtual Met

Concept and Setup
With my background as a VFX artist of 3D animation and post-production, I've always been interested in 3D and the creative ways that it can be interactive. Once I got the chance to intern in the Met's Media Lab and acquired access to 3D models of the Museum's building, I knew that I wanted to use the Oculus Rift virtual-reality headset to walk inside a fantasy version of the Met, enjoying the immersive experience in a completely virtual space.

The Basics
This project requires that one first get comfortable with using Unity, a 3D game engine. The idea is to first set up an environment (e.g., building, floor, stairs), create nonstatic objects in the scene (e.g., fire, characters), and then add a camera in order to navigate the set from a first-person perspective. Depending on the relative positions of the camera and the objects in the scene, I would write script to create the interaction between them, which included possible collisions, as well as sound and light triggers. Tons of basic knowledge about Unity can be found here, and instructions to setup a project from scratch can be found here.

Importing the BIM 3D Models into Unity
This is as simple as placing the FBX file into the Assets folder of the project. While it's not too complicated, there's one thing you should be aware of—the scale. It's good practice to determine the scale right in the modeling application before importing the model to Unity, and note that one Unity unit is equal to one meter. The fewer the game objects the better, and, if possible, use one material for all components.

Screen shot showing the process for importing BIM 3D models into Unity

Adding Collider to Meshes
In order to prevent players walking through meshes (e.g., walls, stairs), we need to add the Collider attribute on the models. First, select the model and at the inspector, then choose Add Component—Physics—Box Collider (or Mesh Collider). Mesh Collider is more specific than Box Collider but is more expensive to use, in terms of processing time.

Occlusion Culling
Occlusion Culling refers to process of removing things you aren't looking into from the software's memory, which ensures the game will run faster. First of all, make sure all the geometries have been broken into sensibly sized pieces—if you have one object that contains all the furniture, either all or none of the entire set of furniture will be culled. Secondly, tag all scene objects that you want to be part of the occlusion to Occluder Static in the Inspector. For more help with this, check out the Unity 3D manual.

Creating and Importing 3D-Scanned Models
First of all, take about twenty photos around the object that you want to 3D scan (be sure you're covering all 360 degrees!). Next, upload the photos to 123D Catch, and then you'll have both the OBJ model file and texture files! Just download the file and then drag the whole folder into Unity's Asset folder. (To learn more about making 3D models of Museum objects, check out Don Undeen's Digital Underground post "Good Photographs Make for Great Digital 3D Models"; to dowload premade 3D models of Met objects, check out the Media Lab's Thingiverse collection.

Note: While all the above steps can be done with the free version of the Unity game engine, integration with the Oculus Rift requires purchasing Unity Pro. Once you've done that, just follow the clear instructions available in this YouTube video!

Conclusion
I had a really great experience as an intern in the Met's Media Lab. My time here confirmed just how much I want to keep working in 3D and also step into the virtual-reality world with Oculus Rift. From this internship, I gained valuable knowledge from the Museum's staff, and also came to know amazing mentors and colleagues from the Media Lab. After working on this project, it occurred to me that some possible uses for the Oculus Rift experience in a museum context might be to gain accessibility for people who can't visit the Met in person, as well as to deliver a useful tool for simulating installation design. I'm sure there are many more possibilities, and I’d love to hear what you think in the comments below!

The author working on the Oculus Rift virtual-reality headset