What We’re Learning about 360 Videos in Museums

Last November at the Museums and Computer Network conference, I was frustrated by the many times we found ourselves conflating different emerging media, as if they offered the same set of affordances. At the same time, I was amazed by the wide-ranging uses of 360 videos in museums (and aquariums) around the country.

Dance at Children's Museum
Every day is a dance party in Let’s Dance! as visitors dance with Féraba – African Rhythm Tap Company on the Dance Portal screen.

Now that I have finished sharing interviews with five different institutions, I wanted to conclude this series with some overarching findings that emerged. First, a reminder:

  1. The Shedd Aquarium is integrating 360 Video captures of their underwater exhibits (with live marine life) into school-based programs, and using ones produced by youth for visitors waiting to buy tickets. They want to learn how 360 video takes their learners to new places and enhance their experience.
  2. The Field Museum is combining footage of scientists in the field with behind-the-scenes in their collection spaces to create a story arc for their visitors to walk alongside their scientists as they engage in a process of exploration and discovery.
  3. The Children’s Museum of Manhattan is combining original and licensed 360 video to offer what they term a Dance Portal, introducing children and families to the delights of dance, in a ½ dome (which is experienced in 180 degrees).
  4. The Monterey Bay Aquarium offers an educational program in which students use Apple devices to capture 360 images of an exhibit or outdoor space then connect them together to create their own 360 tour.
  5. The Carnegie Museums of Pittsburgh offers in the Museum an interactive experience that remixes time lapse imagery taken from outside the Carnegie Museum of Art by a 360 degree camera, to appeal to the photography-curious as well as more expert photographers alike.

Do you see what I mean? If I listed just what they were used for – dance, or marine life, or scientists in the field – and their intended audience – visitors, or youth in schools, or on-site educational programs – it might seem unlikely that the same set of tools were used to support such a diverse set of activities.

Here’s what I took away:

  1. 360 Video is a Swiss Army Knife…: It is being used for so many different things: to show visitors the work of scientists, to let students in a school see inside an aquarium, to create tours of a museum trip, to engage visitors waiting to buy tickets, to teach families about dance, to create an interactive video-based data visualization… and the list goes on.
  2. … But Most Often Used to “Transport” Visitors: While used for a wide variety of reasons, the common threading is using immersive video to let viewers see and pretend to travel to something that would otherwise be inaccessible: behind-the-scenes (in the field, in the collections), within the exhibits, to dance studios. Those I interviewed often repeated the word “connect” – using 360 video to connect people with the Museum, with the work of scientists, with the natural world around them.
  3. There’s a Plethora of Content to License…: The Dance Portal relied heavily on incorporating existing 360 (and 180) video from dance troupes. The app from the Franklin Institute does something similar, aggregating 360 videos from outside sources and disseminating it through both at home use and at facilitated carts in their museum.
  4. … But Tools Are Cheap and Accessible Enough to Make Original Content: Most of the museums and aquarium were making their own original content. Some used expensive (Shedd) or custom-made (Carnegie) tools, but others are using tools cheap and simple enough to put in the hands of youth (Shedd and Monterey).

What do you think? Please read the interviews and share what you take away.

360 Videos in Museums: Shot 5 – Camera as Clock at the Carnegie Museums of Pittsburgh

At the November Museums and Computer Network conference, I marveled at the different ways museums around the country are using 360 videos with their visitors or in their education programs. To highlights these different applications, I started this limited series of interviews  – same questions, different institution – so we can look for patterns.

In our fifth in this series of interviews with museum staff around the country about their use of 360 videos, this time we hear from Caroline Record, Creative Technologist at the Innovation Studio at the Carnegie Museums of Pittsburgh, where she specializes in designing and developing software for custom exhibit-facing projects. Today I spoke with her about their use of 360 videos in  something they call the Light Clock project.

Light clock in the HallWhen did you start exploring 360 video (and if that’s not what you call it, what’s the term round your ways)?

I started exploring 360 video as the web component of the larger Light Clock project. First I will back up and explain the project as a whole.

The Hillman Photography Initiative invited the Innovation Studio to imagine a project that embodied the broad theme of a “camera as a clock for seeing”. We made a town center “clock” in the front plaza of the Carnegie Museum of Art that, rather than telling time, had a single continuously swooping solitary hand. The hand made a rotation every 5 minutes at which point the clock would take a 360 picture of the plaza.

The Light Clock. Image courtesy of Carnegie Museum of Art. Photo by Bryan Conley.
The Light Clock. Image courtesy of Carnegie Museum of Art. Photo by Bryan Conley.

An interactive visualization directly inside the museum lobby placed the museum visitor in a semi circle of screens where they could navigate around the 360 view by physically rotating their bodies left and right. The visualization didn’t display a single static 360 image, instead it displayed a constantly shifting remix of all the imagery the clock had collected up to that point. Every time the clock took a picture it would momentarily flash onto the screen, so that if you took a picture outside with the clock you could have an opportunity to see it.

The project was ambitious and had several interesting technical components. I was the creative and technical lead on the project and we created all the software and design plans in house. One of the biggest challenges was finding a robust 360 outdoor affordable camera that I could interface with through our software. I ended up working directly with a security camera manufacturer and designing a custom rig to stitch the two 180 cameras into a single 360 dome.

I wrote the software in C++ and it required a powerful computer to run well, so making a full web version would have been a significant project. Instead I modified the code I already wrote to export 360 video that could be shared and embedded via Youtube and Facebook.

What is the content you are working to put into 360 videos?

It was a remix of time lapse imagery taken from outside the Carnegie Museum of Art.

What goes into obtaining and editing the footage?

Warning: I’m going to get technical about this!

We created a custom outdoor 360 camera by mounting two 180 security cameras back to back. The cameras are triggered to save a recording to a SD card via a hard wired contact switch that is triggered every time the clock hand makes a rotation. We use the camera’s api to automatically take a screenshot from the recording. Then we use the imageMagick command line tool to mask and edit the photos. Finally we used the hugin command line tool to stitch these two images together into a single equirectangular image. Onsite my software would read in different days based off of the weather data and change the point of view based off the visitor’s movement.

To make the 360 video I wrote another version of the software that remixed all the imagery together and exported each frame as a still. Since the imagery was 4k I couldn’t edit it using my version of Adobe Premiere, so instead I used the FFmpeg command line tool to turn all of these stills into a video that we could upload to Facebook and Youtube as a 360 video.

Who is the 360 video designed for, and where will they encounter it?

The project as a whole was designed for people passing through the museum plaza and museum visitors in general. It was meant to appeal to the photography-curious as well as more expert photographers alike.

 

Wall label for the Light Clock. Photo by Jeffrey Inscho.
Wall label for the Light Clock. Photo by Jeffrey Inscho.

The web version was designed to be shared via the museum’s Youtube and Facebook page. So far the Facebook version has been viewer several thousand times and the Youtube one several hundred.

360 Videos in Museums: Shot 4 – Student Tour-making at the Monterey Bay Aquarium

At the November Museums and Computer Network conference, I marveled at the different ways museums around the country are using 360 videos with their visitors or in their education programs. To highlights these different applications, I started this limited series of interviews  – same questions, different institution – so we can look for patterns.

In our fourth in this series of interviews with museum staff around the country about their use of 360 videos, this time we hear from Katy Noelle Scott, Digital Learning Manager at the Monterey Bay Aquarium, who is not just offering visitors 360 videos but supporting youth to use the aquarium to make their very own.

Katy, when did you start exploring 360 video (and if that’s not what you call it, what’s the term round your ways)?

We started exploring 360 images and video shortly after Google Cardboard was first released about three years ago.

What is the content you are working to put into 360 videos?

As an educator, my goal is to have students create things to communicate their learning. So we work with teachers and students to leverage free and low-cost tools to create their own 360 images and tours. We have students use their own phones (or one of our department’s iPod Touches) to capture 360 images of an exhibit or outdoor space they connect with. We then support them in connecting multiple student images to create their own 360 tour of a place, such as the Aquarium.

What goes into obtaining and editing the footage?

For students and teachers, they can simply use their smartphone or iPod Touch. They can use the free app Cardboard Camera to create an almost-360 image with their own narration. They can also use the free Google Street View app to create full 360-degree images. After their images are created, they can export them into Roundme to connect the images into a tour of the space.

In addition to these options, we have also had students in our on-site teen programs use our Ricoh Theta S and GoPro Fusion cameras to capture higher quality images and videos.

Who is the 360 video designed for, and where will they encounter it?

Some teachers are creating 360 tours for their students, to better prepare them for field trips. This is especially helpful for students with special needs. Students create the tours to share with their peers, other classes, other teachers, and their families.

What are you hoping 360 video will help you or the museum to achieve?

The mission of the Monterey Bay Aquarium is to inspire conservation of the ocean. All of our education programs are designed to help students connect with the natural world. We know that one way to do that is by helping students develop a strong connection to outdoor spaces in their own communities. We hope that, through curriculum like “A 360 Sense of Place,” we can support teachers in helping students develop such connections. We also hope that students who create such tours while on a field trip to the Aquarium will be able to maintain a connection to our exhibits, even when they’re back in their classrooms.

360 Videos in Museums: Shot 3 – Dancing at the Children’s Museum of Manhattan

At the November Museums and Computer Network conference, I marveled at the different ways museums around the country are using 360 videos with their visitors or in their education programs. To highlights these different applications, I started this limited series of interviews  – same questions, different institution – so we can look for patterns.

In our third in this series of interviews with museum staff around the country about their use of 360 videos, I got to work a little closer to home. Ellen Bari, curator and writer at the Children’s Museum of Manhattan (CMOM), invited me to walk the ENTIRE two avenues that separate our two venerable institutions and learn up close what they’ve been up to these past two few year (and, if I was behaved, get to do a little dance).

 

Dance at Children's Museum
Every day is a dance party in Let’s Dance! as visitors dance with Féraba – African Rhythm Tap Company on the Dance Portal screen.

Ellen, when did you start exploring 360 video (and if that’s not what you call it, what’s the term round your ways)?

About four years ago, we were developing an exhibit called America to Zanzibar: Muslim Cultures Near and Far and wanted to give our visitors an immersive experience with Muslim architecture on a very limited budget. We were thrilled when we found Elumenati, a company that specializes in projection design. We licensed magnificent 360 images of mosque architecture from around the world and displayed them on Elumenati’s off-the-shelf GeoDome Panorama. The result – families were able to enjoy a truly unique immersive experience.

At this point, just 360 still images, not video yet, right?

Right. At this point we were featuring still images shot in 360 by various photographers around the world. The response from visitors young and old was so favorable that we applied for a grant using similar technology to create a Dance Portal introducing the world of dance through immersive video footage. We received the Museums for America Award FY2016 from the Institute of Museum and Library Services, and began prototyping. The beta version of the Dance Portal was so well received that we were inspired to develop an exhibition/dance space around it. Let’s Dance!, currently in CMOM’s lower level gallery, introduces children and families to the delights of dance. The Dance Portal is the cornerstone of that exhibit.

What is the content you are working to put into 360 videos?

Dance in all its multicultural glory! We divided the content into simple categories: SEE, for performance clips, DO, for tutorial clips, and LEARN, for cross-cultural explorations of basic choreographic terms.

One of the great challenges to producing and curating the Dance Portal was that there is a very limited amount of dance footage that has been shot in 360. This prompted us to shoot some original video of our own. Working with videographer Paul Anderson, we shot 180 degree video in the studios of a number of world-class dance companies including Camile A. Brown & Dancers, and Elisa Monte Dance. We also filmed a number of dance companies in 180 while they were in performance at the Children’s Museum including Ajna Dance Company, Féraba – African Rhythm Tap Company, Kanu Dance Theater and Thunderbird American Indian Dancers.  In addition, we were able to procure a number of 360 clips and decided to include some HD footage, even though it does not fill the screen in the same way.

CMOM educators dance with visitors in Let's Dance!, while Elisa Monte Dance demonstrates a Soul Train on the Dance Portal screen, which is flanked by mirrors and a ballet barre.
CMOM educators dance with visitors in Let’s Dance!, while Elisa Monte Dance demonstrates a Soul Train on the Dance Portal screen, which is flanked by mirrors and a ballet barre.

Why doesn’t it fit the screen?

The portal screen is a ½ dome – basically a semi circle. It’s looks like a blow-up amphitheater. Video shot in 360 or even 180 will fill the portal screen from end to end.  Standard HD footage, will display in the shape of a rectangle, or traditional video format, which has an aspect ratio of 16 x 9. We were able to stretch our HD footage a bit, and found that these clips have impact as well, in part because the content is so compelling, but also because many of our viewers are small and the screen is big.

Most 360 footage displays, whether in a dome or on a standard screen, allow viewers to scroll around the video to explore every angle. We eliminated the interactive aspect of scrolling so that our visitors will be moved to dance, as opposed to getting caught up in their ability to control the video.

What goes into obtaining and editing the footage?

Beyond filming our own material, lots of research, personal referrals and persistence. A few of the large international dance companies have some 360 footage but the licensing was either unavailable or out of our reach financially. Others were very generous, like Blanca Li Dance Company, Mark Morris Dance Group, Martha Graham Dance Company and Alvin Ailey American Dance Theatre who all gave us permission to use their footage.

I found some wonderful surprises online too, like an authentic hula performance shot in 360 atop Volcanoes National Park in Hawaii. We also had a wonderful collaboration with Mickela Mallozzi who shared more than 30 multicultural HD clips from her Emmy award-winning show “Bare Feet with Mickela Mallozzi.”

Filming Elisa Monte Dance for the Dance Portal in the company's Queens rehearsal studio.
Filming Elisa Monte Dance for the Dance Portal in the company’s Queens rehearsal studio.

Who is the 360 video designed for, and where will they encounter it?

Visitors of all ages can enjoy the Dance Portal daily in the Let’s Dance! exhibit. Each video clip includes a short description of the featured company and dance located on an interactive tablet near the domed screen. Museum educators and adult caregivers use the Portal as a jumping off point to get children moving.

The Dance Portal content is reinforced throughout the exhibit. For example, the Dance Parade photos highlight the diversity and energy of New York City’s dance community. Children can engage with colorful bolster pillows with the same choreographic terms that are featured in the Portal—skip, jump, turn, etc.— and they can enjoy fun dress-up opportunities inspired by the dance groups.

What are you hoping 360 video will help you or the museum to achieve?

By offering our families the opportunity to immerse themselves in dance forms that they may not be familiar with, we are hoping the Dance Portal will inspire our visitors to seek out more dance in their own lives, at home, in school, and in their communities, both as dancers and spectators.

Do you have any initial findings?

The Dance Portal has been very well received thus far. We look forward to adding content and increase the offerings as 360 video becomes more popular and as we continue to host dance performances of all kinds at the Museum.

If you have a 360 dance video clip, please reach out to me at ebari@cmom.org.

 

360 Videos in Museums: Shot 2 – In the Field with the Field Museum

At the November Museums and Computer Network conference, I marveled at the different ways museums around the country are using 360 videos with their visitors or in their education programs. To highlights these different applications, I started this limited series of interviews  – same questions, different institution – so we can look for patterns.

Today we turn to Eve Gaus, the Manager of Digital Learning at the The Field Museum (a natural history museum in Chicago with scientists that conduct research on all seven continents). Centered in the Learning Center of the Museum, Eve connects visitors and youth with their collections and research through technology (games, 3D printing, animated video series, and more).

Eve, When did you start exploring 360 video (and if that’s not what you call it, what’s the term round your ways)?

360 video, which we also refer to as immersive video, has been on our radar since late 2016, and in summer 2017 I was green lit  for the project. I was drawn to immersive video because of its rich storytelling capabilities. Since our scientists are carrying out research across the world, and in many cases, in areas that are difficult to travel to, 360 video offers an opportunity to immerse our viewers in our research in a whole new way.

What is the content you are working to put into 360 videos?

We’re using 360 to explore our collections and research in two different ways. The first is by taking our viewers out into the field with our scientists. Our scientists see and experience the diverse beauty of our natural and cultural world, and we want to bring our visitors along on that journey. And, as any of our scientists will tell you, what they collect in the field is just the beginning of the journey, and so the second way we’re using 360 videos is going behind the scenes to capture our collection. We’re creating a story arc for our visitors to walk alongside our scientists as they engage in a process of exploration and discovery.

What goes into obtaining and editing the footage?

When our scientists head out into the field, we equip them with a 360 camera (we’re currently using Garmins and will soon be adding a GoPro to our collection), a couple of extra batteries and SD cards, and some solar charging panels. Before they leave, we sit together and rough out a shot list. Of course, the reason our scientists are in the field is for research purposes, not to capture amazing video content, so we go in with pretty flexible storylines that we can easily adapt.

Taking the footage behind the scenes is a bit easier, but still includes a pre-conversation and a shot list that we co-build with the scientist so we’re all on the same page about what story we’re telling through the footage. We use an Insta360 for our behind-the-scenes footage. After we collect the footage, we work with our scientist and our video editor to craft and edit the story.

Who is the 360 video designed for, and where will they encounter it?

The videos are designed for visitors to the museum (children through adults) and they’ll be able to encounter them in a variety of ways. The videos will be available on The Field’s Youtube channel, and select videos will also be featured in our Grainger Science Hub, a dedicated space in the museum for visitors to meet our scientists, engage with our collections and learn about our latest research.

What are you hoping 360 video will help you or the museum to achieve?

The purpose of the videos is to help connect our visitors with the science of The Field, and also to explore why we conduct the research we do. It is one thing to talk about conservation and the critical importance of biodiversity, but it is another thing altogether to hike through the Solomon Islands, to be on boat going down the Amazon, or to be working alongside communities in Kenya providing rabies vaccinations for dogs — that has transformative power. We hope that through our 360 videos, visitors to the Museum, and even those who may never walk through our doors, feel excited and empowered to embark on a journey of discovery with us.

Do you have any initial findings?

We’re launching in 2018, so will have some findings by spring/summer 2018!

360 Videos in Museums: Shot 1 – Underwater at the Shedd Aquarium

At the November Museums and Computer Network conference, I marveled at the different ways museums around the country are using 360 videos with their visitors or in their education programs. To highlights these different applications, I started this limited series of interviews  – same questions, different institution – so we can look for patterns.

Today we turn to Miranda Kerr, the Manager of Digital Learning at the Shedd Aquarium in Chicago.

Miranda, What is your role where you work?

As the Manager of Digital Learning, I work with our Learning Group departments to set up digital frameworks and respond to evaluation and research trends as they relate to digital programming and audience needs. I use evidence-based decision making to pilot and integrate digital learning experiences and concepts into Shedd’s on-site, off-site and online programming channels.

When did you start exploring 360 video (and if that’s not what you call it, what’s the term round your ways)?

Part of my job is to follow what’s new in the museum technology space and any digital tools that are trending in other spheres, then consider implications for the work we do in Learning at Shedd.  I don’t remember exactly when 360 video came up on my radar, but I purchased our first equipment in the summer of 2016, a 360 camera and a set of 5 unlocked phones and VR headsets to enable us to pilot both creation and viewing of 360 video with learners.

Then in late summer and fall of 2016, along with a few other staff members, we did a 360 video pilot in one of our programs.  The Park Voyagers program brings educational programming to Chicago Park District facilities often in underserved communities. Shedd works in collaboration with 10 other Chicago museums to provide this free after-school program for children, ages 8 to10, and their families.  The 360 video and headsets were the technology we needed to answer the question, “How can we bring the experience of viewing Shedd Aquarium’s exhibits and animals to children offsite?”  We recorded 360 degree videos in three of our exhibits that would connect to the three ecosystems explored in the program.  To learn more about this pilot, check out the blog we wrote: “Virtual Reality: Bringing Shedd Aquarium to Learners in 360”.

What is the content you are working to put into 360 videos?

Our first attempts at integrating 360 video have been capturing our exhibits to share in other locations.  Because we have fishes and other animals that are swimming around, a simple photo doesn’t capture this in the same way.  We also have done some pilots to see how 360 video can capture our programs in a new way, which includes taking the camera on a snorkel during High School Marine Biology, or on a nature hike during Summer Road Trip.  The most recent 360 videos we’ve captured have actually been inside our exhibits with the animals, which provides a completely new experience.  You can view the underwater Caribbean Reef Exhibit 360 Video here:

What goes into obtaining and editing the footage?

The process for obtaining 360 video depends on where we are filming. If we are hoping to capture a 360 video of an exhibit, we aim to film before the aquarium opens so the footage is just the exhibit without guests in the shots.  In the spring of 2017, we acquired an underwater 360 camera.  To film underwater, we’ve explored a number of creative techniques including  having SCUBA divers and snorkelers hold the camera on a selfie stick, setting up a tripod inside of our exhibits and using zip ties to attach it to an underwater robot.  You can see examples of all of this footage on the Shedd Learning YouTube page.

Editing 360 video footage is a simple process with the software available, but time consuming.  The Ricoh Theta S has a user friendly cell phone app that makes for quick editing, and you can even tweet 360 photos directly from the app.  We shared a 360 photo of campers doing water quality tests at a nature center in a tweet:

The Nikon Keymisson editing works better on a desktop computer, and is a free download.  This process can take hours, even for a video that is just a few minutes long, because of how large the files are.  Although the process is not difficult, each of the steps takes time, transferring from camera to computer, then trimming or editing, and then adding meta-data to be able to upload to YouTube.

Who is the 360 video designed for, and where will they encounter it?

We have used 360 video in a handful of programs to connect with different audiences. In Park Voyagers, Learning staff brought the 360 videos on our devices to be viewed by kids using our VR headsets.  In the Teen Work Study program, teens created their own 360 video, and then shared with guests waiting in line to buy tickets using our VR headsets.  The 360 videos are also posted to our YouTube and shared on social media channels.

What are you hoping 360 video will help you or the museum to achieve?

My big question is, how can 360 video take our learners to new places and enhance experiences?  This year, Park Voyagers participants are viewing 360 videos from inside our exhibits, to have an underwater view and see fishes swimming right at them.  We captured 360 underwater video at the 2017 underwater robotics competition, so that students at the 2018 competition can view a 360 robot-eye view of the underwater course.

Do you have any initial findings?

I have comments from the Digital Learning Staff Reflections completed by Learning staff.  From the Teen Work Study project, “Teens gained skills on the applications of VR, including design goals concerns and troubleshooting of the equipment…Teens were able to bring the Wild Reef [exhibit] out to our guests!”  We also gathered reflections from the teens themselves who did that project:

“Finishing the VR project was a pretty big victory for me. I don’t finish things when I start them, and I finished a lot of things being in the teen lab.”

They were all happy with the outcome of the VR project. They were scared of not doing a good enough job on it or not finishing it. They thought it was cool when the adults they look up to got excited about their VR project. They felt special.

“I was scared of technology – terrified of technology  — until now.”

Is there anything I didn’t ask that you wish I had asked?

What’s next for 360 Video?

Although I don’t know for sure where we’ll take 360 video next, I do know the entire Learning group has this technology tool in their pocket now.  Every year for every program, we do a technology brainstorm we call a SAMR brainstorm.  We can think about how we could use 360 videos to bring Shedd Aquarium to new audiences or give learners additional experiences like 360 videos from inside other exhibits.  We can also think about how to use 360 video assets we collected last year, like from underwater robotics and snorkels in the Bahamas.

Next up: 360 VR at the Field Museum…