A siren screams in the background mixed with another repetitive beeping alarm. The phone rings while over the PA system someone pages a doctor. A group of high school students panic.
“He’s about to die, we’ve got to check his blood pressure” one of them says.
A few seconds later someone realizes the patient isn’t breathing and turns on the ventilator, a rhythmic hissing of air starts in the room and the patient stabilizes as IV drips are initialized.
“Ok, times up.” Dr. Elisabeth Heath says. “Nice job Team Sleep, you were awake for the challenge!”
This group of high school students isn’t really in an emergency room trying to save a real patient. In fact none of the students or Dr. Heath are even in the same place. They’re all connected via virtual reality to an interactive virtual hospital environment.
It’s all part of a STEM career exposure initiative called FocuSStem NextGen started in 2018 in Detroit by high school student Katie Heath. Heath spent a lot of time while she was growing up at the Karmanos Cancer Institute where her mother Dr. Elisabeth Heath works as an oncologist. When she was really young she helped out by putting together binders and doing art projects creating hundreds of glittery pink ribbons for breast cancer walks. But as she got older she got to experience more of the roles played in the hospital and when she realized what a valuable and unique exposure she had access to, she wanted to bring that experience to more students.
“Let me tell you what a hard conversation it was when she came home and said ‘You know, I’m really fortunate to have all of this but I ran into somebody and then I realized, you know, there’s not a lot of people my age doing this.’,” said Dr. Heath. “That was a hard conversation, she’s like “Why not?”” So that really I think spurred her on.”
“She really sort of lead the charge, and put the adults on task saying, you know this is great but my mom’s in the business, what about those who are interested that don’t have that opportunity or even know to volunteer, how about those kids?”
So FocuSStem NextGen was born and high school students were brought into the Karmanos Institute to meet with physicians and researchers as well as clinical staff to learn about the variety of career pathways available in STEM fields. As part of the experience, participating students also got to explore real-world health scenarios and visit different parts of the hospital to get a real sense of what it would be like to work there.
When the COVID-19 pandemic created severe restrictions on who could visit the hospital, FocuSStem’s student-led executive board knew they had to innovate to provide the same immersive experience.
The team turned to CrossComm, a Durham, North Carolina-based app design and development studio, to quickly develop an immersive virtual landscape and experience for that summer’s participants.
“We put two and two together and we said well what if, this is crazy, but what if we could somehow equip all of these high school students with virtual reality and that we all meet in a virtually simulated hospital environment, together in the same virtual space, to simulate that social experience and that shared learning as closely as possible," CrossComm CEO Don Shin said. "This will be one way that we can kind of stand up to the situation that the pandemic has put us in.”
A virtual ICU, operating room and research lab were developed where students could interact with objects and had to complete tasks. Including putting out a fire in a lab, working as a team to keep a patient alive in an ICU, and slicing sample tissue off of a heart and then examining it under a microscope.
“We even went a step further to be able to do something things that they wouldn’t be able to do, like there was a mock patient in the operating room, someone could actually pick up a scalpel – a virtual scalpel of course – and then virtually cut this virtual patient,” Shin said.
Courtesy of CrossComm
Credit
Credit
“When they’re in the modules and they have five minutes to figure out the five tasks that they have to do, and how to do them and in what order, you could feel their stress trying to figure out all of those things, which is so cool!” Katie said. “One of the modules had a person being cut open and there was blood and you like physically jump back […] it’s so realistic. It was just like an exciting thing that was different from the daily zoom lectures and school.”
“People could see each other in this virtual space as avatars and talk and hear each other. And people started to just forget that they were in virtual reality and interact with each other as human beings. So that was an incredible experience,” Shin recalled.
The program was generously supported by a donation from CrossComm, which provided 50 percent of the development at no cost and Oculus Quest Headsets for the students were funded by a donation from Zekelman Industries.
But even with these donations, the cost of the headsets (around $300 each) limits the number of students that can participate every year.
“The Oculus and the VR world is wonderful, but to really reach kids you gotta have something much more mainstream. So I challenged Don and I said we have to come up with a technology that doesn’t require great wifi, which by the way we found out for many kids, was a struggle,” said Dr. Heath.
That’s where extended reality, or XR came into the picture. The CrossComm team developed an asymmetrical collaboration space that brings presenters who are in virtual reality together with an audience viewing and interacting with them through augmented reality.
“That way we could still have an educational experience, not everyone would be in there in VR, but presenters through VR could still talk about three dimensional objects, like a mock enlarged version of a three dimensional heart, and annotate it, rotate it around, and give a talk on it, and audience members can still see the content and the avatar interacting with it, as if that presenter was brought into their living room,” Shin said.
The 2021 FocuSStem Virtual Summit blends virtual reality with augmented reality (AR) to allow participants with a smart phone to access presentations and demonstrations by experts in STEM fields.
“With AR, everyone has a smart phone, so 95% of the students who want to participate but don’t have the funds let’s say for VR, AR is a great substitute. And it gives you that sense of immersion and participation without all the cost. Is it a full immersion like a VR? No. Totally different technology, but I think getting it to a broader audience is important, so that’s why we’re committing to that and exploring that in this summit,” said Dr. Heath.
“All of that leads to really just helping the next generation succeed. That’s our goal. These kids don’t engage, we’re all in trouble. This the future, not just doctors and scientists but like everybody in this world, and it’s huge. It’s a huge world of STEM,” Dr. Heath emphasized.
Shin said in many ways, the isolation and remote work environments created by the pandemic have increased interest in VR, AR and XR combinations of the two.
“I hate using the word positive and COVID together, but I will say that the lock down caused by the pandemic was a catalyst that accelerated the adoption of immersive technologies tremendously,” he said. “Just look at how the pandemic has changed the way we work and changed out relationship with physical spaces, and even changed the way that we interact with each other.”