William Warren's research group is advancing virtual reality technology in the service of studying the science of the swarm: how patterns of crowd movement emerge from individual behaviors. He described his work June 29 in a keynote address to a conference in Vancouver.

The cognitive scientists in the Virtual Environment Navigation lab at Brown University are not only advancing a frontier of behavioral research but also of technology.

Led by Professor William Warren, the group developed a wireless virtual reality system to study a phenomenon that scientists don't yet understand: how pedestrians interact with each other and how those individual behaviors, in turn, generate patterns of crowd movement. It's an everyday experience for all kinds of animals including ants, birds, fish and people.

"When you walk across campus during class change you are, consciously or not, coordinating your movements with the people around you," Warren said. "In some situations coherent 'swarms' form, somewhat like a flock of birds or a school of fish. We want to understand that process."

A well-tuned computer model of such swarming behavior could have many specific applications in human life.

"There are applications to urban planning, architectural design, and evacuation planning," Warren said. "Crowds seem to behave in predictable ways, but the environment is not always built to accommodate people's behavior."

It could also lead to technology to help visually impaired pedestrians. The research is gaining attention. Warren flew to Vancouver last week to give a keynote address June 29 at the annual meeting of the International Society for Gait and Posture Research.

Of swarms real and virtual
His talk was not just about how what we see guides how we walk. It also delved into the kind of technology people normally discuss at video game conferences.

One needs a crowd to study crowds, so Warren and his students recruited volunteers for one called the Sayles Swarm and recorded them using motion capture technology (Sayles Swarm Link). But the most versatile way to assemble a crowd is in a virtual world, where any configuration is possible. In the VENLab, Warren's team including graduate students Kevin Rio and Greg Dachner can immerse up to four real people in a carefully controlled, realistic virtual crowd.

"We're trying to use VR to figure out the rules that govern pedestrian behavior," Warren said. "In experiments, we can manipulate a virtual crowd to see how real subjects respond to the behavior of their virtual neighbors. This allows us to model the visual coupling between an individual and their neighbors. We then use that model to predict what will happen when a group of people walk together in a real crowd."

Walking wireless
To make this possible his team rigged up the 168-square meter VENLab to make their off-the-shelf Oculus Rift VR headsets wireless. The room also has video cameras and a grid of beacons on the ceiling that emit ultrasound pulses denoting their location.

The headset "listens" for those beacons with microphones and combines them with signals from accelerometers, much like the ones in a smartphone, to determine where the subject is and what they are looking at in the virtual world.

Brown has filed for patents on the wireless system. Going wireless has vastly improved the research, because it allows subjects to move unencumbered.

"That's the breakthrough: the freedom of movement," Warren said. "Before we had a bunch of heavy cables running to the head-mounted display. At one point the subject had to wear a big control box in a backpack. But now we've managed to make it quite light and mobile so people can move about freely."

In the future they hope to integrate many people into the same virtual environment, even if they aren't in the same real room. That will require making headsets that can track their location without any surrounding hardware. They are working with engineering Associate Professor Gabriel Taubin to do that.

All along the group has been generating insights. In Vancouver, Warren discussed several "swarm scenarios" his group has studied both in crowds of real people and in virtual reality. In "Grand Central" individuals crisscross the room, as if heading across a train station concourse. In "counterflow" two groups pass through each other.

"We're finding that when people walk together they tend to match the speed and direction of their neighbors. That intrinsically leads to the emergence of collective motion," Warren said.

"What we're showing with the virtual crowd is that as you add more neighbors, their influence simply sums up. Now we are mapping out what we call the 'coupling field' that connects you to neighbors at different distances and different positions. This enables us to simulate the collective behavior of a large crowd."

In this high-tech way, Warren and his team are revealing the science of the swarm.