Cool Jobs: Video game creators

A sword and shield hang on the wall in James Anhalt’s office. Soon, he’ll add a crown to this collection. He is a software engineer at Blizzard Entertainment in Irvine, Calif. This company has created many hit video games, from World of Warcraft to the StarCraft and Diablo series. Swords, shields and crowns abound in these fantasy-themed games. So the company gives out real ones when employees reach work anniversaries. When Anhalt gets the crown, it will represent 20 years with the company.

Anhalt’s job is fun but also demanding. “Kids who play a lot of games might not realize how much work it takes,” Anhalt points out. One major game, 2015’s StarCraft II: Legacy of the Void, took more than 2.5 years to develop. And that was with the help of more than 80 engineers, programmers, designers, artists, writers and others working long hours. Every little detail needed time and attention.

Anhalt worked on the game engine. This software is the framework that developers start with when they create a new game. It includes ways to add artwork, sound, special effects, characters, enemies and more. But programmers often need to modify the game engine. They may fix errors or add new features for artists and designers. That’s where Anhalt came in. He made sure all the parts of the game fit together and ran well.

Clearly, the team succeeded. The game was a big hit, just like the previous two games in the Starcraft II trilogy, which began in 2010. Every year, thousands of elite players from around the world compete in StarCraft II tournaments for prizes of up to $100,000.

But video games are about much more than winning or losing. In this story, we’ll meet two engineers and a neuroscientist who take games very seriously. Anhalt and his colleagues work hard to make their games as fun as possible. An audio engineer is working on a new kind of interactive technology. It’s called mixed reality, and it could transform how people interact with all digital media, not just games. And a neuroscientist builds games to fix people’s brains. He believes that doctors may one day prescribe video games to treat a variety of conditions.

Making the most of memory

StarCraft II is a game of strategy. Players make split-second decisions about which structures to build, where to explore and when to fight or retreat. All these tasks take up space in a computer’s memory. Memory is the space available on a physical chip or disk for storing information. A computer (or tablet or mobile device) only has so much memory. To keep a game running smoothly, developers must pay close attention to how that game is using a device’s memory. If it uses up too much memory, the game will run slowly or even freeze or crash.

Top-of-the-line computers pack in more and more memory every year. But game developers have to make sure that their product will work on everyone’s computers, not just the most powerful ones, Anhalt says. So those developers want to make the game run as efficiently as possible. And that means getting creative with code. Computer code is the set of step-by-step instructions a computer program follows.

Sometimes, following those instructions takes too long. As an example, Anhalt points to graphics (a game’s visual elements). At any moment in the game, each pixel on a display screen needs to be a certain color. Many factors affect that color. A pixel in the sky might be blue, but one on a character’s armor might be gray. Nearby light and shadows or special effects (such as snowfall or an explosion) can affect the color too. “If we have, say, 16 different options that can all be on or off, that generates 65,000 different programs,” Anhalt explains.

While working on StarCraft II: Legacy of the Void, his team found that running through all the color-choosing programs for every pixel on the screen was using too much memory. They managed to compress all those programs down to just 5 percent of their original size. They combined some similar programs together and cut out others completely. They also rewrote sections of the code to make it simpler.  

Making the color-choosing process more efficient gave game artists more memory to work with. That meant they could add even more special effects to the game.

As computers get more and more powerful, graphics and other game features will get more realistic, Anhalt says. More memory means more lifelike detail.

New realities

Soon, games will be so lifelike that they jump off the screen. Anhalt and many other experts agree that the next big things in gaming — and in technology in general — are virtual and augmented realities.

Virtual reality is already here. The Oculus Rift headset came out in March 2016 and the Sony PlayStation VR will come out soon. These headsets both resemble something a deep-sea diver might wear. But when you put one on, you find yourself in another world. Everything you see, hear and touch is a digital illusion.

Augmented reality, in contrast, places digital illusions into the real world. Pokémon Go is a recent augmented reality sensation. The game uses a smart phone’s camera and global positioning system (GPS) to reveal Pokémon monsters sitting on sidewalks or in parks.

But Pokémon Go still uses a screen. “I think screens are going away,” says Gene Rozenberg. He’s an engineer at a company called Magic Leap in Dania Beach, Fla. The company is developing a new kind of augmented reality technology that it calls “mixed reality.”  

“We make you believe that a computer image is really in the world around you,” Rozenberg says. In a demonstration for the magazine Wired, this new technology made text messages float above a person’s desk. It also made three-dimensional jellyfish swim toward the ceiling.

Rozenberg’s role at Magic Leap is to design sound effects. Throughout his career, he has worked on many games, including the Madden NFL series. No matter where he has worked, he’s spent his days composing music and recording sound effects or voiceovers. He also has made sure that these sounds play at just the right moments in a game. That means programming.

“I myself am not a programmer,” he says. But he’s taking a class now to learn more about reading and writing computer code. 

Why does a sound guy need to know about programming? Think about footsteps, Rozenberg says. The sound has to change every time a character steps on a different surface, like grass, wood or stone. Characters might be gigantic or tiny, or wearing different types of shoes. To make footsteps sound good, he says, “I have to think like a computer would.” He has to give very precise instructions to make sure the right sounds play at the right time.

In both virtual and augmented reality, sound gets even more complicated. Developers in these new mediums face a big problem when it comes to sound, explains Rozenberg. A mixed-reality device, for example, plays sounds straight into someone’s ears. But those sounds need to seem as if they are coming from above, below, behind or in front of that person — not from the headphones they are wearing. “This hasn’t been done before,” he notes.

He can’t talk about the steps he might be taking to solve this problem, though. Since Magic Leap’s technology hasn’t come out yet, the company is still secretive about exactly how it works and what it does.

One thing Rozenberg can say is that the human brain is a very important part of the system.

“Your brain is a computer. It processes information and decides how that information is [delivered] to you, the person,” he says. To fool the brain into thinking there really is a jellyfish on the ceiling, developers need to understand how the brain processes information. If they give the brain the right visual and sound signals, the jellyfish will seem like reality.

Fixing brains with games

At his lab in California, neuroscientist Adam Gazzaley is delving even more deeply into the mysteries of the human brain. And it’s not only for the sake of fun. He wants to turn video games into medicine. 

An iPad game nicknamed Project: EVO got its start at Gazzaley’s lab at the University of California, San Francisco. Its players drive through frozen and fiery worlds at top speed. They have to navigate around icebergs and fire pits. At the same time, they perform tasks such as collecting all the red creatures that pop up, while ignoring green and blue ones.

Right now, different teams of researchers are studying whether the game can treat a variety of disorders, including attention deficit hyperactivity disorder (ADHD), autism and depression. If it works, then the U.S. Food and Drug Administration (FDA) might approve the game as a medical treatment.

Imagine going to the doctor and getting a prescription for a video game! This could happen within just a year or so, says Gazzaley. He co-founded a company, Akili Interactive Labs, to turn Project: EVO into a product that people can buy.

And Gazzaley has good reason to think this game could become a success. Before EVO, there was Neuroracer. This early version of Gazzaley’s game also challenged players to drive while collecting objects. In a 2013 study, the scientist and his colleagues showed that the multi-tasking in Neuroracer could give older adults a brain boost.

Attention and memory get worse as people age. So Gazzaley recruited volunteers ages 60 to 85 to practice the game for a month. They got much better at playing the game. More importantly, they performed better in tests of their attention and working memory. (Working memory is the brain’s ability to hold onto information for a short period of time.) Control groups — people who were not being treated — did not show the same benefits.

One control group didn’t play the game. Another control group played the racing part of the game without the collecting task. Six months later, people who had played the full game still had sharper brains compared with the other two groups.

Kids with ADHD also struggle with attention and working memory. EVO might give these kids a brain boost just as Neuroracer did for older adults. In 2015, one of the first studies of EVO got positive results. Kids with ADHD who played the game regularly for four weeks became better at paying attention. They also were less impulsive, meaning they were less likely to act without thinking.

Gazzaley sees Neuroracer and EVO as personal trainers for the brain. “The game is constantly recording how you’re doing and then adjusting the challenge level to be appropriate to your ability,” he says. The better you play, the harder it gets.

Right now, these games use a player’s mistakes or successes to set the level of difficulty. But eventually, Gazzaley hopes to connect games directly to a player’s brain.

In the lab, he’s testing a technology he calls “glass brain.” It lets people see their own brain activity in real time. To use it, a person wears an electroencephalography (Ee-LEK-troh-en-seff-uh-LAH-grah-fee) cap. This cap contains sensors. They record a person’s brainwaves and feed those data to the glass-brain computer system. The system displays a three-dimensional image of the brain. Active brain areas flash with colored light.

A video game could use this real-time brain activity to figure out why a player made a certain mistake, Gazzaley explains. Maybe the person’s reaction time was too slow, for example. The game could then create challenges to train — and then treat — reaction times.

Why would a neuroscientist turn to games to fix people’s brains? Simply put, because games are fun.

Typically, Gazzaley says, when researchers want to study the brain, “We use such boring tests. People don’t engage in them deeply.” People are more likely to learn and build skills if they enjoy the training process.

Anhalt, Rozenberg and other game creators also work hard to create exciting, engaging experiences. When they succeed, you won’t think about how hard they worked behind the scenes. You’ll be having too much fun playing your favorite game.

---

This is one in a series on careers in science, technology, engineering and mathematics made possible with generous support from Alcoa Foundation.