These scientists help rescue ‘broken’ digital art

WASHINGTON, D.C. — To see most art at the Solomon R. Guggenheim Museum, you’d have to travel to New York City. But you only need to go online to see one of the museum’s works: “Unfolding Object.” It’s a 2002 web-based work by artist John F. Simon, Jr. As viewers interact with it, they affect what it looks like. For several years, the art stopped working on updated computers. But thanks to innovative restoration work, computer scientists have put this piece and others like it back in action.

At first, “Unfolding Object” looks like a simple square on your screen. Click on any of its edges and that side unfolds. Click on another edge, and it unfolds. You can repeat the process over and over. Or, go back to the first square or another shape, and click a different edge. As you go, lines appear on each shape. These hatch marks tally up previous viewers’ clicks. The directions represent ones, tens, hundreds and thousands of interactions with viewers. The color of the object and background also change throughout the day. Playing with the work can keep you clicking — again and again and again.

But there came a time when the program wouldn’t work on many newer computer systems.

Such digital — or online — art can’t break in the same way that an ancient vase might shatter. But the art is still quite fragile, says Deena Engel. She’s a computer scientist at New York University in New York City. Recently she and her students worked with Joanna Phillips, a former conservator at the Guggenheim, and others there. A conservator is someone who repairs and preserves artworks. Together they restored “Unfolding Object” art. Engel described how they did it at the annual meeting of the American Association for the Advancement of Science on February 17.

“It’s very difficult to run software from maybe 15 years ago on a current Mac or a current PC,” Engel explains. Operating systems change. Computer programming platforms can change. Hardware gets updated. Old equipment breaks and parts may no longer be available. Other changes affect how colors show up on a screen. All of that can affect what people see in a work of computer art.

“If you can even get it to run at all, it’s most likely going to run differently,” Engel says. “Even things like code for black and white [may] get flipped.”

But fixing broken computer art takes more than a new program that does pretty much the same thing. After all, a copy of the Mona Lisa isn’t the same as the original. And the goal here is to restore the computer art — not to replace it. “Just like you want to be careful with the brush strokes if you’re preserving a painting, we’re very careful with the code,” Engel explains. “We want to preserve the algorithms, and we want to preserve the artist’s hand in the source codes in these kinds of works.”

That’s where Engel and the team had to be innovative. “We are building new models to apply to this rich area of contemporary art,” she explains.

‘We have work to do’

Engel first learned about problems in computer-art restoration some years ago. At that time, conservators from the Museum of Modern Art in New York City approached the head of her university department, who then reached out to Engel. “I must have a reputation for my love of art,” she says.

Engel visited a back room at the museum. There she found stacks of old computer media, including early floppy disks. “I was like, ‘Oh, my gosh, roll up your sleeves. We have work to do!’” 

Engel and others got “Unfolding Object” working again. The Guggenheim museum unveiled this restored version online last November 19. Previously they had restored another web-based work in the Guggenheim’s collection. It’s “Brandon” by artist Shu Lea Cheang.

One task for Engel’s team has been to recreate the computer environment in which the artists had worked. The team set up software that the artists had used. Then the group’s members could see how the art was supposed to look and work. And they could get source code if it was not otherwise available. Finally, the team had to figure out what’s different between the earlier computer environment and what’s in use now. Only then could they begin designing tweaks and add-ons.

For example, Simon originally created “Unfolding Object” in Java. That’s a programming platform. But Java has changed a lot since the early 2000s. In some cases, Simon had used applets — mini-programs within the platform — that no longer worked. So the team had to add new programming language.

The restorers also had to make sure the webpage would run correctly. For that, they worked with Hypertext Markup Language, or HTML. That’s the standardized system for the structure of most webpages.

“We work very hard to allow the original code to run,” Engel says. The approach is almost the opposite approach to how most programmers work, she notes. After all, computer engineers usually aim to update and improve programs. But it’s a matter of ethics for art conservators. They want to keep the art as it was created. So instead of using all modern code, the group tweaks the old programs.

“We add [other features] that remind it to behave the way it did 15 or 20 years ago,” Engel explains. When it is necessary to rewrite code, Engel’s team makes detailed comments in the updated code. The comments note which parts of the original code no longer work. And they state the date and details of what they might of added to solve any problems.

Looking ahead

Meanwhile, artists are creating new computer artworks. Engel’s group is working with the Guggenheim conservators in hopes of making future restorations less tricky. One step is talking with artists when new works come into the collection. Artists can explain what their vision is, and what types of future changes they would be willing to accept. The museum also can get source code at that time for the work. All that info can guide future computer-science teams and conservators if a computer system’s base set of colors or other properties change.

Where possible, there will also be video recordings of computer art with voiceover comments by the artist. Interviews like that can help future restorers see just how the art worked when it was created.

Marc Walton is a materials scientist at Northwestern University in Evanston, Ill. He specializes in applying science to art conservation. He and his team used computer programming to capture 3-D images of “acne” on a Georgia O’Keefe painting. (Chemicals called metal soaps cause the problem, which forms due to chemical reactions between paint ingredients.) Walton thinks Engel’s work is addressing new technical questions and ethical issues in computer art. Computer science and other high-tech fields (such as medical imaging and chemical analysis) will be the future of art restoration, he predicts.

He says, “I think you’re just seeing the tip of the iceberg with these kinds of things.”