

When Klimt first showed his paintings, critics noted his use of a red that was, at the time, rare in the artist’s palette. Even the surprises Smola and Wallner encountered are corroborated with historical evidence. Studying existing paintings from before and after the Faculty Paintings provided clues to the colors and motifs recurring in his work at that time. His research revealed that Klimt’s work during this period tends to have strong patterns and consistency. The key to that accuracy lies in pairing the algorithm with Smola’s expertise. And those kinds of data points, if fed into the algorithm, create a more accurate version of how these paintings probably looked at the time. “The fact that the paintings caused a scandal and were rejected puts us in a better position to restore them because there was so much documentation. “You can call it an irony of history,” says Simon Rein, the project’s program manager. Because the paintings had been considered so sordid and weird, critics tended to describe them at length, right down to the artist’s color choices, he says. But with no color references to the paintings, where did these clues come from? Even Klimt expert Smola was surprised by how much detail the writings of the time revealed. And finally, the AI was fed color clues to specific parts of the paintings. “This creates a bias toward his colors and his motifs during the time period,” Wallner explains. Next, it was schooled specifically in Klimt’s paintings. This helped it understand objects, artwork, and composition. First, the algorithm was fed some hundred thousand images of art from the Google Arts and Culture database. To do this, Wallner developed and trained a three-part algorithm. “The medium of photography is already an abstraction from the real works.” What machine learning is doing is providing a glimpse of something that was believed to be lost for decades. “It's not a process of recreating the actual colors, it is re-colorizing the photographs,” Smola is quick to note. Let's make one thing clear: No one is saying this AI is bringing back Klimt's original works. The results are what Smola and Wallner are showing me-and even they are taken aback by the captivating technicolor images the AI produced. It’s been a laborious process, one that started with those black-and-white photos and then incorporated artificial intelligence and scores of intel about the painter’s art, in an attempt to recreate what those lost paintings might have looked like. Franz Smola, a Klimt expert, and Emil Wallner, a machine learning researcher, spent six months combining their expertise to revive Klimt’s lost work. All that remains today are some black-and-white photographs and writings from the time. During World War II, they were placed in a castle north of Vienna for safekeeping, but the castle burned down, and the paintings presumably went with it. Soon thereafter, the works found their way into other collections.

Professors at the university rejected them immediately, and Klimt withdrew from the project. As soon as he presented them, critics were in an uproar over their dramatic departure from the aesthetics of the time. Commissioned in 1894 for the University of Vienna, "the Faculty Paintings"-as they became known-were unlike any of the Austrian symbolist's previous work. In 1945, fire claimed three of Gustav Klimt's most controversial paintings.
