NVIDIA : Augmented Reality Lets You Slip Inside Magritte’s Surreal Scenes at SFMOMA

San Francisco's Museum of Modern Art wants to bring you closer to paintings. How close? Literally inside.

The iconic museum is running an exhibit of surrealist Rene Magritte that also offers visitors an augmented reality trip into his provocative, often jarring, art.

People can step in front of AR pieces inspired by the artist to enter and interact with famous interpretations of works such as 'Shéhérazade,' which depicts an exotic woman behind pearls in the sky - or a viewer taking her place.

The monitors are designed as window frames reminiscent of one of Magritte's most famous paintings, 'Where Euclid Walked,' which shows an easel painting in front of a window. The original painting begs a contemplation of what is real versus what is perceived - much like AR.

Created with frog design, a global design firm based in San Francisco, the interactive portion of the show uses these traditionally appearing window frames packing NVIDIA GPU computing power and front-facing cameras for visitors to peer into and interact with the artist's world.

Play In the Artwork

The first AR stop, inspired by 'Le Blanc Seing,' displays a children's storybook-like forest scene in which visitors have to position themselves slowly and carefully for the cameras and sensors to allow them into the forest image and be seen. The intent is to force visitors to slow down and really study the interaction to fit in visibly between trees.

Some windows present Magritte-like puzzles for visitors to solve. In one, people's images were not fed into the monitor they were facing but instead an adjacent one, causing confusion and interactions with fellow museum visitors to figure it out. Sometimes people would ask the stranger on the other end for a photo. Voila, just the type of interactivity that was aimed for.

'The paintings are actually looking back and seeing the people - they are turned from visitors to participants,' said Charles Yust, a principal design technologist at frog.

The exhibit itself spills across eight different rooms in the recently expanded downtown museum. Among the more than 70 works of art are some of the Belgian artist's best known pieces, which show his highly idiosyncratic perspective, such as the painting 'Personal Values,' from 1952, which depicts ordinary objects - like a comb and a glass - claustrophobically oversized into a bedroom. Also on display were example of his famous bowler hat paintings, such as the 'Happy Donor,' from 1966 and 'The Son of Man' from 1964.

While the show is a big one for the museum, what sets it apart is its use of AR - which harnesses StereoLabs Zed Mini AR cameras for video and depth perception and NVIDIA Jetson TX2 for processing the real-time interactions.

Jetson for Museum Pieces

Frog design set out to build the interactive monitors as pieces of artwork themselves. The global design firm worked closely with the museum's staff to conceptualize these AR displays.

The AR exhibit was designed to deepen understanding of Magritte's work and get visitors to interact with one another in moments of confusion - even consternation - and reflection at times, in the spirit of Magritte's art.

'We know that playful interactive experiences can create a strong connection between visitors and the subject matter,' said Chad Coerver, chief content officer at the SFMOMA. 'We have been working really hard to think about how tech can advance the museum's mission and connect with the Bay Area around us.'

The show, which runs at the SFMOMA through Oct. 28, includes more than 20 pieces by the Belgian artist that have never previously been shown in the U.S.