Putting the AR in art

There’s a tree stump on a grassy patch in the financial district of Singapore. It’s not of a real tree, but a 3D-printed artwork created in honour of Changi Tree (also called Time Tree), the city-state’s tallest at 76 m. The artwork, the largest 3D-printed structure in Singapore, is meant to reflect the passage of time, and is a fascinating glimpse at what the tree would’ve looked like if it hadn’t been chopped down during World War II, to prevent the Japanese from using it as a ranging point.  And yet, turn a phone on the artificial stump, and suddenly it starts spiralling inward. The spirals are made of text that mention important events in Singapore’s history, and indicate how humans mark the passage of time. 

The Time Tree is one of six street artworks, part of Singapore Art Week, which uses augmented reality (AR), a combination of the real and virtual where physical objects are augmented with perceptual and sensory information using technology. It creates an interactive interface between the artwork and the viewer. This confluence of the world of art and AR have opened up new possibilities, helping free art from the confines of galleries and reach a whole new audience.

“AR offers the chance to contextualise artworks in Singapore by engaging visitors in new ways,” says Kay Poh Gek Vasey, chief connecting officer of technology studio MeshMinds. “Through AR, we can engage viewers to have a deeper curiosity, understanding, and memory of the artist and the artwork. By harnessing digital technologies we hope to increase public participation in the arts.”

Across the world, AR and art are slowly working towards finding a mutually satisfying collaboration. There’s a growing number of digital artists who ‘paint’ using technology. Art exhibitions and artists are using augmented and virtual reality to tell better stories, and museums are using it as a new way to show exhibitions. In 2018, renegade artists took over New York’s Museum of Modern Art (MoMA) to create a virtual exhibition, ‘Hello, we’re from the internet’. Visitors who downloaded the MoMAR Gallery app and went to the Jackson Pollock gallery on the museum’s fifth floor could see the Impressionist’s iconic paintings reproduced or remixed.

New York’s Metropolitan Museum of Art, too, experienced an AR project that animated Van Gogh’s First Steps, After Millet, while the Whitney Museum of American Art enabled visitors to see digital tobacco plants sprouting in the museum, indicating the crops that might have grown there had the land not been seized from the Lenape people.

AR has the power to disrupt museum-viewing experiences, which have traditionally been one-directional. The technology is easy to experience and access, and can create a wholly immersive way to understand an artwork. But it also raises questions related to virtual trespassing: How much can augmentation alter the context of an artist’s work, is augmentation constructive or destructive, and how does privacy fit into AR. At the moment, however, the focus remains on the fun aspect of using the technology.

The Singapore Tourism Board (STB) and Chan + Hori Contemporary, an art gallery, launched Atypical Singapore, a showcase that offers a glimpse into Singapore’s arts ecosystem. Atypical Singapore, which was part of Singapore Art Week this year, had travelled to Russia, Myanmar, Thailand and India (New Delhi) over 2018 and 2019, and featured seven artists, each using Singaporean elements and icons as inspiration. One of the stand-out participants was tech artist Eugene Soh, or The Dude, who created a parody of iconic art works by placing them in Singaporean settings.

French post-Impressionist painter Georges Seurat’s A Sunday Afternoon on the Island of La Grande Jatte became Sunday Afternoon on the Island of Singapore, and Italian artist Raphael’s School of Athens became Food for Thought. To do this, Soh individually photographed each person in the art works over a course of six months at the same location. When viewed through Facebook Camera, the figures in the artwork appear to move and come alive.

“I never saw AR and art as separate in the first place. They are both mediums of creation, tools to realise ideas. Art and tech are both genres under the umbrella of creativity and creation,” says Soh, a graduate of the Nanyang Technological University’s (NTU) School of Art, Design and Media. In 2018, Facebook invited Soh to a closed group that developed filters for the social media app, and he is the second person in the world to upload a filter on Instagram while not being a Facebook employee.

Soh first came into the spotlight around 2012, when one his artworks—a composite of 16 photographs pieced together—called The Last Kopitiam went viral. As a first-year college student in 2010, he was asked by a local publication, Campus Magazine, to provide a photograph for the centrefold of an upcoming issue. He decided to create a version of Leonardo Da Vinci’s The Last Supper, replacing Jesus Christ and his disciples with hawkers at Maxwell Food Centre. “It was supposed to be my goodbye piece to the medium of photography before I went into more coding and tech stuff”, and made him into an “accidental” artist instead. But although the artwork went largely unnoticed in 2010, it got a lot of attention two years later.

Another artist at Singapore Art Week to use AR was Farizwan Fajari (Speak Cryptic), whose S*GATTE was inspired by the five stars on the country’s flag. He painted monochromatic gateways, which activated an AR portal that allowed people to ‘step into’ a view of Singapore’s Marina Bay, surrounded by monochromatic characters referred to as ‘The Tribe’.

“Our intention was to provide an extended lens through which audiences could view and experience art,” says Khairuddin Hori, curatorial director and partner of Chan + Hori Contemporary. “As an example, visitors could view the photographic artworks by Eugene Soh, or the installations by Speak Cryptic, and then view them through the added perspective of AR.” Atypical Singapore also had AR ‘calling cards’ that contained artists’ information and scan-enabled QR codes that let visitors carry a part of the exhibition home and share it.

The technology behind giving static images a new and immersive life is MeshMinds, which believes that by applying emerging technology to the arts, creators can engage with a global audience. Much of their work is focussed on educating people about climate change; they created five original AR experiences for the UN Environment Programme in 2019.

MeshMinds also collaborated with the Singapore Tourism Board to create special envelopes for the Chinese New Year in 2020. Scan the QR code and the paintings reveal information about different aspects of the festival. This year, they created the first public AR walking trail, AR.T Trail, allowing people to experience six works in the Tanjong Pagar district, using Facebook’s Spark AR. “We wanted to ensure that the AR was enticing enough to make people stop and look again at the stories hidden within the public artworks,” says Vasey of MeshMinds.

As with Time Tree, Pembungaan, a bronze mural by Kumari Nahappan, pays tribute to Singapore’s history. Its location has historical significance as it looks over the Singapore River, a key stop on the Spice Route. The mural is an ode to nature and takes inspiration from chillies and nutmeg that were traded along the route. AR lets viewers experience this movement of spices, and the changing of the seasons, indicated by the shifting colours of flora. MeshMinds’ creative technologists had to recreate chillies, nutmeg, lotus flowers and saga seeds before animating them. “We love working at the convergence of art and technology, and enjoy pushing the boundaries of the real and virtual worlds to create showstopping experiences,” says Vasey.

With AR even a blank wall can have secrets. At Facebook’s California office campus, a mundane white wall turned into a waterfall of blue, cascading from the roof and into a puddle on the floor, when viewed through Facebook Camera. The installation was created in 2017 by San Francisco-based artist Heather Day and was the first largescale art work created with Facebook Camera.

Similarly, there are many blank walls on Singapore’s streets hiding secrets that are not visible to the naked eye. In the island-state starved of space, empty walls come at a premium and there’s an ongoing tussle between artists who are out to seek a canvas for their work. Unsanctioned street graffiti is considered vandalism, so artists have to work within state-endorsed spaces. Even then, some art gets wiped off by owners of the walls. It’s what led Rscls, a street art collective based out of the Aliwal Arts Centre (AAC), to find an innovative solution—their painted walls are now blank. Visitors have to scan an image from an AAC postcard or sticker and turn their phone cameras to the wall to watch their graffiti come alive.

This is not the first instance of public art that combines technology to bring murals to life. In 2015, Singapore-based artist Brandon Tay created Cermin Ajaib (The Magic Mirror), which allowed visitors to become a virtual character and change costumes as they moved.

If Soh and MeshMinds are to be believed, the future seems exciting for the AR-art collaboration. “AR is disrupting the art scene globally. Everyone’s getting in on it. It would be cliché if it wasn’t so fun. Right now, there are too many copycats. Over time, we will probably see more original works, as people get more familiar with the platform,” says Soh.

●  The writer attended the Singapore Art week on the invitation of the Singapore Tourism Board.