Zomato embraces deepfake technology and AI in personalised ads starring Hrithik Roshan

YouTube viewers have been seeing a surprising ad from Zomato. The ad, starring Hrithik Roshan features the actor mentioning that he’s craving specific dishes from popular restaurants in different cities. The name of the dish, the restaurant and the city and locality changes depending on the ad’s viewer - and their phone’s GPS. The ad has been created by the agency Enormous Creative.

The ad uses location data to personalise and localise these ads. For example, if a viewer watching the ad in Khanna, Punjab, Roshan will admit that he was craving a burger from OneBite burger - a local restaurant which is popular there.

This ad is made possible thanks to artificial intelligence and deepfake technology. Deepfake technology can be used to modify a video of someone saying something and make it look like they were saying something else instead. Artificial intelligence learns a person’s speech patterns and facial expressions to make this possible.

Zomato’s ad is similar to Cadbury’s 2021 rendition of ‘Not Just a Cadbury Ad’ - which stars actor Shah Rukh Khan. The ad won multiple awards at the recently concluded Cannes Lions Festival of Creativity - including India’s first ever Titanium Lion. Zomato's ad has been made by creative agency Enormous Creative.

Shah Rukh Khan's ad, made by Ogilvy and Wavemaker features AI and deepfake technology, powered by Rephrase.ai. In that ad, Shah Rukh Khan is seen getting ready for a Diwali party and he mentions names of local sweet stores, optician stores, mobile phone stores and so on. Here also the ad uses location targeting to show people ads based on the city they’re at - naming stores at their locality, based on their GPS locations.

Ashray Malhotra, the founder of Rephrase.ai who had worked along with Ogilvy for both versions of ‘Not Just a Cadbury Ad’ recalls that the team took a long time - nearly three years - to perfect the technology used to create the ads.

“Everything is pre-recorded except for the variables. The variables in the Cadbury ad include name of the clothing store, name of the shoe store, name of the optical store, name of the electronic store, name of the sweet store, etc,” says Malhotra. In Enormous’ Zomato ad, the variables were the name of the dish, the restaurant its popular at and the city where the restaurant is located.

Aalap Desai, chief creative officer, Isobar India and Taproot Dentsu says that creativity in using tech is how and where you use it. “The same concept has been used in the same way by Cadbury's and Symphony. The potential is huge but we will only realise it once we start exploring it in newer ways. When we stop repeating its use and start trying new things is when it'll flourish,” he says.

Malhotra says that the level of personalisation that the technology can give is to not just modify existing messages - but to create an entirely new message altogether. Rephrase has also worked on similar ads featuring Tiger Shroff for Castrol and Ranveer Singh for Colors TV.

He adds that AI and ML have a lot of potential in advertising since it helps increase the efficacy of brand communications by giving the brand an opportunity to speak to their audience, one on one. It has made hyper-personalised and localised brand communications possible.

Desai agrees that personalisation makes these ads much more relatable and interesting in a way that appeals to local residents of an idea.

Deepfake technology and personalised videos can also be used to build a better customer journey and relationship. “Imagine after you make a purchase from a brand, you receive a personalised message from the CEO of the company. It’s a game changer,” says Malhotra.

Desai points out that in Zomato’s case, the ad will reinforce the fact that it delivers everywhere. “Apart from that, it's not a unique piece and makes Zomato appear so late to this party that it seems like a mild branding piece. Doesn't excite me at all,” he says.

Behind the scenes - execution and challenges

ZigZag Films was the production house that worked on shooting the raw footage for the AI technology to work on. Abhijit Sudhakar, the co-founder of ZigZag Films recalls that initially the team had given Roshan a 20 minute long monologue to read out so that they could capture information about his voice, its tone, the intonation of the words, etc, and the camera kept capturing him.

Anupama Ahluwalia, founder and producer at ZigZag Films mentions that the line they initially had Roshan say was that he was craving paneer butter masala, and didn’t mention which restaurant or area it was from. Eventually, the dialogue was recreated for different localised contexts to create the personalised ad films.

Desai calls the execution of the ad good. The ad features one of our superstars and the tech is smooth and clean. “It's been done well but when you are the third or the fourth one doing it, there is no first mover advantage and you are just using the tech that's been developed and perfected by others to add to the clutter,” he mentions.

Sudhakar adds that it was a challenging exercise for Roshan as an actor since he had to deliver the line in a certain way to add information when they had to manipulate his voice. “He had to stay in character and deliver the lines with the right expressions to add the three parts of information in that sentence - the name of the dish, the restaurant, and the area. He had to take pauses at certain places to allow the AI tech to add in the information later on,” he says.

Desai says that the fact that the Zomato ad is similar to the Cadbury ad is not at all a reflection of the state of creativity in India; it just shows an openness to working with new formats and technology - an indicator that things are moving in the right direction in the Indian ad industry.

During Zomato’s ad, Roshan is sitting in a plain black truck, surrounded by people armed with guns and bulletproof vests. They hear a noise outside the van and open the doors to an alarmed Zomato delivery man holding Roshan’s order. The ad has two main visual parts - the exterior of the van and the interior, with Roshan and his team.

“Usually when you’re working with a celebrity, it’s difficult to shoot outdoors. The wide shot at the beginning of the van and the industrial-looking area its parked at was created using a miniature version of the van. The setup for the interior of the van was created in a studio,” reveals Sudhakar. The production house had used a similar studio setup while creating PhonePe’s ads with Aamir Khan and Alia Bhatt.