Snapchat, the ephemeral photo and video messenger from Snap, has come a long way since it was initially released in 2011. Over the years, the company has expanded its functionality to include new features powered by augmented reality (AR), leveraging its home-grown wearable hardware. While Snap is far from the only firm working on incorporating AR technology into its products — we recently saw Oppo and Xiaomi unveil wearable AR hardware at the Mobile World Congress (MWC) 2023 — the company is also actively working on helping creators monetise their content by working with brands. Gadgets 360 sat down with Joe Darko, Global Head of AR Developer Relations at Snap, to discuss the company’s plans for AR content, the creator economy in India, and future AR-related hardware offerings.
Gadgets 360: Tell us about Snapchat’s relationship with AR and how Snap has increased the use of AR in the app?
Joe Darko: Eight years ago, we launched Lenses, and they actually help drive engagement on Snapchat. When you think about it, people are using rainbow Lenses and dog-ear Lenses to really engage with their audience and communicate with people. So over time that has evolved, right, we see that Lenses have become more and more innovative. About 250 million Snapchatters use AR on a daily basis, and that is a lot. These are experiences are created by about, let’s say, 300,000 AR creators around the world. These people bring their skill sets to the platform, leveraging Lens Studio’s AR technology platform, which we use to create Lenses, which are used on stories and even on Spotlight.
So that is how Snapchat has been able to grow as well. But we’re moving slowly, when you think about it. When Lenses started out, it was more about fun and entertainment, but it’s now moving more towards innovative use cases like commerce and education. That evolution has actually enabled us to really bring in more advanced AR creators and AR developers who are leveraging the latest technology to really build advanced experiences. Think of multiplayer experiences with connected Lenses. So, you and I can actually play a game within the Lens itself.
Recently, we launched ray tracing technology; the first of its kind on mobile. Now, you can see a Lens become more real due to light estimation. So, for instance, you see light hitting this glossy table. Right now, you’re not going to see the light reflect off the table if you point the Lens at it at night — you can create that experience in AR. And that, in a nutshell, makes stuff like jewellery in AR look so good. Gaming is another use case, which with the help of AR technology has become more advanced, with ray tracing. For instance, gaming looks more realistic including water and lighting and related scenarios.
Gadgets 360: Can a user who is unable to write code create Lenses on their own? Can you explain the process of developing AR Lenses for non-technical users?
Joe Darko: Anyone can create a Lens; we have people from artistic and design backgrounds, to hardcore developers who use advanced code to build AI experiences. Even in our ecosystem, with the programs that we run, we try to cater to both ends of the spectrum.
We also have another tier — partners. These are people who have started businesses, and are actually leveraging creators and developers to build AR experiences. You can open Lens Studio right now and create your first Lens by leveraging our templates. And as you continue to become more competent, you’re going to grow in skill sets and leverage more experiences and build more innovative Lenses.
Gadgets 360: How is Snapchat evolving amid increasing AR adoption? Does AR build on the original idea of disappearing messages or does this broaden its scope to completely different use cases?
Joe Darko: When you look at Snapchat, you’re talking about the formal aspect of the app, where once someone sees a message, it’s gone, right? But people are also using AR to communicate and share their experiences. It’s made communication more fun and more engaging, because it’s not just sending a text message. But when we look at AR, we’re thinking about it from a perspective like commerce.
People are now building these AR experiences from a business perspective; from a commerce perspective. For instance, fashion brands are actually showing what AR can do. You can see how clothes look on you before you try them on. Aside from just chatting, you can also engage with friends or other Snapchatters in AR games.
Lenses also enable you to build outside of Snapchat, with different capabilities using our developer software, Camera Kit. Let’s say you have a dating app. You’re able to build a system that leverages the Camera Kit SDK, which brings in the camera functionality of Snapchat into your dating app. And you can use Lenses to add the same functionality into your app. It’s also Snapchat for Web and the apps. So, it’s not just within Snapchat, it’s outside of Snapchat, which is helping us reach not just more developers, but even more users who are engaging.
Gadgets 360: How important are Lenses to Snapchat’s creator community? Can you share some numbers on how many users in India actively use Lenses on Snapchat?
Joe Darko: So, I won’t be able to go deep into specifics about the data at the moment, but I can start by talking about Lens creators in the country. Snap’s Indian AR creator community, who are also active users of Lens Studio – Snap’s free desktop application designed to help artists, designers, and developers build AR experiences – grew by 60 percent in 2022.
We’re also working to build locally relevant Lenses and hyperlocal revenue Lenses. I know last year, we had about 75 festivals for the year. Lenses are not only for festivals, they’re also for moments like Valentine’s Day; they can also be city-specific, like so Indore has a geofilter for that particular specific day for that city. These are the moments that we’re building for.
And we are always tapping in to our AR creator ecosystem to build some of these experiences, as well as a sense of community for creators. Since January, we’ve had about 20 meetups in multiple cities including Bangalore, Mumbai, Delhi, and Hyderabad. I’ve reached about 1,000 developers at these meetups. Creators who are interested in learning about Lenses and building on them attend these events. We recently had one where approximately 200 Lenses were submitted by about 180 students.
Gadgets 360: What are specific use cases of AR for interpersonal messaging? When and how would the average person use AR in their communications?
Joe Darko: I feel like AR ensures that my communications are more enriched. Communication has to have some level of substance. For instance, my family is a perfect example. I’m not good with design for home and shopping, so I can communicate with my wife — when I’m traveling, she can connect with me and say “Look, that’s how this couch is going to look”. And also, it keeps conversations entertaining instead of being monotonous. My wife uses Snapchat to send me messages with Lenses on our kids’ faces, instead of just basic pictures of them.
AR Lenses makes communication richer, and get more people engaged and involved in communication while adding a lot of colour to the conversation. Because sometimes when we’re not talking face to face, there’s a lot of elements missing. With AR, we are able to add different elements of communication to feel more engaged; more involved. For example, if I use an AR experience while talking to someone else and showing them around, they feel like they are there with me in that moment.
Think about it this way – AR is more immersive. And so if your communication is immersive, you really get in-depth with people, they’re drawn into whatever you’re talking about. And we believe that AR is the most compelling path forward, when we talk about the next reality or the XR industry. We have actually dedicated ourselves to AR because we think it enriches our lives as we’re more immersed in it in a very healthy way. We’re not transported it into a whole new virtual world, we’re still involved in our surroundings — we’re still present.
Gadgets 360: Snapchat users are also creating AR Lenses that are used by brands for marketing purposes. Can you share some examples of creators in India that have monetised their Snapchat AR Lenses?
Joe Darko: Yeah, the first bit I’ll touch on is actually one of the most important things here — monetisation. If you want creators and developers in the ecosystem to build a platform, there has to be a path for them to monetise. And these branded opportunities give developers a chance to really monetise, and continue to be embedded in a platform and grow their skill sets. We’re able to connect our creators to brands and businesses who want AR experiences, and it gives developers a chance to create, make money and build businesses.
There’s a very robust ecosystem in India and lot of our creators have started businesses. They started off as individual creators; individual developers, and then they partnered with others and started a business and now they build for brands and businesses who add monetary value to the ecosystem.
Take the example of Gayathri and Pradeepa, two talented and visionary women who started their journey as AR app developers and quickly evolved into social AR developers. Today, they are successful Snapchat AR experience creators and have founded their own private studio. They started with three people and have now expanded to more than 15 full-time employees along with eight freelancing consultants — these include developers, designers and modelers. And this is a business they started about a year and a half ago.
They started by developing simple AR apps and gradually worked their way up to creating complex AR experiences for major brands. Their company has created over 150 AR experiences for some of the biggest names in the industry, including Google Lens, Pepsi, Disney+ Hotstar, Vijay Television, Zee Entertainment, Neutrogena, Flipkart and many more.
Meanwhile, 24-year-old Kavin Kumar started social AR in 2018 as a hobby and worked for a creative studio making social Lenses for global brands. Now he runs a Coimbatore-based XR Studio, which helps brands use social XR for marketing campaigns and engage with their audience.
Recently Kavin started a London-based firm to target brands and opportunities in the European AR market. He employs five full-time employees and three freelancers, up from a single employee since last year. His team has worked for HBO for House of the Dragon, Camera Go, PS1 LyCA production, Zara, Coldplay Music, Tesco, The Great British Bake-Off, and Old Spice.
Gadgets 360: Do creators in India get a chance to work with international brands? Does Snap facilitate partnerships between creators and brands in the country?
Joe Darko: We have a creator marketplace where AR creators can get in touch with brands. Sometimes we have brands coming to us and we tell them to work with some creators to build out experiences, because we trust our creators who are closest to us to build these robust AR features for brands and businesses. This marketplace is pretty much like a self-service system where brands can go and find them.
Sometimes we connect brands that have relationships with Snap to AR creators and developers for new projects. We are in the business of democratising AR and that really started with Lens Studio being an easy and free-to-use developer platform. And as they get more competent and more comfortable, creators are able to grow with AR and are able to work with companies and make money.
For example, a brand that wants to build an experience for users in India because they want them to build a local, relevant experience for the Indian market, would want to find someone in India. Then, we would connect them to creators in India. If they really want to hyperlocalise it, they can definitely do that. Take for example the partnership with HBO for the TV series House of the Dragon. Our teams were involved in picking the creators who would build that experience. And it was just amazing to see the global creator ecosystem come together and build it for HBO.
Gadgets 360: What about monetisation for Snap as a company? How will the company use AR for commercial product and brand placement on Snapchat in the future?
Joe Darko: First and foremost, we want to serve the ecosystem of brands, businesses and advertisers. So, when developers and creators work to build great AR experiences, they are possibly tied to an ad campaign. And that is how Snapchat comes in with the potential to monetise. But I’ll be very candid and say it’s all about the health of the ecosystem is not just about Snapchat making money.
We’re looking to build a healthy ecosystem, to help developers, make sure we have the right platform, while ensuring advertisers get what they want to continue to drive revenue for them. The key message here is not just Snapchat making money off AR as the ecosystem, but everyone benefiting from AR.
Gadgets 360: What are the challenges of supporting AR features on older or low-end hardware? Is there a set of minimum requirements that a user’s phone must have to use these features?
Joe Darko: Whenever we roll out a feature, we do a lot of testing on multiple devices in different parts of the world to understand how it will work. And sometimes we may launch something to see how it does on a specific phone. We conduct testing behind the scenes to understand how those experiences or features will fare in regions where there may be, let’s say, lower-end devices, because we want to make sure anyone can pick up Snapchat to build experiences for themselves.
And we know not everyone has the high-end devices, so it’s a lot of testing. Even when we launch ray tracing for shopping features, we knew that ray tracing is very powerful experience, right? We went back to work with the product team to better understand that we make sure this feature really works across all devices, and that the experience was constant for everyone.
Sometimes we release products for high-end devices, then we go back retroactively and figure out how we can make them better and easier for other devices. For instance, Lenses built with specific megabyte sizes — obviously you want to make sure there’s this threshold to work across all devices.
We also have to do our homework to make sure we’re not releasing features that only work on specific devices. It’s more about testing, monitoring, and understanding feature adoption and capabilities and how would it work. That’s not something we’ve spoken about openly, but we do the work to make sure they work on different devices.
Gadgets 360: Snap hasn’t announced any new hardware since the Pixy flying camera was shown off last year. Is the company working on any AR-specific hardware? Will we see new AR spectacles soon?
Joe Darko: I’m actually very glad you brought this up. We did bring Spectacles 4 to developers and they were actually AR glasses, but it was a dev kit for developers to build with us. We believe in being open with our developer community – when you do that, you’re able to get feedback from the community and you’re able to build with them and build experiences they will love.
So, we gave the developers who applied through the Spectacles programme a chance to test these AR glasses and build their experiences. And they’re just like AR glasses. It wasn’t available to consumers, just developers that we picked to build with us.
As Snap we believe that users experience AR through their phones, but we also understand that the future of AR is in wearables and AR glasses. So, we’re definitely trying to iterate in that area and actually build glasses that we believe that developers and consumers will love.
I can’t tell you when we’re going to have an announcement for the launch of new AR glasses, but they are actually a part of our roadmap as Snap doesn’t want to just ensure that AR works well on phones but also on head-mounted displays in the future.
Gadgets 360: We saw prototypes of new AR glasses from companies like Xiaomi and Oppo at the recently concluded Mobile World Congress (MWC) 2023. Does Snap plan to work with other hardware makers to support Snap AR features on their devices in the future?
Joe Darko: I’ll be honest and say we are open to the idea. We’re definitely open to build alongside the ecosystem, with ecosystem enablers. But I believe that we’re probably going to take on the same approach that we set first for the Spectacles 4, which is the AR glasses for developers.
How do we build something for our community and our ecosystem first based on how we know it? If there’s room to work with different partners in the ecosystem, then I believe that from that perspective, we don’t want to leave any stone unturned, as a company. But at the same time we’re figuring how to do this ourselves. We want to learn from it and do better ourselves.
Some responses have been condensed and slightly edited for clarity.
