AR in the Beauty Industry

The Sephora Virtual Artist app first came out in the spring of 2017, first starting out with a feature for trying on eyeshadow before moving onto other products. If you’ve ever gone out shopping with your friends in Sephora, you’ve also probably seen some version of this in the store, where instead of your phone screen, you’re greeted with iPads and an even larger screen to give you product recommendations. If you think that they feel a little like Snapchat filters, you’re not totally off — both apps use the same technology to fix up your look on the screen. But how does this all work? I’m going to be discussing a couple of key buzzwords that you may have heard tossed here and there so you can really get to know what’s behind the blush and lip shades you play with on the app.

AR and VR: What’s the difference? 

To many out there, you’ve probably heard of these letters thrown out when talking about video games, but games aren’t the only applications for these experiences. And while they’re similar, they’re not exactly the same — but it’s easy to remember which is which.

Augmented reality is, in layman’s terms, digital things in our world. To augment something means, “to make larger; enlarge in size, number, strength, or extent; increase,” according to Dictionary.com. With this definition, you can imagine that augmented reality are digital items that “enhance” or “augment” the real world. Pokemon Go is one of the most popular examples of augmented reality out there for video games. An option you can have when catching Pokemon is that you can turn on your front camera so that the Pokemon, your Pokeball, and any other items and animations are all against the background of the real world. Another option you probably know is Snapchat. The filters you can use for your selfies use augmented reality to add butterflies on your face, flower crowns on your head, and dog tongues when you open your mouth. Sephora’s Virtual Artist app, which I’ll talk about below, is another example of augmented reality.

Virtual reality is, also in layman’s terms, ourselves in digital worlds, or in a “virtual reality.” They’re usually used to immerse the user in whatever activity the VR app provides. Anything that’s trying to put you in another world digitally is an example of virtual reality. Lots of people also think of video games when you think of VR: you’ve probably heard of headsets like the Oculus Rift and the HTC Vive, and tales of horror games like Emily Wants To Play. However, VR is also used for things other than gaming, like for training simulations for the military and practice surgeries for medical students. I’m not aware of Sephora or any beauty companies using virtual reality with their services, but it’s still a pretty cool thing to build and experience!

How does Sephora Virtual Artist work?

As mentioned before, Sephora Virtual Artist uses augmented reality to have users try on thousands of lipstick and eyeshadow shades without having to commit to purchasing the products and physically trying them on. It can also guide you through digital make-up tutorials personalized for your face, and match color shades to your skin. To do all of this, AR uses computer vision to understand what’s going on in the real world. 

Our brains and eyes can process the real world just fine — but as smart as we think computers are, image recognition is actually a really huge issue for them. Entire labs are actually dedicated purely for computer vision. To be able to map out the makeup on the face, the app needs to know the depth of the face, where your eyes are, how big your lips are, etc. First, the camera is going to take in your face and map out what features are present and where they’re located. After you pick out a shade of lipstick, the app uses the camera data to show your new look on the screen. 

That paragraph might make the whole process seem pretty obvious — you’re basically just looking at the picture and then plastering a picture on top of it. In practice, it’s not that easy. Making sure the computer knows exactly where to put where is an incredibly complicated task, not to mention making sure it looks the way we want it to. This is why Snapchat still won’t make your face totally perfect, even after all these years. 

It’s pretty amazing to think that all of this is happening in a matter of a few seconds. 

• augmented reality
• beauty
• makeup
• sephora
• tech
• technology