The Sephora Virtual Artist app first came out in the spring of 2017, first starting out with a feature for trying on eyeshadow before moving onto other products. If you’ve ever gone out shopping with your friends in Sephora, you’ve also probably seen some version of this in the store, where instead of your phone screen, you’re greeted with iPads and an even larger screen to give you product recommendations. If you think that they feel a little like Snapchat filters, you’re not totally off — both apps use the same technology to fix up your look on the screen. But how does this all work? I’m going to be discussing a couple of key buzzwords that you may have heard tossed here and there so you can really get to know what’s behind the blush and lip shades you play with on the app.
To many out there, you’ve probably heard of these letters thrown out when talking about video games, but games aren’t the only applications for these experiences. And while they’re similar, they’re not exactly the same — but it’s easy to remember which is which.
Augmented reality is, in layman’s terms, digital things in our world. To augment something means, “to make larger; enlarge in size, number, strength, or extent; increase,” according to Dictionary.com. With this definition, you can imagine that augmented reality are digital items that “enhance” or “augment” the real world. Pokemon Go is one of the most popular examples of augmented reality out there for video games. An option you can have when catching Pokemon is that you can turn on your front camera so that the Pokemon, your Pokeball, and any other items and animations are all against the background of the real world. Another option you probably know is Snapchat. The filters you can use for your selfies use augmented reality to add butterflies on your face, flower crowns on your head, and dog tongues when you open your mouth. Sephora’s Virtual Artist app, which I’ll talk about below, is another example of augmented reality.
How does Sephora Virtual Artist work?
As mentioned before, Sephora Virtual Artist uses augmented reality to have users try on thousands of lipstick and eyeshadow shades without having to commit to purchasing the products and physically trying them on. It can also guide you through digital make-up tutorials personalized for your face, and match color shades to your skin. To do all of this, AR uses computer vision to understand what’s going on in the real world.
Our brains and eyes can process the real world just fine — but as smart as we think computers are, image recognition is actually a really huge issue for them. Entire labs are actually dedicated purely for computer vision. To be able to map out the makeup on the face, the app needs to know the depth of the face, where your eyes are, how big your lips are, etc. First, the camera is going to take in your face and map out what features are present and where they’re located. After you pick out a shade of lipstick, the app uses the camera data to show your new look on the screen.
It’s pretty amazing to think that all of this is happening in a matter of a few seconds.