UNIVERSITY PARK, Pa. — A trio of enterprising Penn Staters will travel to Paris to compete in the world finals of the 2018 L'Oréal Brandstorm on May 17. In the competition, they will pitch their "smart mirror" prototype and compete against teams from around the world for awards for the best innovative technology, most socially responsible campaign, and top overall brand.

Team SalonAI, which was selected as one of the top two teams in the U.S., is made up of 2018 Penn State graduates Vincent Trost (data sciences) and Vamshi Voruganti (industrial engineering), and senior Yuya Ong (data sciences).

Started in 1992, Brandstorm is L'Oréal's annual competition that challenges teams of college students to develop innovative ideas and business cases for some of the French beauty and personal care company's 24 international brands. This year's event included more than 34,000 registrants representing 50 countries, and asked teams of students to invent the professional salon experience of the future by using modern technologies that attract and engage millennials and increase customer loyalty to their stylists.

The making of a smart mirror

As members of Nittany Data Labs, Trost, Voruganti and Ong have spent several years exploring new applications of data sciences. Recognizing that salon interactions are most often based around a mirror, SalonAI is aiming to bring the mirror into the 21^st century by using computer vision technology, machine learning algorithms, and data collection infrastructure.

To create the mirror, Ong programmed facial recognition software that identified key points on the user's face — such as the distance between the eyes and the curve of the jawline — to determine face shape. Trost used an algorithm to determine which hairstyles were seen on models with different face shapes featured in various beauty catalogs. By combining the two, the mirror can recommend and simulate hairstyles and cosmetic products that are popular among others with similar features.

"With this system, we can better analyze many features including facial shape, age, skin tone, and other critical information about a customer in real time," said Ong in the team's video pitch. "By leveraging this data we can better deliver a higher quality of services than ever before and will generate tremendous brand loyalty."

Ong built the mirror in his dorm room over two days using an old computer monitor and spare parts he found online. He then created the entire facial tracking algorithm in 24 hours during HackPSU.

Not only for looks, the mirror also serves as a pipeline for data collection, which the group views as the most valuable element of their product. For example, by indexing individual preferences, companies like L'Oréal can anticipate and develop products that respond to trends, and stylists can improve their performance.

"The entire salon experience is seen in the mirror, which allows us to capture the subtler points," said Ong. "This lets us solicit real-time feedback using facial reactions to see whether the consumer likes the style or not. The stylist can record the user's preferences and get feedback on how well they performed the services they provided."

"We are not trying to replace the stylists," said Trost. "They are the reason it works, but we are trying to enhance the experience and make it more collaborative."

Creating a shared salon experience

L'Oréal's internal research has shown that 97 percent of stylists believe they provide consultations to their clients, but only seven percent of clients feel like they receive one. This disconnect helped the group members hone in on their proposal.

"This is a problem we identified, and the way to fix that problem is more engagement," said Voruganti.

The team also surveyed more than 100 women to gather feedback about the salon experience. Their research showed that most of their peers either asked the stylist to recreate a hairstyle based on a photo from their smartphone or simply let the stylist choose which style would look best. Neither is ideal, the group proposed.

"When you get a haircut, you spend the whole time staring in the mirror," explained Trost. "Instead of trying to describe it or zoom in on a photo on your phone, why not just display the style on the mirror? Then you and the stylist are working together, identifying what you do and don't like, simulating it on you in real time, and everyone is happy because you can see exactly what you're going to get before you get it."

Added Ong, "This isn't automated intelligence or artificial intelligence; it's augmented intelligence."

Reflecting on an idea

Nearing the end of the competition, the team believes much of their success has come from their ability to capitalize on individual strengths.

"To run a successful startup, you need three people," said Trost, recalling advice given to him by Steven Haynes, associate teaching professor of IST. "One should be completely devoted to product development — that's Yuya; one should be completely devoted to marketing the product and securing funding — that's Vamshi; and one should be focused on making things run smoothly — which has been my role."

Along the way, the team received hands-on coaching and regular feedback from L'Oréal executives, industry experts, and Penn State faculty. They credit Colleen Connolly-Ahern and Lee Ahern, both associate professors in the Donald P. Bellisario College of Communications, for helping them reach this point.

"They have gone above and beyond what we could ever expect from them," said Voruganti. "They've helped us refine our pitch, served as sounding boards, and filled in our blind spots."

To make it to the world finals, more than 1,000 U.S. registrants submitted video pitches, with 10 teams selected to present their ideas at the national finals in New York City. The top two teams from that national competition — including SalonAI — were selected to participate in the final competition in Paris. In the upcoming final round, the group is one of 100 global teams that will pitch their idea and answer questions from a panel of judges.

And while they're excited to be in the finals, the group is focusing their attention on turning their idea into an entrepreneurial business. Noticing the potential for smart mirrors in industries like high-end retail, fitness and personal health, they're looking forward to receiving additional feedback and exploring potential opportunities through L'Oréal's beauty tech accelerator.

"We're ultimately trying to start a business," said Trost. "If we lose this competition we lose nothing, but we've had the opportunity to pitch our idea to CEOs of Fortune 500 companies, refine it, and make it better."

Concluded Voruganti, "It's risk-free entrepreneurship."