Mirror, Mirror

The SalonAI team was selected as one of the top two U.S. teams in L'Oréal’s Brandstorm—an annual competition that challenges college students from around the globe with inventing the salon experience of the future.

For centuries, the design behind the mirror has remained largely unchanged—just a simple piece of polished glass without much room for innovation. But a team of recent Penn State alumni and a current student have reimagined not only the future of the mirror, but of the salon experience as a whole. 

Made up of 2018 Penn State graduates Vincent Trost (data sciences) and Vamshi Voruganti (industrial engineering) as well as senior Yuya Ong (data sciences), the SalonAI team developed a smart mirror that uses facial recognition software to analyze faces and recommend hairstyles and cosmetics based on previously collected data. 

The mirror was developed as part of L'Oréal's Brandstorm, an annual competition that challenges international teams of college students with modernizing the professional salon experience while engaging millennials and increasing customer loyalty to their stylists. SalonAI was selected as one of the competition’s top two U.S. teams, and the team presented the idea and prototype at the international finals in Paris in May 2018. 

Team SalonAI at the L'Oreal Brandstorm

Team SalonAI - L'Oreal Brandstorm

Team SalonAI (Vince Trost, Yuya Ong, and Vamshi Voruganti) receive their award as one of the top two U.S. teams in the L'Oréal Brandstorm from David Greenberg, president of L'Oréal's Professional Products Division. The team earned the award for their creation of a "smart mirror" and will compete in the world finals on May 17 in Paris. 

IMAGE: Penn State

“Penn State is a great place for student entrepreneurs, mostly because of the massive network of individuals that we were able to leverage.” —Yuya Ong, Penn State data sciences student

Creating salon solutions

As members of Nittany Data Labs, Trost, Voruganti, and Ong have spent several years exploring new applications of data sciences. Recognizing that salon interactions are most often based around a mirror, SalonAI is aiming to bring the mirror into the 21st century by using computer vision technology, machine learning algorithms, and data collection infrastructure.

As part of the Brandstorm competition, L'Oréal provided the SalonAI team with internal research stating that while 97 percent of stylists believe they provide consultations to their clients, only 7 percent of clients feel like they receive one. This disconnect helped the team members hone in on their proposal. 

"This is a problem we identified, and the way to fix that problem is more engagement," said Voruganti.

The team also surveyed more than one-hundred women to gather feedback about the salon experience. Their research showed that most of their peers either asked the stylist to recreate a hairstyle based on a photo from their smartphone or simply let the stylist choose which style would look best. Neither is ideal, the group proposed.

"When you get a haircut, you spend the whole time staring in the mirror," explained Trost. "Instead of trying to describe it or zoom in on a photo on your phone, why not just display the style on the mirror? Then you and the stylist are working together, identifying what you do and don't like, simulating it on you in real time, and everyone is happy because you can see exactly what you're going to get before you get it."

To develop this solution, the SalonAI team turned to Penn State’s entrepreneurial resources and vast network of faculty and alumni for support. 

“Penn State is a great place for student entrepreneurs, mostly because of the massive network of individuals that we were able to leverage,” Ong said. “Connecting with alumni and people in the industry that come from Penn State not only allowed us to create powerful connections but also established a support system for us to share ideas, resources, and effortlessly join solutions together.” 

From dorm room to global stage

To create the mirror, Ong programmed facial recognition software that identified key points on the user's face—such as the distance between the eyes and the curve of the jawline—to determine face shape. Trost used an algorithm to determine which hairstyles were seen on models with different face shapes featured in various beauty catalogs. By combining the two, the mirror can recommend and simulate hairstyles and cosmetic products that are popular among others with similar features.

“With this system, we can better analyze many features including facial shape, age, skin tone, and other critical information about a customer in real time.” —Yuya Ong

Yuya Ong demonstrating facial recognition technology

Yuya Ong - L'Oreal Brandstorm

Yuya Ong demonstrates the facial recognition technology that powers team SalonAI's smart mirror. The technology maps the user's face by recognizing key points, such as the jawline's shape and distance between eyes, and displays popular hairstyles and colors seen on similar face shapes.

IMAGE: Penn State

"With this system, we can better analyze many features including facial shape, age, skin tone, and other critical information about a customer in real time," said Ong in the team's video pitch. "By leveraging this data, we can better deliver a higher quality of services than ever before and will generate tremendous brand loyalty."

Ong built the mirror in his dorm room over two days using an old computer monitor and spare parts he found online. He then created the entire facial tracking algorithm in twenty-four hours during HackPSU.

Not only for looks, the mirror also serves as a pipeline for data collection, which the group views as the most valuable element of their product. For example, by indexing individual preferences, companies like L'Oréal can anticipate and develop products that respond to trends, and stylists can improve their performance.

"The entire salon experience is seen in the mirror, which allows us to capture the subtler points," said Ong. "This lets us solicit real-time feedback using facial reactions to see whether the consumer likes the style or not. The stylist can record the user's preferences and get feedback on how well they performed the services they provided."

"We are not trying to replace the stylists," said Trost. "They are the reason it works, but we are trying to enhance the experience and make it more collaborative."

Reflecting on an idea

The team believes much of their success has come from their ability to capitalize on individual strengths and Penn State’s network of resources. 

"To run a successful startup, you need three people," said Trost, recalling advice given to him by Steven Haynes, associate teaching professor of IST. "One should be completely devoted to product development—that's Yuya; one should be completely devoted to marketing the product and securing funding—that's Vamshi; and one should be focused on making things run smoothly—which has been my role."

“We've had the opportunity to pitch our idea to CEOs of Fortune 500 companies, refine it, and make it better." —Vincent Trost, Penn State alum

Along the way, the team received hands-on coaching and regular feedback from L'Oréal executives, industry experts, and Penn State faculty. They credit Colleen Connolly-Ahern and Lee Ahern, both associate professors in the Donald P. Bellisario College of Communications, for helping them reach this point.

“The innovation process is hard—we know this firsthand from the endless hours of late nights and the constant stress and hustle,” Ong said. “But knowing that we had a large family of staff, faculty, peers, and community supporting us, we were able to confidently work hard and strive for the best.”

Noticing the potential for smart mirrors in industries like high-end retail, fitness and personal health, the SalonAI team is looking forward to receiving additional feedback and exploring potential opportunities through L'Oréal's beauty tech accelerator.

"We're ultimately trying to start a business," said Trost. "We've had the opportunity to pitch our idea to CEOs of Fortune 500 companies, refine it, and make it better."

Looking back, Ong has one key piece of advice for other Penn State entrepreneurs looking to start their own journey from idea to prototype.

“Be resourceful as much as possible—you are sitting on a gold mine of people, ideas, community, and opportunity waiting to be unearthed,” Ong said. “It's up to you to find them and put them to work to create something novel.”

Penn State Entrepreneurs Featured on LiveBIG Instagram

Innovating—from working with new materials to creating products that can revolutionize industries—Penn State students and alumni from diverse backgrounds are using their creativity, dedication, and entrepreneurial mindsets to identify problems and create solutions. For more stories about Penn State entrepreneurs and innovators, check out Penn State’s BTN LiveBIG Instagram takeover, running from Sunday, October 7, through Saturday, October 13.