Mixed realities: Kosé’s makeup simulator reproduces ‘ultra-realistic’ colours and textures that ‘AR cannot achieve’

The Mixed Reality Makeup (MR Makeup) Colour Machine employs the use of high-speed projection mapping technology developed by the Yoshihiro Watanabe Laboratory of the Tokyo Institute of Technology.

This technology is what allows the makeup simulation to be projected on the face instantaneously.

“MR Makeup allows the makeup to smoothly follow the human body's movements and changes at the facial expression. The high-speed face recognition and projection technology, updated 1000 times per second, completes the process from facial movement to MR Makeup projection with a few milliseconds delay or less,” ​explained the Japanese cosmetics firm.

At the same time, because it is a projection, it can adapt to any area or shape, it highlighted. “Because this is projection mapping the areas and shape of the makeup can be easily changed without removing the makeup.​”

The high-speed projection mapping technology combined with colour correction technology allowed the company to achieve realistic makeup simulation instantaneously.

“By combining these, we have succeeded in developing an immersive makeup simulator that can achieve a natural finish that looks like you are actually applying makeup, even on a moving face.”​

The results, said the firm, trump the effects of AR-enabled virtual makeup try-on applications we have seen on smart mirrors and smartphones.

“MR Makeup is a technology that reproduces natural colours and textures that AR cannot achieve. Unlike AR filters, the colours are projected directly onto the skin, enabling realistic makeup colours to be produced according to the skin tone of the person.”​

With these characteristics, the company believes the technology has applications beyond makeup, and even beyond cosmetics.

“In the near future, everyone will be able to try out makeup anytime and anywhere. We are looking to expand this technology not only to makeup but also to hair colour and clothing design.”​

Global launch

MR Makeup Colour Machine debuted in August 2022 at the Maison Kosé Ginza flagship store and it was the first time beauty consumers could experience this mixed reality simulation.

A survey of 119 people conducted by Kosé from August to October revealed that it was highly rated across the board with a satisfaction level rated at an average 9.8 points out of 10.

The perceived naturalness was also highly rated, scoring 9.3 points out of 10.

The makeup simulator was also well-received at CES 2023​, which was held from January 5 to 8.

Organised by the Consumer Technology Association, the exhibition is considered one of the most prominent tech trade shows in the world.

During the show, Kosé surveyed 197 visitors who tried the machine and 91% said that they had a “surprising or moving experience”.

Additionally, 86% said that they felt that it offered the possibility of a new way of self-expression.

“We will continue to promote further technological development and business development in order to meet the needs of society and enhance our corporate presence on the global stage.”​