Hello readers this is the first of several short articles where I talk about some of the research that goes into making my images.
In this particular post I will be looking into the Uncanny Valley of digital characters. To start you off with a simple explanation –
The uncanny valley refers to our ability to differentiate between real human faces and things that TRY to look like real human faces. This in essence is a defense mechanism that millions of years ago allowed us to spot predators or other dangers that could potentially try to lure you with a familiar sight.
Fast forward several million year however and the uncanny valley is your ability to differentiate between real and CGI characters. More than that studies find that certain characters like cartoons or stylized drawings and paintings do not trigger this uncomfortable feeling – however more advanced models with accurate skin and eye textures seem to trigger this feeling.
Stay tuned for more science and (dare I say) research on the matter (by ME!!!).
Needless to say this field is extremely interesting to any artist with character artists like me even more so as this can have a direct impact on the quality of our work. As such I have looked at several journal articles on the matter.
Facial Expression Of Emotion And Perception Of The Uncanny Valley In Virtual Characters (11th Otc 2016)
This paper examines the effect of detailing on different sections of a character’s face when it comes to how believable they are. The paper looks at how certain expressions and emotions can be more difficult to reproduce or are more prone to causing the Uncanny Valley Effect. The paper concludes that it is detailed animation of the upper part of the face that plays the most vital part, due to fact that its lack could “inhibit effective communication of emotion”. As such some emotions like anger or happiness are less affected by this while fear, sadness, disgust and surprise can be more difficult to replicate.
This research does have some limitations like the reliance of subjective data to measure the response of the subjects – it is mentioned that future studies should implement methods of measuring physiological changes in the subjects bodies to increase the accuracy of the data. Another thing to consider is the age of the article – 2011. This is important as recent developments in photogrammetry have led to the creation of some of the most realistic and accurate models that still struggle to appear believable.
As a character artist this research can help me better manage polygon counts and concentration of detail on the upper face while in some cases having me avoid trying to replicate certain emotions if time is a major factor of the project.
Tinwell, A., Grimshaw, M., Nabi, D. and Williams, A. (2011). Facial expression of emotion and perception of the Uncanny Valley in virtual characters. Computers in Human Behavior, 27(2), pp.741-749.
Too Real For Comfort? Uncanny Responses To Computer Generated Faces (14th Otc 2016)
This paper looks at the effects of the model quality on the perceived Uncanny Valley Effect. The paper looks at principles in other areas of study to help identify crucial design principles that can minimize the effect of the Uncanny Valley. These include psychological studies in empathy, mate selection, threat avoidance, cognitive dissonance and psychological defenses.
Several studies show that things like texture photorealism and polygon count can increase believably however it also places more emphasis on the facial proportions, which if found to be atypical can magnify the Uncanny Valley Effect, while mismatches in the texture and size of the eyes can be extremely unsettling.
The principles outlined in the end suggest that photorealistic textures should be avoided unless photorealism is specifically pursues and that a more stylized approach should be used instead. Should photorealistic textures be used it is important to keep all facial proportions within a reasonable limit with minimal stylization to avoid making the character eerie and unsettling.
Some of the limitation the paper are that they used their own creation – an “eerie index” to measure participant feedback. This makes many of the findings unreliable and future research needs to be done in creating a standardized “eerie index” to normalize findings and compare them with other research in the field. Another limitation is the fact that only one base model with many variation was used – that of a 30 year old male, as such some of the guidelines may not apply to characters from the opposite sex or different age groups given the fact that some facial features can become more exaggerated with age.
This paper is useful to me as it not only teaches us how to minimize but also maximize the effect of the uncanny valley – something that can be extremely useful for creature design where a subtle uneasiness could be sought after.
MacDorman, K., Green, R., Ho, C. and Koch, C. (2009). Too real for comfort? Uncanny responses to computer generated faces. Computers in Human Behavior, 25(3), pp.695-710.
To Stylize or not to Stylize? The Effect of Shape and Material Stylization on the Perception of Computer-Generated Faces (17th Otc 2016)
This paper looks at the effects of shapes and materials on the appeal of human faces. Tests with different material properties that simulate skin are done as well an exploration into the appeal of facial shape.
The studies find that facial shape and proportions play the most significant part when it come to creating a realistic and believable characters. On the other hand texture and materials had the most impact on character appeal and believably.
Some of the limitations of the paper are that when it came to exploring styalizations in textures and shapes are that the styles explored were quite limited, as such these findings may not apply to certain styalized characters. This paper while exploring emotions does not include an animated face as such it could be more difficult to correctly perceive an emotion from the participants point of view.
This paper is useful to me as it directs some focus on the materials on the face. This means that emphasis on skin textures as well as effects like sub-surface scattering and specular maps should be carefully considered when making a face.
Zell, E., Aliaga, C., Jarabo, A., Zibrek, K., Gutierrez, D., McDonnell, R. and Botsch, M. (2015). To stylize or not to stylize?. TOG, 34(6), pp.1-12.
Real-Time High-Fidelity Facial Performance Capture 5th Nov 2016)
In this paper the authors propose a method of high quality facial and performance capture techniques using a single low quality camera. What’s more interesting is that the method described does not only capture facial shape and structure but is also able to detect area with wrinkles and create wrinkle maps on the fly as well as capture a respectable amount of other skin and pre detail.
What interesting is that some of the limitations describes include subject obstruction (obviously) but also light changes which can be difficult to control should the subject move or turn during a performance.
I find this technology very interesting as it seems that videogrammetry could soon prove to be a more accurate and affordable method of scanning objects when compared to photogrammetry, something which I am currently experimenting with.
*Personal photogrammetry experiment… (please kill me)
Cao, C., Bradley, D., Zhou, K. and Beeler, T. (2015). Real-time high-fidelity facial performance capture. ACM Transactions on Graphics, 34(4), pp.46:1-46:9.
Breathing Life into Shape: Capturing, Modeling and Animating 3D Human Breathing (17th Nov 2016)
In this paper researchers use data from 58, 3D scanned subjects to determine human body deformations that occur during breathing. The aim of this is to create a simple and efficient animation rig that can applied to any character and have them behave in a realistic method.
As a character artist I am always interested in new animation methods that can bring my work to life as it often means that I will have to adapt my workflow or work under new constraints so that my models and characters can be used in a wide range of situations. While character animation in video games still comparatively rough against film character animation (with their implementation of bone and muscle structures in the characters) I do believe that it is only a matter of time until this technology reaches video games.
Tsoli, A., Mahmood, N. and Black, M. (2014). Breathing life into shape. ACM Transactions on Graphics, 33(4), pp.1-11.
High-Quality Capture of Eyes (5th Dec 2016)
In this paper researchers propose a method for capturing 3D data from photos of the human eye! This is a massive breakthrough as up until this point capturing the human eye through photogrammetry has proven to be impossible. In the paper the researchers no only manage to capture the overall shape of the eye exterior but also manage to make a 3D reconstruction of the iris inside!
The reason why this is a major breakthrough is that photogrammetry is severely hampered by reflective and transparent surfaces – to get around this issue some practitioners have taken to using 3D sprays designed to remove the gloss and transparency of an object so it can be easily scanned ie. a glass vase. However for obvious reasons this is not possible with human eyes.
This breakthrough is very exciting for me as it can open up the possibility of no only scanning human eye but also transparent objects like sea waves or water ripples – something that I have wanted to incorporate in my work for some time now.
Bérard, P., Bradley, D., Nitti, M., Beeler, T. and Gross, M. (2014). High-quality capture of eyes. ACM Transactions on Graphics, 33(6), pp.1-12.