Sintel: Facial Blendshapes
on February 24th, 2010, by angela
Sintel shapes, 12 examples
some quick tests combining shapes, screencaptures
Sintel pupils dilate, test renders
Note: This blog post is only focused on facial shapes. Texture is still in progress.
Our main character Sintel has a total of 55 shapes. These shapes have all been hooked up to Nathan’s facial rig. Tomorrow I will start creating various poses (phonemes & emotions) for our facial pose library.
Some of my workflow when creating facial shapes:
– I start by shaping each in its symmetrical form using X mirror and proportional edit. For better precision and polishing, I move vertex by vertex.
– While the shape is taking form, I constantly scrub through 0-1 (0-100%) range value to ensure natural motion from default to shape. I also check to make sure motion looks natural from all view angles of the face (front, three-quarter front, side, three-quarter back, top, bottom).
– Overriding the shape beyond full value of 1 will help magnify any trouble areas and/or continue building from it.
– For shapes that need to mix well together, I activate two shapes at their full value and continue editing on the active selected shape.
– For mouth shapes, I open the jaw for inside mouth cleanup, ensuring the mouth is not intersecting through the teeth.
– Each shape is exaggerated beyond what is considered natural. This allows for better flexibility while animating. For closer shots, the animator can go from subtle to a maximum of about 60-70% of the shape. Pushing beyond 60-70% range is reserved for distant shots where the extreme maximum 80-100% of the shape can still be read from far away.
– As a final step I split the shapes up (example: left side & right side) using vertex groups and “Mirror Shape Key”.
p.s. Shape key creation in Blender 2.5 is awesome! 🙂
– Angela
February 24th, 2010 at 12:46 am
WOOOOPIE I’m the first! Keep up blending Angela.
February 24th, 2010 at 12:51 am
Angela, this is an amazing job ! I’m really impressed.
Now I understand why you were so hardly working in the edgeloops of the face topology. It really paid off 🙂
It’s also nice to see you guys/gal manage to get wet eyes in render (or is this gimp/post-pro?).
The “sad” face looks so pity … nice to see our warrior can develop warm feelings.
Congratulations !!!
February 24th, 2010 at 12:51 am
Nice work. One thing I do is to create a simetrical shape using X-Axis mirror edit and then use a vertex group to create single side shapes.
Keep it up !
February 24th, 2010 at 12:59 am
Great job. Sintel looks wonderful with these shapes.
February 24th, 2010 at 1:03 am
mmm… morphing is the future of facial animation. The main problem is that you cannot reuse facial animation libraries so easily than bone animation. Impressive work, very thorough.
February 24th, 2010 at 1:12 am
Looks like Sintel is going to be a lively character! It will be interesting to learn how these will be incorporated into the rig.
February 24th, 2010 at 1:15 am
Angela, your job is absolutely awesome congratulations!
P.S.: It’s really weird to see a bald woman…
February 24th, 2010 at 1:19 am
Post a blend of the model + rig once you’ve got it hooked up so we can play with it and do dialogue tests!
February 24th, 2010 at 1:20 am
Those pupil renders look fantastic.
February 24th, 2010 at 1:21 am
Great Job indeed.
Keep it up.
February 24th, 2010 at 1:31 am
i love you… it, i love it.. that’s what i ment.
good job (as in awesome job).
February 24th, 2010 at 1:35 am
Wow! Looks excellent, so far. And the WIP render is awesome to see, too. (I’ve been dying to see a test render of Sintel.)
February 24th, 2010 at 1:35 am
Love those renders, especially how lashes turned out soft. 🙂 The browSup.L shape looks like a whole facial expression itself ^^
February 24th, 2010 at 1:50 am
those eyes… ooo….
February 24th, 2010 at 1:58 am
Impressive!
I think most 3D characters being made suffer from rigid and limited face movement. A lot of the time it seems as the goal is only to make the viewer understand if it’s a happy, sad or surprised face.
This is one of a few cases I have seen where the expressions actually look real. The 4 combined poses are fantastic.
February 24th, 2010 at 2:03 am
You’re doing a very nice job. I liked the tip about opening the jaw to check if the skin crosses the teeth. Thanks for sharing your workflow.
Is there any set of “standard” shape keys that a character should have?
Keep it up!
February 24th, 2010 at 2:16 am
Hey guise, are you thinking of using any mesh-deform stuff at all to push the expressions? There were a couple excellent examples in the original pixar paper.
February 24th, 2010 at 2:28 am
“Angela for president”, Looks great.
Will you have to do this for all the characters or just Sintel & the Shaman?
February 24th, 2010 at 3:32 am
Yay, Angela’s back! 😀 And the awesomeness exudes! XD
55 shapes are like more than the number of FACS action units, aren’t they?! Whatever, I love your work and the results so far! And Sintel has lovely dark pigmentation around her eyes. ^^ *swoon*
February 24th, 2010 at 4:19 am
the eyes wow, that’s about all I can manage to get out. this looks really good
February 24th, 2010 at 4:41 am
thanks for sharing the workflow
February 24th, 2010 at 4:43 am
Hey guys! I don’t normally comment on these things, I am more of a game engine guy my self so I don’t really know this…but when does the animation process start?
Happy blending! 🙂
Sunjay03
February 24th, 2010 at 5:01 am
Amazingg!!! 🙂
February 24th, 2010 at 5:15 am
Awesome stuff Angela!
I think the shader is very very good and also the facial shapes!
February 24th, 2010 at 5:33 am
I love your posts, Angela. Always so inspirational! I can only imagine how amazing your tutorials on the DVD will be!
By the way, is the wireframe that is shown in the renders the actual mesh you work with or does it have extra layers of subdivision applied to it?
@Victor: I agree that the major drawback of using morphs for facial animation is reuse, but remember the tool that Campbell wrote to transfer BlendShapes from one mesh to another of identical topology? He said that he wanted to extend it to work for meshes with different topology, but that did not have priority for Durian. That may have changed now that BlendShapes are such an integral part of the facial animation for this project.
This is how I imagine it could work, if incorporated into Campbell’s existing script:
1. Shrink-wrap Sintel’s head to fit onto the Shaman’s head. Ideally, you would want this process to be interactive so you could proportionally push the Sintel head points around on the surface of the Shaman head.
2. Since each point on the Shaman head now lies on a face of the Sintel head, it you could represent each Shaman head point as a distance-weighted sum of the corresponding Sintel face points. You would then be able to use these weights to apply BlendShape offsets from the Sintel head to the Shaman head. You may want to have an extra parameter to exaggerate or de-exaggerate each BlendMorph as you transfer it.
That’s pretty much it. You would of course keep tweaking the shrink-wrap process until you got a decent result for one of the BlendShapes, then Bingo! You have a way of instantly creating 55 BlendShapes for the Shaman head, ready to be tweaked!
February 24th, 2010 at 5:59 am
Seriously, beautiful work. Your Animators are gonna have fun! Your work heightens the entire project.
Pixar or Dreamworks are def in your future.
=]
February 24th, 2010 at 8:20 am
Top-Notch Angela!!!!! Your work is just awsome! I can really learn alot just of your images you post here on the Durian Blog!
I am working at the moment on a lowpoly Model Facerig, and already “cloned” some of your shapes! I really love it!
One question. Do you have problems with the bone controls for the deform values? I’ve added driver for my shapekeys and mapped them to my bones. When I say now local for the bonespace and move the Head(facial controls are parented to the headbone) of the girl, the shapes are driven too. In 2.49 I dind’t have that problem.
February 24th, 2010 at 10:44 am
Doing great Angela! I’m pretty impressed with the mesh and especially the pupil renders; those literally suck you in the Sintel character!
February 24th, 2010 at 11:52 am
Victor: Actually, you could always re-use animation. I’m not the animation expert neither the rigging one, but all the shapes are driven by a face rig. So there are little controllers on the side of the mouth which let the animator ‘drive’ the smile shape like if he was animating a smile with bones.
If we extrapolate this, all the characters will have a very similar face rig, and if they all have the same shapes as expression, the animator could always copy the animation from the face of the shaman, to let say, the face of Sintel. And the controller moving to create a smile, will still do a smile on Sintel 😉
This is probably not what we’ll do, since all the shots have different expression. But I do think it would be possible and easy to do! 🙂
But yeah, you indeed have to create facial shape for each characters, because hey, they have different personality and won’t smile the same way 🙂
February 24th, 2010 at 1:09 pm
Angela, your work is impressive, I’m almost speechless….
The shapes bring a whole load of sensation, they’re awesome.
But the absolutely extraordinary thing is that you felt the need to specify those are not final textures… I think eyes render itself is something simply exceding yet the best static renders you can see over cgtalk…
I really can’t imagine how you could push them further 😉
February 24th, 2010 at 1:16 pm
Oula, sintel is angry.
February 24th, 2010 at 2:06 pm
keep it up angela 🙂
February 24th, 2010 at 2:40 pm
Indeed, fantastic work Super Angela!!
February 24th, 2010 at 2:41 pm
I don’t like bald women. lol 😛
February 24th, 2010 at 2:57 pm
Can we get a render with the default expression and the raised eyebrow. As someone who’s been on the receiving end of that look too many times to count its a personal favourite. Love the work Angela and Team!
February 24th, 2010 at 3:18 pm
Wow! Very, very impressive, and your tips are great! Whenever I’ve tried facial shape keys I somehow always manage to tab out of edit mode at the wrong time and lose all my work, but this definitely gives me the impetus to get right back into it and learn it properly. Really inspirational stuff, thank you for sharing!
February 24th, 2010 at 3:47 pm
This looks nice 🙂
But if I may…
I think the movements are too local. (maybe they have to be local, for bether combination?) For example: By the nasalSneer, the mouth should also being “pulled” up etc…
Anyway, great work!
February 24th, 2010 at 5:16 pm
Great work!!
You model very nice the expressions on her, it follows well the topology as well.
IMPORTANT Question.
-Are you guys have also on the facial rigging workflow a way to make corrective shapes for the shapes..?..
which means that let say that you have an smile shape and an “o” shape. They work great individually, but then when you activate both at the same time they look wrong, here is where a “corrective shape for shapes” TOOL will be useful, because you can sculpt/model the correct shape when two or more shapes are active at any given time. ESSENTIAL FOR ANY HIGH END FEATURE QUALITY FACIAL RIG.
Character like gollum for example use this technique.
Thanks
February 24th, 2010 at 5:49 pm
Hey mom,
I agree with everyone, amazing work! I can’t wait to see more. Love you!
February 24th, 2010 at 5:51 pm
This shapes are amazing!
Keep up your great work.
February 24th, 2010 at 6:09 pm
Looking good! I love the moving pupils and the showing teeth.
And so many shapes, when it’s their turn, the animators can go wild!
February 24th, 2010 at 6:16 pm
Excellent work Angela! All of shots looks good and perfect. WIll be nice to see these expression in the final short. Personally I think the middle image showing dilated pupils is a bit too exaggerate even if it made like this to be seen from far away. But is my only crit. Other than that I think it is a flawless design and excellent work!
Congratulations! 😀
February 24th, 2010 at 7:27 pm
Darn… I wish I knew shape keys in 2.49 FINALLY! So I’ll just figure them out in 2.5 if this works better 🙂
Anyway: looks great, feels great, and your reel, Angela, gives us all the perception that Sintel is going to be incredibly good!
February 24th, 2010 at 7:37 pm
wow, that is awesome, Angela!
so, Shape Keys are great in 2.5… cool, i’ll learn them in Alpha1 instead of 2.49b 😀
GO SINTEL TEAM!!! 😀
February 24th, 2010 at 8:08 pm
NICE. I can’t figure the stupid thing out in 2.49, so I hope it’s easier in 2.5.
February 24th, 2010 at 8:46 pm
@beat25: That’s the animator’s job.
February 24th, 2010 at 8:48 pm
She’s gorgeous!
February 24th, 2010 at 9:52 pm
guys, you know that when pupil close the eyelids also close a bit 😀
Awesome work so far, it seems that you have a lot of progress behind our backs 😀
February 24th, 2010 at 9:54 pm
sorry for having to post again… can you do something like this next time 🙂
http://data68.sevenload.com/slcom/wg/xg/losjjkc/wjrjngltpmjd.jpg~/Shrek-Cat.jpg
…just kidding
February 24th, 2010 at 9:57 pm
When I saw those eyes I almost fell over in my chair of excitement. Super-duper-awesome job Angela!!!
February 24th, 2010 at 11:21 pm
Angela!! I want a tutorial from you!!
February 25th, 2010 at 1:46 am
(sorry, 2nd post, but i forgot something)
the eyes are SO awesome 😀
February 25th, 2010 at 2:03 am
Nice!
Only the mouth down looks a little odd, but I guess it is needed for easily readable expressions…and you said that it should normally be used only at 60%…
February 25th, 2010 at 6:43 am
Angela, I just wanted to pop in to say that I hope you’re also keeping the neck in mind some of these expressions…especially for cornerOut.L; a tendon in the neck stands out when you pull the corner of your mouth back like that. It’s not very visible, but it’s there.
With that aside…love the eyes and I can’t wait to see the final product! 😀
February 25th, 2010 at 7:07 am
@El’endia Starman: that would be the platysma muscle. While this muscle does play a part in widening the lower mouth, it isn’t usually noticeably activated except in exagerrated expressions or ones of extreme emotion. The same cornerOut action can be created purely by the risorius muscle in the face.
Based on this, I would propose a seperate facial shape for the platysma muscle by itself for use in aforementioned extreme situations.
February 25th, 2010 at 7:10 am
As a rule of thumb, use them in Wallace and Gromit shaped mouths. :p
February 25th, 2010 at 8:43 am
Mmmmm… What a beautiful brown eye’s! ;-D
February 25th, 2010 at 9:06 am
Those expressions are awesome 🙂
Thanks for sharing these images and the intro to your workflow.
It helps me a lot with my understanding of shape key and how to use them more effectively.
February 25th, 2010 at 9:50 am
I think everyone in the blender team neads to make a tutorial depending on their roles.
even Ton will need to make a tut as being a producer an all.
we demand it!!!
February 25th, 2010 at 11:33 am
Good Work Angela:)
I have a question. Recently, using 2.49b(not 2.5), I had serious problems mixing shape keys for emotions with lipsync(composed of more shape keys and animated bones). I’m assuming Sintel will have lipsync so, is the workflow of the facial animation ready for adding the lipsync? In my experience I had to convert back all the shape keys of emotions into facial rig poses (losing a lot of time and effort).
February 25th, 2010 at 7:08 pm
Great work!but I missed buck bunny.it was a wonderful project.
February 25th, 2010 at 7:19 pm
Wonderful Angela, absolutely wonderful.
take care
February 25th, 2010 at 9:28 pm
Facial animation is just huge and vastly important to any project, it would be great to see some focused tutorials/tips/dirty tricks on the Durian DVD concerning this. Somebody grab a camera and shoot over Angela’s shoulder all day long 🙂
February 26th, 2010 at 12:55 am
that would be worth alot i presume. this in itself to me is a valuble resource. if i am to understand blender even more, id like to see more experience users work on it for a prefessional production project.
February 26th, 2010 at 3:38 pm
Thats right people come from all works of life, we all poses different persona’s like a DURIAN it is hard spiky and a dark color but when you open it up-which a durian is a fruit …the inside is the complete opposite.
February 26th, 2010 at 9:15 pm
I’ve got a question about the choice of shape keys — specifically the mouthTwist key. I saw this key in the sample Guardians file posted earlier and again in the picture here, and it strikes me as an unusual choice.
I’m not a facial rigging expert or anything, but I’ve read Jason Osipa’s book, and the set of keys he proposes are roughly analogue to ones you’ve shown, except for mouthTwist. And furthermore it’s not a shape I’ve seen in reality, nor is it an expression I can make easily make with my own mouth.
So, I was wondering if there was a animator or rigger there who could speak on how useful this key is, or describe situations in which it is used? (And, if it’s not used, then perhaps you could save yourself some time by not modeling the key on other characters, or replace it with a more vital shape.)
February 27th, 2010 at 12:55 pm
this is kind of off topic but didn`t know where to put it
someone had an idea of rendering durian stereoscopicaly
would be cool 🙂 no need for 3D glasses and if you`re used to it, your eyes don`t hurt or anything….you just watch it like any other stereoscopical animation
and again, sorry for the off topic 😛
February 27th, 2010 at 5:51 pm
Thanks for all your encouraging comments! 🙂
@ Dalai Felinto: the pupils are all raw renders, no post-pro.
@ Grem: difficult to say, as the set of standard shape keys will vary from production to production. Time/quality, character style, character importance, whether they have a speaking role, etc, are all factors considered. For example, while I worked on a feature animated film 2 years in production, characters with speaking roles were required as much as 100+ split shapes. One of the books I highly recommend for practical 3d animation application is “Stop Staring: Facial Modeling and Animation Done Right” by Jason Osipa. He describes a simple setup to a more complex setup. Hope this helps!
@ John Fraser: Mesh-deform is not being implemented for facial expression.
@ awratten: Sintel, our main character received “the works” with 55 shapes. The Shaman, who is a secondary character but also has a speaking role, will receive just about the same attention. Secondary but non-speaking Dragon Baby’s facial expressions are now completed, receiving 22 shapes. The plan for background characters are simple facial rigs.
@ Sunjay: animation process has been ongoing from the start of the project, along with development of animation tools in 2.5.
@ Rudiger: The wireframe shown is the actual mesh I work on. Subsurf modifier applied, subdivision view set to 2, and optimal display turned on.
@ Andreas(ndee): Nathan sets up the rig, bone controls, and drivers. I’ll look into the issue you describe.
@ VicenteC: Animators will key the facial controls that drive the shapes, and not animate the shapes directly.
@ jcl: mouthTwistL/R shapes are meant to be mixed with mouthL/R, allowing the animator to tilt/twist the mouth as they move the mouth to the side. Of course, to simplify one can always combine both actions in one shape. However, it was decided on this production to keep them separate. This separation had also been implemented and favored by animators on some of the other productions I’ve worked on.
February 28th, 2010 at 1:21 am
thanks angela for all the replies.
i dont think orange project was this detailed. thanks to Ton i assume for connecting project – community. good luck.
i wonder how long left untill the completion of the project?
February 28th, 2010 at 10:58 am
hey thanks for your answer angela! And thanks also for forwarding my question to nathan!
Greetings
February 28th, 2010 at 6:39 pm
Love them, they look very natural 🙂
March 1st, 2010 at 1:19 am
wonder where minuits of friday night meeting went?
March 1st, 2010 at 2:49 pm
@Andreas(ndee):
The local space drivers not working is a bug. I believe it has been fixed in a recent commit, though. So if you wait for some new builds, it should work now.
In our rigs I’m driving (almost) everything with “Single Property” variables rather than “Transform Channel” variables, so I never ran into the bug. With single property, you can give it the rna path for any piece of data, including transform properties. You could also do this to work around the bug if you need to. Although this won’t account for the effects of constraints on local transforms.
March 1st, 2010 at 4:50 pm
thank you for the information nathan!!! I will try it! 🙂
March 1st, 2010 at 8:29 pm
hi, amazing work!, i wanna know if the pupil is controled by a automatic model. I sent a paper about it in a past comment. Anyways below a video showing realtime rendering using this photorealistic model.
http://www.vimeo.com/3545229
Paper info and source code:
http://vitorpamplona.com/wiki/Photorealistic%20Models%20for%20Pupil%20Light%20Reflex%20and%20Iridal%20Pattern%20Deformation
thanks
March 1st, 2010 at 8:34 pm
WHERE’S THE BLEND.
March 2nd, 2010 at 11:47 am
would it be possible to see the blend file ?
March 2nd, 2010 at 5:13 pm
Angela: Very informative, thanks! I’m looking forward to seeing the keys in action. 🙂
March 4th, 2010 at 1:04 am
Hey: Angela, totally unrelated, but I saw in the durian twitter stream that you solved a problem with textures seams by using NODES. How is this done, exactly?
March 4th, 2010 at 1:16 am
take a look at some more of the rigginig control for Sintel.
http://www.youtube.com/user/BlenderFoundation#p/u/0/V63VDd8DFCQ
http://www.youtube.com/user/BlenderFoundation#p/u/1/o3LFZJjfRI8
March 4th, 2010 at 6:15 am
Sintel looked mixed, between Chinese and German. Bruce Lee is also mixed: 75% Chinese and 25% German.
March 4th, 2010 at 7:07 am
NO! Sintel go see a doctor you eyes are yellow you have Jaundice!
March 4th, 2010 at 1:05 pm
@John Fraser – Basically I posted the svn commit because Angela and I were tag team solving some problems. There are multiple nodes that make up the skin material and I solved seam issues on each and every node. I just didn’t make that clear. I don’t often read those commits so I usually just type enough to refresh my memory.
That being said, if you wanted to solve them with nodes it is possible using multiple uv sets, baking and so on but it’s more trouble than it’s worth. Painting in blender paints across multiple seams on multiple textures on the model itself so it’s getting simpler to remove seams in general.
March 4th, 2010 at 4:34 pm
Ben: Oh, excellent. Thanks for replying.