Sintel Face Rig, The Beginning!
on February 15th, 2010, by nathanSo, the face rig for Sintel has pretty much been a big train wreck up until now. I started off trying to do a primarily bone-based face rig, since it would be easier to automate and adapt to other characters. I made some progress with this, and it’s still a direction I’d like to pursue after Durian for background characters in future works. Here’s where I managed to get things:
Unfortunately, the R&D for such a system was just taking too much time, and was progressing way too slowly. The controls were very under-constrained (not in the good kind of way) and way too finicky. And the weight-painting was positively painstaking. And contrary to what I might wish, we have to finish this movie within the time allowed by our budget.
So in the end we scrapped that system in favor of the tried-and-true shape-key method. Angela quickly blocked in some of the basic shapes, and I hooked them up to the rig. Here are the results so far:
(Keep in mind that there are still more shape keys to be added, and the ones that are there are not final yet.)
I’m not at all sure about this control interface. I’m rather averse to the slider-and-box gui’s of most face rigs, but I’m also finding it very difficult to come up with an alternative that isn’t a horribly cluttered mess. There’s so much to control on a face. So we’ll see.
Unfortunately, whatever I’ve got in place when we start facial animation in earnest is what we’ll be stuck with, so the control scheme may end up being somewhat less-than-awesome. But such is the way of production rigging.
You’ll also note that there are occasionally some nasty artifacts in the deformations around the eyelids (particularly noticeable in the above video when I’m blending in and out of the last face pose). Rest assured, this will be fixed before the movie comes out. 😉
–Nathan
February 15th, 2010 at 9:12 pm
wonderful 1st
February 15th, 2010 at 9:25 pm
The last Shape looks horrible, like a Deamon 😀
February 15th, 2010 at 9:40 pm
truth are very slow
por fabor mas rapides
February 15th, 2010 at 9:54 pm
Thanks a lot for the update. I hope updates will be a bit more frequent than just once a week in the future 🙂
Keep up the good work!
February 15th, 2010 at 10:01 pm
I find myself pulling faces along with the video, nice work Nathan 🙂
February 15th, 2010 at 10:06 pm
Keep up Blending.
Thanks.
February 15th, 2010 at 10:06 pm
WOW!! Fantastic rig!!
February 15th, 2010 at 10:11 pm
sweet ;]
Keep it up! I’m counting on you! (epic story in 10min? this will be hard)
February 15th, 2010 at 10:14 pm
So Nathan,
I guess this is a bit more serious than BBB, you’re not rigging a big bunny that will be jumping rope…lol. It looks like you are using bone driven shape keys. What I would like to know is: do you think this is a better way to go with facial rigging? I’d like to know how facial rigging with bones became a mess, too many bones and weight painting/vertex groups sharing bones??
It seems you have achieved better results with shape keys and they can be changed on the vertex level to suit your need, which is not possible with a facial rig using bones for deformation….
C’mon, share your thoughts and insight…
Randy
February 15th, 2010 at 10:17 pm
Wow…. cant wait for the final 😀
February 15th, 2010 at 10:19 pm
It’s a shame about the first rig. That one seemed to have awesome potential. For the second one, will you be saving a lot of actions and just layering them for animation? Seems logical, but I”m far from an expert, and I’d love to hear what you’re going to do.
February 15th, 2010 at 10:24 pm
Finally a blog post!!! awesome stuff. So cool to see the other side of movie making 😀
February 15th, 2010 at 10:53 pm
It’s a bit of a pity that time constraints ultimately forced the shapekeys approach to prevail in the end 🙁
I’ve personally avoided shapekeys at all costs with my own rigs so far, since creating (and recreating) the shapes rather often wasn’t exactly a viable option IMO, and/or the risks of being constrained to Proog/Emo style emotive ranges were ever present with this approach.
On the other hand, we get to see Angela’s shapekey-fu in action! (and at least someone stress-tests the shapekeys)
—
Regarding the control interfaces:
I don’t like box-and-slider interfaces either 🙂
Some interesting approaches/experiments I’ve seen have involved less visible controls, with some of those used to deform geometry being embedded in/hidden within the geometry but still being selectable. Although these have the disadvantage of requiring a bit more hit+miss when getting used to the rig, particularly if the controls are small and spaced far apart, it’d be interesting to experiment with this approach a bit (though once again, probably time constraints prevail).
February 15th, 2010 at 10:58 pm
I had a hearty laugh at the first video when you made her eyelids close upwards. I was instantly thinking of Wilhelm screams and horror movies.
February 16th, 2010 at 12:00 am
In 2.5, is it possible to have the character linked and be able to animate the shapekeys in the .Blend file dedicated to the animation?
I mean, is this possible directly animating the shapekeys’ values or through bone driven shapekeys?
February 16th, 2010 at 12:02 am
Looks amazing! We use shape-key based animation on a lot of our commercial work in Maya and the animators here seem to like it a lot. I know as a modeler it can be fun to use this approach as it gives a lot more customized character expressions from project to project. Keep up the good rigging!
February 16th, 2010 at 12:08 am
Yeah, the lower-lid blink was just creepy.
On another note, keep up the good work, guys.
More frequent updates would be well-appreciated.
February 16th, 2010 at 12:11 am
Looks like you have bones corresponding to the mayor pulling muscles around the mouth more or less (http://www.cg.tuwien.ac.at/hostings/cescg/CESCG-2001/MSimunek/pictures/image002.gif) What made you chose these directions particularly?
Also seconds Revolt_Randy’s request for more details 🙂
February 16th, 2010 at 12:59 am
Bravo!
February 16th, 2010 at 1:03 am
Hi Nathan, those both look great, I think the way the bones are set up in the second video looks quite intuitive, like really manipulating facial muscles. I am setting up a face at the moment and also tried rigging it with deforming bones but have now replaced with almose entirely shapekeys (exept for the jaw). Am thinking of possibly adding some deforming bones for smaller detail to manipulate the basic shapes later but don’t know yet if this will work. Are you still using a jaw bone in the second video, or is it all shapes? Anyways, great work, thanks for sharing.
February 16th, 2010 at 1:30 am
Lol @ 1:20 And great job Nathan!
February 16th, 2010 at 1:39 am
I guess it’s somewhat hard to do things, basically only done by muscles with bones. Of course, bones have a different context in that case but the behaviour is quite the same. fixed bones and joints which can be restricted in degrees of freedom… Something based on muscles you can put over bones would be nice… Shapekeys can simulate muscle-stuff of course but that’s not quite the same as muscles…
Too bad you had to drop the old setup… But the new one looks great too 🙂
Will you go on perfecting the bone-based rig after the film maybe? It would be useful to have one basically preset for the next film already, just needing minor changes to fit the new characters…
February 16th, 2010 at 1:40 am
Nathan: Look at Malefico works. It’s the same?
http://mercator.licuadorastudio.com/?p=302
February 16th, 2010 at 1:54 am
Sintel with glasses looks nice 😛
Good job!!!!!! 😀
February 16th, 2010 at 3:35 am
Great Job! Looking for some crinkle on the nose during growls/snarls/Frowns 🙂
February 16th, 2010 at 4:56 am
good !!
🙂
February 16th, 2010 at 5:19 am
I wish there could be an easy way to make such nice and easy to use controls with a minimum of efforts on blender…
But I guess that will be for 2.52… :/
February 16th, 2010 at 5:28 am
good, but with the wire looks a little scrapy
February 16th, 2010 at 6:47 am
Excellent work Nathan!
February 16th, 2010 at 7:15 am
Great work Nathan and Angela. 😀
I personally thought shapekeys were the way to go for facial posing.
February 16th, 2010 at 8:32 am
One question! I’ve played around with the shape keys for some time now. Why isn’t it possible to undo when shape keys are enabled?
You have to be VERY carefull to not mess up your mesh. And if you do so it is not possible to undo.
Is this still work in progress?
Greetings from germany
February 16th, 2010 at 12:20 pm
This looks great, but will still need shape keys to make it look truly natural.
February 16th, 2010 at 1:29 pm
Love the first video’s effort, couldn’t really see a problem till the last couple of features when you raised the upper lip and the centre didn’t move properly with the bones. Shame bones can’t be set up to work like muscles.
Keep up the brilliant work guys!
February 16th, 2010 at 4:41 pm
Why not do as with the guardians? Post a blend file for us to see. The work looks good, but who knows the community can help. I also do not like to use ShapeKeys.
Maybe that way, do not save some time, since this is the biggest problem.
February 16th, 2010 at 9:15 pm
Well, I’m coming from a Maya background, and have recently explored several face rigging solutions. Unfortunately none of these tools are available to you in blender, but looking into them may help you develop the best possible system for this project.
I’ve experiment with both the Face Machine (similar to your current setup) and Facial Animation Toolset (completely joint based). The Face Machine has a lot of flexibility, like your current setup. One thing you may want to look into developing is a system to save a library of poses, because animating without them will be nothing short of a nightmare. Also, you’ll probably want a way to scale the contribution of each pose, I know you’re trying to avoid sliders, but there is really no faster way when animating.
The Facial Animation Toolset alternatively is based completely on sliders. You loose some control, but what you gain in speed animating and in quality is well worth it. A lot of this is due to having actual motion cap data drive the expressions, so when you change expressions it doesn’t get there in a linear fashion. Really helps add to the believability. I’d take this system any day of the week over the Face Machine.
If you want to use this approach, if you have Maya available, you could try try exporting the facial animations for each pose from the Facial Animation tool set and bring them into Blender. You’ll then have to replicate the sliderUI in blender to drive the animations. Also, you’ll need a way to scale the animations and key any changes you want to make. Of course, that’s a ton of work and scripting you’d have to do, but if you’re shooting for exceptional results that are easy to animate, it’s one of your better options.
Alternatively, if you go with the pose system with the current controls, you could build a system that allows you to hand animate the face moving into the expression to avoid the linear unnatural movements that hurt the first video. I’d take a decent amount of hand animating, but once it’s in place you’ll get far better results. You may even be able to look at the mocap data from the Facial Animation system as reference.
Hope I was able to help. Just wanted to toss around some ideas.
February 17th, 2010 at 4:41 am
Hey Christopher Emerson..
I think Nathan don’t have any idea about Maya. They doesn’t seem to like how stuffs are done with other software. They go for their own ideas.. 🙁
I am stumbling on Bone based face rig for my project, yes its slow to setup, but truly an artist (animator) friendly it would be.
Link for Research on Facial animation for nathan.
http://research.animationsinstitut.de/44.0.html
February 17th, 2010 at 6:30 am
@Chris – This is a Blender project, not a Maya project. These open movie projects are intended to develop the functionality and usability of Blender, not to explore the use of other tools. Furthermore, all the tools which are used for the project (even the non-Blender ones) are open source, enabling the film itself to be open source.
February 17th, 2010 at 9:51 am
Duuh, I didn’t realise there was more to the post. The bone-controlled shape-key rig is absolutely phenomenal! Well done to you, Angela, and of course Blender!
February 17th, 2010 at 11:07 am
Hold on a second here, I never said anything about this being a Maya project. I only merely made the suggestion that the team “look” into the other facial setup systems available in other software and perhaps use some Mocap data (either directly or as a reference) to add more life-like, non-linear animations for each expression. Also to go with sliders so they can slide through these animated deformations, because it’s faster to animate in the long run and makes for more naturalistic facial movements.
The only reason I mention all this is because, well… I’m getting far better results. I’m not saying they should necessarily copy an existing tool, but rather study the strengths and weaknesses of existing methods to see what the next step should be.
They should be aiming for a solution that has non-linear deformations, but doesn’t take forever to animate with. The shape key method is a start, but it’s still going to be very time intensive with without a pose library, and lack a more naturalistic movement without animations leading up the the target pose.
February 17th, 2010 at 3:52 pm
Hi Chris, thanks for your comments.
We’re already planning on creating a facial pose library (blender has a built-in system for pose libraries).
Unfortunately we can’t export anything from Maya, as that would constitute using non-open-source software for the visuals of the movie. And one of the primary points of these projects is to provide the community with all the tools we used afterwards. 🙂
I’ve looked at both face machine and face robot as best I can without actually having access to either. At no point have we been aiming for such comprehensive systems as those, though. It would take even more R&D than the simpler system we were originally trying.
Regarding specific rigging features, probably the the most useful feature I’ve seen from Maya (wrt faces) is the ability to slide clusters along a curve. Seems like that would come in handy in a lot of scenarios.
I’ve seen some very good shape-key (blend-shape) based rigs. A good example being:
http://www.youtube.com/watch?v=VK2N3PvaFcs
And Gollum from LOTR is also mostly shape-key based, IIRC. As well as all the faces seen in Angela’s reel. So I’m not one to write off shape keys. In fact, I’m quite certain that short of doing some kind simulation, shape keys will always be a necessary component of any really good face rig. Even in the bone-based rig we were trying, the plan was to supplement it fairly heavily with shape keys.
So for me it’s not a matter of trying to get away from shape keys, but rather it’s exploring what mix of methods gets the best results and the most flexibility. Although what that means depends on requirements.
You may be right about sliders for controls. But I’m not ready to give up yet. In general, I prefer rig controls that feel like you’re directly manipulating the character. And I’d like to find a usable facial control scheme that fits that, if possible.
I try to limit sliders to things like ik/fk switching, where you’re managing the behavior or settings of the rig itself.
February 17th, 2010 at 8:34 pm
No .blend ? 🙁
February 17th, 2010 at 11:16 pm
Is there something to say about lipsync and Sintel or blender 2.5 ?
great work so far.
well work needs necessary time
greetings
Anton
February 18th, 2010 at 9:25 am
Nathan,
It would be great if you started a thread on blenderartists or something, with more information about what didn’t work and why. That way the community could help you, and offer relevant information. We could share ideas as a community and achieve something great together… you don’t have to figure everything out on your own, hundreds of people are eager to help and offer suggestions 🙂
February 18th, 2010 at 3:25 pm
Hi Gord,
Good idea. 🙂
I’ve started a thread here:
http://blenderartists.org/forum/showthread.php?t=179673
February 20th, 2010 at 8:51 pm
0:07 – return of the banana mouth