“Wing Simulation” or “Gaaaaaah…”
on July 5th, 2010, by nathanOn Big Buck Bunny we did the wings of both the flying squirrel and the bird with hand animation. In the case of the bird, that went really well. But for the flying squirrel it imposed a lot of limitations on us and made things just generally a huge pain for both animation and rigging.
So for Durian I decided to use cloth simulation for the membrane of the dragon wings. This has led us down a long, winding, extremely annoying road. I thought it was going to be easy. Little did I know…
—-
The early tests of the dragon wings went quite well, as you can see in the video below. The wings were simulated in a separate mesh, and then mesh-deform was used to transfer those deformations to the dragon mesh.
This worked okay, although there were intersection issues within the wing membrane itself since the geometry was double-layered:
Here is a crappy diagram of a cross-section of the arm/wing geometry, and what happens when the wing membrane gets deformed:
There were also intersections of the wing membrane with the body.
For the final setup I added a body collision object so that the wings wouldn’t intersect with the body, and I also added a displacement modifier to “puff” the wings out to resolve membrane self-intersections:
This sometimes looks weird, but it seems we’re able to get away with it in most shots.
We also used an awesome new feature of Blender’s cloth sim that lets you specify a shape key as the “rest” shape of the cloth. This, for example, lets me make the wing membrane taut (in the initial tests above it was too loose) even in the default pose of the model simply by making a shape key where the membrane is shrunk and assigning that as the rest shape. It also helped me tweak other simulation issues, where the wing membrane would sometimes bend in ways that looked weird.
Here are some silly tests of the feature. The first shows a cube whose base shape is quite small, but I’ve set its rest shape to be quite large. So when the simulation begins, it explodes.
The second shows the opposite. The base shape is large, but the rest shape is small. This causes the cloth to shrink-wrap over the monkey head when the simulation starts.
—-
After all of this I figured it would be clear sailing! But not so. Doing simulations in the actual scenes turned out to be a hair-pulling exercise.
First we had to make wing sim work with linked models. Fortunately this turned out not to be quite so bad. But we had to set things up with Python. You can only modify linked properties through python, and even then they don’t get saved to disk and will revert upon reloading the file. So what Campbell did was he created some scripts that run every time the blend file is loaded that set the appropriate properties to the appropriate values. It works like a charm, although it’s quite production-specific. At this point Campbell has made scripts that even go through and bake all of the wing sim in a batch process. It’s pretty sweet. But, again, very techie and production-specific.
It would be nice to find a more general and user-friendly solution to all of this in the future.
Anyway, once we got that working our problems weren’t over. Managing collisions for cloth sim is extremely tedious, and requires making sure that there are no intersections at the beginning of the simulation, and that no forced intersections happen during the simulation.
A simulation can easily start with intersections. We have very basic rigging on the wings that just gets them in roughly the right position as a starting point for the simulation. Often this basic rig causes the wings to intersect with the body or themselves depending on the dragon’s starting pose.
To combat this we’ve been going into every animation file with the dragon and–before the start of the animation–creating a 30-ish frame transition between the default pose (guaranteed no intersections) and the start pose of the animation. And then we include those frames in the simulation. This is pains-taking and annoying, and it makes our simulations take longer. But it usually gets the job done.
The other problem that can happen is forced intersections. This can happen a lot, since animators cheat a lot. And animators should be able to cheat. But unfortunately they can’t in this case. What can happen is, say, the rigid hand-animated parts of the wings might penetrate the dragon’s body, and the cloth sim has no way to deal with the properly. It literally forces the wing membrane to intersect.
(Yeah, this is contrived, but it shows the problem more clearly than the actual production examples.)
And once that intersection happens, it gets stuck. It can’t get free without quite a bit of force. It “snags” on the dragon.
To combat this problem we do the simulations once, see if anything snags, and then go try to fix the forced intersections in the animation. This is also really pains-taking and annoying.
And part of the problem with these two issues is that no matter how correct or accurate the collision detection in Blender is, these two issues will persist.
A shout-out to the Blender devs: if anyone is interested, there is a cool paper from Pixar and a related paper from Stanford that address precisely these issues. If we could get that sort of functionality into Blender, it would go a huge way towards making Blender cloth really production-ready. Far more so than having more “correct” or “accurate” collision detection in the traditional sense. Forced intersections and starting intersections are a fact of life in production, and really need to be addressed directly. I will be forever indebted to anyone that takes this on.
But even with all of that taken care of we still have lots of problems. Like the wings disappearing or exploding.
And in those cases I’ve just been running the exact same simulation over and over again. It gives different results somehow. And I just wait until I get one that doesn’t explode or disappear. It would be cool to get this fixed as well. 😉
In the end, I think it’s been worth it. We’ve identified some important areas of cloth sim that need to be improved, and this will be helpful to developers in the future. And with a bit of hair-pulling we’ve managed to get some pretty good results:
—Nathan
July 5th, 2010 at 8:03 pm
No wonder Joe Eagar was hired to help with cloth!
July 5th, 2010 at 8:20 pm
Yay! Finally a cloth post! 😀 Thanks for the in-depth insight, Nathan! Thanks to all the team for going through all that so we one day won’t also have to!
And finally a dragon video! Awesome! 😀 Hoping that was a sped-up version of the dragon snapping.
July 5th, 2010 at 8:44 pm
arf! arf! Cloth bending sure is fun, right 😀 –sorta like airbending, water bending and fire bending.. oh ooh!! and earth bending too 😀
oh yeah.. fabric bending can go places >.>
-on a more serious note : Congrats guys!
July 5th, 2010 at 9:12 pm
Seems to be a hell of a job, but I have full confidence that it’s gonna look awesome on the silver screen!
July 5th, 2010 at 9:15 pm
@Vending:
Coming soon: The Last Dragonbender
Starring: Nathan Vegdahl 😀 😀 😀
July 5th, 2010 at 9:21 pm
Still, compared to Franky’s wings.. visually cloth simulation > hand animation. And I’m quite willing to sacrifice some of your hair for the result. 🙂
July 5th, 2010 at 9:32 pm
Then I hope he washed it! 😉
July 5th, 2010 at 9:41 pm
Well done guys! This all looks challenging im still wondering how the hell to model reali complicated models than render them. Bt u guys rock whoop whoop!
July 5th, 2010 at 9:49 pm
“But even with all of that taken care of we still have lots of problems. Like the wings disappearing or exploding.”
It sounds like you guys have been having a lot of fun over there.
It looks well worth it.
July 5th, 2010 at 9:54 pm
Nevertheless, the dragon itself looks awful!!
Just like a bat!! A lordly dragon looks different.
July 5th, 2010 at 10:26 pm
Interesting and entertaining post I must say. I always thought “pros” like you don’t cheat around, but you guys also got tons of problems which seem insurmountable. It makes me feel a bit better 🙂
@JuppieOnTheRocks: Yes, I’d also took another style for the dragon, also I would want Sintel to be more realistic, not this slighty comic/anime style. But overall it isn’t bad in my opinion.
What would be interensting too would be a blog post about smoke-sims *hint* 😉
July 5th, 2010 at 10:48 pm
wow!!
thanks for this tour on the produccion, really usefull info!
i hope that cloth sim would be improve by some one too, also would be nice to have tearing cloth.
great work
July 5th, 2010 at 11:00 pm
wow excelent. ive been waiting for more developement in this field.
July 5th, 2010 at 11:22 pm
The final reslut is awesome!
How much time was schedule for this part(the wing simulations)?
You must have went far beyond what you had in mind at the first place. Strong of you to getting trough such a issue. I’m impressed.
July 6th, 2010 at 12:04 am
Wow Nathan!
How on earth did you manage to blog that, AND smile to Beorn’s camera at the same time!?
You are the greatest!
July 6th, 2010 at 12:54 am
Thanks for documenting this so well. So many of us have had troubles with cloth its really nice to see you working hard on the system.
Soooo: could you make a list of tips for us? eg things to avoid, ways around certain problems (eg your pre-roll for the sim start) etc.
July 6th, 2010 at 3:34 am
Good to see you all mentioning the Pixar untangling paper, I was reading the problems you were having and immediately thought “Huh, sounds like no-one’s implemented Pixar’s flypapering/detangling methods yet”.
Here’s hoping someone gets it coded for everyone- I remember a classmate in college going through all the frame-by-frame fixing you’re dealing with – it was a staple chore of clothsims prior to Pixar’s paper.
(Hey, if you have any idea how they did the drag-the-mesh rig towards the end of http://graphics.pixar.com/library/HarmonicCoordinatesB/HarmonicCoordinates.divx – at 4:32 – that’d be awesome to know too! Never seen a rig mesh where you could just grab any part and it would move-as-if-jointed before. Not the actual harmonic coordinates thing, but where he says “I crudely articulated the cage” and then makes it behave better than ANY animation rig I’ve ever worked with)
July 6th, 2010 at 3:51 am
Oh, thank you Nathan that you pulled that out! I didn’t want to bother coders for a ‘feature requests’ when i used cloth sim once.. 😛
July 6th, 2010 at 4:57 am
I don’t want to trouble you but… I guess your experiencing the problem already. I have this problems since months ago. I was trying to simulate a thick quilt draped to a person sleeping. I thought it would be quite straightforward, just apply cloth simulation. How hard could it gets? … It was near impossible!
I made tests first with a subdivided plane that I extruded to add thickness and drape it over a sphere… and all test went horribly wrong. There are intersections happening all over, and if I turn on self collision it explodes into a pillow form and then rest to a shapeless mess. I think I made a request at thinker code about simulating thick fabrics for situations like this.
I wonder if softbodies will work better but I get tired of experimenting and deemed the venture a failure.
July 6th, 2010 at 5:15 am
Interesting article, light on the technical details (though I’m looking into the two papers linked for my mathematical fix -laugh-) but showing some interesting improvements on the basics.
The rest shape idea is pretty cool and I will be looking into how it can be used in other ways. I will be most interested in seeing how the cloth performs visually in the movie. The videos here are somewhat minimal on that (for good reason), so seeing how the tweaks and hacks worked out in the end will be nice.
The reason I mention this is that I’ve seen the use of “cloth as a membrane” simulations look pretty sub-par before (in non-Blender animations), so a decent result here would be good to validate the method.
July 6th, 2010 at 7:14 am
this time I have the dragon project ..
thanks for Confidently
July 6th, 2010 at 9:06 am
Oh my, this is like a dejavu. I can relate to each and every point you made here, especially the hair pulling.
I had hoped that the blender 2.5 implementation is more usable than in 2.49, but it seems you have to fight the same issues, although the shape key definition is a col thing to have.
I have gone through the exact same iterations and solutions in my current project (python scripting substituted with manual work 🙂 ) and your post is a good confirmation, that there doesn’t seem to be a better way.
Any way the result you produced was worth the effort I would say.
And I wonder how many your hair you left on your scalp for the (I guess) similar tedious work with the hair simulation ….
July 6th, 2010 at 12:32 pm
If I’m reading it correctly, the proposed algorithm requires quite robust computation of mesh self-intersection. If I had the time, this is where I would start. With robust implementations of all the mesh intersection requirements, the rest of the algorithm looks straightforward (but time-consuming, of course).
July 6th, 2010 at 1:04 pm
See, that’s really the problem with these open movies – problems are found, but not fixed! 🙁
July 6th, 2010 at 2:20 pm
@maskotti: they do find and fix *many* problems, for bigger ones as this one, the time doesn’t allow to fix them properly, but before these open movies the problems were not found even!
July 6th, 2010 at 2:58 pm
@maskotti:
What notmaskotti said.
A lot of issues have been addressed in Durian. And especially we’ve gotten 2.5 pretty much back up to 2.4x levels, which was a huge undertaking. You have to think of 2.5 itself as being one of the “features” of Durian in this case.
But yes, there is still a long way to go before cloth simulation (and I dare say simulation in general) in Blender is really production ready. I would not want to use cloth/hair sim in its current state on another production.
But we’ve managed to identify a lot of the issues that need to be resolved, and we have demonstrated that they need to be resolved, and that’s pretty huge. I am confident this issues will be resolved in due time, at which point we’ll have pretty kickin’ cloth simulation.
July 6th, 2010 at 3:03 pm
Nathan, every Blender user in the world thinking of using cloth, even users who don’t know it yet, appreciate the pain you’ve gone through so that somehow, some day, we won’t have to! Seriously, thank you.
I’d find it very interesting to hear a similar level of detail on how you all got on with controlling (and/or rendering) Sintel’s hair, as I’m sure someone has mentioned issues with that, and the Pixar reference above reminded me of their Volumetric Hair paper which I’d read with great interest last year.
July 6th, 2010 at 3:31 pm
where’s Joe Eagar ?, working hard for cloth?
July 6th, 2010 at 3:34 pm
I would have used cages, and shape keying, with some cloth sim blending…..but that is a workaround and hardly the point…thanks for sharing.
July 6th, 2010 at 4:14 pm
@ dd:
yes, I second that, where is Joe? Is he still at the institute working on particle/cloth? Or when he left, if so?
July 6th, 2010 at 4:25 pm
Joe would only be here for a few months, he’s back home again!
July 6th, 2010 at 4:59 pm
Oh, bit of a sad news, I think his help to the team has been very important past months, but………. this should mean he can come back to coding B-Mesh, right? 😉
Yu-huuuu!
July 6th, 2010 at 5:55 pm
Wow! As a blender newbie I am amazed at the amount of work that has to go into something like this!
Now that I have revealed that about myself I have a question someone will hopefully be kind enough to answer. I have skimmed the whitepapers by pixar/stanford and am imagining a totally different solution by implementing a cloth “thickness,” “fold” and “crease” algorithms – as I write this I am looking at the flabby part of skin between thumb and index finger moving my fingers around. Basically the idea would not be to eliminate the problem but improve control over it.
Would alternative workarounds like this be welcome?
July 6th, 2010 at 6:19 pm
Blender is slowly tipping the scales *hint hint* in 3d land.
July 6th, 2010 at 7:53 pm
Hey guys, I was so anxious about this post… cloth is one of my favorite features in Blender.
I’m pretty sure you all are having a lot of trouble on solving this new version issues, yet I know you can do it! =D
That’s it, keep up the good work and this awesome animation will be as cool as we all expect =]
Regards
July 6th, 2010 at 10:54 pm
Was doing the hair also as “Gaaaaaah…” as cloth? I hope it gets better eventually… This shows how Durian is not only a movie project, but also a way to test the software in a real production environment and iron out all the bugs.
July 6th, 2010 at 11:10 pm
WARNING
Silly comment follows, read at your own risk:
Something cool for cloth would be a force field that would be a mix of the Cloth, Solidify and Lattice modifier. Cloth Simulation algorithm would run only in a simple flat mesh and then the lattice would deform everything we put inside the “force field”, cloth, pockets, zips, etc accordantly 🙂
Please don’t complain you were warned 😛
July 7th, 2010 at 12:47 am
Should you add some wind facing the direction of the movement to simulate air resistance, since wind gonna fill the wing when moving quickly.
July 7th, 2010 at 2:20 am
I ran into a lot of these same issues when doing my own personal animation in 3ds max. But in max there were a lot of ways to control the simulation. It was still a pain in the ass.
July 7th, 2010 at 3:47 am
I think the bottom line is that cloth is hard: not all that surprising. Fortunately, it seems like all this mucking around has allowed the team to collect significant data on a number of ways the cloth sim can fail spectacularly :P. This ought to help the devs make the cloth sim fail less spectacularly :P.
I have to imagine simulation requires a lot of time, testing, and expert manpower to get right. I’m eager to see how the Blender devs can improve it in the future, because it looks very promising, even if it is a bit flaky in the real world.
July 7th, 2010 at 4:35 am
“To combat this we’ve been going into every animation file with the dragon and–before the start of the animation–creating a 30-ish frame transition between the default pose (guaranteed no intersections) and the start pose of the animation.”
I’ve see this technique on Star Wars (or some other major production) behind scenes XD. Is not too bad 😛
July 7th, 2010 at 10:21 am
Nathan,
thanks for the post, it has a lot of interesting stuff in it. I was wondering if the clothsim was designed to have a a “cloth intensity texture” and a “initial simulation force texture” to identify what parts of a mesh should be influence by the clothsim. How much problems would be reduced. You have one manageable mesh, the full mesh would be inside the clothsim. And parts can be influenced by animators.
You still need to solve tons of probs, but perhaps it will be more suitable for skins-like simulations.
Cheers!
July 7th, 2010 at 6:51 pm
good luck!
July 7th, 2010 at 8:26 pm
Nice article. I can remember facing the same self-intersecting wing problems half a year ago, when I’ve done the trailer for ZOTAC. Luckily I didn’t had to do as many dragon shots as you. 🙂
July 7th, 2010 at 8:52 pm
Couldn’t you only simulate a single side of the wing and then simulate the other one with the previous side only working for collision? It wouldn’t be so realistic, but from the camera you usually see only a side of the wing…
July 7th, 2010 at 10:53 pm
Ahh. For a minute I was expecting to hear the bad news:
“Soo… we decided to just scrap the ‘dragon with wings’ idea and go with a ‘land’ dragon. The cloth stuff can wait til next project.”
I’m glad you all decided to just pull through.
July 8th, 2010 at 2:14 am
solution: go film a real dragon and comp it in… :/
July 8th, 2010 at 7:40 am
The Kaboom and the Shink-wrapping could actually be used for animation effects. Haven’t thought of one practical purpose, yet, but the effect is there.
Good job on accidentally producing those effects!
July 8th, 2010 at 11:36 am
@jirms:
I think you may misunderstand the problem. The problem is forced intersections. Like, imagine a character wearing a shirt, and another character poking him in the chest. If the finger intersects with the chest even a tiny bit (and animators often want to do this to give a better sense of contact), it would force the shirt to intersect with either the chest, the finger, or both. These sorts of forced-intersections are important for the cloth system to be able to handle gracefully.
@Giovanni Lanfiuti:
We already only simulate a single layer. But we have to transfer the deformation of that single-layer simulation to the double-layer membrane of the model. And it’s that transferring process that introduced the subtle self-intersection.
If we simulated both layers then we would get all kinds of weird problems, since they would move independently.
July 8th, 2010 at 1:21 pm
I’ve been having some fun using the soft body instead of the cloth for creating some wings, the length value can keep it tight and you can also set some of the verts to bones, to gently persuade the membrane to go the direction you want… I am a complete noob to this though, so there probably are many reasons why you need to use the cloth instead, I’ll see if I can come up with some good examples.
July 8th, 2010 at 2:39 pm
I hope there will be a decent tutorial on all this exciting stuff!
July 8th, 2010 at 3:57 pm
Had a quick play with the soft body for wings during me dinner break, took about 30 mins: http://vimeo.com/13179083
Kopi, I’d say the best way to learn the cloth and soft body is set up a test rig, and change the values one at a time, its a shame that the soft body doesn’t have a preset like the cloth… been working on my jiggle physics 😀
July 8th, 2010 at 5:31 pm
Thank you Nathan! Sounds like my plan of building an AI that uses weight painting to control cloths was a fairly dumb idea. From a programming standpoint it sounded fun though.
If I understand the problem correctly now we are talking about polymorphism / decoupling Object Oriented concept issues. If you want to see programmers cry tell them this should be easy …
July 8th, 2010 at 5:55 pm
hey. i found some secrets. this is AWWEESSSSSOOMMMMMA’H http://www.licuadorastudio.com/blog/wp-content/uploads/2010/07/sintel_5.3b_0129_compara.jpg
http://www.licuadorastudio.com/blog/wp-content/uploads/2010/07/sintel_5.3a_0076_compara.jpg
post MOAR please
July 8th, 2010 at 6:36 pm
SECRETS YEEEAAAAAHHHHHHH!!!!!!!!
😀
July 9th, 2010 at 12:14 am
off the top of my head, this would be a nice use-case scenario for the use of bones with dynamic constraints. One could rig the wing membranes with bones, animate it, bake the animation, then simply tweak the bad parts. Might be something more feasible for the mechanical-minded among us. If anyone is interested in this concept I can make up a mock up.
July 9th, 2010 at 4:33 pm
hey
first thing to be said is that you all are doing a great job there in Amsterdam(n).
second thing is in the screenshots it appears to be some rain in Sintel… will we be able to animate rain easier than now? Because the reactor particle system is gone now in 2.5 I don’t really see how you guys do that.
greats cedal
July 13th, 2010 at 9:11 am
Tried using cloth, it was a real pain, any reason you diddn’t use soft bodies? it was quite easy to do this batwing, and diddnt have the intersection issues:
http://vimeo.com/13280878
August 17th, 2010 at 12:09 pm
Nice technique