Render farm tour
on June 20th, 2010, by ideasman42Today Colin and myself recorded a tour of the renderfarm, this shows how its used by artists and the systems which render the frames.
Apologies for accidentally having areas of the video masked (would have been a spoiler).
– campbell
June 21st, 2010 at 12:36 pm
GIven that LuxRender is looking a being able to render on CPU/GPU/Network is there still a need for the internal render. Could Luxrender/Yafaray and the Internal render groups come out with one kick ass Renderer?
So are there going to be any out-take shots in the final release?
June 21st, 2010 at 12:53 pm
They have more than 16 motherboards, namely the 6 Dell servers, plus they also render on workstations in their ”spare time”. (a.k.a. artist not sitting at his desk)
So when everyone is to bed they have near 30 computers rattling and humming!
June 21st, 2010 at 5:53 pm
I second the suggestion of releasing the render management software for the community. I’d love to look into the code and maybe try to generalize the application for more common uses… 🙂
Best wishes!
June 21st, 2010 at 6:40 pm
It will get released, (I heard it myself in the video)
but it’s not a bad idea to wait a moment and first let them tweak/finetune everything in their environment, let them write a manual, organize all the data etc. etc.
Open source should not be ”grab some files, put them on the internet, find out everything by yourself, well…. enjoy… if you can…
”Haastige spoed is zelden goed” (Hurry is seldom good) do we say in Holland! 😉
June 21st, 2010 at 7:18 pm
I would really recommend using the netrender script that is now in 2.5. We decided to use our own scripts because they were already working and easier to fit into svn and file management we had in place here, switching to a very different system was a bit risky at the time we got the renderfarm. If you want something that works out of the box, netrender works great, it also has a web interface to manage things. I’ve tested it on the render nodes here and it worked, so it would definitely be good to use that on future projects.
June 21st, 2010 at 7:31 pm
Provided that SVN gets implemented in it, right?
June 21st, 2010 at 9:53 pm
@rickyblender, rendering so many images helps a lot with quickly finding render problems but we have enough on our hand with blenders rendering engine.
We’re aware the air conditioning is pretty make-shift :S
@ajedros, as fair as I know we dont have plans to do GPU rendering at the moment at least.
@horace, in retrospect we should probably have had a stronger fan at the base of the justa-cluster, while not ideal, pulling some draws out works for now.
@Jogai. I didnt use farmer joe, but one of the reasons we have our own farm is so we can quickly integrate new features – like svn integration, rendering AVI’s from EXR files, scanning metadata from EXR’s and showing average rendertimes etc.
@Homeworkbad, this video was edited in blender, for the blog people edit video with whatever they like, we’re not as strict about being totally opensource when it comes to making content for the blog.
@patrox, I have asbolutely no idea about how blender compares to other software. One would need to run side by side tests. However brecht has been working on optimizations, some of the scenes now have half overall render times :D. Future plans havnt been made yet AFAIK.
@The Fatsnacker, yes we use the render branch sometime after durian this branch will be merged into 2.5x.
@J. Only 5 of the studio workstations are powerful enough to be used as render nodes. animators have systems with 4gig of ram and only 2 core CPU’s so they are too slow compared to quad core i7’s. (so a total of 27 nodes max)
netrender/svn integration is possible but would be more appropriate if netrender was already well tested in production. testing netrender AND adding new features would loose too much of our time.
June 22nd, 2010 at 12:57 am
That render farm is hot! Love the little GUI interface you have.
June 22nd, 2010 at 1:24 pm
ideasman Campbel said:
“However brecht has been working on optimizations, some of the scenes now have double overall render times”.
ouh shit… probably another optimization will help to get back to half the render time, maybe even less?
😉
Or are here any Language issues ?
edit from a durian guy: thanks for noticing, fixed! 😀
June 22nd, 2010 at 1:37 pm
Not wanting to belittle your efforts but here is a video from the BBC news site (hope you guys can see it) which amongst the fluff about Toy Story 3 has a look at the Pixar Renderfarm, something to aspire to 😉
By the way, regarding the shower by the render farm, I think you guys may have misunderstood liquid cooling 😀
June 22nd, 2010 at 1:38 pm
D’Oh – forgot the link
http://news.bbc.co.uk/1/hi/entertainment_and_arts/10308227.stm
June 22nd, 2010 at 5:54 pm
@matt:
Holly crap!!! It sounds like surfing on a tsunami 😀 It’s even intimidating.
One question:
Quote: “we might donate render cycles to cool & high profile projects that go on with Blender”
This render cycles donations would have to be done in house, like one would have to send/take the movie files physically to the blender institute or it would be possible to submit the renderings jobs from the Internet?
June 23rd, 2010 at 12:53 am
Great video! Especially for us TDs out there. Colin, I’m curious as to what codec the AVIs are compressed with after the renders are done. Are you using ffmpeg? Also, are the scenes rendered in layers? If so, do you have some sort of script that composites the rendered sequences into an EXR? Maybe this is a question more for Tom.
June 23rd, 2010 at 2:06 am
Tom? You mean Ton! 😉
June 23rd, 2010 at 11:50 am
Very informative indeed. For still images I have really come to love Lux render. They have a network rendering integration that is very cool and easy out of the box. Not something for Sintel, that’s for sure, but anyone else should have a look.
Back on topic: keep up the cool work!
June 23rd, 2010 at 5:36 pm
@matt: good link about pixars renderfarm, it gave a perspective on the rendertime – a complex scene in Toy Story 3 takes like 7-14 hours to render… (can’t blender have a neon-sign for its renderfarm too?;) )
June 23rd, 2010 at 6:11 pm
Hey Ideasman,
I’m running DrQueue now on a Linux machine, it’s OK, functional, but buggy as you’ve mentioned before. They also have a few web interfaces, but Blender’s interface looks much more promising. Sweet. I’m looking forward to making use of your farm tools, on my system.
I have a question, how does the server-slave communication happen? I’m curious how the server communicates with each node. Is it through SSH command line calls?
Loved the video 🙂 Thanks
June 24th, 2010 at 1:49 am
Hi, which render manager do you use for the render queue???
June 24th, 2010 at 12:20 pm
Thanks this was an awesome behind the scenes look, i think the render farm software would be useful for students from network admin backgrounds to get their feet wet.
June 24th, 2010 at 2:26 pm
🙂 Just dreaming of my own cluster now.. Thanks guys 😉
June 25th, 2010 at 4:53 pm
Wow… I have no idea what most of that meant… Any possibility that we could get a tutorial about renderfarms? (After this is finished, of course) I’ve got several computers, but the only reason I can use them is that LuxRender is awesome and easy. Most of us are modelers/animators, not sysadmin, so anything that you guys could teach us would be really awesome!
Keep up the great work.
June 25th, 2010 at 6:29 pm
this render farm seems to be a very interesting subject indeed
it would be interesting to have a special page for this system of render farm
including a general description for
software use
hardware system included
how many processsor and type speed
memory in GB & hard Disk GB
and give an idea of overall peformance for that system
from a rendering point of view
and if possible give an idea of the electrical power consumption
per machine and total power
i mean $1000.00 / month seems to be a huge electrical bill
don’t know what your local cost is for Kw/ hour?
and add a link for a small video tour of the render farm too!
compare that to the power of PC over the last few decades
and it’s pretty amazing to see where we are at and how fast it is evolving over time – hardware and software wise
and it’s gone get even better and faster over next few years
happy 2.5 with high speed rendering
June 25th, 2010 at 10:03 pm
@rickyblender: Don’t forget the power consumption of the 2 airconditioning systems…
June 27th, 2010 at 12:50 pm
Thank you for video tour
June 30th, 2010 at 12:33 pm
Well, I would love to see some of the renderfarm scripts you guys have built up. I’m currently running a drqueue setup and it is slow. As a bonus I might be able to lend some of the CPU’s to speed up your 4K render — I have 1 farm running with about 220 cores and in the next little bit will hopefully be bringing on another 2000 cores 🙂
I run the render farm for student projects, but it only gets used during class time is idle the rest of the time.
PS. love the renderfarm in a bathroom
July 1st, 2010 at 5:35 pm
@maddes, when the code isnt as well tested generally you can find bottlenecks and make bigger improvements but at some point it becomes a lot harder to improve render speed.
@BlenderBoy, we use avi-jpeg mainly because its very fast to seek and we dont get codec problems, the ffmpeg integration in blender isnt that great still 🙁
@roydude & artramirez3D, there is a fairly simple python script which loops on a timer and sends commands to systems via ssh.
@chris, will release with the production files.
@Matt Hurley, saddly no, setting up a render farm like this still is fairly involved however you can use netrender which comes with blender 2.5, as saied before it doesnt have the features we needed but these were mostly durian spesific.
@rickyblender, this info would definetly be useful but with such a small team its hard to find time to pay attention to these details when the artists find bugs on a daily basis 🙁
@Brent, wow, thats a massive cluster!, so I expect you know what your doing :), before durian I read up on Condor – http://www.cs.wisc.edu/condor/ its been used with blender before, looks like it could be used as a more generic way to distribute jobs.
July 2nd, 2010 at 8:42 am
Very Impressive I am using farmerjoe -the render farmer.
Working perfect on Ubuntu workstations.
I am also reading good things about corefarm.
there is now also a commercial renderfarm for Yafray.
Is there anyone familiar with corefarm?
http://www.corefarm.com/
July 2nd, 2010 at 8:59 am
sorry forget what I wrote about commercial corefarm it is online rendering
August 23rd, 2010 at 8:54 am
I’ve got a question it’s about the rendertime. How well does the Blender’s rendertime compared to commercial solutions as Maya and 3D Studio Max? if I want to make a short movie like 1 minute with the same quality as Sintel, would it also take several days to render on my single computer? Isn’t the rendertime for sintel like 20 days??? – if we assume it the the results becomes 27 days…
17000 frames / (24 frames/seconds) / (60 seconds/minute) = 11.8 minutes
1 minute / (11.8 minutes) * 20 days * (16 motherboards) / (1 motherboard) = 27 days
How much effort will the Blender development put on rendertime optimization in future releases?
i get all the information from the google search renderrocket
January 28th, 2011 at 12:33 pm
Hi,
I’ve got about 5000 .EXR files (rendered by Blender 2.56a r34481) that I’d like to convert into .PNG:s, so I can create a nice .AVI file.
Please tell me what tool you are using to extract png:s from EXR files. I tried the exrtools suite, and Photoshop, and some other tools but the result is a black image. However, the EXR-file IS properly rendered, because I can see it look OK in the Compositor using an Image input node connected to a Viewer node. But adding the EXR files in the video editor doesn’t help either. Somehow I need to batch-import them, and I would into a very happy guy if you – or someone else could give me a hint!
Thanks
BTW Your work with Sintel is simply amazing!
Cheers!
January 28th, 2011 at 2:42 pm
Heh, it wasn’t that hard. Once I added a single Image input node, and added the first .exr file – a new dropdown option appeared, where I could select “sequence”. Then I added a composite output node, and could render the output as an animation.
October 18th, 2011 at 3:21 pm
Hi,
where can I find the php(?) script shown in the video for managing the render jobs? Is it on the DVD set?
regards
Darki