Greetings from the audio trenches!
on June 8th, 2010, by jan
Hey there! Jan Morgenstern here. I’m responsible for the scoring, sound design, and re-recording mix of Sintel, and I just realized that I’ve been cowardly ducking any blogging duties that might have arised ever since I started working on this project. Clearly, I’m due. For my blog inauguration, I’ve chosen the proven subject of “what am I doing here anyway”.
Quick note: I boldly assume that most of you are more at home in the realm of 3D and video than audio, and seeing that I regularly go “wait, rig what with the what now?” when I read posts from the rest of the team, I’ve put a little glossary at the end of this article. I hope that this keeps things more readable than liberal use of parentheses would.
Taking care of all audio aspects of a film by oneself provides some great opportunities, but the sheer number of things that need to come together to build a cohesive soundtrack can be a bit daunting, at least to me. Sintel is a relatively complex production in that the action is pretty condensed and takes place in a lot of different settings during a short amount of time. Because of this, I’ve been looking for strategies to avoid wearing too many hats at once. Here’s what seems to work for me:
– Split the whole task into smaller, more manageable chunks, loosely modelled after the audio workflow of a feature film production.
– Keep these chunks in separate DAW projects.
– Work on each of them in isolation until it can stand on its own.
– Worry about glueing them together later on.
Apart from keeping me from going bonkers, this approach has another big advantage, which has to do with the fact that scoring and mixing, while traditionally being post production tasks (and thus taking place after the film has been shot), are happening alongside the main production itself in our case. We’re trying our best to follow a strict process for controlling which aspects of the movie are “frozen” and thus ready for post production, but it’s still inevitable that some things slip through the process and get reconsidered at a later point. If I kept everything in one place, it’d make accommodating these late changes an exercise in frustration.
Whenever I’m finished with a subproject (for reasonable values of “finished”), I bounce it to one or multiple submixes, called stems; these are just audio files, which then get incorporated into the next stage of production. It’s pretty similar to the process of rendering scenes in layers, which are then being post-processed and combined in compositing. In this manner, the subprojects form a dependency graph, which in the case of Sintel looks like this:
If something changes way up in the graph – say, in the Dialogue Edit –, it takes a bit of work to deploy those changes up to the final mix; in the example (which does occur from time to time), it comes down to bouncing the dialogue tracks of the characters that appear in the affected scene, bringing these changes up in the respective Scene Mix, bouncing the dialogue stem of that mix, and picking up the new stem in the Master Mix. You can figure that it’s a good idea to get the topmost elements as close to final as possible before advancing to the next stage.
Here’s what happens in each stage:
Dialogue Edit. This is a fairly simple 3-track project in which I assemble the 200-ish takes from our dialogue recording session into contiguous dialogue tracks for each of the characters. Each character’s dialogue is then bounced to a simple mono file with no automation or further processing (these happen within the Scene Mixes, as I need context from the rest of the soundscape in order to do that properly.)
Sound Design. Some of the special effects are a bit more complex, as they’re made up of layered and processed sounds; this is especially true for the Dragon utterances, which also use granular synthesis a lot. It would be rather impractical to handle this in the actual Scene Mixes, as it would drive their track counts even higher and make them more confusing and prone to mistakes. For this reason, I’m building “palettes” of raw source sounds inside separate projects, which I then pick up as stereo spot effects in the…
Scene Mixes. Here’s where everything that makes up the soundscape of a scene (except the music) comes together for the first time. These projects are by far the most complex and automation-heavy in the whole workflow; they often have around 60-80 multichannel tracks. Each track in these projects falls into one of 4 categories: Dialogue, Spot Effects, Foley, or Ambience (as you can see in the screenshots, these are color-coded.) When I’m done with a scene, I bounce the tracks in each of these categories into a 5.1 stem, which is then combined with the other Scenes in the Master Mix.
Scoring. Here’s where I compose and arrange the film music for each scene (or combination of scenes, in case a music cue spans several scenes). These are just traditional stereo music production projects, but they can get pretty entangled all on themselves (my basic orchestral template consists of 100-ish MIDI tracks, each of which addresses an average of 15-20 switchable instrument articulations), so I’ve been moving away from handling the mixing within the same project. Instead, I’ve begun sourcing the mix out into yet another project (wheee!) and bouncing the orchestral sections (Woodwinds, Brass, Percussion, Choirs, Strings…) into separate, unprocessed stereo stems, which then go into the…
Score Master Mix. In this single audio-only project, I’m picking up the 7 to 8 stereo stems from each of the music cues and combine them into a continuous music stem for the whole movie. This is a pretty linear process, not unlike working with a tape machine. This is also where the stereo stems are upmixed into a 5.1 stem. It wasn’t quite easy to let go of the idea of mixing the music in 5.1 natively from the beginning, but I experimented with that during Elephants Dream and Big Buck Bunny, and I just didn’t feel that the end result justified the extra effort.
Master Mix. Here I combine the output stems from the scene mixes and the score mix into the final soundtrack. In order to accommodate smooth scene transitions, some of the stems need to be dovetailed into one another (instead of cut abruptly), so I have at least two tracks for every kind of stem.
Once the Master Mix is finished, it’s almost time for me to whip out the 30 year old scotch. The 5.1 output of the mix ends up on the DVD and the surround web releases, and is downmixed to stereo for other release formats. For the theatrical release, I’ll deliver the stems to a Dolby-certified mastering lab that prepares SR-D and SR printmasters for duplication. I’ll also try to get access to a cinema-style dubbing stage for the final mix, as that always gives a much more accurate picture how things will sound in cinemas out there than my humble studio could.
Finally, some big words which you can use to impress audio engineers at the next cocktail party (I’m kidding, we’re all miserable social misfits who don’t get invited to cocktail parties):
Automation: A programmed variation of a parameter over time. Commonly automated parameters are level, panorama position, equalizer or filter settings, reverb send levels, and so on. Automation can be pretty useful in music production; in post production, it’s absolutely essential.
Bounce: The audio equivalent of “render”.
DAW: Digital Audio Workstation. The central piece of software that an audio guy uses to do stuff; in highly proprietary systems such as Pro Tools, the term often includes the hardware, too. My DAW of choice is Steinberg’s Nuendo. Nuendo isn’t free software, which makes me kind of a cool renegade underdog.*
Dolby SR-D and SR: Widespread systems of getting multichannel sound on physical 35mm copies. SR-D is the professional equivalent to Dolby Digital on DVDs (which provides 5.1 discrete channels), while SR refers to the old method of encoding 4 non-discrete channels into an analog optical soundtrack. If you get SR-D on a copy, you always need to provide an SR soundtrack as a fallback, and both need to go through the hands of a Dolby-approved voodoo priest before a duplication lab will print them.
Downmix: Catch-all phrase for reducing a multichannel signal to fewer channels. When you’re watching a DVD with a 5.1 soundtrack on your stereo TV set, your DVD player downmixes the signal on-the-fly, so that information from the center and surround speakers doesn’t get lost.
Foley: Footsteps, cloth movements, subtle sounds of characters interacting with things. The stuff that even film audio geeks don’t notice unless it’s either missing or done wrong.
Stem: A submix of tracks that usually belong to a common category, such as dialogue, ambience, or music. Usually, everything up to the final mix is being handled in stems, which is a good compromise between having total control and keeping things practical. A lot of audio post contractors, such as mastering studios, expect their clients to deliver material in stems instead of monolithic mixes so that they can do finishing touches.
* It was a joke. Please don’t pummel me and take my lunch money again.
June 8th, 2010 at 4:53 pm
This might have already been answered, but what software are you using for audio? Since this is an open-source project, are you also using open source software for audio mixing and composing the score?
June 8th, 2010 at 4:59 pm
“DAW: Digital Audio Workstation. The central piece of software that an audio guy uses to do stuff; in highly proprietary systems such as Pro Tools, the term often includes the hardware, too. My DAW of choice is Steinberg’s Nuendo. Nuendo isn’t free software, which makes me kind of a cool renegade underdog.*”
June 8th, 2010 at 5:02 pm
Great post (including the humor…)
Seriously though, it was really interesting and I am looking forward to the final score as BBB’s was so excellent (no pressure)!
@blkhatRaven:
It said “My DAW of choice is Steinberg’s Nuendo” 🙂
June 8th, 2010 at 5:04 pm
He seems to be using either Cubase or Nuendo. Most probably Nuendo, because of its multichannel features, and being most post-production oriented.
June 8th, 2010 at 5:05 pm
OK, I was late, Andrew replied faster. 🙂
June 8th, 2010 at 5:05 pm
Thanks! I’ve been waiting for this one…very interesting.
Since this project is open, and we get the .blend files… will we get midi files (or some more flexible format)? Personally, I’d think it would be great if us music people could learn from you, that way.
June 8th, 2010 at 5:05 pm
I’m talking about the scores, of course.
June 8th, 2010 at 5:07 pm
It’s easy to understand why Ton picks Morgenstern every time, his work is always fantastic!
June 8th, 2010 at 5:11 pm
Coolius Maximus. noting more to add. :3
June 8th, 2010 at 5:12 pm
Note for all: for the CG (2d and 3d graphics) we restrict ourselves to using Linux workstations with free/open source programs, obviously Blender mostly. That is a target we can manage ourselves quite well, with direct access to developers and support.
For everything not CG-related and not done in the studio here we’re pragmatic, and expect people to use what they’re comfortable with.
-Ton-
(BTW: I use OSX on an old PPC mac still!)
June 8th, 2010 at 5:16 pm
Am I the only person that thinks that Sintel’s voice sounds like she’s 50?
June 8th, 2010 at 5:18 pm
Is there any reason for using proprietary software for audio? I’m not complaining, I’m just wondering if such a huge task would be possible with the currently available open source tools. What are they lacking if they’re available but not good enough?
Thanks for the post, makes a change from all the 3D posts! It’s interesting to see the other side of the movie. I look forward to the sound track, if it’s anywhere near as good as BBBs we’re in for an excellent film!
June 8th, 2010 at 5:48 pm
W: MIDI files would be kind of cool, but useless… as the instruments are mapped and programmed, I doubt anyone will be able to get an audible result with these MIDI-files (unless Jan releases the project file and you have an exact copy of his setup).
Jan: Nice blog, thanks for that and keep up that amazing work – heading for the last milestones, tschakka! 😉
June 8th, 2010 at 5:54 pm
Great enormous post! fantastic! very instructive. With that huge post Jan doesn’t need to post again 🙂 Ton! force the rest of the team who haven’t posted to do it 😛
June 8th, 2010 at 6:05 pm
@Robert:
i don’t think there is any open source movie sound mixing project at the moment.
There are open source editors and open source sound mixers, yet what remains untouched is the idea to put a video stream next to an audio stream with the great capabilities of sound editors. There are threads in the audacity forum in order to request such funcionality, but there hasn’t been done anything about this within audacity development… anyone knows any project about open source sound mixing where such things have been discused?
June 8th, 2010 at 6:10 pm
@ blenderman345, no, you’re not. I agree 🙁
June 8th, 2010 at 6:14 pm
@Biel Linux Multimedia Studio aka LMMS . it seams to FL studio but open source
June 8th, 2010 at 6:17 pm
@blenderman345:
… I do not think sintel sounds like she is 50. Considering that she lives alone, provides (steals?) her own food and generally takes care of herself a more mature voice than what an average twelve year old girl would have is perfect.
As a side note I really am glad that the durian crew has not felt the need to make Sintel into a “princess.” Let her be dirty and tough to give her character a contrast with the nurturing side as “scales” is introduced.
June 8th, 2010 at 6:17 pm
@Biel http://lmms.sourceforge.net/ but its comunity is small so its development is slow.
June 8th, 2010 at 6:21 pm
@blenderman345:
You are, like, the 50th one to say that.
June 8th, 2010 at 6:31 pm
These Blender Institute projects were a brilliant idea and have really helped to advance Blender. How awesome would it be if these projects could be expanded to include other communities to help further the progress of other open source tools, such as Audacity, Ardour, Rosegarden, or GIMP.
I understand resources are limited, but this would also expand the pool of potential funders to include these other communities. I realize not every community is as active or supportive as Blenders, but maybe being involved in these types of projects would help engage these communities.
June 8th, 2010 at 6:34 pm
And it will take much time until Audacity reaches the level of Nuendo or Cubase. They are just so advanced… Ardour is on a good way, but since there are simply no good VST´s and Samplers for Linux, your dream will have to wait. On Windows there is the “Shortcircuit”-Sampler which has been released as Open Source after they stopped to use it commercial, and it´s pretty usable, but i can´t find any equivalents for Linux. Linuxsampler looks nice, but it don´t gives you the control you need to edit Soundtracks.
And so im happily working with Cubase… 🙂
June 8th, 2010 at 6:37 pm
What a shame that the audio is proprietary… Still, if it sounds as good as the screenshots look, I guess I’ll have to live with it.
June 8th, 2010 at 6:41 pm
Thx fot share your workflow !
June 8th, 2010 at 6:45 pm
hey guys. don’t be fanatic with open source software. to be open doesn’t mean only use open software but to be open to everything in order to succes a project or be productive. if you’re closed just to open software then you aren’t open. it’s some kind of paradox :S
June 8th, 2010 at 7:04 pm
Thanks WO262 that made my day!!!
June 8th, 2010 at 7:13 pm
Great post Jan. Very interesting as I dabble in audio as well as graphics (very amature). It sounds like you will have earned that 30 year old scotch by the time you’ve finished so have one for me while you’re there 🙂
June 8th, 2010 at 7:22 pm
Love the Ice Age movies, super animation.
June 8th, 2010 at 8:07 pm
Back of the Mike ( http://www.archive.org/details/Backofth1938 ) is an interesting look into old school foley techniques. Fun stuff like a funnel + ping pong ball + sand = rain in there. Just be glad you don’t have to do things in real time these days 😉
June 8th, 2010 at 8:37 pm
hey jan nice explanation, (gotta work hard on the audio when doing an movie i guess…)
June 8th, 2010 at 8:54 pm
Jan, would you be willing to export some of your work to OMF or similar?
On a separate note, I realize that your sequencer is Nuendo but what plugins do you use? Are all the plugins from Steinberg or do you use others such as Waves?
Would you be willing to write up some advanced breakdowns of your techniques for scene mixes? For example, I would like to hear how you adjust reverb for different scenes and what reverb software you like best and why.
Thanks,
loopDuplicate
June 8th, 2010 at 8:57 pm
P.S. Are we gonna get any audio snippets before the movie comes out?
P.P.S. Below you post it says, “* It was a joke. Please don’t pummel me and take my lunch money again.” What is that all about? 🙂
June 8th, 2010 at 9:27 pm
Great, what a lot of work, especially if doing everything by yourself!
Question:
I assume the 35mm prints will again be made with the Cinevator filmprinter?
June 8th, 2010 at 9:55 pm
no, you’re not the renegade underdog. all other durian members are 😉
June 8th, 2010 at 10:19 pm
@Domino Marama: great link!
But on the making-of DVD’s of LOTR extended edition I learned that they still use classical foly artists and techniques including fiddling about with pieces of leather etc. and stepping in boxes filled with all kinds of stuff along side all the digital techniques.
For the battle at the Pelennor they actually recorded a block of concrete being dropped from som twenty metres with several microphones some which were buried. The drops caused a crater which they started filling up with all kind of rubbish. These recordings were used for the sounds of the catapult-battle.
June 8th, 2010 at 10:27 pm
Thanks a lot, Jan! 😀 This is a wonderful post.
I wonder, how do mixing the score natively in 5.1 and upmixing from stereo stems differ that makes the former more practical for you? What are the pros and cons of either method?
June 8th, 2010 at 10:28 pm
Why not use Blender sequencer to lay out the audio tracks and do any noise reduction or other audio post in audacity?
June 8th, 2010 at 11:55 pm
Oh, its here! 😀
June 9th, 2010 at 12:17 am
In case I missed the answer to this question, what software was used to record the actual music, or was it midi or live recorded?
June 9th, 2010 at 12:40 am
Will you distribute the .npr with all the samples?
Maybe not the whole project, but just for a scene, you know.
It will be a highly valuable ressource for the amateurs.
I wish the next Blender project will start a federation with other tools to get a full opensource pipeline for movie creation.
June 9th, 2010 at 12:44 am
Very interesting post. Nice to see such an organized workflow. Given that you have computed the dependency graph, are you using makefiles to update/bounce appropriate stems when something is changed?
(If not, try asking Campbell about it, he might enjoy setting something up for you.)
Midi files of the music would definitely be useful, I think. If not, then perhaps some auto generated score…
Any other hints at all about how you compose would be wonderful, but I can see that you may have very little time.
About the open source issue, Jan has always been very open to the idea, but is is a tricky discussion:
http://www.blender.org/forum/viewtopic.php?t=15608&postdays=0&postorder=asc&start=0&sid=5fc1008b8e23faaac2f7cb8796dbaad3
In that thread I felt he was quite misunderstood by some people. It would be interesting to hear if he thinks that open source has progressed and is closer to being usable, though…
June 9th, 2010 at 1:23 am
For anyone interested. BPyJack: a python script to sync playback video with your opensource DAW
http://www.youtube.com/watch?v=xvysT6Ak4y4
If with an opensource DAW we can get the same quality than the current graphic level, and honestly it can. I think it’d be pretty good, more “honest” and everyone will be happy! 🙂
We don’t care about 5.1 stuff yet or high orchestral sounds yet.
But hey, we like Jan’s works and i think we really appreciate his involvement in the project, it’s just a matter of making the Open Movie project more coherent (it could be a good topic for the next BFconf :))
Anyway, your first choice is totally understandable!
June 9th, 2010 at 1:33 am
Jan Morgenstern’s quote “And if you look at the available projects pragmatically and without an ideological bias, you’ll notice that most of them are still a far cry from their established commercial counterparts.”
This is exactly what it is said about Blender outside the community and the opensource world. 🙂
However…
June 9th, 2010 at 2:39 am
But Jan has it’s own tools carefully choosen by a lot trial and error, ups and downs etc. by the years.
It’s pointless to me to give a butcher a blunt knife, if you know what I mean. 😉
The guys at the Blender Institute are Blender guys and Jan is a Nuendo guy.
So never change a winning team out of principles that make no one happier.
Open source is a great thing, but is has to be useful at first place.
June 9th, 2010 at 2:42 am
yoff,
I do not think that we missunderstood him in that thread. I think the truth was told and it was not well recieved by the closed source users.
Thanks
June 9th, 2010 at 2:56 am
Impressive post.
June 9th, 2010 at 3:10 am
Although it’s a shame that you wern’t able to use open source tools to make the audio for the project, it’s still appreciated that you took the time to write about your process for making all the audio stuff. Its true that, at first glance, there isn’t any rational reason for using some 2nd-grade tool for doing a job when you already have a tool that you know how to use well (ever tried convincing a handyman that using a knife is better than a hammer to put in a nail ?) but, in my opinion, part of the reason for starting ‘open’ projects is to be able to take a look at what’s already out there, and if it doesn’t to what you need it to do, what does it take to make it better. Not only does this bring benefits to the software, it also benefits everyone in the same situation who are looking for open alternatives (if you want me to put in nails with a knife, make the end more blunt and heavy. Thenanyone will be able to use a knife to put in nails). People who only swear by open source software might by rightfully called fanatics and closed-minded, but people who just call out names and bring nothing to the table to make the situation better, for the benefits of others, are just hypocrites
June 9th, 2010 at 3:16 am
gelendir:
http://www.openoctave.org
We have been working on the tools for a professional open source audio pipeline for some time.
At the moment our efforts are focused on an up and coming audio editor with some of the best work flow I could imagine. As well as enhancing sampler support for SFZ scripting.
So we keep plugging away.
June 9th, 2010 at 6:16 am
Thanks Jan! I hope this gives people a better idea about all the work that goes into audio in a project like this. As for the open source questions, I guess it’s time more coders started joining the ranks of those working in that field.
June 9th, 2010 at 7:34 am
How about next open movie, you ask other software projects like the GIMP, Inkscape, Ardour, Dirac, Ubuntu, and GStreamer if they want to send a developer over so they can improve their stuff too?
Can you IMAGINE what would happen with one developer from each of those projects present? I’m thinking Blender, at this point is mature. The other software you guys rely on? Maybe not so much. Why not help them help you?
June 9th, 2010 at 7:54 am
Hehe, it hardly seems fair! You and you alone have to create a soundtrack that can live up to the visuals that were created by a team of 10-15 full-timers, plus hundreds of community contributors! From the clips we’ve been shown so far, you seem to be well and truly holding your own, though.
Also, perhaps you could lend your expertise to one of these open-source audio projects by coming up with a wish-list of what you would require for professional work.
June 9th, 2010 at 8:35 am
Please, papa! Please release Blender Open Workshop DVD by Jan Morgenstern!!!!
June 9th, 2010 at 9:08 am
# yetanotherone Says:
June 8th, 2010 at 17:07
“It’s easy to understand why Ton picks Morgenstern every time, his work is always fantastic!”
I thought it was Jan picking Ton everytime, but that could be just my perception.
June 9th, 2010 at 9:17 am
* It was a joke. Please don’t pummel me and take my lunch money again.
I’m always thinking in a project like this: What if…
What if these great guys used real software, wouldn’t that what they come up with be 10x as great? It probably would.
But…, then, where are these great real software projects?
Blender Institute projects are open content projects. Even if only closed software where used the project would still be cc3. There is no more open than that.
Of course it would be great if some open source audio project would hire a bunch of artists for a month or 6 to make an open content album, I have not seen that yet, please post a link if you know of such a project.
June 9th, 2010 at 10:12 am
Hey Jan, nice to see your workflow. I’ve recently experienced something similar in an end of year project (10 minute audio production in 5.1. With foley, fx, sound design, vox etc.) but my small team ended up using alot less stems as we had planned, so basically our session was hell. I think being able to work it out how you do would’ve helped us tremendously. Thanks a great deal for the insight, especially on using different sessions for different scenes.
I’ve just read that whole argument thread about open source audio software you were into… I feel for you man 😉 Personally I’m more PT right now, but the price of all the add-ons ( especially for 5.1) will eventually get me into Nuendo I think. I just prefer the whole PT workflow and interface though. Keep it up and I hope you get to keep your cool about all this Open Source linux audio pressure (for the time being though, hope that scene gets better).
June 9th, 2010 at 11:15 am
Unfortunately the audio stack in Linux is like a disaster area. Many thinks need to be improved in many areas to make a musician comfortable.
To sum up: virtualization has huge latencies, Wine is partly broken (so can’t take advantage of many VSTi, or other audio apps), and native apps are lacking features. Kernel is not tuned for desktop (again latency), doesn’t support pro audio HW and PulseAudio is mostly broken (other things like Jack are not automatic and require extra steps to work). Also you may already have a $500 sound library, but you can’t use it because it’s not compatible, and there aren’t Linux equivalents, commercial or non commercial.
The only way you can make music in Linux today is with either electronic music, or using external sound sources (samplers, keyboards). But forget about orchestral hollywoodesque type of music.
June 9th, 2010 at 11:50 am
I think blender needs to sell it self and proprietary does audio best. Part of this project is just to improve blender.
On the other hand there should be the “Open Multimedia Foundation” which will focus on different open projects to improve them. Though some will be harder than other. An open project on a picture really wouldn’t go that well, unless it was to do say an entire comic. Though workshop dvds for anything would do really well.
Though if the open audio project does happen I want Jan to do it with an army of minion programmers following his directions.
Also if he did do a workshop on audio it shouldn’t focus on the software. it would be more about writing movie music and methods used to produce it. Which i would buy though i don’t know if blender should be the one doing it.
June 9th, 2010 at 1:14 pm
ccherrett: You would be wrong. Jan’s post at the top of the second page of that thread sums up what was wrong with the tone of that thread and his first post (3rd on page 1) neatly ties up why there is no production-ready open-source audio package.
Jan, thanks for the insight into the audio world – the scene mix screen-shot is a bit of a jaw-dropper!
June 9th, 2010 at 4:57 pm
wow a little complex for me but equal sounds interesting. i hope a sound editig tutorial in de dvd set of durian with an opersource will be wonderful lear at least the basisc for score short animation pieces
June 9th, 2010 at 7:44 pm
CorsairX,
Well if Jan said it then it must be true. End of conversation. 🙂
June 9th, 2010 at 7:57 pm
david,
I admit there is a setup invloved with Linux (as with other platforms). Once you have your rug fine tuned there is nothing better for performance.
You are correct about the VSTi situation, however there are alternatives, one of which I have setup. My setup if far more expensive than the $500 example you gave and it all runs and routes for realtime playback with over 900 connections between the different applications. I have a full professional orchestral setup that can rival the output of anything in a hollywood studio.
5.1 mixing is a shortcoming of linux. I am hoping Ton and Jan would tackle this one together. I am sure they would have it figured out in no time. Till then there are some hopefulls in the land of ambiosonics, and at some point the non-mixer should have the ambiosonic panner finished. Ambiosoinics can convert to 5.1
June 9th, 2010 at 7:58 pm
Also Windows can be used as a sampler with midi over lan. So you could use a Windows rig to run all your VSTi plugins if needed. Just an option
June 9th, 2010 at 8:02 pm
@joeri67
Maybe this is similar to what you imagine?
http://www.the-album.cc/
People use linux audio to make all kinds of music from purely recorded to purely electronic and everything in between and certainly also orchestral music. Sure, there aren’t a lot of companies involved in linux audio, but it’s slowly increasing, renoise and pianoteq are good examples. The main problem I see for film music is 5.1. There’s hardly anything there and the codec patent/license situation doesn’t help. So surround development on Linux revolves mostly around the old (patents expired) but technically superior ambisonics.
June 9th, 2010 at 8:23 pm
ccherrett, would you mind pointing me at a piece of music that you’ve created with your orchestral setup? I’m genuinely interested.
June 9th, 2010 at 9:52 pm
Hey Jan,
Just wondering. What libraries do you use at the moment? Do you still use ewqlso? Did you expand with sam, vsl, lass, si, kh, … ?
The scores I’ve heard sound amazing! Keep up the great work 🙂
June 9th, 2010 at 9:54 pm
Jan,
I will tell you what. You come and join us in #openoctave in irc, we can give you the full run down of the setup in a conference. We can share some scores in there. Possibly Alex would be willing to share some of his classical work with you at that time.
For the record, I cannot compose my way out of a wet paper bag. I code for my wife and for Alex. I am simple a coding monkey for the musicians. 🙂
Thanks!
June 9th, 2010 at 10:47 pm
Excellent article Jan. I am an audio engineer and would just like to note that directly after reading this I was invited to a cocktail party. So that’s put that to bed 🙂
(the fact that I will be drinking alone in the corner means nothing)
June 9th, 2010 at 11:07 pm
LMMS has many more plugins than are initially apparent. I’ve been using the Soundfont 2 processor lately, but the other options are nice as well.
I agree that it would be nice to have a working JACK setup out of the box, tho. Its a real pain to setup, and it seems that most common hardware setups won’t handle it either. Perhaps the medibuntu package will garner a proper audio app?
Great post Jan!
June 10th, 2010 at 12:32 am
Thanks for the comments, everyone!
@loopDuplicate: I’m a big fan of the Oxford and UAD plug-ins, and pretty happy with Altiverb. Not too crazy about the built-in Nuendo stuff (not that it’s bad, just a little patchy.)
@D: A native 5.1 mix would require placing all instruments on the surround plane one-by-one. I did that for BBB, and it made everything horrendously complicated. This time I’m basically preparing the score in stereo, then blowing it up by using 5.1 reverbs in mixdown, anchoring a bit of the mid channel to the center, and splitting off the bottom end of some percussion tracks to the LFE.
@V1ntage: I’m using EWQLSO rarely these days. My current go-to libraries are VSL, LASS, a bit of Symphobia, and Tonehammer, True Strike, Stormdrum 1+2 for percussion.
@ccherrett: We should do that someday. I’m still inclined to say that statements like “I have a full professional orchestral setup that can rival the output of anything in a hollywood studio” shouldn’t be made without an equally bold way of backing up that claim. No offense.
June 10th, 2010 at 12:47 am
” Nuendo isn’t free software, which makes me kind of a cool renegade underdog ”
Yes, exactly.
June 10th, 2010 at 1:41 am
Jan,
Come by some time and we can go through things. At that time you can tell me if the statement is true. I believe the statement to be true.
We could use your support as well to gain more development help.
June 10th, 2010 at 6:52 am
Haha! Nice explanation (and a bit daunting and scary). Other than that, keep up the great work and looking forward to “hearing” what you have for the whole world. ^_^
-Reyn
June 10th, 2010 at 7:13 am
Ha.. I like jan
June 10th, 2010 at 10:40 am
\o/
June 10th, 2010 at 2:11 pm
It would have been better if the audio software was free and open as well. But nontheless Jan is really good.
June 10th, 2010 at 2:28 pm
Perhaps Jan could spend some of his free time testing OpenOctave and help it shape up for production use.
June 10th, 2010 at 6:44 pm
I think you guys are all doing a great job and I know it can be hard but you have my two tumbs up keep it up everyone.
June 10th, 2010 at 7:38 pm
OpenOctave in short:
1. Take a working open source music app: Rosegarden
2. Strip notation interface
3. ???
4. Profit!
No offense, but music notation is THE way of entering/visualizing notes for every musician. Moreover, making a lot of small apps that have to be combined to have a working setup is time demanding and can make your inspiration disappear soon. And making a whole new distro for just composing music has the disadvantage of… well… installing it and rebooting just to compose something.
Just make a new app with everything in it. Zero config.
And take time to make the kernel devs understand that we need low latencies in the default kernel (BFS scheduler, etc). Don’t make a separate distro for every different usecase, dammit.
Blender doesn’t need a new distro to run, and doesn’t need to config a lot of separate apps to work.
June 10th, 2010 at 7:57 pm
David,
……… hmm what to say………….. Well for starters…………. hmm not sure where to go with you…………… You win, you figured it all out 🙂
June 10th, 2010 at 10:56 pm
No problem to use proprietary software. Since this one is acessible for most people. I use ArtRage2 whit the GIMP. But we are “free users” too ^_~
June 11th, 2010 at 12:13 am
Having just started to seriously dabble in the audio production side of animation and film making, I would LOVE to see a video on the DVD’s covering the audio production process from start to finish including the dialogue recording sessions and processing, foley/sound design and all the mixing/mastering with explanations of what was done and why.
Although most of the ‘serious’ audio software is proprietary, the likes of Pro Tools, Cubase and Ableton LIve have ‘Lite’ versions that come free with hardware purchases (and I can’t actually see what limitations these Lite versions have yet) like Audio Interfaces and Midi Keyboards/Controllers etc
And for those who suggested Audacity – it’s just an editor. Audio is so much more than just editing…
June 11th, 2010 at 12:16 am
Hey Jan,
Thanks for doing this blogpost!
I am a Composer/RecordingTech/Musician/Producer myself so I am always eager to see the workflow of others!
I support your usage of Nuendo btw, I am not against the use of proprietary software (Logic Studio 9 user myself) as long as it isn’t PT kind of stuff 😉
I am currently working for a Game Developer from Norway on some Orchestral compositions using EWQLSO and others, so its interesting to see how you do everything in a movie environment.
Are you going to use external 19″ rack units for mastering or do you work entirely digital?
Also I want to give a reaction on all the people asking about why not using open source for the sound track.
There are quite some reasons for this.
– First of all, the lack of high quality instruments and samples.
On proprietary software there are tons of instruments and samples created by really high end audio technicians that can be used with VST/AU/RTAS plugins.
On linux, we currently have one project OpenOrchestra that is aiming to create a full orchestra library for linux, the problem?
Money and time, the project has been dead for over 900 days.
– Second, and this could be argued about, the DAW software.
I am aware of projects like Ardour (using it for simple recording), Audacity (using it for simple mastering), LMMS, Rosegarden etc, but it just isn’t far enough to use as a main piece of software!
I am not saying the projects aren’t capable, I am just saying the projects aren’t capable yet!
There are lots of things missing, or to instable, and the lack of support for existing Samplers and Instruments makes it hard to use in a Midi environment.
I am not saying Ardour (lets take the biggest one) can not be used, I use it on regular base for Multitrack Audio recording (with some complications here and there i must say), and I really like the capability’s of LADSPA (Linux Audio Developer’s Simple Plugin API) for the complete replacement of proprietary mixing and mastering plugins, but for the complete midi/audio package the best option for audio on linux would be using the DAW: Reaper, and thats also Proprietary software (although with fair licensing) and official support on linux by Wine.
For now, this is just not possible.
@N.A.: There is a really good sampler for linux using GIG files (with really awesome tools for converting and creating sample library’s) called: LinuxSampler(.org)
I use it in quite some projects and even converted some Proprietary samples towards GIG to use it on my “on the go” laptop for simple composition.
June 11th, 2010 at 3:50 am
David: I challenge you to post audiofiles of your “Hollywood class setup”. And not to some IRC – channel, but on a web page anyone can download and listen. I have not, to this day heard anything even half decent made with Linux music setup.
This is seriously interesting, since I would love to not to spend any more money on proprietary music software. Seems to me tho, that Linux world is lagging some ten years behind of what’s happening on other platforms.
Your claim is a bold one, and it would be nice to hear the bells and whistles to prove it. Also some kind of a rundown on how the results are produced should be included.
So, cards to the table! Amaze us with your Hollywood-rivalling-system!
I believe it when I hear it.
June 11th, 2010 at 3:54 am
I said David… I meant ccherrett.
Doh.
June 11th, 2010 at 4:19 am
JoaT,
I had a chance to share some music (which belongs to my wife) in irc today. She is not comfortable sharing it publically before we release or movie that it is for. If that is not good enough then I guess you are stuck in Windows or Mac 🙂
In order to fully do your homework before posting, go to the website and watch the video tutorials. They outline the setup. Ask for help if you would like to learn more. There is even a quick demo of open octave midi in action if you have the time to watch them.
Thanks!
P.S. you really need to be nicer to David. I am glad you did not talk to me that way 🙂
June 11th, 2010 at 10:58 am
“this is especially true for the Dragon utterances, which also use granular synthesis”
Did anyone else catch the term “utterances” here? Does that mean the dragon speaks?! (:-O
June 11th, 2010 at 1:00 pm
It’s just a noun of the verb ”to utter”.
In other words:
Just the roar of the dragon.
June 11th, 2010 at 3:03 pm
JoaT: We challenge you to share with us one of your compo! We will judge if your ears how appreciate music. 🙂
You see, it’s easy to challenge everyone. Where Jan is honest, respectful and well-known professional, you come here as an arrogant artist who ask to be paid to use something free.
JoaT’s quote “This is seriously interesting, since I would love to not to spend any more money on proprietary music software.”
Free Software is not about “free tax” only, it’s about sharing! You are welcome to spend money on it if you don’t like to give it to Proppelhead or Steinberg. :p
It’s a two-way sharing experience, involve yourself in a project even though you are a so overbooked professional having no time to “play” with the under social class, could be a good proof of your real will to use it. 🙂
Then, we will love if you take time to experiment with free software even though it’s hard at the beginning by reporting bug, suggesting workflow, sharing demo music(with sources), well, by helping to improve the software itself, you know…
Pinchies – in the both ways – are not really interesting for the opensource world. 🙂
Thank you!
PS: i’m not related to the Openocatve project.
June 12th, 2010 at 2:05 am
Man, being an audio engineer/music guy myself, I’m overwhelmed by the amazingness of what you’re doing. Scoring alone is a big task to take on, but mixing everything together on top of that…dang dude. You’re incredible.
I use ProTools for my work at Syntax Records, but have been very curious about Nuendo. From the images you’ve uploaded it looks a bit less limited than PT when it comes to track numbers. How do you like it compared to others you’ve used? Obviously it’s your favorite weapon of choice…but why?
I’m excited to hear the final product. The audio on the trailer (assuming that was your work) moved me to tears. Such beautiful work indeed.
What sort of background are you coming from? I did a course at SAE Institute (Nashville, TN). It was much fun, but I only use about 10% of what I learned.
Do you exclusively use Nuendo or are there other toys in the mix too? It’s frowned upon by some, but FL Studio is my fav for production.
Anyway keep up the incredible work!
-Shane Newville
June 12th, 2010 at 8:38 am
Wow – it’s interesting that there are so many following the Sintel project that are both CG artists and musicians. I thought I was relatively alone on this.
Perhaps I missed this and it’s buried in the replies above, but I was wondering whether all the orchestration is being done with sample libraries, or will there be some live instruments used. I’ve always noticed that adding even one live instrument (perhaps even percussion) adds a huge impact to the overall effect of the score.
June 15th, 2010 at 6:36 pm
Ruchir,
I think he’s using virtual instruments since he mentions midi being part of the production. But of course, this isn’t an official response. Only guessing. But I have heard some incredibly impressive sounds come from orchestra v-instruments. And from what I heard on the trailer, it’s hard to tell the differences without intensely trained ears.
Hey by the way Ruchir, do you have a site with music I can hear? You can hear some of my music on http://www.myspace.com/shanenewville or in some of my animations:
Little Ninja Project: http://www.youtube.com/watch?v=KJX2pjbxAsc
My music is more sample based and program drum heavy.
Take care.
June 15th, 2010 at 7:55 pm
would be possible that you make a small video tutorial about designing the dragon sound, and the use of th surround sound techno
thank you.
June 16th, 2010 at 7:13 am
Looks really awesome. Looks like you’re using Nuendo, or maybe Cubase, which I’ve used for years. Great software and workflow. Keep the good work up.
-James
June 16th, 2010 at 10:09 am
Ok, should have read more of the posts. I see you’re using Nuendo (YAY).
To those complaining about opensource multitracking/mixing software, the truth is that it just isn’t there yet. For one Linux has a lot of issues with the various audio cards out there. Sometimes the drivers work great, other times not. When working with a lot of channels, presumably at high sample and bit rate, driver buffers really become important. I’ve looked at this many times when having to upgrade my license for Nuendo (anything but cheap). Reaper is pretty good, but it lacks something that makes it just not sound as good as Nuendo.
The variety of plugins also isn’t great. VST is a very restricted format (having developed some in the past) which doesn’t work well with the idea of opensource. I look forward to the day real quality opensource mixing/multitracking software comes, but I don’t fault anyone for using commercial applications now. Especially when they sound so good.
When I first started doing post-sound for some indie film groups with a much older version of Nuendo the video player was pretty horrible. I ended up syncing Nuendo on one system to Sony’s Vegas on another via midi clock. It worked pretty good, and I still do similar setups for really complex mixes, but I much prefer keeping everything in the same application.
MIDI tracks would be great, but I think accompany them with track renders, maybe at lower resolution to fit everything on the disks, to remix everything if desired. This is of course if including such things is even on the horizon. Again, good work. Keep it up.
-James
June 16th, 2010 at 2:01 pm
I’m generally exclusively Linux these days except in one area: Music production. Sadly it’s the one real bottleneck I run into, not being able to get quality music apps in linux. I don’t think the people designing the core linux systems are really that interested in making them work in a standard way when it comes to audio interfaces. For some reason there seems to be quite a lot of effort in codecs and playback but very little in audio/video production. Don’t get me wrong, there are great projects out there as mentioned previously, such as Audacity (Great cross platform audio editor) Rosegarden (Modeled after early Cakewalk versions), LMMS (a simple clone of Fruityloops) and Ardour (a dedicated multitrack recording solution) which work but require a lot of finagling to get working correctly. The big problem area is that the audio drivers themselves are not exactly standard and by default don’t support low latency audio – the biggest requirement for any music production. Add to this a hodgepodge of midi support and you have a real issue trying to get all this working in Linux.
The basic sound requirements of these applications is generally support for JACK, a low latency patch bay application which allows you to do all sorts of nifty tricks and is now even supported by Blender. Unfortunately it isn’t in and of itself an audio driver – you must get JACK working with one of the drivers such as ALSA, Pulse Audio, OSS, etc. with pulseaudio still heavily in development and not interested yet in being used for DAW style applications but being adopted by most of the linux distributions.
Jack also supports MIDI patching, however my experience is that many midi interfaces aren’t supported at the driver level and usually aren’t supported well by the driver creators. This means that the vast majority of audio interfaces out there simply don’t work. It also means that projects that rely on midi such as Rosegarden get a lot less use and support since all but the most dedicated users can get it configured correctly, and then once they do they are using software that hasn’t grown with the equivalent windows programs such as nuendo.
Interestingly a big reason many audio apps don’t work in Wine is the software protection – Nuendo and Cubase SX require dongles, and that software can’t reasonably be emulated. However other programs work well inside of Wine, namely Sound Forge and amazingly Fruityloops Studio. But as mentioned before, you will only get as far as your ability to get the underlying audio and midi drivers working correctly. God forbid you want to set up adats , specialized midi patchbays or boutique hardware interfaces.
Until hardware manufacturers directly support linux, or agree on open standards, there will probably be a lot less adoption of Linux than there should be as a serious widespread production platform. It’s simply too easy to get the job set up in Mac or Windows environments via programs like Acid, Fruityloops, Ableton Live, Nuendo etc. It also means there is very little commercialization available to coders and teams tend to be small and decay down to a few core members.
If Blender keeps up it’s trend, it may end up becoming the best video editing suite in Linux, and perhaps also audio production as well out of necessity?
June 16th, 2010 at 5:33 pm
Let me suggest Ardour for Linux users. Actually I’ve only used it on OS X and it looks pretty impressive. Especially for being free. I think, technically it is not open source, but the guy asks for donations. Check it out:
http://ardour.org/
June 18th, 2010 at 11:36 am
I might have come across a bit edgier than I intended…
It was not my intention to be rude to anyone or to downplay Linux or open source software, but to raise a question about the real capability of it. I have understood there’s some serious structural problems with Linux audio (The low latency issue, the drivers for soundcards, the lacking implementation of connecting protocols / APIs a’la VST, ReWire and DirectX).
I have used Ardour and checked out Rosengarden and Audacity, but somehow they are more along the lines of “good starts” than full grown tools to substitute the current proprietary tool choices. For any DAW today the real force behind them is a working plug-in architecture with plenty of available good quality plugins. Without them it doesn’t matter how good the DAW itself is. It would still be just a multitrack recorder without usable mixing capability.
I admit not doing my homework on OpenOctave. In fact, this is the first time I even hear about it. The reason I would like to see the demos on a web-page is that if they are of as impressive quality as claimed, there is a lot of audio professionals out there who would be very interested in hearing them. Professionals who mostly would not be bothered to IRC to find out if there’s any truth to these claims.
Think of this like “Elephant’s dream” was to a wider cg professional audience. Before that no-one took Blender seriously. After that a lot of people started to take Blender seriously.
If we get to hear and see examples of pro quality work done with Linux / Open source systems, we are more likely to invest our time in the platform too. To jump in at the early stages takes faith and devotion. And for a lot of full time professionals investing that time is just not an option. There’s far too much things to do just keeping up with the work and the constant learning it requires.
That being said, I am not a full time audio professional. I used to be, but now it’s merely a hobby of mine. I would probably be willing to invest some of my free time in Open source audio the minute I would see it offering something I don’t already have. As a hobbyist I am not even running the latest software, so it doesn’t even have to be “cutting edge” for me.
June 18th, 2010 at 12:45 pm
I watched all the videos on OpenOctave.org, so now I think I have as good an idea about what OpenOctave is and does as you manage to pass with these videos.
I try to summarize for everyone else here:
You have a customized Linux build with:
1.Ardour
2. Linux sample player with Giga sampler capability
3. Convolution reverb
4. Self made midi editor / Librarian(?)
5. A piano roll editor (a spinoff of Rosengarden)
6. A couple of scripts
7. A massive configuration effort to place the instruments on your sample library to their respective places in an orchestra and the convolution reverb settings to mimic the concert hall so that you can tailor the reverb of each instrument to be more realistic to the place it’s supposed to be located in.
I’d say a novel idea, but a one trick pony if I’ve ever seen one.
Vienna string library sounds professional played with any sample player.
Convolution reverbs sound as good as the impulses used.
The scripting part seemed like something I would not want to spend time with in context of music making. The graphical routing parts seemed every bit as cumbersome as they usually are in almost any software I have ever used.
Since you are the builder this thing, I’d suggest you take it a few steps further and make a graphical interface where we see the seating plan of the orchestra from above and can select the instruments by clicking the seats from there. Sure would beat the dropdowns, and perhaps even raise some interest amongst composers. Like in your Open Octave Verb, but combine the midi there too so you can select and play instruments.
Oh, bugger.. You would need the symphonic library to distribute with your build to make it work with anyone but yourself.
What if someone needs other things than string library on a concert hall? I didn’t get it from the videos. How any of the things that separates the Open Octave from using Ardour, Rosengarden and reverb & sampler as separate applications benefit these setups?
And what is it with a million separate builds anyway? To me it is a tell-tale sign of fundamental flaws in the OS if there’s a need to carve out a special version of the OS itself for every single application.
Might sound a bit rude, this one. It is, however, a reality check. The idea is somewhat novel, and you have done some work to implement it. But I doubt anyone who knows what a convolution reverb is or has heard Vienna libraries before is willing to trade their current tools for OpenOctave.
June 22nd, 2010 at 4:16 am
Hi, Jan.
I’m your fan. I’m a musician. I play, bass, acoustic guitar, piano, sax… and now I’m studing Music Production. I use the Blender to make my littles animations, just for fun. Music it’s my priority. I’d like someday make the job that you do: the soundtrack of a movie.
You’re an inspiration!
June 23rd, 2010 at 12:57 pm
Hi all,
I’m a lecturer in Music with a degree in Maths and Computing and user of Blender since 1999. Good to see so many cross-over users here!
For what it’s worth, the Muse Receptor uses a stripped-down, customized version of Linux, but they have managed to sort out the latency issues and get it down to 32 samples (0.7ms) according to the ‘Sound on Sound’ review at http://www.soundonsound.com/sos/jun10/articles/receptor2promax.htm, so somehow the Linux OS can be fixed, it just may need a lot of hacking and/or other functionality removed to make it compatible with one system!
I currently work with Cubase / Logic / PT /Ardour on Windows/OS X/Ubuntu Linux and am an advocate of open source software, but currently have not found anything nearing professional standards. There are lots of interesting, inspirational and usable pieces of OSS out there, but they do very specific things, and aren’t all-round DAWs with good architecture and MIDI and audio handling with the ability to handle a wide range of professional plug-ins and accommodate pro sound cards.
I’m sure there are plenty of us with a genuine interest in OSS who would love to help in the development of DAW software, but most projects seem to come to a standstill very quickly, and we probably need the impetus of a project like the blender ones to forward development. Not sure if anyone has the drive and tenacity to see something like that through though. I’m sure an audio version of Ton would be very welcome!!
Jan, thanks for the info, it’s really interesting to see how other people work too, and I’m sure everyone appreciates the time and effort it takes to compose all of the beds/scores/foley etc (regardless of choice of software). Would love to see videos of you at work, but I’m sure it’s taken more than enough of your time writing a blog.
Thanks again, your stuff is always great to listen to.
July 4th, 2010 at 6:07 pm
Sorry for asking this, there will be a version of no-voice audio to make a fan dubbing?
I searched at forums but no response.
Best regards.
July 8th, 2010 at 9:58 am
@Negrólder!! – usually ALL of the elements are released under Creative Commons, so yes, you can change all the mixes yourself providing you have Nuendo 😉
July 30th, 2010 at 5:54 pm
I am also a composer, and beyond the technical side of things, I look forward to listening to the music of Jan (I loved the music of Elephants Dream and Big Buck Bunny …). Thank you Jan.
August 9th, 2010 at 11:06 pm
@N.T. Audacity is not a DAW and does not aim to be.
@Robert Ardour? http://ardour.org/
August 9th, 2010 at 11:11 pm
@david not when it comes to pro-audio, there you just use JACK and be done with it, and really the ‘disaster’ is not nearly what people make it out to be, and it’s mostly behind us now.
September 1st, 2010 at 5:06 am
@joat the fundamental problem of linux is that you have myriads of different kernel configs, patched code etc. at the enduser’s systems. on windows, you got some different OSes, yeah, but the API and behaviour (ASIO) remains constant or at least backwards-compatible. this is definitely NOT the case on linux and this is the reason why linux in general won’t be an option for DAWs ever. at least not if you don’t make your own linux build fitting to your DAW software…