Sintel, the Durian Open Movie Project » Blog Archive » Encoding Issues – a call for help.

    

Encoding Issues – a call for help.

on September 18th, 2010, by beorn

Hey all, Beorn here, the animator dude who’s been roped into the thankless task of encoding the movie into various formats. (Actually I was crazy enough to volunteer!)
Fortunately the DivX files will be encoded by the good people at DivX, who are actual professionals (as opposed to a forum trawling script kiddie like me).
Despite what Ton has said on this blog before, this is not my area of expertise. I’m really learning as I go and would very much appreciate any advice from the community. I just thought I’d share my current progress and ask for any suggestions on how to improve things.

The goal:
To provide h264 quicktimes and ogg video files in 480p, 720p and 1080p, with subtitles and surround sound (except the 480p versions which will be stereo), and to encode the video for the dvds (NTSC and PAL versions).

I’ve been supplied:

  • directory of PNG files, 2048×872 in size
  • a stereo wav file
  • an audio only quicktime file with 5.1 sound in the aac codec. (there is also an ac3 file for the dvd)
  • a directory of subtitle files in .srt format

Please note: I’ve compiled my own versions of ffmpeg and x264 in order to make the quicktimes.

My current issues:

  • I haven’t been able to add 5.1 audio to the ogg files in any other codec than flac, which means the files are HUGE.
  • The ogg video files are about 3x bigger than the quicktimes, even with stereo sound
  • I haven’t been able to add subtitles to the quicktimes.
  • I’m using mplayer to do the scaling/cropping/letterboxing, but it keeps crashing on the 1080 versions when trying to crop on the fly. Notice the hack I’ve included on those versions; making cropped copies of the PNGs, encoding them and deleting them. It works, but it’s… inelegant.

Here are the shell scripts…

Ogg versions:
title="Sintel"
author="Durian Open Movie Team"
year="2010"
license="Creative Commons Attribution 3.0 license"
copyright="(c) Copyright 2010 Blender Foundation, all rights reserved"
description="Sintel open movie project"
au_stereo="input/audio/sintel-master-st.wav"
au_surround="input/audio/sintel-master-51.mov"

echo "********************"
echo "Ogging the audio"
echo "********************"
ffmpeg -i $au_stereo -acodec libvorbis -ab 448k -ac 2 output/ogg/tmp-audio-st.ogg
ffmpeg -i $au_surround -acodec flac -ac 6 -ab 128k output/ogg/tmp-audio-51.ogg

echo "********************"
echo "Ogging the subtitles"
echo "********************"
kateenc -t srt -l en -c SUB -o output/ogg/tmp-subs_en.ogg input/subtitles/sintel_en.srt
kateenc -t srt -l de -c SUB -o output/ogg/tmp-subs_de.ogg input/subtitles/sintel_de.srt
kateenc -t srt -l es -c SUB -o output/ogg/tmp-subs_es.ogg input/subtitles/sintel_es.srt
kateenc -t srt -l fr -c SUB -o output/ogg/tmp-subs_fr.ogg input/subtitles/sintel_fr.srt
kateenc -t srt -l nl -c SUB -o output/ogg/tmp-subs_nl.ogg input/subtitles/sintel_nl.srt
kateenc -t srt -l pl -c SUB -o output/ogg/tmp-subs_pl.ogg input/subtitles/sintel_pl.srt

echo "********************"
echo "Starting 480 ogg"
echo "********************"
rm output/ogg/stream.yuv
mkfifo output/ogg/stream.yuv &&
mplayer mf://input/png/*.png -mf w=2048:h=872:fps=24:type=png -sws 9 -vf expand=2048:1152,scale=854:480 -vo yuv4mpeg:file=output/ogg/stream.yuv&
ffmpeg2theora --keyint 24 --framerate 24 -v 8 output/ogg/stream.yuv --title "$title" --artist "$author" --copyright "$copyright" --license "$license" -o output/ogg/tmp-480.ogv
oggz-merge output/ogg/tmp-480.ogv output/ogg/tmp-audio-st.ogg output/ogg/tmp-subs_en.ogg output/ogg/tmp-subs_de.ogg output/ogg/tmp-subs_es.ogg output/ogg/tmp-subs_fr.ogg output/ogg/tmp-subs_nl.ogg output/ogg/tmp-subs_pl.ogg -o output/ogg/sintel-480p.ogv
rm output/ogg/stream.yuv

echo "********************"
echo "Starting 720 ogg"
echo "********************"
rm output/ogg/stream.yuv
mkfifo output/ogg/stream.yuv &&
mplayer mf://input/png/*.png -mf w=2048:h=872:fps=24:type=png -sws 9 -vf expand=2048:1152,scale=1280:720 -vo yuv4mpeg:file=output/ogg/stream.yuv&
ffmpeg2theora --keyint 24 --framerate 24 -v 8 output/ogg/stream.yuv --title "$title" --artist "$author" --copyright "$copyright" --license "$license" -o output/ogg/tmp-720.ogv
oggz-merge output/ogg/tmp-720.ogv output/ogg/tmp-audio-st.ogg output/ogg/tmp-subs_en.ogg output/ogg/tmp-subs_de.ogg output/ogg/tmp-subs_es.ogg output/ogg/tmp-subs_fr.ogg output/ogg/tmp-subs_nl.ogg output/ogg/tmp-subs_pl.ogg -o output/ogg/sintel-720p.ogv
rm output/ogg/stream.yuv

echo "********************"
echo "Starting 1080 ogg"
echo "********************"
cd tmp
mplayer mf://../input/png/*.png -mf w=2048:h=872:fps=24:type=png -sws 9 -vf expand=2048:1080,crop=1920:1080 -vo png:z=3
cd ..
ffmpeg2theora --keyint 24 --framerate 24 -v 8 tmp/%08d.png --title "$title" --artist "$author" --copyright "$copyright" --license "$license" -o output/ogg/tmp-1080.ogv
oggz-merge output/ogg/tmp-1080.ogv output/ogg/tmp-audio-51.ogg output/ogg/tmp-subs_en.ogg output/ogg/tmp-subs_de.ogg output/ogg/tmp-subs_es.ogg output/ogg/tmp-subs_fr.ogg output/ogg/tmp-subs_nl.ogg output/ogg/tmp-subs_pl.ogg -o output/ogg/sintel-1080p.ogv

rm tmp/*.png
rm output/ogg/tmp*.*
echo "Ogg versions encoded"

H264/AAC Quicktimes:

title="Sintel"
author="Durian Open Movie Team"
year="2010"
copyright="(c) Copyright 2010 Blender Foundation, all rights reserved"
license="Creative Commons Attribution 3.0 license"
description="Sintel open movie project"
au_stereo="input/audio/sintel-master-st.wav"
au_surround="input/audio/sintel-master-51.mov"

echo "********************"
echo "Starting 480 quicktime"
echo "********************"
rm output/h264/stream.yuv
mkfifo output/h264/stream.yuv &&
mplayer mf://input/png/*.png -mf w=2048:h=872:fps=24:type=png -sws 9 -vf expand=2048:1152,scale=854:480 -vo yuv4mpeg:file=output/h264/stream.yuv&
ffmpeg -y -f yuv4mpegpipe -i output/h264/stream.yuv -i $au_stereo -acodec libfaac -ab 448k -ac 2 -vcodec libx264 -vpre slow -crf 22 -threads 8 -metadata title="$title" -metadata author="$author" -metadata year="$year" -metadata copyright="$copyright" -metadata license="$license" -metadata description="$description" output/h264/sintel-480p.mov
rm output/h264/stream.yuv

echo "********************"
echo "Starting 720 quicktime"
echo "********************"
rm output/h264/stream.yuv
mkfifo output/h264/stream.yuv &&
mplayer mf://input/png/*.png -mf w=2048:h=872:fps=24:type=png -sws 9 -vf expand=2048:1152,scale=1280:720 -vo yuv4mpeg:file=output/h264/stream.yuv&
ffmpeg -y -f yuv4mpegpipe -i output/h264/stream.yuv -i $au_surround -acodec copy -vcodec libx264 -vpre slow -crf 22 -threads 8 -metadata title="$title" -metadata author="$author" -metadata year="$year" -metadata copyright="$copyright" -metadata license="$license" -metadata description="$description" output/h264/sintel-720p.mov
rm output/h264/stream.yuv

echo "********************"
echo "Starting 1080 quicktime"
echo "********************"
cd tmp
mplayer mf://../input/png/*.png -mf w=2048:h=872:fps=24:type=png -sws 9 -vf expand=2048:1080,crop=1920:1080 -vo png:z=3
cd ..
ffmpeg -y -r 24 -i tmp/%08d.png -i $au_surround -acodec copy -vcodec libx264 -vpre slow -crf 22 -threads 8 -metadata title="$title" -metadata author="$author" -metadata year="$year" -metadata copyright="$copyright" -metadata license="$license" -metadata description="$description" output/h264/sintel-1080p.mov
rm tmp/*.png
echo "Quicktimes encoded"

DVD versions:

# NTSC version
mencoder mf://input/png/*.png -mf w=2048:h=872:fps=24:type=png -sws 9 -vf expand=2048:1152,scale=720:480 -oac lavc -ovc lavc -ss 0 -endpos 883 -mpegopts format=dvd:tsaf -lavcopts vcodec=mpeg2video:vrc_buf_size=1835:aspect=16/9:vrc_maxrate=9800:vbitrate=4400:keyint=15:acodec=ac3:abitrate=192 -srate 48000 -ofps 30000/1001 -of mpeg -mc 0 -noskip -o "output/dvd/sintel-ntsc_noaudio.mp2"
echo "NTSC-DVD files encoded"

# PAL version
mencoder mf://input/png/*.png -mf w=2048:h=872:fps=24:type=png -sws 9 -vf expand=2048:1152,scale=720:576 -oac lavc -ovc lavc -ss 0 -endpos 883 -mpegopts format=dvd:tsaf -lavcopts vcodec=mpeg2video:vrc_buf_size=1835:aspect=16/9:vrc_maxrate=9800:vbitrate=4400:keyint=15:acodec=ac3:abitrate=192 -srate 48000 -ofps 25/1 -of mpeg -mc 0 -noskip -o "output/dvd/sintel-pal_noaudio.mpeg2"
echo "PAL-DVD files encoded"

Please try to limit comments to those that are actually useful to this process.
Also, please do not ask us to send you the files to encode yourself. After the premiere, all can be released.

Cheers,
Beorn

164 Responses to “Encoding Issues – a call for help.”

  1. Ton Says:

    You all shouldn’t forget that we have a mirror that’ll host all 2048 original pngs. For both ED and BBB this has been a great resource for many codec testing everywhere, including by Microsoft! Providing the original frames for comparison is very important, no resampled scaling should be done on that.

    What we talk about now is just what we’ll release here initially, and what will go on the DVDs we ship. I like to have this project public now, further delays are not needed.

    When talking to Colin he mentioned he prefers to keep the width, and likes the idea to provide a 2048 pixel version as “HD” movie, so I suggest we go for that.
    A scaled down 720p and 480p will also be made, including versions on youtube etc. We won’t letterbox frames.

    The 4k version still has to be rendered, a process that’s estimated to take 2 to 2.5 months. It’s being postponed until we’re fully done here the online launch and DVD mastering.

    BTW: I’ve asked DivX about this topic, and got this back from their chief encoder “We can take care of this, we have experience cropping uncompressed sources as well as resizing, but I think cropping is best to preserve quality.” ๐Ÿ˜‰

    That’s also my two cents!

    Anyway, don’t worry for now. The movie itself is worth watching in any way, scaled or cropped alike. Thanks for all feedback, we’re back to all the other todos that are awaiting us… Oh, just 7 days before premiere, yay!

  2. J. Says:

    Of course cropping preserves the quality the best, but artistically it’s kind of not done.

    And yes, the premiere, can’t wait! (projected in BlenderScope ๐Ÿ˜€ ๐Ÿ˜€ ๐Ÿ˜€ )

  3. kardargo Says:

    First of all, its great to read that there will be no letter-boxing on the HD-versions.

    I did a quick test with a 2048k version of BBB (scaled to 2048×872, x264-codec with bitrate set to 10.000 in an mkv-container) in order to see if it would play on the popcornhour (c200).
    I can report, that it played with just one little hick-up (but i think that was due to some bad rendering of x264, since this also occurred in VLC and Mplayer at exactly the same point).
    So it seams that at least the sigma-based media-players should play the 2048k version just fine.

  4. Scott Says:

    Ton Says:
    September 20th, 2010 at 20:05
    When talking to Colin he mentioned he prefers to keep the width, and likes the idea to provide a 2048 pixel version as โ€œHDโ€ movie, so I suggest we go for that.

    Great!

  5. gundampilot001 Says:

    LOGAN,

    OGG, WebM and Dirac are not there just because “we can”, but because they are the only unencumbered by patents. Quality and compatibility isn’t the only thing that matters. There are tons of information on this issue in the Internet, and you should probably read some of it. For starters:

    http://www.fsf.org/news/blogs/community/google-free-on2-vp8-for-youtube
    http://www.fsf.org/news/free-software-foundation-statement-on-webm-and-vp8

  6. Pete Says:

    My two paid-for cents are that I trust the Durian folks. The 2k version sounds like a good compromise for me to use since I know how to set scaling and cropping in my player of choice, but I would also be perfectly happy with a 1080 crop. As I see it, that 3% on either side REALLY doesn’t matter for framing, unless Sintel spends all her time halfway out-of-frame…

    And to give the neglected audio side some attention, I would absolutely love to see a stereo FLAC track make it to the DVD, but as long as that’s available alongside the PNGs and Jan and everyone are happy with the on-disc audio quality, I won’t have any reason to complain.

    Thank you, Beorn, for taking on this thankless task!
    -Pete

  7. weilynncg Says:

    I am not really skilled at encoding files, but I know a free software that encode files well (as I eared) which is “handbreak”. For subtitles I am afraid to be not very helpful in the aim of using free softwares, I’m not really really aware with them… Sorry.

    Excuse my poor english. I hope I could have help you.

  8. FishB8 Says:

    Everybody is talking about the quality with scaling. If you want good quality in scaling, it should probably be done in a 32bit linear color space. Most of the scaling routines listed thus far are done in 8bit sRGB (or even YCbCR colorspace *ick*) and that presents issues with creating aliased lines and gamma error issues. You would probably get best quality by doing the scaling for each destination format by creating PNG sequences scaled with imagemagick (or graphicsmagick). You can create scripts for them and they can do the scaling in float linear color space.

    Interesting read: http://www.4p8.com/eric.brasseur/gamma.html

  9. John Bรคckstrand Says:

    Is mplayers built-in scaling good (if thats whats used)? Say, as good as imagemagicks scaling which actually filters them before downscaling?

    If not thats bad.

  10. Spudd86 Says:

    Might be a good idea to look into doing a WebM version, open codec, better video compression than Theora.

    Also for Theora you could email the developers at Xiph, they are pretty accessible, probably Monty would be a good person to talk too: monty@xiph.org (got that from http://www.xiph.org/contact/ ) he knows Theora inside out.

  11. tommy Says:

    Spudd86 i agree, xiph even supports webm!
    WebM (VP8+Vorbis) is superior compared to Ogg Theora (VP3+Vorbis)

    its also supported by Chrome 6, Opera 10.60, Firefox 4 and even Internet Explorer 9 with webm codec installed

  12. Martin Says:

    WebM/VP8 is a part of Ubuntu 10.10, Fedora 13, Fedora 14, and current Debian testing.

    Ubuntu 10.10 just upgraded to the latest 0.9.2 release of the libvpx vp8 library. http://www.ubuntuupdates.org/packages/show/211504

    Fedora 13 and Fedora 14 includes the 0.9.1 version of libvpx at the moment.

    Ubuntu, Debian, and Fedora communities supporters versions of gstreamer, vlc, ffmpeg, mplayer, web browsers, and other applications with VP8 and WebM support.

    libvpx 0.9.2: https://groups.google.com/a/webmproject.org/group/webm-discuss/browse_thread/thread/affaf0069c199ca4

  13. Uan Guz Says:

    Congratulations to the Durian OMP & Blender Foundation for the Sintel Premiere!!!

    I can not wait to see the movie when I can, whether online or when I get my own DVD.

    Greetings from a mexican Blender fan & visual artist who learn Blender every day and hope some day can show my 3D stuff to you!

  14. moncler uk Says:

    Hi there, You have done a fantastic job. I’ll definitely digg it and individually recommend to my friends. I am sure they will be benefited from this website.