Recent Changes - Search:
Project /

DVD Ripping

Notes specific to source discs:
Teaching Company
Commercial DVDs
On-Demand DVDs
Editing DVD content
DVD audio ripping
Quad-Ripper
MythTV

Notes specific to target device:
Nook Color
Nook Tablet HD+
Machspeed TRIO

General Notes about DVD discs

References:

In my work I've developed an understanding of DVD layout using the following definitions:

  • "TITLE" refers to a VTS_xx_y.VOB set of files under /VIDEO_TS/
  • "CHAPTER" refers to subsections of a "TITLE"
  • If a "TITLE" has a menu, it is found in VTS_xx_0.VOB and should not be included in a conversion. Only VTS_xx_1.VOB and above contain audio/video data.

Tools

I use Puppy Linux to do my work, and I found a very helpful PET containing 'lsdvd' which describes a DVD disc's layout, lsdvd-0.16-p5-20101216.pet.

I have found the following commands useful:

  • lsdvd -x /dev/sr0
  • mplayer dvd://<title> -dvd-device /dev/sr0 -chapter <first>-<last> -alang en -slang off -dumpstream -dumpfile <VOB filename>
  • ffmpeg -y -threads 8 -i <infile> -b 800k -bt 1000k -vcodec mpeg4 -aspect <ratio: 4:3 16:9> -acodec libfaac -ac 2 -ar 44100 -ab 128k -f mp4 <outfile>
  • ffmpeg -y -threads 8 -i <infile> -b 800k -bt 1000k -vcodec libx264 -vpre default -vpre baseline -aspect <ratio: 4:3 16:9> -acodec libfaac -ac 2 -ar 44100 -ab 128k -f mp4 <outfile>
  • ffmpeg -y -i <infile> -itsoffset <secdelay> -i <infile> -map 0:0 -map 1:1 -vcodec copy -acodec copy <outfile>

Teaching Company

The TC has used more than one authoring system to produce their DVDs. I have come across the following arrangements:

  1. each lecture is contained in its own "Title" (VTS_xx_y.VOB set)
    • Argumentation
    • Art of Critical Decision Making
    • Building Great Sentences
    • Change and Motion
    • Classical Archeology of Ancient Greece and Rome
    • Early Middle Ages
    • Experiencing Hubble
    • Foundations of Western Civilization
    • Games People Play
    • Great Ideas of Philosophy
    • Great Minds of the Western Intellectual Tradition
    • High Middle Ages
    • Joy of Mathematics
    • Late Middle Ages
    • Latin 101
    • Mathematics from the Visual World
    • Mathematics, Philosophy, and the Real World
    • Particle Physics for Non-Physicists
    • Quantum Mechanics
    • Understanding Calculus
    • Writing Great Fiction
  2. each lecture is a series of chapters within a single "Title" on the entire DVD disc (VTS_01_y.VOB)
    • From Monet to Van Gogh
    • Great Authors of the Western Literary Tradition
    • History of Ancient Egypt
    • Italian Renaissance
    • Understanding the Universe
    • Understanding the Universe, What's New in 2003

I used lsdvd to discover the layout of each course's discs.

NOTE: When working with History of Ancient Egypt and other Type 2 courses I was not able to extract multiple sequential chapters at one time successfully. At the end of each chapter mplayer would terminate with a core dump, thus I had to extract each chapter on its own, then I could cat the set of chapters into ffmpeg for a single MP4 conversion. Unfortunately, this resulted in a momentary blanking at the join points and the audio was not synced with video. Each join's blank exacerbated the out-of-sync problem. This same problem resulted in both the case where the chapters were extracted to VOBs and joined, then converted to MP4, and when each chapter VOB was first converted to MP4, then joined. Ultimately my best (though hardly perfect) solution was to combine all the VOBs together, converting into a single, huge MP4, then extract the individual titles using ffmpeg with -ss and -endpos options. Unfortunately, even with this solution, which eliminates the blanking problem, the audio is still not in sync, and I had to use the -itsoffset option also. Judgment is called for in determining the audio offset and it didn't always result in accurate syncing. '''This problem has resolved with fine-tuning to the ffmpeg commands in the helper utility tcrip.sh (see next NOTE2).

NOTE2: I wrote a rather involved bash shell script to assist in working with both types of authoring called tcrip.sh. It helps to automate all of the steps to extract, convert, and name each lecture and associated files.

NOTE3: The script tcrip.sh works only if the DVD is copied to the hard disk, first. The steps for a Class 1 type of DVD from the Teaching company are:

  1. insert the DVD into the drive
  2. mkdir XXXX (i.e., LATIN_101 or other appropriate course name)
  3. cd XXXX (each DVD will be copied into this directory, into a subdir named from each DVD's disc title)
  4. # tcrip.sh copy /dev/srX
  5. # tcrip.sh play <disc_title>
    1. note the content of each VTS title, creating a .nam file in the DVD's subdir
  6. # tcrip.sh convert <disc_title> OR # tcrip.sh convert16 <disc_title> depending on 4:3 or 16:9 aspect ratio
  7. # tcrip.sh rename <disc_title>

2018-08-15 -- I have been experimenting with better quality rips and with an Intel Core i5 (quad-core) computer, I have discovered that H.264 video encoding produces very nice quality for a much lower bitrate. I have started to re-rip my Teaching Company video collection using this stop-gap command on Puppy Linux 5.7.1: 

mplayer dvd://<vtsnum> -dumpstream -dumpfile /dev/stdout | \
vgtmpeg -threads 16 -y -i - \
  -vcodec libx264 -preset medium -vf scale=480:360 -aspect 4:3 \
  -acodec libfaac -ab 128k \
  -f mp4 <outfile>.mp4

I need to update tcrip.sh to adapt to this new encoding method, AND, perform all its functions without having to first copy the DVD to local hard disk.

Commercial DVDs

I have found that many commercial DVDs have purposeful encryption/corruption beyond the CSS encoding that makes the re-encoding with ffmpeg produce unusable files. The key, I found, was to first 'dump' the title using mplayer, which overcomes this extra layer of encryption.

Another aspect of commercial DVDs that surprised me is that some contain multiple 'titles', not just chapters, within a single VTS_xx_y.VOB fileset. lsdvd shows all of these 'titles', whether they are each a separate VTS fileset or not, and that is the best way to discover what the DVD contains.

Yet another aspect of commercial DVDs that caused trouble initially are the multiple soundtracks and subtitle streams. mplayer doesn't always select the correct ones, and I had a hard time figuring this one out. The only way I found that I could determine to stream indexes was to play the VOB with ffplay and experiment with different streams until the correct stream is identified. Here are the two commands:

  • ffplay $source.VOB -vst $videostream -ast $audiostream -sst 0

Then, with the stream indexes, the solution is to use mplayer specifying the -map option twice, first to indicate the video stream, the second time to indicate the audio stream, and this usage of -map also eliminates subtitles.

So, at first, I used a command to 'extract' each of the 'titles' from the DVD regardless of the VTS fileset structure, each 'title' into its own VOB, like this:

  • mplayer dvd://$title -dvd-device $source -dumpstream -dumpfile $source/${source}_$title.VOB

But this creates yet another pile of VOBs that are fairly large files. And, it is a complete copy of the VOB stream, including all video, audio, and subtitle streams.

I wanted to create a single pass solution, without the intermediate VOB files and allow for the selection of the correct audio stream while eliminating the subtitles. Here are the commands that works in one pass. First, find the correct audio index by trial and error, then do the conversion: 

  • mplayer dvd://$title -dvd-device $source -dumpstream -dumpfile /dev/stdout | \
ffplay - -vst $videostream -ast $audiostream -sst 0

  • mplayer dvd://$title -dvd-device $source -dumpstream -dumpfile /dev/stdout | \
ffmpeg -y -threads 8 -i - \
-map 0:$videostream -map 0:$audiostream -b 800k -bt 1000k \
-vcodec mpeg4 -s vga \
-acodec libvo_aacenc -ac 2 -ar 44100 -ab 128k \
-f mp4 $source/${source}_$title.mp4

Sadly, this method of piping from mplayer to ffmpeg is problematic since there are text messages from mplayer that corrupt the stream being piped to ffmpeg. This method should probably not be used, though it appears to work.

Jerky Video

Many of my commercial DVDs result in jerky video (an extra layer of copy-protection beyond CSS?). mplayer will play the video fine, but when converted with ffmpeg, the resulting mp4 video is jerky. The audio is fine. At first I thought it had something to do with NTSC interlaced format versus non-interlaced (or progressive) format, but this didn't bear out in my many trials. Some interlaced videos exhibited the jerkiness, while others didn't. I now guess that this conversion problem related to the purposeful 'corruption' of the DVD files as another layer of copy-protection, and ffmpeg can't deal with it; perhaps related to which libraries are available to each program, namely libdvdread, and perhaps others.

I struggled with this for days, and I now believe I have a solution. I came across vgtmpeg which is an 'extended' ffmpeg that includes the needed libraries. It is a drop-in replacement for ffmpeg using all of the same command parameters, which makes using it easy. Files converted with vgtmpeg are smooth and all of my tests with it proved successful. The only disadvantage of vtgmpeg is that it requires a CLIB version that is higher than the one on Lucid Puppy 5.2.8. Precise Puppy 5.7.1 does have it and is the platform for using it.
NOTE: vgtmpeg fails with 'invalid instruction' on AMD Athlon+ processor.

Display Aspect Ratios

Many videos have a display aspect ratio that is different from the DVD specification's 720x480. A 4:3 video would render as 640x480, for example, and a widescreen 16:9 video renders as 854x480. There is a Display Aspect Ratio (DAR), some kind of metadata that is somehow encoded in the video, but some of my playback devices do not abide by it. For example, my Palm Pixi Plus smartphone and B&N Nook Color display DVD videos in the encoded pixel by pixel aspect ratio, that is 720x480 rather than the desired 640x480 (DAR 4:3) or 854x480 (DAR 16:9). The solution is to 'scale' the video when encoding the MP4. This is done in ffmpeg or vtmpeg using the video filter 'scale', ala:

  • ... -vf "scale=640:480" ... OR ... -vf "scale=854:480"

Cropping letter-boxed videos

Another important issue when converting videos is dealing with letter-boxed or black-framed videos. I have several commercial DVDs that have the image (encoded at 720x480, of course) with black bars or black frames as part of the encoded data. It is desirable to remove these black regions when encoding to MP4 so that the video is displayed in it's fullest width or height on alternate devices. This is accomplished in a two-step process. First the black regions are identified using the video filter 'cropdetect', then the video is cropped during the conversion to MP4 using the video filter 'crop':

  • ... -vf "cropdetect=24:16:0" ... (temporary conversion to dummy output file)
  • ... -vf "crop=w:h:x:y" ...

Combining 'scale' and 'crop' then produce a completely clean and full-display video file. The easiest way is to use the cropdetect figures along with the DAR to generate the scale figures. That is, use original width minus double the crop, 720-2x, and original height minus double the vertical crop, 480-2y to arrive at the sample dimensions. Then apply the Display Aspect Ratio to either height or width, which ever is most appropriate. For example, if the cropdetect returns crop=704:272:8:102 with a DAR of 16:9 then cropped size is 720-(8*2):480-(102*2), that is, 704:276. A DAR of 16:9=1.77778, and we apply this to the width (704*1.77778) giving 396, so our scale values are 704:396. The entire video filter definition would be:

  • ... -vf "crop=704:272:8:102,scale=704:396" ...

Subtitles

Generally, subtitle streams do not start until the first bit of dialog, which can be many minutes into the film. ffmpeg and vgtmpeg only analyze the first 1/2 second of the stream by default which is much to short for detecting the subtitle stream. A longer analyzing period is needed, perhaps using a value of '300000000' (5 minutes) with the following:

  • ... -analyzeduration <millisecs> -probesize <millisecs> ...
NOTE: these options must appear before the -i <inputfile> element.

Also, adding the stream will require specifying the scodec and a third map element:

  • ... -scodec copy ... -map 0:<substream> ...

"On-Demand DVDs

Some DVDs are burned onto Write-Once DVDs and have an unreadable menu structure by mplayer. A work around (other than the MythTV method below) is to simply dump the dvd sector-by-sector to a file. These "From the Vault" or "From the Archive" DVDs often have the menu, if there is one, and studio splash screen at the beginning of the DVD sectors. Once the movie start point is located, a dump from that point to the 'end' of the DVD is all that is needed.

Locating the beginning of the movie is a bit fiddly, but this method works:

  • use this command, adjusting the skip=XXX value to locate the beginning of the main feature:
    • dd if=/dev/sr0 bs=16384 skip=XXX | mplayer -
  • once the proper XXX value is determined, copy the movie thusly:
    • dd if=/dev/sr0 bs=16384 skip=XXX > ''movie name''.DD-DUMP.DVD.vob

Editing DVD content

On occasion it is nice to edit a DVD video and then re-burn it to a new DVD disc, perhaps to scale it and/or to crop a four-sided letterbox.

The key is to rip the DVD to .mpg format, scaling and/or cropping at the same time, then remaster the video for burning to a new disc.

A key is to ensure that the video is de-interlaced or it will look terrible when re-encoded and burned to the new DVD disc.

Another key is to ensure that the output resolution is exactly 720x480 (for NTSC) otherwise the disc will not play in a DVD player.

The command to rip and scale/crop that works for me is:

  • ffmpeg -y -threads 8 -i <VOBs> -b 6000k -bt 1000k -vcodec mpeg4 -vf "<crop=widthnew:hightnew:xstart:ystart><,scale=720:480>" -deinterlace -acodec ac3 -ac 2 -ar 48000 -b 256k -target ntsc-dvd <outfile.mpg>

For my work on Abilene Town, which required 4-sided cropping (4-way letterbox) and then rescaling, was:

  • cat /mnt/sr1/VIDEO_TS/VTS_01_?.VOB | ffmpeg -y -threads 8 -i - -b 6000k -bt 1000k -vcodec mpeg4 -vf "crop=528:352:92:56,scale=720:480" -deinterlace -acodec ac3 -ac 2 -ar 48000 -b 256k -target ntsc-dvd abilene.mpg

Once the video has been ripped, and possibly cropped and/or scaled, it is time to re-encode, titled, and burned using these commands:

  • export VIDEO_FORMAT=NTSC
  • dvdauthor -o <outputdir> -t <outfile.mpg>
  • dvdauthor -o <outputdir> -T
  • mkisofs -dvd-video -o <new.iso> <outputdir>
  • growisofs -dvd-compat -speed=1 -Z /dev/sr1=<new.iso>

For my Abilene Town project these commands were:

  • export VIDEO_FORMAT=NTSC
  • mkdir ABILENE
  • dvdauthor -o ABILENE/ -t abilene.mpg
  • dvdauthor -o ABILENE/ -T
  • mkisofs -dvd-video -o abilene.iso ABILENE/
  • growisofs -dvd-video -speed=1 -Z /dev/sr1=abilene.iso

There you have it!

There is no disc menu, the movie just starts playing when inserted into a DVD player.

DVD audio ripping

Ripping audio from DVDs is has similar issues to those listed above in the Commercial DVDs section. Fortunately, ripping only audio is easier and can be done using ffmpeg, without having to resort to using vgtmpeg so all can be performed on Lucid Puppy. Helpful commands include:

  • lsdvd -c
  • mplayer dvd://$track -chapter $chapter_start[-$chapter_end] -dvd-device /dev/sr0 -dumpstream -dumpfile /dev/stdout | ffmpeg -y -threads 8 -i - -map 0:1 -acodec libmp3lame -ac 2 -ar 44100 -ab 128k -f mp3 $output_file
    • $chapter_start and $chapter_end are usually the same (e.g. 5-5) when a single chapter is wanted

Sometimes the desired audio stream, say '2-channel stereo', varies from chapter to chapter (track to track), and the following will automatically select the stereo stream while processing all chapters:

  • track=1 ; for chap in 01 02 03 04 ; do stream=`mplayer dvd://$track -chapter ${chap}-${chap} -dvd-device /dev/sr0 -dumpstream -dumpfile /dev/stdout | ffmpeg -y -threads 8 -i - -map 0:1 -acodec libmp3lame -ac 2 -ar 44100 -aq 3 -f mp3 junkme.mp3` ; mplayer dvd://$track -chapter ${chap}-${chap} -dvd-device /dev/sr0 -dumpstream -dumpfile /dev/stdout | ffmpeg -y -threads 8 -i - -map 0:${stream} -acodec libmp3lame -ac 2 -ar 44100 -ab 128k -f mp3 $output_file ; done

Quad-Ripper

2016-09-11

A multi-drive DVD ripping configuration occurred to me over the past few days of doing DVR-made DVD ripping. I have built a couple of tools, one for Linux, another for Windows, that assists in ripping one-at-a-time DVDs and saving to a file named for the movie and it's year. But it is a tedious process and we have many, many more to go. Thus this concept.

Hardware

  • A standard motherboard with four (4) DVD drives attached to the two IDE ports. System will boot from a USB thumb drive.

Software

  • Puppy Linux version 5.2.8 (or 5.7.1) booted from USB thumb drive
  • A custom automation script that does the ripping:
    • reads a text-file list of titles and years (created/edited before use with script)
    • writes a text-file log of titles ripped
      • Movie title
      • Movie year
      • MD5 hash of the first 10240 bytes of the disc (used latter for keeping script in sync)
    • starting with /dev/sr0
      1. confirms that a disc is in the drive
        • if not, move on to the next drive
      2. calculates the MD5 hash of the disc
      3. scans the log for matching hashes
        • if matched, moves on to the next drive, considering this disc already copied
      4. reads the next(first) name from the 'movie' file
      5. rips the disc to <title>_(<year>).SAT.vob
      6. writes to log: Title, Year, MD5 hash
      7. beep tones to indicate disc ripped, and which drive (1,2,3,or 4)
    • (repeat sequence for next drive (/dev/sr1, etc)

Nook Color

H.264

This is a command that I found on various web postings that is specifically for the Nook Color. Of course, change the input from AVI to VOB if ripping a DVD.

  • ffmpeg -y -threads 8 -i myvideo.avi -b 800k -bt 1000k -vcodec libx264 -vpre default -vpre baseline -acodec libfaac -ac 2 -ar 44100 -ab 128k mynewvideo.mp4

The key is the use of the H.264 encoder. Since the Nook Color has a graphics chip that can decode H.264 video in hardware, this produces excellent video playback with a very reasonable file size, approximately 7MB/minute.

The trouble with H.264 encoding is that it is extremely slow. Here are some transcoding rates, all when running Puppy Linux LuPu 5.2.8:

  • Presario20: 2 frames / sec
  • HP Compaq t4410 : <1 frame / sec
  • Amppro Quad-core: 14 frames / sec

Encoding a 2-hour DVD takes almost seven hours on the Amppro quad-core!

Another downside with the H.264 encoded videos is that they play poorly on a basic computer setup (i.e., Presario20, etc.), since they have no special H.264 hardware.

MPEG4

I did further investigation and experimentation and discovered that the Nook Color plays MPEG4/AAC encoded MP4 files well also. Using this command I produced some test files that turn out to have almost identical file sizes to the H.264 encoded files:

  • ffmpeg -y threads 8 -i <infile> -b 800k -bt 1000k -vcodec mpeg4 -acodec libfaac -ac 2 -ar 44100 -ab 128k <outfile>

The encoding is very fast, only 26 minutes for a 2-hour video on the Amppro quad-core. Thats good! The downside is that the video quality is lower. By looking closely and comparing an H.264 sample with the MPEG4 sample it can be seen. Under normal usage, though, I don't think it will prove to be so bad as to be unusable. I intend to encode all video content in the future using the MPEG4 codec. These files will then play on the Nook Color and a computer as well.

Further experimentation shows that halving the video bit-rate to just 400k produces smaller video files, but the quality is noticeably poor. I'll stick with 800k or better.

Also, the key to making videos that play on the Nook Color is to have the audio settings as given in the above commands. Using audio bit-rates higher than about 196k/s creates a file that the Nook Color refuses to play. The 128k setting that I use is sounds perfectly good.

Nook Tablet HD+

Uses the same settings as for the Nook Color.

One gotcha that I have yet to sort out is that files that play well (with correct aspect-ratio) on the Nook Color, also play on the HD+, but the aspect is wrong, almost a perfect square, rather than the 4:3 or 16:9 that shows on the Nook Color. Further experimentation with aspect-ratio settings in the FFMPEG command may correct this issue.

Machspeed TRIO

The TRIO is one of the so-called 'Chinese' MP3 players, aka 's1mp3'. It has a small LCD screen which can be used to show video content at 160x120 resolution. These 's1mp3' players can only play AMV-formatted videos.

The unit came with a Windows-based tool (or at least a link for downloading) for converting, but it didn't work at all. I never had a successful execution of that tool. I found an online site that would do the conversion, but it placed a watermark on each frame. Ultimately, I learned that there is a version of FFMPEG that is purpose-built with the AMV encoder statically linked in. Alas, this version of FFMPEG is from 2007, so is pretty old, and it can't directly decode DVD VOB files. So, my method consists of first converting the VOB to AVI using the standard FFMPEG, then converting the AVI to AMV using the purpose-built FFMPEG.

Using Puppy Linux 5.2.8, here are the two commands used to convert VOBs to AMVs:

  • ffmpeg -i <infile>.vob -s qqvga -vcodec mpeg4 -r 16 -acodec mp2 -ac 1 -ar 22050 -qmin 3 -qmax 3 -y -f avi <tempfile>
  • amv-ffmeg -i <tempfile> -s qqvga -r 16 -ac 1 -ar 22050 -qmin 3 -qmax 3 -y -f amv <outfile>.amv

NOTE: the size parameter (-s qqvga) is shorthand for 160x120.

NOTE 2: converting 16:9 video requires the image to be 'letterboxed'. This is done with this command:

  • ffmpeg -i <infile>.vob -vf "scale=160:90, pad=160:120:0:15, setdar=4:3" -vcodec mpeg4 -r 16 -acodec mp2 -ac 1 -ar 22050 -qmin 3 -qmax 3 -y -f avi <tempfile>

I have written a shell script that will perform these two conversions with a single command, called vob2amv.sh, which currently lives on the 'music' storage disk on Puppy Player TATER.

MythTV

Some 'from the archive' DVDs are burned on a one-by-one basis, using what appear to be DVD-RW media, though they have a professional silkscreened label. Some of these are unreadable in a computer (PuppyLinux 5.7.1 "Precise") and I have resorted to playing the disc through a standard DVD player (which does fine) and capturing the output via MythTV. There is some loss of quality, as would be expected, but it is sufficiently good to consider this a non-issue.

When re-assembling the MythTV rig, here are the steps:

  1. connect video from DVD player to Video-In on Hauppauge card in Myth box (Silver), using RCA-to-RCA patch cable
  2. connect audio from DVD player to Audio-In on Hauppauge card in Myth box, using RCA-to-Mini cable
  3. connect speakers or headphones to Audio-Out on Myth box's sound card
  4. connect monitor to VGA or DVI Out on Myth box's video card