Transfer and transcode to iPod/iPhone
This post describes how I copy video files from my Humax 5050C DVR, and how the result is transcoded for use on my iPod/iPhone/Apple TV. And: how these two things are done in parallel, to save time (in my case: 35%) and disk space.
It has been a hobby project putting multiple pieces together. As with many things in life, the trip itself is often more interesting than the destination. So, I decided to share the details of the trip. If you are interested in the destination: you can download the complete (be it proof-of-concept) script as well. For that, scroll to the bottom of the post.
Suggestions, thoughts, etc? Let me know and/or leave a comment.
Pipe FTP transfer to ffmpeg
Not interested in all of the details?
To pipe the output of the FTP transfer to ffmpeg, I use:
wget -q "${url}" -O - | ffmpeg -i - $ffsettings -y "${ofile}"
For an explanation, read on.
Objective
- copy video file from DVR to computer
- transcode to iPod/iPhone compatible format
- minimize time needed for these two actions by combining them
- add metadata
My environment
- Humax 5050C DVR
- Apple iMac (quad core)
- MacPorts on the iMac (which in turn requires Xcode, Apple’s development environment)
Workflow
- use free (and excellent!) FTP application Cyberduck to select the file I want to transfer (to be optimized in the future: either select from command line or use watch folder)
- copy file from Humax 5050C using FTP (using wget)
- pipe the output of the FTP transfer to ffmpeg, which transcodes to H.264 format
- use iFlicks to add the metadata
Alternatives
I am sure there are plenty of alternatives to the above workflow. It was intened as a hobby project to find out more about transfers and transcoding.
From the looks of it (have not downloaded it), Video converter for Mac is a user friendly GUI way to convert,
the script on Slated looks more polished than mine (but no remote transfer), although I did not want to bother with the Perl and gpac requirements.
Minimize time needed for FTP and transcode
This was the main reason for starting this project.
Two benefits: 1) no large file stored locally and 2) while the transfer is taking place, the transcoding is being done in parallel, which saves time.
The actual result depends on a) the transfer speed between Humax and iMac (in my case: slow) and b) the transcoding speed of the (in my case) iMac. For me typical results: 35% faster and the 8GB source file becomes a 900MB file.
Numbers
I used a MPEG-2 SPTS file with duration 2:14:57 and size 8.1GB (resolution 720×576, 15000 kb/s video, 192kb/s audio) and measured the clock times needed for both transfer and transcode. The result: a 920MB H.264 file.
- sequential: transfer (0:59:13 @ 2.17MB/s) + transcode (0:32:14) = 1:31:27 (ffmpeg: ~100fps, ~900kbits/s)
- parallel: transfer + transcode = 0:58:49 (ffmpeg: ~53fps, ~900kbits/s)
So: 59 minutes instead of 91 minutes, a saving of 35%.
Humax 5050C details
The Humax 5050C has an FTP interface, to access use user humaxftp and as password use the admin PIN (default: 0000).
Files are stored as single program transport streams (SPTS), as an indication: a 2 hour HD file is about 7.5GB.
Related: for Windows users, there is a tool to send files from and to the Humax 5050C. Even when not using the tool, the man page is an excellent read to find out more about the internals.
Easy FTP access
I have put the credentials in a .netrc file, this way the client application does not need to ask for the username + password. Make sure you set the right permissions, otherwise the file will be ignored.
In my case:
$ more ~/.netrc
machine 10.0.0.132
login humaxftp
password 0000
$ chmod 600 ~/.netrc
Text encoding on the HUMAX 5050C
Two issues:
- use ISO-8859-1 as the text encoding (for files like “Top 2000 à gogo.ts”)
- wget does not like character codes %28 and %29 (but no problems with e.g. %20 for space and international characters)
The Humax 5050C uses ISO-8859-1 as text encoding, make sure your client uses the proper text encoding, otherwise it will have a problem retrieving files with international characters.
Here are my Cyberduck bookmark settings:
The second item to be aware of: wget (or Humax?) does not like it when the URL contains %28 instead of “(“. When I use Cyberduck to copy the URL, it looks like:
ftp://humaxftp@10.0.0.132/Video/Top%202000%20%C3%A0%20gogo%20%28NPS%29_20091227_0053.ts
This works fine for wget, except the %28 and %29 part. After replacing them it works fine (make sure you put the url in quotes):
"ftp://humaxftp@10.0.0.132/Video/Top%202000%20%C3%A0%20gogo%20(NPS)_20091227_0053.ts"
Prepare ffmpeg
The easiest way to install ffmpeg on the Mac is by using Macports.
By default ffmpeg will install without support for H.264, Lame MP3, etc. There is a good description of how to install ffmpeg with the additional libraries.
In short:
sudo port install ffmpeg +gpl +lame +x264 +xvid
Make sure you uninstall or at least deactivate previous ffmpeg installs.
Use pipe between wget and ffmpeg
How to achieve this? Actually, that turned out to be quite simple. In the most basic form:
wget -q "${url}" -O - | \
ffmpeg -y -i - $ffsettings "${ofile}.m4v"
wget:
-q: run wget in quiet mode, no progress bar (otherwise you have two applications producing output at the same time)
-O -: save the result in standard output (=to the pipe)
ffmpeg:
-y: overwrite existing files
-i -: get the input from standard input (=result of the pipe)
ffmpeg settings
Now the most subjective part, what ffmpeg settings to use to convert to H.264 that can be used on the iPod/iPhone?
There is no right or wrong here, it really depends on your taste and perception. When asking Mr. Google, you get many many different suggestions. I decided to stick with presets as much as possible.
Below is my selection as used in the script, with ffmpeg v0.5, where I have grouped the various settings so it is easier to see what they do:
set ffmisc = "-threads 0" set ffcrop = "-croptop 2 -cropright 6" set ffaudio = "-acodec libfaac -ar 48000 -ac 2 -ab 128k" set ffpreset = "-vpre hq -vpre ipod640" set ffvideo = "$ffcrop -deinterlace -vcodec libx264 $ffpreset -s 640x480 -b 768k" set ffsettings = "$ffmisc $ffaudio $ffvideo"
ffmisc
-threads 0: use all available power, in my case: all 4 cores of the iMac
ffcrop
Remove the ugly pixel row at the top and the black on the right (depends on the source of course, this worked for my Humax files).
ffaudio
Pretty standard settings for the audio.
ffpreset
Many sites have a long list of video specific settings. I decided to use presets, they make life easier… Usually “-vpre normal” is fine, “-vpre hq” takes quite a bit longer. In my case that is not an issue as the FTP transfer speed is the bottleneck anyway. Details of the presets can be found in /opt/local/share/ffmpeg/*ffpreset.
ffvideo
De-interlace the video and use the H.264 codec, use 640×480 as the size and 768kb/sec as the bitrate.
Collect the metadata and add to iTunes
iFlicks (shareware) is a great application to convert videos for iTunes and to add metadata (episode info, covert art, etc.).
Update (Oct 2011): in recent versions of ffmpeg (e.g. v0.7), there is a different preset system for libx264. Instead of e.g. “-vpre hq” you are now supposed to use “-preset hq”.
Unless you are confident the default settings do exactly what you want, it is best to run the GUI. I pass the name of my video file to the iFlicks application using Applescript. I have not figured out how to define application settings in Applescript, which is why I run it with the GUI.
Two tricky parts here:
- Applescript expects absolute path names, so I need to make sure this is done.
- I need to convert the UNIX file path to an Applescript friendly POSIX path.
The snippet to create the absolute path (I am sure there is a prettier way of doing this):
set D = `dirname "$ofile"` set B = `basename "$ofile"` set abspath = `cd "$D"; pwd`/$B
To call the Applescript from within the shell script:
/usr/bin/osascript > /dev/null << EOT set mypath to "$abspath" set macpath to (POSIX file mypath) as string tell application "iFlicks" import macpath with gui end tell EOT
Putting it all together: the script
Click to view the complete script, right-click to save (remove the .txt extension).
[Download not found]
Leuk artikel, ffmpeg parameter ga ik proberen!
Ik gebruik ipv. wget het commando lftp met de mirror actie. Dat werkt feilloos, op een dingetje na.
Ik heb een heel raar issue met de 5050c. Ik zie met ftp (ls) 20 van de 80 files niet. Die 20 files kan ik via de humax interface wel zien en kan afspelen. Gisterenavond heb ik de firmware geupgrade, maar dit effect blijft bestaan.
Heeft iemand iets vergelijkbaars meegemaakt?
Meer een vraag dan een reply.
N.a.v dit artikel CyberDuck voor mijn Mac gedownload om de pas aangeschafte Humax5050c te kunnen ftp’en.
Bij maken verbinding wordt door CyberDuck de volgende foutmelding afgegeven en de verbinding komt niet tot stand:
listing directory failed
/media r-xr-x Music
Als ik echter in Terminal verbinding maak dan geeft ls voor de Music directory gewoon: drwxr-xr-x 2 root root 4096 Jan 01 00:00 Music
Enig idee wat hier fout kan zijn gegaan? (text transfer heb ik als aangegeven al op ISO-8859-1 gezet.
Vast bedankt voor de moeite.
Hans van der Meer
Ik krijg dezelfde melding als ik het “Path” veld leeg laat bij m’n humax bookmark. Als ik “/Video” gebruik als standaard pad, dan is er niets aan de hand, en kan ik ook gewoon naar de Music map. Zie screenshot in het artikel voor mijn instellingen.
Thanks for the hints! This will enable me to use our 5050c properly!