Bembel-B Blog

2008/03/19

Fixed-point Ogg Vorbis and Musepack Decoder Packages for Debian Etch ARM

I’m using Firefly Media Server on a Linksys NSLU2 running Debian Etch to stream my whole music collection to my Pinnacle SoundBridge HomeMusic. Many files in my collection are in Ogg Vorbis format and a few in Musepack too, which unfortunately aren’t natively supported by the SoundBridge. Luckily Firefly can transcode audio files on the fly e.g. using the ssc plug-in and any console application via shell scripts. But the problem with the Debian packages is that they don’t take into account the missing floating point unit (FPU) of the NSLU2 hardware, and that makes oggdec/ogg123 and mpc123 take ages – way below real-time – to transcode. Fixed point versions of the transcoders aren’t available in the Debian Etch package repositories even the sourcecode already exists, so I had to build them myself.
Linksys NSLU2
I first searched the Debian packages for fixed point versions.
I found the libvorbisidec package, which is the fixed point version of libvorbis also going by the name Tremor. The vorbis-tools is only available as normal floating-point version linked to libvorbisdec. For Musepack there are (floating-point) packages libmpcdec3 for the library and mpc123 for the application.
In the Firefly forums I found patches for vorbis-tools (providing the ogg123 and oggdec transcoders), so that I could build Tremor versions of the transcoder apps.
In the Musepack sources there’s already a define variable to build with fixed point math only. So I would just have to enable that and rebuild the library and transcoder.

To make a long story short, here are the Debian Etch ARM binary packages for fixed-point Ogg Vorbis and Musepack:

I will gladly provide the sources. Feel free to ask for them! You can download the Deb sources for libmpcdec and vorbis-tools further below.

Update

These are the integer Lenny ARM binaries. Libvorbisidec is available through the official repos.:

Update 2

Here are the Deb sources (.dsc, .diff.gz and .orig.tar.gz) to build your own binary Deb packages.

Performance now is very good on my 266 MHz NSLU2. From what I’ve seen in top, mpc123 and ogg123 stay below 50% CPU usage and decoding is done faster than real-time.

Only to show off how unbelievably cool I am ;), and in case somebody would like to reproduce these builds on another distribution, I’ll roughly describe the steps I’ve taken.

Build libvorbisidec

Pretty straight forward. As far as I remember I only had to rebuild the source package:

mkdir -p ~/debuild/libvorbisdec
cd ~/debuild/libvorbisdec
dget -x http://ftp.de.debian.org/debian/pool/main/libv/libvorbisidec/libvorbisidec_1.0.2+svn14261-1.dsc
cd libvorbisidec-1.0.2
debuild -rfakeroot -uc -us
cd ..
sudo dpkg -i libvorbisidec1_1.0.2+svn14261-1_arm.deb libvorbisidec-dev_1.0.2+svn14261-1_arm.deb

Patch and build vorbis-tools

This one was trickier. The patches found on the forum aren’t complete. The older one is blindly changing the endianess of the resulting audio data, thereby producing only noise on the little endian Debian I use. The newer one misses linking mpcdec to libvorbisidec.

mkdir -p ~/debuild/vorbis-tools
cd ~/debuild/vorbis-tools
dget -x http://ftp.de.debian.org/debian/pool/main/v/vorbis-tools/vorbis-tools_1.1.1-6.dsc
cp -pr vorbis-tools_1.1.1 vorbis-tools_1.1.1.wrk
cd vorbis-tools_1.1.1.wrk
[apply and manually fix the tremor patch]
[create tremor patch file against vorbis-tools_1.1.1/]
[copy patch file to vorbis-tools_1.1.1/debian/patches]
[add patch file at bottom of vorbis-tools_1.1.1/debian/patches/series]
[add libvorbisidec-dev dependency to vorbis-tools_1.1.1/debian/control and document Tremor versions in description]
[increase version and add changes in changelog via dch -i]
cd vorbis-tools_1.1.1
debuild -rfakeroot -uc -us
cd ..
dpkg -i vorbis-tools_1.1.1-7_arm.deb

Patch and build libmpcdec

Here I add a patch to enable the fixed-point define. Beware, the following steps are just written down from memory in a hurry. So expect flaws. :)

mkdir -p ~/debuild/libmpcdec
cd ~/debuild/libmpcdec
dget -x http://ftp.de.debian.org/debian/pool/main/libm/libmpcdec/libmpcdec_1.2.2-1.dsc
cp -pr libmpcdec-1.2.2 libmpcdec-1.2.2.wrk
vim libmpcdec-1.2.2.wrk/include/mpcdec/math.h
[uncomment #define MPC_FIXED_POINT]
mkdir libmpcdec-1.2.2.wrk/debian/patches
diff -Naur libmpcdec-1.2.2 libmpcdec-1.2.2.wrk > libmpcdec-1.2.2.wrk/debian/patches/enable_fixed_point.diff
cd libmpcdec-1.2.2.wrk
dch -i
[increases version in changelog. add some notes.]
debuild -rfakeroot -uc -us
cd ..
dpkg -i libmpcdec3_1.2.2-2_arm.deb libmpcdec-dev_1.2.2-2_arm.deb

Build mpc123

This one has only to be rebuilt with the fixed-point libmpcdec version above installed.

mkdir -p ~/debuild/mpc123
cd ~/debuild/mpc123
dget -x http://ftp.de.debian.org/debian/pool/main/m/mpc123/mpc123_0.2.1-1.dsc
cd mpc123-0.2.2
debuild -rfakeroot -uc -us
cd ..
dpkg -i mpc123_0.2.2-1_arm.deb

I haven’t found a fast enough way to transcode Monkey’s Audio (aka. APE) files yet, but that’s mainly because I don’t have any (which haven’t been transcoded to mp3 by myself already). For your information: Building the Monkey’s Audio Codec (MAC) for Linux did work.
I think when looking for more low resource or fixed-point codecs, a good place to look for is the Rockbox Wiki. That’s what I used too.

To make this Firefly Transcoding post more complete, here’s my current version of the ssc script:

#!/bin/sh
#
# script to facilitate server-side transcoding of ogg files
#
#
# Usage: mt-daapd-ssc.sh   
#
# You may need to fix these paths:
#

WAVSTREAMER=/opt/mt-daapd-ssc/wavstreamer
OGGDEC=oggdec
OGG123=ogg123
FLAC=flac
MPC=mpc123
FFMPEG=/opt/mt-daapd-ssc/ffmpeg-quiet.sh
MAC=mac

ape_file() {
    $MAC "$FILE" - -d | $WAVSTREAMER -o $OFFSET $FORGELEN
}

ffmpeg_file() {
    $FFMPEG -i "$FILE" -acodec pcm_s16le -f wav - | $WAVSTREAMER -o $OFFSET $FORGELEN
}

mpc_file() {
    $MPC --quiet --wav - "$FILE" | $WAVSTREAMER -o $OFFSET $FORGELEN
}

ogg_file() {
#    $OGGDEC --quiet -o - "$FILE" | $WAVSTREAMER -o $OFFSET $FORGELEN
    $OGG123 -q -d wav -f - "$FILE" | $WAVSTREAMER -o $OFFSET $FORGELEN
}

flac_file() {
    $FLAC --silent --decode --stdout "$FILE" | $WAVSTREAMER -o $OFFSET $FORGELEN
}

FILE=$1
OFFSET=${2:-0}

# this is nonsense!
if [ "$3" != "" ]; then
  FORGELEN="-l ${3%.*}"
fi

case "$1" in
  *.[sS][hH][nN])
  ape_file
  ;;
  *.[sS][hH][nN])
  ffmpeg_file
  ;;
  *.[mM][pP][cC])
  mpc_file
  ;;
  *.[oO][gG][gG])
  ogg_file
  ;;
  *.[fF][lL][aA][cC])
  flac_file
  ;;
  *)
# here you could cat a generic "error" wav...
# cat /path/to/error.wav
  ;;
esac

Oh, FYI I’m using mt-daapd 0.2.4+r1376-1 from the Debian repo. I got the wavstreamer binary and the ssc script example by extracting them from a newer Debian package. The Firefly Wiki provides more information on transcoding implementation.

ChangeLog

[2008-04-11: Add Lenny Debs.]
[2008-06-06: Add Deb sources for libmpcdec and vorbis-tools.]
[2009-04-26: Fix box.net links.]

Advertisements

10 Comments »

  1. This looks as if it might solve my current problem (rhythmbox crashes my mt-daap server whenever it asks for an OGG file) but I am baffled by the references to wavstreamer and ffmpeg-quiet.sh neither of which came with the mt-daap i installed from debian package, version 0.9~r1696-1.3 — where do i get them from?

    Comment by Lou — 2008/05/08 @ 00:01 | Reply

  2. Hi,

    Thanks for this info, I’ve been looking for something like this for a long time.

    Sadly I run into two problems:

    1) if I use

    $OGG123 -q -d wav -f – “$FILE” | $WAVSTREAMER -o $OFFSET $FORGELEN

    I get “Unable to Play” messages on the SounBridge and the mt-daap log has ‘Got CLD Signal. Reaping’ error messages.

    2) if I use the original line

    $OGGDEC –quiet -o – “$FILE” | $WAVSTREAMER -o $OFFSET $FORGELEN

    the ogg files play but all I get is white noise, which would suggest endianess problems.

    FYI My slug runs on Debian Lenny.

    I’d be grateful for any ideas you might have.

    Cheers
    Emile

    Comment by MrE — 2008/05/17 @ 19:05 | Reply

  3. Hi Lou, sorry for my late answer. The Aksimet Comment Spam Filter installed on wordpress.com blocked your comment, so I “accidently” saw that today.

    Wavstreamer is part of the mt-daapd unstable and nightly releases found at http://nightlies.fireflymediaserver.org/, but not included in the official Lenny packages. I think I extracted it from the nightly little-endian ARM Deb for Debian Etch.

    That ffmpeg-quiet.sh isn’t needed. It was just some part of my experiments using a ffmepg wrapper script (with i/o redirects or fifos) for transcoding, as ffmpeg doesn’t have a –quiet paramter. I didn’t take a deeper look into using ffmepg. I don’t think there are fixed-point builds around.

    Another thing to keep in mind, the Lenny Deb of mt-daapd comes with the ffmpeg transcoding plugin. As far as I understand, this will prevent the ssc script plugin from being run. I simply moved that ffmpeg .so out of the plugin directory, as with the mt-daapd version I’m currently running, there’s no way to enable/disable certain plugins with the config file.

    Comment by FrankZabbath — 2008/05/18 @ 22:59 | Reply

  4. Hi Emile, what version of ogg123 and oggdec are you running? Did you install the integer ones I provide? These are running fine for me with the mt-daapd-ssc.sh script. It would surely be better to use oggdec instead of ogg123, as it is more lightweight.

    Here’s some version info from my Debian Lenny NSLU2 system:
    scheff@nslu2:~$ ogg123 –version
    ogg123 (w/Tremor hack) from vorbis-tools 1.2.0
    scheff@nslu2:~$ oggdec –version
    oggdec (Tremor) from vorbis-tools 1.2.0
    scheff@nslu2:~$ dpkg-query –list “*ogg*”
    Desired=Unknown/Install/Remove/Purge/Hold
    | Status=Not/Inst/Cfg-files/Unpacked/Failed-cfg/Half-inst/trig-aWait/Trig-pend
    |/ Err?=(none)/Hold/Reinst-required/X=both-problems (Status,Err: uppercase=bad)
    ||/ Name Version Beschreibung
    +++-==============-==============-============================================
    ii libogg-dev 1.1.3-3 Ogg Bitstream Library Development
    ii libogg0 1.1.3-3 Ogg Bitstream Library
    ii liboggflac-dev 1.1.2-8 Free Lossless Audio Codec – C development li
    ii liboggflac3 1.1.2-8 Free Lossless Audio Codec – runtime C librar
    scheff@nslu2:~$ dpkg-query –list “*vorbis*”
    Desired=Unknown/Install/Remove/Purge/Hold
    | Status=Not/Inst/Cfg-files/Unpacked/Failed-cfg/Half-inst/trig-aWait/Trig-pend
    |/ Err?=(none)/Hold/Reinst-required/X=both-problems (Status,Err: uppercase=bad)
    ||/ Name Version Beschreibung
    +++-==============-==============-============================================
    ii libvorbis-dev 1.2.0.dfsg-3 The Vorbis General Audio Compression Codec (
    un libvorbis0 (keine Beschreibung vorhanden)
    ii libvorbis0a 1.2.0.dfsg-3 The Vorbis General Audio Compression Codec
    ii libvorbisenc2 1.2.0.dfsg-3 The Vorbis General Audio Compression Codec
    ii libvorbisfile3 1.2.0.dfsg-3 The Vorbis General Audio Compression Codec
    ii libvorbisidec- 1.0.2+svn14261 Integer-only Ogg Vorbis decoder, AKA “tremor
    ii libvorbisidec1 1.0.2+svn14261 Integer-only Ogg Vorbis decoder, AKA “tremor
    ii vorbis-tools 1.2.0-2 several Ogg Vorbis tools
    scheff@nslu2:~$ grep -i ogg /opt/mt-daapd-ssc/mt-daapd-ssc.sh
    # script to facilitate server-side transcoding of ogg files
    OGGDEC=oggdec
    OGG123=ogg123
    ogg_file() {
    # $OGGDEC –quiet -o – “$FILE” | $WAVSTREAMER -o $OFFSET $FORGELEN
    $OGG123 -q -d wav -f – “$FILE” | $WAVSTREAMER -o $OFFSET $FORGELEN
    ogg_file
    scheff@nslu2:~$ dpkg-query –list mt-daapd
    Desired=Unknown/Install/Remove/Purge/Hold
    | Status=Not/Inst/Cfg-files/Unpacked/Failed-cfg/Half-inst/trig-aWait/Trig-pend
    |/ Err?=(none)/Hold/Reinst-required/X=both-problems (Status,Err: uppercase=bad)
    ||/ Name Version Beschreibung
    +++-==============-==============-============================================
    ii mt-daapd 0.9~r1696-1.3 iTunes-compatible DAAP server

    Oh, and you might need to disable the ffmepg transcoding plugin, that comes with some mt-daapd builds. Read the last paragraph of my previous comment and the Transcoding HOWTO in the Firefly Wiki.

    Comment by FrankZabbath — 2008/05/18 @ 23:07 | Reply

  5. Hi, thanks for your reply.

    Yes, I did install the versions of the

    aerion@YGGDRASIL:~$ oggdec –version
    oggdec (Tremor) from vorbis-tools 1.2.0

    aerion@YGGDRASIL:~$ ogg123 –version
    ogg123 (w/Tremor hack) from vorbis-tools 1.2.0

    aerion@YGGDRASIL:~$
    Desired=Unknown/Install/Remove/Purge/Hold
    | Status=Not/Inst/Cfg-files/Unpacked/Failed-cfg/Half-inst/trig-aWait/Trig-pend
    |/ Err?=(none)/Hold/Reinst-required/X=both-problems (Status,Err: uppercase=bad)
    ||/ Name Version Description
    +++-=============================-=============================-==========================================================================
    ii libogg0 1.1.3-3 Ogg Bitstream Library
    rc liboggflac3 1.1.2-8 Free Lossless Audio Codec – runtime C library (ogg)

    aerion@YGGDRASIL:~$ dpkg-query –list ‘*vorbis*’
    Desired=Unknown/Install/Remove/Purge/Hold
    | Status=Not/Inst/Cfg-files/Unpacked/Failed-cfg/Half-inst/trig-aWait/Trig-pend
    |/ Err?=(none)/Hold/Reinst-required/X=both-problems (Status,Err: uppercase=bad)
    ||/ Name Version Description
    +++-=============================-=============================-==========================================================================
    un libvorbis0 (no description available)
    ii libvorbis0a 1.2.0.dfsg-3 The Vorbis General Audio Compression Codec
    ii libvorbisenc2 1.2.0.dfsg-3 The Vorbis General Audio Compression Codec
    ii libvorbisfile3 1.2.0.dfsg-3 The Vorbis General Audio Compression Codec
    ii libvorbisidec1 1.0.2+svn14261-1 Integer-only Ogg Vorbis decoder, AKA “tremor”
    ii vorbis-tools 1.2.0-2 several Ogg Vorbis tools

    aerion@YGGDRASIL:~$ dpkg-query –list mt-daapd
    Desired=Unknown/Install/Remove/Purge/Hold
    | Status=Not/Inst/Cfg-files/Unpacked/Failed-cfg/Half-inst/trig-aWait/Trig-pend
    |/ Err?=(none)/Hold/Reinst-required/X=both-problems (Status,Err: uppercase=bad)
    ||/ Name Version Description
    +++-=============================-=============================-==========================================================================
    ii mt-daapd 0.9-svn-1586 iTunes-compatible DAAP server

    aerion@YGGDRASIL:~$ grep -i ogg /usr/bin/mt-daapd-ssc.sh
    # script to facilitate server-side transcoding of ogg files
    OGGDEC=oggdec
    OGG123=ogg123
    ogg_file() {
    # $OGGDEC –quiet -o – “$FILE” | $WAVSTREAMER -o $OFFSET $FORGELEN
    $OGGDEC -q -d wav -f – “$FILE” | $WAVSTREAMER -o $OFFSET $FORGELEN
    ogg_file

    I don’t seem to have a ffmpeg plugin, maybe that’s because I’m running svn-1586? I tried to install svn-1696 but had to remove it again as the nightly Ron currently provides was not compiled with sqlite3 support. I could of course compile it from source, which I did do for a while, but I got a little fed up with the long time it takes to compile Firefly on the slug :-) so I switched back to the nightlies binary packages that Ron provides.

    aerion@YGGDRASIL:~$ mlocate plugin | grep mt-daapd
    /usr/lib/mt-daapd/plugins
    /usr/local/lib/mt-daapd/plugins
    /usr/share/mt-daapd/plugins
    /usr/share/mt-daapd/plugins/out-daap.a
    /usr/share/mt-daapd/plugins/out-daap.la
    /usr/share/mt-daapd/plugins/out-daap.so
    /usr/share/mt-daapd/plugins/rsp.a
    /usr/share/mt-daapd/plugins/rsp.la
    /usr/share/mt-daapd/plugins/rsp.so
    /usr/share/mt-daapd/plugins/ssc-script.a
    /usr/share/mt-daapd/plugins/ssc-script.la
    /usr/share/mt-daapd/plugins/ssc-script.so

    I noticed that, unlike you, I don’t have libogg-dev 1.1.3-3 and liboggflac-dev 1.1.2-8 installed. Are these files require or is that something you’ve got installed for other purposes? Also, I don’t seem to have libvorbisidec-, I’ve only got libvorbisidec1

    Thanks for your assistance!
    Emile

    Comment by MrE — 2008/05/24 @ 01:05 | Reply

  6. Hi Emile,

    I don’t know what’s wrong with your ogg setup. Have you tried to decode an ogg to wav on your slug and play it on your pc? Maybe that will work. As for the white noise, the binary package I built is Little Endian aligned, which I’m sure is the type you’d need on your slug too. Just to be clear, here’s my uname -a output for Debian Lenny:

    scheff@nslu2:~$ uname -a
    Linux nslu2 2.6.24-1-ixp4xx #1 Sun Apr 20 07:37:13 GMT 2008 armv5tel GNU/Linux

    Regarding the dev packages you mention, they aren’t needed for running the vorbis tools. I installed them because of build dependencies. The “libvorbisidec-” package is actually called libvorbisidec-dev; the output got trunctated because of the small terminal screen I used. You don’t need this package too.

    Well, that’s all I can come up with for now. And sorry for my late answer. Oh, one last thing: I could make the source package(s) (esp. vorbis-tools with tremor) available, in case you’d like to try and build them yourself. Good luck!

    Comment by FrankZabbath — 2008/06/02 @ 20:42 | Reply

  7. Hi Frank,

    No problem about the late reply!

    The white noise doesn’t seem to happen anymore, but my SoundBridge simply won’t play Ogg and Flac files anymore.

    uname -a reports
    Linux YGGDRASIL 2.6.18-6-ixp4xx #1 Fri May 9 01:24:25 UTC 2008 armv5tel GNU/Linux

    I’m running a slightly older kernel than you, but that shouldn’t make a difference, I think.

    Perhaps indeed I’ll have more luck trying to build the packages myself, so I’d be grateful if indeed you could make them available.

    Thanks again,
    Emile

    Comment by MrE — 2008/06/04 @ 01:14 | Reply

  8. Hi Emile,

    sounds like problems with your Firefly Server to me. Maybe with the SoundBridge Software too. Are the .ogg and .flac files shown in the Firefly Collection? Are you sure you have setup the Firefly Configureation and Plug-Ins properly? Are you running the latest SoundBridge Firmware? My SoundBridge Home Firmware is some Months old and I got it using the Beta Firmware access found on the Roku Website; Telnet shows me:
    SoundBridge> version
    version: software version 3.0.51 Release
    blackfin revision 5

    The Kernel version shouldn’t matter, as I’ve used other/older Kernels without problems.

    I’ve updated my Blog posting and added download links to the libmpcdec and vorbis-tools sources, ready to be rebuilt with debuild and alike.

    Good luck again. May the source be with you! :P
    Kristian

    Comment by FrankZabbath — 2008/06/06 @ 16:36 | Reply

  9. Hi Kristian,

    I hope you you’ll get to read this message as this is quite an old post.

    It’s been a while since I visited this post and therefore did not see the links to your Lenny binaries and source files. Unfortunately the links have expired and I’m unable to find this software elsewhere.

    I’ve tried patching vorbis-tools myself using the latest version of the patch available on the Firefly forum but I’m running into problems applying the patch.

    I’d be really grateful if you could make the files for Lenny available again as I’ve not been able to play FLAC and Ogg files on my Soundbridge/NSLU2 for over 2 years now and would really like to do so as a significant portion of my music library is stored in those formats.

    thanks
    Emile

    Comment by MrE — 2009/04/20 @ 00:57 | Reply

  10. Hello Emile,

    thanks for notifying me about the broken links. I’ve (hopefully) fixed all of them now in this blog. The files are still available, but the trick I used to get direct downloads doesn’t work anymore.

    I also sent you an e-mail last week about this. I don’t know if my packages will still work without problems with the current Lenny packages. I haven’t used my fixed-point decoders for a while now, because most of my music collection is in MP3 format.

    Cheers
    Kristian

    Comment by FrankZabbath — 2009/04/26 @ 17:45 | Reply


RSS feed for comments on this post. TrackBack URI

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

Blog at WordPress.com.

%d bloggers like this: