Bembel-B Blog

2008/03/19

Fixed-point Ogg Vorbis and Musepack Decoder Packages for Debian Etch ARM

I’m using Firefly Media Server on a Linksys NSLU2 running Debian Etch to stream my whole music collection to my Pinnacle SoundBridge HomeMusic. Many files in my collection are in Ogg Vorbis format and a few in Musepack too, which unfortunately aren’t natively supported by the SoundBridge. Luckily Firefly can transcode audio files on the fly e.g. using the ssc plug-in and any console application via shell scripts. But the problem with the Debian packages is that they don’t take into account the missing floating point unit (FPU) of the NSLU2 hardware, and that makes oggdec/ogg123 and mpc123 take ages – way below real-time – to transcode. Fixed point versions of the transcoders aren’t available in the Debian Etch package repositories even the sourcecode already exists, so I had to build them myself.
Linksys NSLU2
I first searched the Debian packages for fixed point versions.
I found the libvorbisidec package, which is the fixed point version of libvorbis also going by the name Tremor. The vorbis-tools is only available as normal floating-point version linked to libvorbisdec. For Musepack there are (floating-point) packages libmpcdec3 for the library and mpc123 for the application.
In the Firefly forums I found patches for vorbis-tools (providing the ogg123 and oggdec transcoders), so that I could build Tremor versions of the transcoder apps.
In the Musepack sources there’s already a define variable to build with fixed point math only. So I would just have to enable that and rebuild the library and transcoder.

To make a long story short, here are the Debian Etch ARM binary packages for fixed-point Ogg Vorbis and Musepack:

I will gladly provide the sources. Feel free to ask for them! You can download the Deb sources for libmpcdec and vorbis-tools further below.

Update

These are the integer Lenny ARM binaries. Libvorbisidec is available through the official repos.:

Update 2

Here are the Deb sources (.dsc, .diff.gz and .orig.tar.gz) to build your own binary Deb packages.

Performance now is very good on my 266 MHz NSLU2. From what I’ve seen in top, mpc123 and ogg123 stay below 50% CPU usage and decoding is done faster than real-time.

Only to show off how unbelievably cool I am ;), and in case somebody would like to reproduce these builds on another distribution, I’ll roughly describe the steps I’ve taken.

Build libvorbisidec

Pretty straight forward. As far as I remember I only had to rebuild the source package:

mkdir -p ~/debuild/libvorbisdec
cd ~/debuild/libvorbisdec
dget -x http://ftp.de.debian.org/debian/pool/main/libv/libvorbisidec/libvorbisidec_1.0.2+svn14261-1.dsc
cd libvorbisidec-1.0.2
debuild -rfakeroot -uc -us
cd ..
sudo dpkg -i libvorbisidec1_1.0.2+svn14261-1_arm.deb libvorbisidec-dev_1.0.2+svn14261-1_arm.deb

Patch and build vorbis-tools

This one was trickier. The patches found on the forum aren’t complete. The older one is blindly changing the endianess of the resulting audio data, thereby producing only noise on the little endian Debian I use. The newer one misses linking mpcdec to libvorbisidec.

mkdir -p ~/debuild/vorbis-tools
cd ~/debuild/vorbis-tools
dget -x http://ftp.de.debian.org/debian/pool/main/v/vorbis-tools/vorbis-tools_1.1.1-6.dsc
cp -pr vorbis-tools_1.1.1 vorbis-tools_1.1.1.wrk
cd vorbis-tools_1.1.1.wrk
[apply and manually fix the tremor patch]
[create tremor patch file against vorbis-tools_1.1.1/]
[copy patch file to vorbis-tools_1.1.1/debian/patches]
[add patch file at bottom of vorbis-tools_1.1.1/debian/patches/series]
[add libvorbisidec-dev dependency to vorbis-tools_1.1.1/debian/control and document Tremor versions in description]
[increase version and add changes in changelog via dch -i]
cd vorbis-tools_1.1.1
debuild -rfakeroot -uc -us
cd ..
dpkg -i vorbis-tools_1.1.1-7_arm.deb

Patch and build libmpcdec

Here I add a patch to enable the fixed-point define. Beware, the following steps are just written down from memory in a hurry. So expect flaws. :)

mkdir -p ~/debuild/libmpcdec
cd ~/debuild/libmpcdec
dget -x http://ftp.de.debian.org/debian/pool/main/libm/libmpcdec/libmpcdec_1.2.2-1.dsc
cp -pr libmpcdec-1.2.2 libmpcdec-1.2.2.wrk
vim libmpcdec-1.2.2.wrk/include/mpcdec/math.h
[uncomment #define MPC_FIXED_POINT]
mkdir libmpcdec-1.2.2.wrk/debian/patches
diff -Naur libmpcdec-1.2.2 libmpcdec-1.2.2.wrk > libmpcdec-1.2.2.wrk/debian/patches/enable_fixed_point.diff
cd libmpcdec-1.2.2.wrk
dch -i
[increases version in changelog. add some notes.]
debuild -rfakeroot -uc -us
cd ..
dpkg -i libmpcdec3_1.2.2-2_arm.deb libmpcdec-dev_1.2.2-2_arm.deb

Build mpc123

This one has only to be rebuilt with the fixed-point libmpcdec version above installed.

mkdir -p ~/debuild/mpc123
cd ~/debuild/mpc123
dget -x http://ftp.de.debian.org/debian/pool/main/m/mpc123/mpc123_0.2.1-1.dsc
cd mpc123-0.2.2
debuild -rfakeroot -uc -us
cd ..
dpkg -i mpc123_0.2.2-1_arm.deb

I haven’t found a fast enough way to transcode Monkey’s Audio (aka. APE) files yet, but that’s mainly because I don’t have any (which haven’t been transcoded to mp3 by myself already). For your information: Building the Monkey’s Audio Codec (MAC) for Linux did work.
I think when looking for more low resource or fixed-point codecs, a good place to look for is the Rockbox Wiki. That’s what I used too.

To make this Firefly Transcoding post more complete, here’s my current version of the ssc script:

#!/bin/sh
#
# script to facilitate server-side transcoding of ogg files
#
#
# Usage: mt-daapd-ssc.sh   
#
# You may need to fix these paths:
#

WAVSTREAMER=/opt/mt-daapd-ssc/wavstreamer
OGGDEC=oggdec
OGG123=ogg123
FLAC=flac
MPC=mpc123
FFMPEG=/opt/mt-daapd-ssc/ffmpeg-quiet.sh
MAC=mac

ape_file() {
    $MAC "$FILE" - -d | $WAVSTREAMER -o $OFFSET $FORGELEN
}

ffmpeg_file() {
    $FFMPEG -i "$FILE" -acodec pcm_s16le -f wav - | $WAVSTREAMER -o $OFFSET $FORGELEN
}

mpc_file() {
    $MPC --quiet --wav - "$FILE" | $WAVSTREAMER -o $OFFSET $FORGELEN
}

ogg_file() {
#    $OGGDEC --quiet -o - "$FILE" | $WAVSTREAMER -o $OFFSET $FORGELEN
    $OGG123 -q -d wav -f - "$FILE" | $WAVSTREAMER -o $OFFSET $FORGELEN
}

flac_file() {
    $FLAC --silent --decode --stdout "$FILE" | $WAVSTREAMER -o $OFFSET $FORGELEN
}

FILE=$1
OFFSET=${2:-0}

# this is nonsense!
if [ "$3" != "" ]; then
  FORGELEN="-l ${3%.*}"
fi

case "$1" in
  *.[sS][hH][nN])
  ape_file
  ;;
  *.[sS][hH][nN])
  ffmpeg_file
  ;;
  *.[mM][pP][cC])
  mpc_file
  ;;
  *.[oO][gG][gG])
  ogg_file
  ;;
  *.[fF][lL][aA][cC])
  flac_file
  ;;
  *)
# here you could cat a generic "error" wav...
# cat /path/to/error.wav
  ;;
esac

Oh, FYI I’m using mt-daapd 0.2.4+r1376-1 from the Debian repo. I got the wavstreamer binary and the ssc script example by extracting them from a newer Debian package. The Firefly Wiki provides more information on transcoding implementation.

ChangeLog

[2008-04-11: Add Lenny Debs.]
[2008-06-06: Add Deb sources for libmpcdec and vorbis-tools.]
[2009-04-26: Fix box.net links.]

Advertisements

2007/05/23

Some More Ubuntu Feisty Fixes and Findings

This is an update to my previous posting about Ubuntu Feisty Fawn problems. Most have been resolved, some only further analyzed.
Ubuntu Logo

Automount for smb shares (preferably with cifs)

Mounting of Windows XP and Fedora Samba (Browse and Domain Master and Wins Server) shares could finally be solved by using smbfs as filesystem type.
The strange thing is, when mounting via commandline neither specifying smbfs nor cifs did work correctly. But when leaving the filesystem option out, it worked and was mounted automatically as smbfs:

# mount //192.168.0.111/roms ~/mnt/roms -o username=usersmb,password=passsmb

The automount map provided with Ubuntu did not work (uses cifs). As I already had tried another map as described at HowtoForge, I used that one and simply set it to use smbfs instead of cifs. And voila, shares being correctly mounted!

This is my /etc/auto.master

#
# $Id: auto.master,v 1.4 2005/01/04 14:36:54 raven Exp $
#
# Sample auto.master file
# This is an automounter map and it has the following format
# key [ -mount-options-separated-by-comma ] location
# For details of the format look at autofs(5).
#/misc  /etc/auto.misc --timeout=60
#/smb   /etc/auto.smb
#/misc  /etc/auto.misc
#/net   /etc/auto.net

/samba  /etc/auto.smbfs --timeout=300

And the working /etc/auto.smbfs

#!/bin/bash
# $Id$
# This file must be executable to work! chmod 755!
key="$1"
# Note: create a cred file for each windows/Samba-Server in your network
#       which requires password authentification.  The file should contain
#       exactly two lines:
#          username=user
#          password=*****
#       Please don't use blank spaces to separate the equal sign from the
#       user account name or password.
credfile="/etc/auto.smb.$key"
mountopts="-fstype=smbfs,file_mode=0644,dir_mode=0755,uid=scheff,gid=users"
smbclientopts=""
for P in /bin /sbin /usr/bin /usr/sbin
do
        if [ -x $P/smbclient ]
        then
                SMBCLIENT=$P/smbclient
                break
        fi
done
[ -x $SMBCLIENT ] || exit 1
if [ -e "$credfile" ]
then
        mountopts=$mountopts",credentials=$credfile"
        smbclientopts="-A "$credfile
else
        smbclientopts="-N"
fi
$SMBCLIENT $smbclientopts -gL $key 2>/dev/null \
   | awk -v key="$key" -v opts="$mountopts" -F'|' -- '
        BEGIN   { ORS=""; first=1 }
        /Disk/  { if (first) { print opts; first=0 }; 
                  sub(/ /, "\\ ", $2); 
                  print " \\\n\t /" $2, "://" key "/" $2 }
        END     { if (!first) print "\n"; else exit 1 }
        '

Note there’ll have to be credential files for each host named /etc/auto.smb.hostname and you’ll have to set the needed user/group in the mountopts variable above.

To have the other clients (than the Browse Master) mounted by host name and not IP address (i.e. /samba/someclient instead of /samba/192.168.0.123), I had to configure Samba on my Ubuntu box to use the Fedora machine as Wins Server. Not sure if that’s really necessary. Might be the Firewall blocking broadcasts, but on the other hand I had disabled the Firewall completely during tests IIRC.

I’m still wondering, why it didn’t work right out of the box. Other people don’t seem to have these problems. The error was quite strange. The mount command succeeded, but the shares only contained infinite empty folders like this:

ls /samba/host1
share1
share2
share3
ls /samba/host1/share1
share2
share3
ls /samba/host1/share1/share2
share1
share3

Kind of weird reciprocal directories.. :)

Improve keyboard configuration like inputrc, tab completition, vim as setup in Fedora

What’s still missing in my keyboard setup for the terminal is the possibility, to switch between insert and overwrite mode using the Ins key. Also the special character keys (like “ß” or “°”) using the normal text terminal (not via X) are not accessible. But scrolling to the first and last bash history entry with PageUp and PageDown does work now with the following changes.

diff /etc/inputrc.orig /etc/inputrc

12c12
< # set convert-meta off
---
> set convert-meta off
23a24,27
> # Completed names which are symbolic links to
> # directories have a slash appended.
> set mark-symlinked-directories on
> 
37,38c41,42
< # "\e[5~": beginning-of-history
< # "\e[6~": end-of-history
---
> "\e[5~": beginning-of-history
> "\e[6~": end-of-history

Other things still prevail, but I didn’t investigate yet. Whats annoying with the Tab Completition is, that it wouldn’t complete something like ~/*.txt. When there are possible alternations in the middle of a word/path, completition won’t work. But it was working under Fedora.

And regarding vim, whenever I’m in Insert Mode, using Ctrl+Left or Ctrl+Right results in a new line with some Escape Sequences garbage, instead of jumping to the previous/next word. There’s probably more differences, but I’ll simply have to compare my old Fedora vim config to the Ubuntu one.

A working ssh-agent and gpg-agent setup (preferably with Seahorse)

The integration of Seahorse (seahorse-1.0.1-0ubuntu1) seems to be in an early stage and buggy. I couldn’t manage to get it coexisting with gpg-agent and ssh-agent. The agent wouldn’t be found, would crash or stall, or simply won’t process the passwords I typed. The best thing would be, to drop Seahorse and simply use the two standard agents.

Crackling sound with Amarok and web streams (xine engine)

I found the noise to be coming from the ogg decoder plugin of xine (libxine1-1.1.4-2ubuntu3) and only for files with low sample frequency. When playing the same file or stream (used this for testing) with mplayer it sounded perfect. I played a little with the ~/.xine/config lowering the priority of the ogg plugin and also removed /usr/lib/xine/plugins/1.1.4/xineplug_dmx_ogg.so, but there’s no other plugin available for ogg decoding on my system (shouldn’t ffmpeg be able to do that?). Then I rebuilt the libxine1 packages, but sadly the sound was as bad as with the official build. So, I hope there’ll soon be a fixed libxine1 package or alternatively build my own with the needed modifications (new ogg/vorbis libs?).

ChangeLog

[070530 More details on xine ogg decoder noise. Fix markup of inputrc diff.]
[070923 Fix backslashes (”\\” markup was meanwhile rendered as “\\” and not “\” as before.]

Create a free website or blog at WordPress.com.