BIOS Recovery

A couple of months ago, I was looking at a friend's laptop. It had definite motherboard problems and a dead HDD. As part of the service, I updated the BIOS. Unfortunately, it died mid-flash.

I tried the local Fujitsu-Siemens service center, but they said the motherboard had to be replaced. On an aging laptop, that's just not worth it. New HDD, yes. New Mobo, no.

I asked around UCT everywhere, looking for someone with a PROM programmer, but the only one I could find was an ancient device attached to a 286. The "new" programmer (not much newer) was broken... I waited a couple of months, but it still isn't fixed.

So, I tried a mail-order BIOS flashing service, biosflash.com. They found a compatible chip, programmed it (with the update I'd been trying to install), and put it in the (registered) post within 24hrs. 2 weeks later, I've got it, and it installed it in the laptop.

It works!

Usually this kind of kind of thing means the laptop is written off. (Desktops normally have some kind of bad-flash recovery procedure, but it's rare in laptops, and not foolproof anyway). I'm very impressed with biosflash.com: for only €15, the machine is no longer a brick.

Now to replace that HDD...

Automated backups to external disk

I remember somebody asking how to do this on the CLUG lists a while back. But here's the problem:

You've got an automated backup system, but you want offsite backups. DVDs are too small, external hard drives are the only option. You want the user to be able to plug in the firewire disk, have the backup start automatically, and let them know when it's done.

Here's how I implemented it:

The backups are implemented with backup-manager, they backup into /mnt/backup-tmp/

The external hard drive connects by firewire. Running udevinfo -a -p /sys/block/sdd on it showed me it's ID:

ATTRS{ieee1394_id}=="0090a9787b339de6:1:0"

I created this UDEV rule file /etc/udev/rules.d/local-backup.rules:

ATTRS{ieee1394_id}=="0090a9787b339de6:1:0", SYMLINK="backupdisk", RUN+="/usr/local/sbin/backup-to-external.sh"

And the relevant fstab entry:

/dev/backupdisk /mnt/backup-disk vfat   sync                    0       0

And the backup script /usr/local/sbin/backup-to-external.sh:

#!/bin/sh -e
LOCKFILE=/var/run/backup-to-external.lock
logger "Backup disk detected"
# Test for expired locks
if [ -e "$LOCKFILE" ]; then
  if ! kill -0 `cat "$LOCKFILE"`; then
    rm "$LOCKFILE"
  fi
fi
lockfile -r0 "$LOCKFILE"
echo $$ > "$LOCKFILE"
sleep 5s
logger "Backup to external begun"
mount /mnt/backup-disk
rsync -a /mnt/backup-tmp/ /mnt/backup-disk/
umount /mnt/backup-disk
beep -l 1000 -f 3000 -r 5
echo -e "You can disconnect the disk now.\nThank you.\n\nThe backup System." | mail -s "Backup completed" the-secretary@email.address
rm "$LOCKFILE"
logger "Backup to external completed"

CLUG Talk videos

CLUG is one of the more active LUGs around, and has had fortnightly meetings with technical talks consistently since at least 1999 (the extent of our records, and my involvement).

While this heritage is a testament to a strong LUG, there are only about 20 people who regularly attend meetings, and thus get the benefit of these talks. With a LUG membership of around 800 (my best estimate from mailing list figures), this is a tiny fraction of our community. People who can't make it have been requesting videos of talks for a while, and recently Jonathan Carter brought his camera, and we started playing with videoing them. He has been stuck in Johannesburg for a few weeks and left me his camera, so I've been playing around with encoding.

So far, the lessens learned:

  • Video quality: A camera, high up at the back of the room produces reasonable, compressible video without having to have someone pan around following the presenter.
  • But the built-in microphones on a cheap camera just aren't effective at that range (especially when you've got a quiet speaker).
  • Audience questions are hard to record.
  • Audio editing is do-able, and necessary. But so far, we haven't ventured into video editing (on linux, this isn't trivial).
  • You can't normally read all the slides (or demos) with SD video. At full PAL resolution, most slides are legible. Screen-capture is an option, but most presenters make their slides available, which is far more bandwidth efficient.

Our progress so far is these procedures, and these videos: UK, ZA. We are using Ogg/Vorbis/Theora, and 3 different qualities of video. The three qualities are overkill, but I'm still experimenting with settings. I'd like some feedback - especially from a codec expert :-)

We lost Wikimania 2008

We lost Wikimania 2008, quite badly (by points). I'm obviously sad, because I was hopeful, and had invested the odd weekend into comparative flight pricing, bid editing, and sitting on the phone...

According to the published criteria, we should have done well. Somehow we managed to score lower than Alexandria on "rotation" which is odd because Wikimania has never been held in the Southern Hemisphere.

I'd have liked too see the IRC log from the judges meeting, to see where we lost our points, but of course that isn't going to happen, so we'll have to work it out ourselves, by comparing the bids.

In retrospect, our bid wasn't that great. We were pinning more hopes on our location, and Cape Town's attractions, then on the bid itself. It was expensive, last-minute, and lacking sponsership & good accommodation. By losing, we got our chance to get it right next time.

The good news, is bidding opens for Wikimania 2009 almost immediately. I don't know what the chances are of 2009 being in Africa again, but I'm assuming we are going to bid, if everyone can pick themselves up and get going on it. To do it, w e are going to need firm sponsors. Find any friends you have corporate contacts, and twist their arms.

BIOS Flashing with memdisk

I've just discovered memdisk. It's part of the syslinux package on Debian/Ubuntu, and hides in /usr/lib/syslinux/memdisk.

Memdisk lets you boot a floppy image, via grub or pxelinux. In this modern era of computers without floppy drives, it means you can do BIOS updates without having to go through the whole procedure of turning a floppy image into a bootable CD.

In PXELINUX, the config file would look like this:

DEFAULT memdisk initrd=FILENAME.img

In Grub, like this:

title     Bios Flash
kernel    /boot/memdisk
initrd    /boot/FILENAME.img

Thanks ThinkWiki for the idea.

Caveat emptor: apparently some flash tools don't like memdisk, so YMMV

Dovecot shared mailboxes (the correct way)

I've just implemented shared mailboxes in dovecot (which rocks, btw). It isn't difficult, but I don't think it's very well documented...

The preferred way to do this is with IMAP Namespaces. My natural approach would be to create something like a Maildir tree /srv/mail/shared, and make this the "public" namespace. Then set filesystem permissions on subtrees of that, to define who can see what. Unfortunately, dovecot uses strict Maildir++, and won't let you create mailboxes inside each other (on the filesystem) /Foo/Bar is stored as a Maildir called .Foo.Bar, so subtrees don't exist, so this isn't an option. The up-comming dbox format should allow something like this, but it isn't usable yet.

My solution was to create multiple namespaces. One for each shared mailbox. Users are given permission to use them via file-system permissions (i.e. group membership), example:

# Default namespace, needed if you add namespaces
namespace private {
    prefix =
    separator = /
    index = yes
}
# Office inbox, available to receptionists, office managers, and directors:
namespace public {
   prefix = office/
   separator = /
   location = maildir:/srv/mail/office/.Maildir:CONTROL=~/.Maildir/control/office:INDEX=~/.Maildir/index/office
   hidden = no
}
# Umask for shared folders
umask = 0007

Setting CONTROL and INDEX mean that dovecot's metadata is stored in the user's personal Maildir, so users who don't have permission to see the shared mailbox don't get errors.

The permissions of the mailbox should be done as follows:

# touch /srv/mail/office/dovecot-shared
# chown -R mail.office-mailbox /srv/mail/office
# find /srv/mail/office -type d -print0 | xargs -0 chmod 2770 
# find /srv/mail/office -type f -print0 | xargs -0 chmod 660

If you want a common subscription list, you have to manually symlink:

# ln -s /srv/mail/office/subscriptions ~luser/.Maildir/control/office/

Seems to work well. (at least with thunderbird)

Non-free codecs on gutsy

Finally, it's got easy to install the codecs we can legally use in this country:

Medibuntu now has a non-free-codecs package, an all-in-one virtual package.

Nice job!

Personally, I'm still in favour having them there by default, and the installer removes them if you select a patent-encumbered country as your time-zone, but I can see why people are wary of that approach....

NFS Locking + Kubuntu

This afternoon, I went off to UCT, to help Arno out with a sudden problem in TSL: When you try and log into KDE, it just hangs. And if you try and run a KDE application from Gnome, it hangs...

I straced startkde, and spotted a hang that looked lock-related:

[pid  5726] open("/home/stefanor/.qt/.qt_plugins_3.3rc.lock", O_RDWR|O_CREAT|O_LARGEFILE, 0600) = 9
[pid  5726] fcntl64(9, F_SETLKW64, {type=F_WRLCK, whence=SEEK_SET, start=0, len=0} <unfinished ...>
[pid  5725] <... select resumed> )      = 0 (Timeout)
[pid  5725] gettimeofday({1190991469, 364681}, NULL) = 0
[pid  5725] ioctl(3, FIONREAD, [0])     = 0
[pid  5725] gettimeofday({1190991469, 364908}, NULL) = 0
[pid  5725] select(9, [3 4 6 8], [], [], {0, 120271}) = 0 (Timeout)

After some trial and error, it turned out, that the problem was the NFS mounted /home. KDE wants it to be mounted with the options async,lock. Then it's happy. I don't why how this turned up in a routine nightly Ubuntu feisty update. Maybe it was a backport?

Anyway, there you go: KDE is fussy about its locks.

Petrol Theft?

Yes, petrol is now expensive enough, that it's worth the risk to steal it from someone else.

On friday night, some bastard cut the fuel lines of all 4 cars parked outside our house. We are probably the only people on our street who don't park behind an electric gate, because our driveway only has space for my parents cars on the property. My car lives on the council-owned part of our driveway.

I found out because my sister (who'd been visiting to use our Internet connection) broke down on the way home. With her car, this is perfectly normal, and to be expected. I went out to tow her home (she was only about a 500m away). I couldn't find the tow hook on my car (my Golf has removable tow hooks, and somebody once broke into my car and stole the jack-set containing the removable hook), but I did notice that my car was trailing a line of petrol... odd... I went back to borrow my father's car (with a tow-bar), and towed her home with that.

She was going to borrow my mother's car (she usually does that when my mother is away, and her car breaks down), but noticed that there was a pool of petrol under it, too. Very odd...

At this point, I put 2 and 2 together, and found that all of the cars had cut lines. In the case of my father's car, they hadn't got a fuel line, but had god a brake sensor line instead. (Apparently VWs all have the fuel lines on the outside for safety, so they are the easy targets). On my car, they had only cut the fuel return line, so it could still drive, but would trail petrol.

The police were quick to come, but not very helpful: they said they'd never seen anything like it. They were thinking vandalism or neighbour issues. The questions were "Do you have a dog?" "Do you get on with your neighbours?" etc. You can tell what the normal problems in our area are :-)

The AA refused to come and help or tow the cars to my mechanic, saying the damage was an "insurable risk". I thought most car problems were "insurable risks" and the reason one has AA membership is to sort things out afterwards? This refusal by them is unacceptable, what if it had happened when I was parked somewhere else? Would I have to get my car home by myself?

Fortunately my grandmother's car was in storage in her garage, and I could rescue it. My sister stole her mother's car, too. But I've now got a whole lot of paperwork to sort out, and 4 dead cars to get repaired... As I said, "Bastards!"

Photos here

Update:

  • None of the cars have empty tanks, so if they were trying to steal petrol, they didn't succeed.
  • The damage came to R500 for the VWs, which isn't bad at all. (Replacing the lines would be a mission, my mechanic repaired them). Excluding many hours of towing, phone calls, e-mail, and SMSs, though.

Local only DSL

Update: Debian/Ubuntu version

I've finally jumped onto the local only DSL bandwagon. If you haven't done it yet, it's a great way to save some bucks. The idea is that you get a local only account like this, which costs a fraction per GiB compared to normal account. Then you get your router to connect to both simultaneously, and route intelligently between them.

Most ADSL routers won't let you connect 2 concurrent ADSL connections on the same ATM circuit. The solution is to use a separate modem and router. I'm using a basic Billion modem, in bridged mode, and a WRT54GL, running OpenWRT/kamikaze, as the router.

OpenWRT doesn't support 2 PPPoE connections out of the box, but I've found the problems, and got a few changes committed upstream, that solve them:

The firewall (/etc/init.d/firewall) needs to be modified with "WAN=ppp+" somewhere near the top, so that it masquerades all the ppp connections. This was a hack, apparently the firewall is being re-written soon.

There is also a bug that resets existing PPPoE connections on a ethernet interface when you fire up a new connection. This will apparently be fixed by the future interface aliasing support. For now, I just hacked around it in /lib/network/config.sh:

prepare_interface() {
    local iface="$1"
    local config="$2"

    #SR: We don't want to reset any pppoe connections
    config_get proto "$config" proto
    [ "$proto" = "pppoe" ] && return 0

and /sbin/ifdown:

config_get ifname "$cfg" ifname
config_get device "$cfg" device

[ ."${proto%oe}" == ."ppp" ] && device=
[ ."$device" != ."$ifname" ] || device=
for dev in $ifname $device; do
    ifconfig "$dev" 0.0.0.0 down >/dev/null 2>/dev/null
done

I got my local routes list from cocooncrash's site (he gets them from local-route-server.is.co.za, aggregates them, and publishes every 6 hours). OpenWRT already has a static routing configuration system, but it's very verbose. So I wrote my own, adding the new configuration option routefile. I used these hotplug scripts to set up routing and source routing, with the help of iproute2:

$ ipkg install ip

$ mkdir /etc/routes
$ wget http://mene.za.net/za-routes/latest.txt -O /etc/routes/zaroutes

You'll probably want to update that route file regularly. I don't run cron on my WRT54GL, so I do it manually. Up to you.

/etc/config/network:

# ...local interfaces...

#### WAN configuration
config interface    wan
    option ifname   "eth0.1"
    option proto    pppoe
    option username "xxxxx@international.co.za"
    option password "xxxxxx"
    option routefile "/etc/routes/exceptions"
    option defaultroute 1

config interface    localdsl
    option ifname   "eth0.1"
    option proto    pppoe
    option username "xxxxx@local.co.za"
    option password "xxxxxx"
    option routefile "/etc/routes/zaroutes"
    option defaultroute 0

/etc/iproute2/rt_tables:

#
# reserved values
#
255     local
254     main
253     default
0       unspec
#
# local
#
1   wan
2   localdsl

/etc/hotplug.d/iface/20-split-routes:

case "$ACTION" in
  ifup)
    . /etc/functions.sh
    include /lib/network
    scan_interfaces
    config_get routefile "$INTERFACE" routefile

    # Does this interface have custom routes?
    if [ -e "$routefile" ]; then

      # Add routes for this interface
      cat "$routefile" | while read route; do
        ip route add "$route" dev "$DEVICE"
      done

      # Set up source routing
      peer=`ip addr show dev $DEVICE | sed -n '/inet/ s#.* peer \([0-9.]*\)/.*#\1# p'`
      address=`ip addr show dev $DEVICE | sed -n '/inet/ s/.* inet \([0-9.]*\) .*/\1/ p'`

      ip route add "$peer" dev "$DEVICE" src "$address" table "$INTERFACE"
      ip route add default via "$peer" table "$INTERFACE"
      ip rule add from "$address" table "$INTERFACE"
    fi

    # Make sure this interface is present in all the custom routing tables:
    route=`ip route show dev "$DEVICE" | awk '/scope link  src/ {print $1}'`
    awk '/^[0-9]/ {if ($1 > 0 && $1 < 250) print $2}' /etc/iproute2/rt_tables | while read table; do
      ip route add "$route" dev "$DEVICE" table "$table"
    done
    ;;
esac

/etc/hotplug.d/net/20-split-routes:

case "$ACTION" in
  register)
    . /etc/functions.sh
    include /lib/network
    scan_interfaces

    # If this interface doesn't have a link local route, we don't need to bother
    route=`ip route show dev "$INTERFACE" | awk '/scope link  src/ {print $1}'`
    [ ."$route" = ."" ] && return 0

    # Add this interface's route to all custom routing tables
    awk '/^[0-9]/ {if ($1 > 0 && $1 < 250) print $2}' /etc/iproute2/rt_tables | while read table; do
      ip route add "$route" dev "$INTERFACE" table "$table"
    done
    ;;
esac

Now, lastly, it won't bring up both interfaces by default. That will be fixed by aliasing in the future, but for now:

/etc/init.d/network-multiwan:

#!/bin/sh
ifup wan
ifup localdsl
$ chmod 755 /etc/init.d/network-multiwan
$ ln -s ../init.d/network-multiwan /etc/rc.d/S49network-multiwan

That's it, and it's working beautifully :-)

What is source routing, people ask? The problem is that your router now has 2 WAN IP addresses. IP1 is used for local traffic, and IP2 for international. So if somebody in ZA tries to connect to IP2, the reply (local destination) will go out of Interface 1. The ISP on the other end of Interface 1 will drop this reply, because it's not coming from IP1.

Source routing tells the router that replies must go out of the same interface that the request came in on. I'm doing it by creating separate routing tables for traffic origionating from each WAN interface.

Gotchas

  • If you are using 2 different ISPs (say SAIX international and IS local), you must make sure that DNS queries get routed out the right interface. SAIX won't accept queries on their servers from IS, and vice versa.
  • SAIX Web proxies, Mail servers, and News servers don't accept traffic from local accounts. (especially from another ISP)