DBMail, the hard way

I’ve been using DBMail as my own personal mail server for a long while now. I’ve known the software ever since it’s 1.x release, and been using it for its great benefits – Quick and accessible large IMAP folders (and I have over 60,000 items in there).

DBMail is a filtering backend for server based mail storage. It allows saving e-mail into a Database, and accessing it both via IMAP4 or POP3 protocols. This allows for large volumes of mail, with high load and traffic to be stored via faster-than-plain-files storage mechanism. It is great solution when you decide to use server-based-storage (IMAP4, in our case), and when you have tons of e-mail you don’t tend to delete for the last 6 or so years. I like it.

During the last few years of upgrades, I’ve seen the product grow, and get better. I’ve seen it get developed to a level where it can actually be the back-end of a real production services supplier. I’ve heard about some who actually used it for this, via the mailing lists of DBMail users. During this time, during the upgrades I’ve applied, something with my DB schema was broken. Nothing which prevented actually using the system, but it caused all kinds of small-time minor side effects, such as, in my case, subscribing to IMAP4 mailboxes broken, and the such. No show-stopper.

After consulting DBMail people, I’ve decided to change my schema, using a procedure suggested by them. I’ve “dumped” my DB into a file, “drop” it, and recreate the DB, brand new and all, according to their specifications. Afterwards, I’d import the dumped DB into it, and get it over with.

I’ve failed. It’s either there’s some major difference, or something’s broken with the procedure. I was able to login, but failed to get my e-mail!

So, I’ve restored my DB to its origin, and waited for some more spare time.

It takes a while doing this, BTW, as this DB is 17GB in size!

It took few weeks, but I was able to find the free time to start initiating another transfer. I’ve failed doing so, working on the level of the DB itself, then I’ll go on the applicative solution – stinks, but works.

It’s stinks, because it means every client of you (me and my wife) would have to transfer mail between mailboxes via our mail client. It stinks because it takes hours.

With over-bloated DB files as is, I’ve decided to create a new Database server, with new and separated store. I’ve added another MySQL server on this specific machine, running using different socket, different port and different store, lock and pid files. I’ve run an additional dbmail-imapd, my IMAP4 server, using the alternate socket, recreated the (empty) DB, and the empty tables, and started populating them. It took almost a day.

One weird thing I’ve noticed, but did not investigate any further, is that ThunderBird 1.0.6 on Linux did this same job faster than on Windows. Network settings of both computers are rather similar, and I could not really explain it, nor did I try much further.

I’ve almost forget to recreate our mail aliases and addresses, when recreated the rest.

After it was done, I’ve stopped both dbmail-imapd servers, and both MySQL servers, and replaced the stores between them. I’ve re initiated this dual-configuration, and moved the mails accumulated during the 10 hours delay. Afterwards, I’ve removed the older account, and shut down the alternative mail server.

It worked flawlessly, and although slow (and I’ve made few mistakes, including holding both stores on the same disk array), it was straight forward procedure. Stinks, but works.