Posts Tagged ‘mail server’

Microsoft Exchange, data replication

Sunday, November 27th, 2005

Here’s a little issue. If you were to replicate MS Exchange DB from one machine to another, how/what would you have done?

The scenario goes as follows: You have your own domain, and you use, for your own core services AD and MS Exchange for the whole organization. While AD supports some built-in replication, so you could hold a second
site and join a Windows server machine into this domain, and assuming you have some sort of link, you would have a replica of your AD on the secondary site. In case something happened to your primary site, your secondary site would have a complete and correct replica of your primary site.

Exchange doesn’t act that nice. Replicating MS Exchange is not simple, even assuming you have a method of transferring the data from one site to the other.

Two methods I can think of – The first would be to somehow proxy the information getting into it, and transport it (maybe rewrite the headers to point to a different recipient) to another site as well. In this case, you are actually agnostic to the store. You don’t care if you maintain your mailboxes on MS Exchange, Windows, Unix, or whatever system. You have two copies, and you’re fine with it. You should find some logical method to make your primary site’s mail server transport all mail, even internal mail, through this proxy mechanism.

The second method is a bit more interesting. If we could have had a real-time replica of the primary site’s Exchange DB, we would have been able to “mount” it on the secondary site. It is not trivial, but it can be done using some
not-so-common software or hardware based solutions. However, the secondary site would require few things, per this list:

1) Similarity:

I would strive to similarity between both sites. MS recommend similarity (when dealing with cold-recover), per this site, especially with the Exchange patch level, and somewhat with Windows patch level. The more similar, the better chances we’ll have. So:

a. Similarity in patch level.

b. Similarity in Storage group settings, especially transaction logs’ location.

c. Similarity in the structure of each store under each storage group

d. Similarity in the absolute path of each store’s edb and stm location between source and target

2) Recoverability:

To make things recoverable, you should make sure each store has, under the “Database” tab, under properties, the option which will allow restore to overwrite the DB.

3) Flexibility:

You need to find some flexible script which will be able to change, as fast as possible, the pointers in your AD, to point to the new Exchange server, in the users’ Exchange attributes. I have such a script, but I cannot disclose it here.

Armed with these three, you can copy, transfer, replicate, or whatever, your DBs from one location to the other. Make sure you replicate the logs as well, else it’ll get messy, and will require tons of time for DB recovery.

To recover, you should only mount the stores. Assuming you have followed the prerequisites, you would be able to mount the stores in no time. Otherwise, you would need to run eseutil on the stores. It might get messy.

Afterwards, only one thing to do – mass change the attributes of the users in question to point to the alternate Exchange server, and the alternate Exchange store. Should work like a charm.

DBMail, the hard way

Saturday, September 17th, 2005

I’ve been using DBMail as my own personal mail server for a long while now. I’ve known the software ever since it’s 1.x release, and been using it for its great benefits – Quick and accessible large IMAP folders (and I have over 60,000 items in there).

DBMail is a filtering backend for server based mail storage. It allows saving e-mail into a Database, and accessing it both via IMAP4 or POP3 protocols. This allows for large volumes of mail, with high load and traffic to be stored via faster-than-plain-files storage mechanism. It is great solution when you decide to use server-based-storage (IMAP4, in our case), and when you have tons of e-mail you don’t tend to delete for the last 6 or so years. I like it.

During the last few years of upgrades, I’ve seen the product grow, and get better. I’ve seen it get developed to a level where it can actually be the back-end of a real production services supplier. I’ve heard about some who actually used it for this, via the mailing lists of DBMail users. During this time, during the upgrades I’ve applied, something with my DB schema was broken. Nothing which prevented actually using the system, but it caused all kinds of small-time minor side effects, such as, in my case, subscribing to IMAP4 mailboxes broken, and the such. No show-stopper.

After consulting DBMail people, I’ve decided to change my schema, using a procedure suggested by them. I’ve “dumped” my DB into a file, “drop” it, and recreate the DB, brand new and all, according to their specifications. Afterwards, I’d import the dumped DB into it, and get it over with.

I’ve failed. It’s either there’s some major difference, or something’s broken with the procedure. I was able to login, but failed to get my e-mail!

So, I’ve restored my DB to its origin, and waited for some more spare time.

It takes a while doing this, BTW, as this DB is 17GB in size!

It took few weeks, but I was able to find the free time to start initiating another transfer. I’ve failed doing so, working on the level of the DB itself, then I’ll go on the applicative solution – stinks, but works.

It’s stinks, because it means every client of you (me and my wife) would have to transfer mail between mailboxes via our mail client. It stinks because it takes hours.

With over-bloated DB files as is, I’ve decided to create a new Database server, with new and separated store. I’ve added another MySQL server on this specific machine, running using different socket, different port and different store, lock and pid files. I’ve run an additional dbmail-imapd, my IMAP4 server, using the alternate socket, recreated the (empty) DB, and the empty tables, and started populating them. It took almost a day.

One weird thing I’ve noticed, but did not investigate any further, is that ThunderBird 1.0.6 on Linux did this same job faster than on Windows. Network settings of both computers are rather similar, and I could not really explain it, nor did I try much further.

I’ve almost forget to recreate our mail aliases and addresses, when recreated the rest.

After it was done, I’ve stopped both dbmail-imapd servers, and both MySQL servers, and replaced the stores between them. I’ve re initiated this dual-configuration, and moved the mails accumulated during the 10 hours delay. Afterwards, I’ve removed the older account, and shut down the alternative mail server.

It worked flawlessly, and although slow (and I’ve made few mistakes, including holding both stores on the same disk array), it was straight forward procedure. Stinks, but works.