Data Migration – Part 1: Exporting Data

Part of moving to a new mail system is the migration of user data. Arguably this is the most important part as this is what your customers (users) see and feel on the ‘new’ system (new to them; if you are at this point you have hopefully been playing with the new system for a while, due diligence and all…). If you don’t believe me in saying this is the most important part to test, plan and execute, try moving your CEO’s mailbox in the middle of the day somewhere around year-end or shortly prior to a board meeting from an old system (Exchange or otherwise) to your new Exchange 2007 system. Enough Said.

Of course there are several factors that need to be taken into consideration when moving a USER (not just the data) from one system to another like installing/configuring clients, re-pointing mobile devices etc. For the purpose of this series, I am going to focus on techniques on getting your user’s data from one spot to another.

Data migration can come in many flavors:
- In-House Exchange 2000/2003 to In-House Exchange 2007 (all part of the same Exchange Org)
- In-House Exchange 2000/2003 to In-House Exchange 2007 (new Exchange Org, or AD forest if you would like)
- In-House POP/IMAP based system to In-House Exchange 2007
- In-House GroupWise to In-House Exchange 2007
- In-House Lotus Notes/Domino to In-House Exchange 2007
- All of the above except replace the last part with ‘Hosted Exchange 2007’ (By the way, Hosted Exchange and Software as a Server (SaaS) in general are the best! End Shameless Plug.)

The above list is a subset of possibilities, but these are the most common situations.

Exporting the Data

Lets deal with the first instance, but only because it’s the easiest to handle. Moving mailbox data between Exchange Servers in the same Exchange Org (or Forest) is as simple as using the Move Mailbox from the Exchange 2007 Exchange Management Console (EMC) or through the Exchange Management Shell via the Mailbox-Move cmdlet. The speed in which data can be moved depends on a few factors, as always, like server power/utilization, network connectivity etc. A good rule of thumb we use when quoting customers at the high level is 1GB/hour. For a great article on how to use Exchange’s built-in tools to move mailboxes between servers in the same org, see the following link:
http://www.msexchange.org/tutorials/Transitioning-Exchange-2000-2003-Exchange-Server-2007-Part3.html

The rest of the instances involve a little bit more work. The hard part is generally getting the data out of the old system in such a way that is useable/importable.

The primary goal is this: get the data to .PST format!

If you’re pulling the data from an Exchange Server (even Exchange 5.5), Exmerge is your friend. Exmerge for those that don’t know, is a tool that runs against the Mailstore and exports all the data on a specified user or group of users mailbox(s) to a *.PST file format. This includes all mail (including the folder tree), deleted items, contacts, calendar items, notes/memos, tasks. Some 'gotchas' that don’t get exported include Mailbox Rules (most of them are client-side anyways), archives (again usually client-side). Another KEY gotcha for Exmerge is that the PST files it exports are in ANSI format, not Unicode. This means that they are subject to the 2GB size limit. So if you exporting the CEO’s 5GB mailbox, you’ll need to take a few runs at it to try and get multiple files down to less than 2GB each. You can filter by date of creation on items to try and narrow down the export. PLAN THIS CAREFULLY!

Exmerge is run on a domain Windows XP/2000 or Server 2000/2003 member (there are older versions available that run on NT). Because of the way inherited permissions are set up on the Exchange Stores by default, you have to be careful which account you run Exmerge as. For Exmerge purposes, I would suggest a dual processor server with lots of storage that is dedicated to this process. Exmerge is very capable of running multi-threaded processes so it can export up to 4 accounts at the same time. For detailed instructions on how to set up Exmerge with the appropriate permissions, see this link:

http://www.exchangeinbox.com/articles/024/exmergesetup.htm

For POP/IMAP type systems, this can be really easy or really difficult. Remember that in a POP setup, generally all the user’s mail is pulled down to the client when the user ‘POPs’ her mail. IMAP does leave the mail on the server, but that’s all it leaves. If the user has contacts, calendar items, tasks, notes (no matter which client software they are using) they are generally ALL stored on the local machine. This is because the POP and IMAP protocols where designed for EMAIL, not the rest of the stuff. The bright side is that if your users are using Outlook or Outlook Express already, there is an export to PST function right at the client. Although not the most glamorous use of time, this could mean the Jr. Admin gets to sit in the server room with two or three different machines in front of her each performing an export to PST on a different user for the next week. Hey, it’s good training.

As for the ‘other’ systems (GroupWise, Lotus Notes/Domino etc.), you generally have to use some sort of third party tool to export the data. We have used Quest Software’s tools for both GroupWise and Notes with success (http://www.quest.com). As part of our Due Diligence when planning and preparing for a customer data migration using a third party tool, we do the following to ensure a smooth migration:

1) We perform test migrations of large, non-production mailboxes (during off hours if at all possible) for the sole purpose of finding out how long it takes to migrate a unit of data. We generally like to metric how much data we can move in an hour and through experience have been able to guess it is usually around 1GB/hour.

2) We perform content test migrations. What this means is to create an empty mailbox on the old system, add one or two items of each item type that is available (so a few mail items, calendar items, recurring appointments, meetings with a few attendees, contacts, tasks with reminders or any other ‘everyday use functionality’ you can think of), and following the process of the tools you are using, move the mailbox to the new system. Coming up with a detailed test plan including documenting what was in the original mailbox, and a test script of what is expected in the new mailbox will allow you to plan for exceptions. There will be exceptions. Some of them can be fixed; some of them users will need to live with. For the latter, if the users are told well in advance, it results in less of an impact on your help desk when you actually move people over.

So at this point you should have a big hard drive full of PST files with your user’s data. You are half-way there. Stay tuned for Part 2 in which I will discuss getting your newly exported PST files into Exchange 2007.

Richard