It doesn’t matter if you have 100 records or 1 million records, the data import phase of a project can seem like a daunting task. It’s kind of like moving a 5 bedroom house. There’s extracting the data from your old system, or gathering it from various places it may live. There’s prepping it for import by removing duplicates and making sure that email and phone number fields are formatted properly. And let’s not forget figuring out what pieces of data you actually want to bring over into the new system. Once these steps are completed, along comes the biggest piece – actually importing the data. This is either done by someone within your organization or by an outside consultant. A variety of tools are used, some magic is performed and then you have a new system with data that looks familiar to you.
How to do you know that everything made it in? How can you be sure that all your old data is in the new system? As someone who spends a lot of time with data sheets and databases, I can tell you that perfection is the goal. Here are some tips for pre, during and post data migration.
1. Take Inventory
Before you even start in the new system, take a look at the records you have in the old system. Get an accurate count of how many there are. If you have different data types that are going into different record types in Salesforce, get counts on how many of each exist. Get summations as well. If you’re migrating from a different system, run reports that sum up dollar amounts and that count records. If you’re coming from Excel documents, use Sum functions and Count functions to get accurate information on your source data. For some useful Excel tricks, take a look at this post.
2. Have a Data Plan
Having a plan of which objects should be imported first is very important, especially in a relational database like Salesforce. You can’t import Opportunities without Accounts and you can’t import Contact Roles without Contacts. Creating a plan about which objects to import first, then second, then third, and so on is important. It also gives you a good understanding of how long the migration will take. If you approach the migration piece like its own mini project by assigning dependencies and deadlines, you’re in good shape. For some questions to ask on what to import, take a look at this post.
3. Delta or No Delta
If your team is working in an old system and you’re doing a migration into a new system, the question arises, what to do with the data that’s input into the old system while the new system is being built? The idea of a delta data load is that you grab a snapshot of the data in the old system first, then let your users continue to use the old system. Once the switch is made to the new system, you grab all the ‘delta’ data that your users have since input into the old system. While this may be a necessity in an organization, my recommendation is to avoid this type of setup, as it causes confusion and basically doubles the data migration time. If your organization can handle it, lock your users from the old system the second that the snapshot is taken. Have them keep records in a different way and when the new system is up, the data can be entered manually. Of course, this is not an ideal solution for all organizations.
4. Have a Data MAP
It may seem that you can wing the mapping as you go when doing an migration, but that only works for data sets that have very few fields and where the map is obvious. Having a map of fields from the source data into the new system is a really good way to make sure that all pieces get imported. Grab all fields from the source data, throw them in an Excel sheet and go through each one, making decisions on whether to import each item or not. If you do want to import it, then put which field you’ll map to next to the source name. This is your map moving forward and refer to it often, whenever you aren’t sure which field goes where.
5. The Actual Import and Records that Fail
Use whatever tools you want to do the import, as there are many out there. Keep track of all records that fail. We’re striving for perfection here, folks! Even if one record fails, figure out why. Sometimes it’s a problem that, if fixed early, will stop other problems from happening. For instance, if a record is missing from the Contacts object, and the Opportunity record import fails, then you may have other Opportunity records that will fail. If you import the Contact now, you’ll save yourself headaches later. As my grandmother likes to say, “a stitch in time saves nine.”
6. Post Migration Check
Now that you’re confident that all the data has made it over, run some reports. Check count totals and sums totals. Be sure to check the different record types and make sure they line up with the counts you put in your data inventory.
Just like moving a house, you go through the process and ultimately you’re in a nice new system with clean data, and only the data that you want. You have to make a plan, take inventory before you move, figure out the timing of the move, execute the move and then take stock of what was brought over and what is missing. If you follow these steps, your data migration won’t feel so daunting.
You may also want to read:
Implementation Best Practices: Data
Keeping It Clean: The Data Import Basics
Data Migration: The Apex Data Loader
