Do I HAVE to Migrate Any Data Into My New ERP?

Anyone preparing for an ERP implementation will have to confront performing data migration for their new ERP. We hear clients’ woes day in and day out who cannot understand why it is so hard and wonder why they can’t just skip migrating the data. Then there is the other extreme CEO’s practically cry when they find out it is not a good idea to bring over all the historical data. How can there be such a vast amount of misunderstandings about data migration in ERP implementations?

Let’s take the one extreme – the executive who is contemplating migrating nothing into their new ERP. They may suggest this extreme because they have the scars from a previous implementation and know how difficult making the data decisions are, how difficult transforming the data was, and ultimately how hard it was to persuade users to leave their historical data in their old systems. Thus, they suggest “starting fresh” with “nothing.” While this viewpoint has some merits, it just isn’t workable for a company of any substantial size. Think about the most basic kinds of records – Customers and Vendors. If you don’t “migrate” anything new into your system, you will have no data to perform transactions with! Perhaps your employees could type it all in fresh from scratch. But then you still must decide which customers to type into the system by hand, which customers you don’t need, which related customer records to bring over, and which ones are outdated or no longer necessary, like old credit terms or some custom field. What about your inventory and part list? You will need to decide on a method for eliminating archived, old, duplicate, or otherwise irrelevant items from the new “migration.” And unfortunately, these are the kinds of decisions you will have to make whether you “start fresh” or do the more usual and customary method of extracting, cleansing, transforming, and loading via a CSV or Excel upload.

Then there is the other extreme – clients who assume they will bring over ALL of their data, all 20 years of historical accounting transactions, historical sales orders, purchase orders, work orders, inventory, and the list goes on. The difficulty with accommodating a company wanting to migrate all history lies in the fact that data is interrelated. For instance, bringing over all historical closed-out sales orders requires including your product list for older items no longer sold. Or in the case of migrating all historical purchase orders, necessitating the migration of items and vendors you no longer use. Therefore, you inadvertently migrate inactive customers, parts, BOMs, supplies, etc. that you do not actually need in your new ERP! We liken it to an empty-nest couple who builds their custom dream house after the kids are out of college but when it is time to move in, they plan on moving all their old furniture, all the junk in their basement, garage, and attic which has accumulated from 30 years of raising kids and grandkids! No, no, no! You want to get new furniture for that new dream home! And it’s time to ditch all the old kid clothes, toys, and broken down, stained furniture. But what are we supposed to do about the audit or historical reports? It is common to keep a license to your legacy system, or perhaps a read-only copy, or worst-case scenario, dump everything into a data warehouse so you can query it for historical reports. But don’t kludge up your new ERP with all the data of the past. It is way too painful to cleanse, transform, and validate, with little value to show for it. And frankly, it is too expensive, whether it is your own team spending the time or you are hiring technical resources to perform the work.

So, what is the “middle ground” for a logical ERP data migration approach? There are four key sets of data you will need. These include:

Master records or Entity records – these include Customers, Vendors, and related fields.
Open transactions – this means AR and AP that are not closed out at the time of go-live; therefore, you need to make sure the related AR and AP records are accounted for in the new system.
Trial Balances – best practices include about two years of monthly trial balances; this is also the time to consolidate multiple ERPs into one new trial balance for each of your new accounts. You won’t have “click-through” detail on these which disappoints many Controllers, but you should keep a license to your old system for audits or detailed comparative reports.
Historical Transactions – these are the ones loaded with dynamite; proceed carefully! Depending on your industry, we frequently see clients needing historical Sales Orders, Purchase Orders, or Projects.

Then, ensure you account for the required time during your implementation to do the following major activities:

Extraction: establish the extraction method for each data source and pull a copy of the database into a working environment to clean and manipulate.
Cleansing: this is a process to deduplicate and standardize the nomenclature across your various data fields and eliminate data you won’t need in the new system.
Transformation: you may need someone with SQL development skills for this step; it is best to write scripts and save them for re-running on the data as you make changes throughout your implementation. You will need to transform your data into the template for your new ERP and these transformations are not necessarily a “one-for-one” match. After all, many companies upgrade ERP to benefit from new accounting dimensions, more sophisticated BOMs, etc. so the transformation may not be linear.
Loading: Of all these steps, this is the only one that your implementation partner usually performs. Check your statement of work, the hours are usually pretty limited to small loads for testing data and one full load at go-live. That means you and your team are responsible for the earlier steps (Extraction, Cleansing, and Transformation.)

We recommend starting your Data Migration Plan as early as possible in your implementation and we even begin it during the Selection phase with our clients. We tell our clients it is the wildest variable in the total cost of your implementation, so we help them scope it as a major cost prior to seeking final approval to purchase software and services. If you need help creating a Data Migration Plan, contact us for help. Our data migration resources are all US-based and extremely experienced, technical consultants in extracting data out of most legacy ERPs and transforming it into all the prevalent, modern applications.