Ecommerce Replatforming Success Hinges on Seamless Data Migration

For ecommerce retailers, a successful replatforming, or platform upgrade, hinges on a key component for success: the migration of data. This means bringing important business information – namely products, customers, and order history data – into your new site.

In Episode 7 of Recommerce, Sara and Tiffany explore what merchants need to consider in handling your data prior to launch on a new platform. You’ll learn:

• How to think about your data and what you’ll need moving forward
• Why an ERP may (or may not) make the process easier
• What types of data may not be necessary for your new site
• Possible solutions for the biggest challenge in data migration: order histories

Full Episode Transcript

Sara: Welcome to another episode of Recommerce. I’m Sara, the Founder of Command C, and I’m here with Tiffany, our Tech Lead.

Tiffany: Hi, everyone.

Sara: Hey Tif, good morning. We’re up pretty early to record this podcast this morning. So we’re gonna try to stay awake. Today we’re gonna talk about the importance of having a strategy for data migration when replatforming or upgrading your site to a newer version. When we’re talking about data migration, the core elements that we’re talking about are products, customers, and order history. There are some other items that fall into this phrase as well, but typically the primary items we’re considering are the three I just mentioned.

When we’re talking about data migration, the core elements are products, customers, and order history.

Tiffany: Yeah, those are the three that really define an ecommerce store certainly, but there are a lot of other important, smaller things. The one that most immediately comes to mind is page content. So any sort of marketing materials, things like that, usually your strategy when you’re building a new site or replatforming it to a new platform, probably involves some amount of change to those. In many cases, you’re not actually migrating them [content pages], you’re sort of recreating them in the new environment.

But there are other smaller pieces of any e-commerce business that should really be migrated and considered, like coupon codes and gift certificates. I think there’s nothing that makes a customer sadder than trying to use a gift certificate that they’re pretty sure they have a balance on that doesn’t exist anymore. So coupon codes, gift certificates, any sort of other automation that you have around returns processing or things of that nature are going to come into play. Most of the rest of the data that was really tied to customer’s products and order histories.

Sara: Reviews too, that’s a big one.

Tiffany: Yes, yes. Thank you for mentioning that. That’s a really good point, and reviews are interesting because often times when we replatform, we’re also choosing a new aggregate reviews provider. Or we’re using the platform’s own reviews provider, and it doesn’t seem to be a lot of mapping out there for mapping between them. So we have to think about how much of that is coming over. Then that actually jumps into the whole SEO realm as well because now those reviews are potentially tied to URL that also potentially no longer exist. So there’s a lot to think about there.

Sara: I’ve been on projects where this can be a bit of an afterthought, and I’ve also been on projects where clients sort of just assume that they can use a migration tool that’s inherent to the platform. I know that Shopify has a migration tool that bring over customers and products from Magento. So it can be easy to fall into the trap of not addressing this upfront.

Tiffany: That trap is an easy one to fall into. I think it’s also exacerbated by the fact that it’s so obvious. The product data has to be there. This obviously something that we’re doing. The advertising and the marketing around “one click” says this is very easy, and it can be. We’re not suggesting that you can never use a tool like that, but there are really some points to consider.

Tiffany: If you’ve been on your platform for a while, it has forced you to mold all of your product data. We’ll just use products as our example because it’s the most variant of the data. It has forced you to mold all of your product data into whatever data structure that platform has provided for you. A lot of times we see that over time, that data becomes a little bit skewed. So now we need a flag for drop shipping so that we can indicate to customers on the front end that this item is drop-shipped, but there really isn’t one in the platform, so let’s use this other thing. Or let’s piggyback it on top of the fact that this is a product that can only be shipped using X shipping method. So all this logic exists inside the product data, or tangent to the product data, and that logic doesn’t necessarily need to be there anymore, especially if the new platform is more modern and has better handling of product attribute data.

Migration is the absolute perfect opportunity to take a step away from your platform and think about your data holistically.

Tiffany: Migration is the absolute perfect opportunity to take a step away from your platform and think about your data holistically. These are the pieces of information that I need to store about my product. These are the pieces that my customers need to know. These are the pieces that can be automated based on other factor and logic, and thinking about all of that also informs the development process in addition to the migration process.

Sara: I guess what I hear you saying is that data migration is really an opportunity to optimize your content in a way and be strategic about what you actually need to bring over versus just sort of bringing over all these potentially legacy information for the sake of bringing it over.

Tiffany: Yes, and depending on your platform, that trimming down of the data into its most perfect form sort to speak, is also gonna have a performance impact. So this is a really smart change to make all around. In addition to providing you with the opportunity to really think about how you want to structure your product data and what structure you’re bringing over, there are two other really important factors that come into play when you think about your product data.

Tiffany: The first one is that you need to think strategically about how you’re going to use that information in the future. A really good example of this is the difference between tags and meta fields in a platform like Shopify. How are you going to use that information on the front end of the site? How are you going to update that information later? If you are planning to continue to use standard import tools for creating your products, for example, you may want to opt for something that can be more readily updated via those tools. So thinking strategically not only about how you’re using it now, and how you’ve historically used it, but how you plan to use it in the future is going to have a big impact.

How are you going to use your data moving forward?

Tiffany: The second other important item is that you should never do this more than once. It is really, really important. Your product data, your customer data, and order history – they have baggage. It’s really important that you only make this move once. You’re already shifting the entire foundation of your business underneath your customer base. Even if we do the most seamless job imaginable, there are still differences. What you don’t want to do is migrate everything in, launch, get rolling, and realize that you’re missing a bunch of information that you need, or that you have too much information weighting you down and negatively impacting performance. Now you’re stuck in the same route you were in with your old platform, where it’s not really as mutable as it would normally be. So we advocate for thinking strategically about it so that you don’t have to do it again, so that you don’t have to sort of move the entire foundation again.

Sara: Yeah, that’s compelling. As you were talking I was thinking, what if there’s an ERP in place? How much more important does strategic planning become upfront in that scenario?

Tiffany: On one hand, having an ERP is a huge blessing to data migration because the ERP accesses a central hub. It’s already pushing data back and forth and we can use it in sort of a push in a one direction, right? We can take the information, we can tell the ERP to push all the information into the new platform, and now we’re not going platform to platform. We’re going from a central hub that’s already kind of designed to push data into an e-commerce platform. This will work from a lot of your information, depending on your set up – your mileage may vary.

But the flip side of that is that now you also have an ERP to deal with. Migration becomes not just about getting the data in the new place. But it’s also about how do we make the cut over at launch time to make sure that you’re not losing information? How do we test this ahead of time without polluting your production data, which something that we spend a lot of time working through the mechanisms for? A lot of safeguards go into place because it’s obviously much less than ideal if your test orders from all your QA process land inside your ERP and get processed by your shipping team.

Sara: Yeah. That’s not really that fun.

Data migration should only happen ONCE.

Interlude: You were listening to Recommerce, a podcast for e-commerce wearable brands navigating technical complexity and change. Brought to you by Command C, a development team that saves e-commerce retailers from outdated tech and ineffective operations, with a strong focus on Magento and Shopify Plus. You can learn more about how we help at commandc.com.

Sara: So I’m hoping that we’ve made the case for why an upfront strategy is so crucial in this scenario. Let’s kind of dive into the nitty gritty a little bit about the top level points of data that we mentioned, starting with products and customers.

Tiffany: So when we’re migrating product data, the strategy that we use for accomplishing the actual migration of the data really depends on how you can get it out of your old system. So how does it come out? Does it come out in a spreadsheet? Do we have access to the database that houses it? Are there existing tools, like an ERP for example, that would allow us to fetch that data in an alternate way? But the biggest determining factor as to how we accomplish the migration tends to be how clean is the data and how does it come out of the existing system? So if you tell me the only thing you have is a 70,000 rows CSV full of all of your product data, as long as that is in a machine-readable format, and by that I mean as long as it’s pretty standardized and there aren’t a lot weird things in it, we’re likely looking at a scenario where we are either rewriting that CSV into the format that the platform will take, or we are writing our own integrating with an API that’s gonna migrate all of that information in.

Tiffany: Most platforms that we work with have a CSV importer. It just seems to be the way that product data gets passed around, and it’s usually pretty easy to get the data into that format, and so a lot of times that’s the approach that we take. That also gives the merchant the opportunity to really scrub that data ahead of time without needing to modify it in their existing platforms. So there is some plus sides to using that CSV approach, although it’s probably the only time you will ever hear me say or advocate for a spreadsheet ever.

Sara: I am having déjà vu, I feel like this isn’t the first time you hated on spreadsheets.

Tiffany: Sorry.

Sara: Just kidding.

How do you plan to export your data for migration? CSV file? Or is there a platform tool available?

Tiffany: But it does give the merchant the opportunity to really scrub that data and remove whole columns that you don’t even need, and rename columns, and then it becomes a very collaborative migration process. There are also the sort of one-click tools. They’re designed to work platform to platform, and in many cases this is accomplished with the direct database access, and we provide access credentials for each platform, and then they click the button and it runs the import. This is something that if you have a small store, one that would almost be on the edge of maybe possibly you would consider migrating the products by hand, like 50 or less. That one click tool might be an excellent place to start without any testing. Just run it and see what it does, because removing 50 products from the new platform and starting over again with a more robust solution isn’t going to be a big deal.

Sara: Does everything that you just said apply to customers as well? Are those pretty kind of similar in structure and process?

Tiffany: Customers tend to be the most straightforward of the data migration that we run. I like to think it’s because we’ve been storing personal information about people forever. I mean, phone books, you name it, so we pretty much got that nailed. There are some interesting things to pay attention for. Some platforms won’t store multiple addresses, some platforms won’t let you assign default shipping and billing addresses, so there are some things to pay attention to, but for the most part, customer import tends to be really straight forward.

The only thing, I’ll give one little gotcha to that I guess, which is passwords. Usually, passwords don’t migrate well between systems. If you’re upgrading within the same system, like if you were going from version one of a platform to version two you might be fine, but if you’re migrating between two completely different platforms you may need a strategy for notifying your customers that their passwords have been changed. This is a pretty common security practice. So it’s not a huge pain point for most people as far as we’ve experienced.

Sara: Alright, the elephant in the room. What about order history?

Order histories can be tricky because typically an order is stored as a snapshot of product data in time.

Tiffany: I have a fraught relationship with this particular elephant. Order histories are very tricky. The reason order histories are tricky is that typically an order is stored as a snapshot of product data in time. It’s a snapshot of customer data in time. At the time the order was placed this product cost this much, the shipping method was this, the customer address for this particular order was this. So inside a single platform it’s easy to sort of run the tentacles out from that data to the existing current data or whatever the most recent iteration is.

But when we move it, that becomes a lot harder, and we have to think about what all about that order, just like with products, what all about that order do we even need, and should people be able to go and look up their order histories and be able to interact with them in the new platform the same way that they could in the old platform. The best example of this is a reorder function. A reorder function is very difficult to achieve on an order history migration, again because we’re talking about snapshots of products in time, products may not even exist in the new platform anymore.

Tiffany: So order histories, clear why they’re complex, they also tend to be large. If you’ve are in and been on already been on the same platform since 2013, that is a whole lot of order data, and I think the most compelling reason to consider an alternative to the order history migration is how frequently do your customers actually access this information? For some business models not having order history is a no-go, you absolutely must have it, but for many others, customers really aren’t accessing that data, and if they need it, they have your customer support team to reach out to. So in those instances, I would sometimes advocate for not migrating order histories at all.

There’s also a middle ground solution: we can write a tool that fetches order history on demand from the old system.

Tiffany: There’s also a kind of a hybrid middle ground solution where we don’t migrate the order histories, but we write a tool that fetches them on demand out of the old system. It does mean you have to keep posting live for the old system, but if we can fetch that order history on demand when a customer logs in and access it and we verify that they are who they say they are, then you’re needing to clutter up your new system with all that old data, but you’re still providing a layer of customer support at roughly the same level of functionality because again, we’re not really gonna accomplish reorders in any readily available way, and then we’ve covered the basic needs of customers when it comes to order history and we sort of backfill with our customer support team who can always look up the old data in the old systems.

Sara: Well, that’s incredibly informative and I think we’ve made our case. So thanks for the chat this morning. It’s great to talk to you as always.

Tiffany: As always. Thanks, Sara.