Welcome to the third article of our Digital Transformation Series!

We covered what could go wrong in telco digital transformations before going into what the ideal telco-to-techco company could look like in our earlier blog posts.

Here we’ll cover an equally important topic, handling data migration from the old software to the new.

‍

What Does a Telco Migration Look Like?

As telcos, we deal with a lot of data. 

There are customer details, mobile plan types and their different value-added services like caller ID and roaming, just to name a few. Older data like legacy plans, or missing customer details from an older time can throw a spanner into the works if the legacy plan data is not properly migrated.

All this data needs to be transferred from legacy business support systems (BSS) onto the new infrastructure without any data going missing. One approach to this is the ETL (Extract, Transform and Load) approach.

Data is kept in various locations in different formats. During the extraction process, all of this data is identified, mapped out, then extracted. 

During the transformation step, the data needs to be converted into a format that the new database can use. During this phase, data will be validated and observability reports will be run to minimise bad data being loaded into the new system. 

The loading stage refers to loading the cleaned and transformed data onto the new platform. 

At Circles, we run both our own telco brand and also provide our telco software as a base for international partnerships, such as with KDDI and e&. We also provide telco data migration services for our partners as well. 

We use the ETL approach to data migration as it lets us migrate large volumes of telco customer data without compromising quality. Telcos come in various sizes and no two migration journeys are the same, which is why we’ve incorporated extensive discovery discussions, scenario testing, and roll-backs and rehearsals into the process. 

‍

Circles’ End-to-End Migration Journey

1. Exhaustive Analysis and Discovery

Before data extraction can begin, we need to analyse what data needs to be transferred, mapping where all the data is and needs to go and figure out all the dependencies in the data migration plan.For telcos, this means understanding data like plan types, add-ons and value-added services, customer profiles, and bill run cycles. To ensure that we don’t miss anything, our teams will work with domain experts and business teams in your telco to identify and map out all these details.A lot of things can be missed without proper planning, such as unclear scopes and roles and responsibilities (RACI), poorly mapping out data and dependencies and much more. Speaking to as many key stakeholders as we can and aligning everything in the beginning is vital.

2. Migration Planning

Planning involves everything from roles and responsibilities, the scope of work, success metrics and error margins. These aren’t done in a vacuum and plans will heavily involve the client telco’s domain experts and business leaders. It also needs to factor in service disruptions and slowdowns while considering the costs of running the systems: do you run both the legacy and new infrastructure together, or do you migrate everything one-shot?The following will also be covered during migration planning:

• Migration schedules and timelines
• Resource planning
• KPI definitions e.g. stability periods (time between data migration batches to see if everything is stable), success metrics, error tolerance
• Roll-back plan definitions
  
  

3. Design and Develop

Here’s where we’ll design the customer and plan configuration setup. Once that’s done we’ll start extracting and transforming the telco data onto a staging environment, where the following is done:

• Staging data population & correction
• Staging data enrichment

The following will be developed at this stage as well:

• Data loading script
• Testing script
• Roll-back script
  
  

4. Functional Testing

Once the tools and scripts are developed and the extracted data is placed in a staging environment, we can begin transforming it into meaningful formats that the new system can use.After the data transformation, the following is done:

• Data validation tests
• Scenario testing
• Observability reporting and migration reporting

For visibility and proper project management, migration reports will be done at every stage from this stage onwards.

5. Prep and Rehearsals

This stage involves data loading and wrapping up data extraction. The following is also done:

• Dry-runs of the migration 
• Rehearsing Roll-backs
• Data Validation tests
• Data and environment cleansing based on the validated data from the dry-runs
• Fine-tune the migration process based on rehearsal results
• Migration reports will continue being done throughout this stage as well.
  
  

6. Execution

At this stage, the data loading begins.This stage can be stretched or compressed according to business priorities and how the data migration is structured. If it’s a “big bang” migration, there will be down time and all the data will migrated in one single phase from the old system to the new system. For phased migration, we can stretch the phase accordingly to fit in the pilot test, multiple migration phases and also stability periods between batches to monitor the stability of the systems during the migration.Validations and audits are conducted after each migration batch to ensure that everything is there, and the roll-backs that were rehearsed earlier can be carried out if anything goes wrong.During this phase, we’ll work with the telco to manage customer communications during the migration too. 

‍

7. Post Migration Activities

Once the whole migration is done, we’ll validate the migrated data in the new environment and monitor the systems during the stability period.We’ll iron out and address any post-migration issues and optimize performance while we operate and maintain the new system.When the project is complete, we’ll also run a post-mortem on it.

Potential Risks

Without careful planning and proper processes, issues that affect the customer experience, like this anecdote about two banks merging in Malaysia, can occur.

A bank was acquired in Malaysia by a competitor, and its credit card data needed to be migrated to the acquirer’s database. However, the migration was incomplete. 

Some customers weren’t able to pay their credit card bills and had their cards deactivated but couldn’t pay the bills because the billing information wasn’t correctly migrated. People getting punished for a goof up during a migration is not a great customer experience.

Careful planning and processes helps to mitigate the following outcomes:‍

Loss of Revenue

Wrongly Migrated Charging Logic: For example, if a rate is mistakenly set to $1 per minute instead of $1 per second, it could result in significant revenue loss.

Recurring Add-ons or VAS Subscriptions Not Migrated: Subscription services may be disrupted if they are not properly transferred.

Incomplete Business Rules/Logic Migration: For example, if only Plan A can subscribe to Add-on B, this rule must be correctly implemented in the new system to avoid service issues.‍

Loss of Data

Legacy Plan and Business Logic Loss: The nuances of old plans and business rules might not be fully captured in the migration, leading to potential data gaps.

Disruption of Service

Incomplete Plan Migration: Not all service plans may be fully transferred, affecting customer access. This ties in with the earlier bank data migration example.

Missing Customer Data: Critical details or past records could be lost during the transfer.

‍

Risks

These risks should be accounted for when planning and executing the migration:

Data Loss and Semantic Risk 

When migrating, we need to ensure consistency between old and new databases. All relevant data from the old system needs to be migrated and transformed into usable formats that the new database can use.

Connections must also be maintained between interdependent data, such as a customer’s data consumption, call duration, frequency of use and more being tied to that customer’s profile.

Semantic Risk refers to data being incorrectly mapped or interpreted in the new system, which makes the migrated data unusable. This can happen if the data is not properly mapped and transformed.

Interference risk during the migration process

Interference risk refers to multiple stakeholders using the application simultaneously during the migration process. For example, if one party locks a customer details table, that prevents other parties from using the table.

Two stakeholders migrating the same data at the same time can also result in duplicate data.

Incomplete business information: 

If you have mapping gaps between business processes and the metadata they need, the migrated data cannot be accessed by relevant business units.

Metadata refers to providing information about the data itself, such as what it is, when it was made and by whom, where it’s stored, who can access it, and who owns it.

This can be prevented during the planning process by meeting all key stakeholders from departments like commercial, IT, finance or revenue assurance, operations and the rest.

Scope and Dependencies Clarity and Unexpected Changes

The data migration project’s scope includes parameters like the database’s object types, source objects in scope and more. However, like most IT initiatives, this process is prone to scope creep which can quickly burn a hole in the project’s budget.

Different data is depended on by different applications and departments. Without clarity on this, the migration could negatively impact different departments.

Unexpected change requests can result in multiple iterations of the data migration, slowing down the project and increasing costs.

These are just some of the risks associated with data migration. Now we can cover some of the mitigation strategies that we can apply.

‍

A Non-exhaustive List of Risk Mitigation Strategies

These strategies are incorporated into the planning and execution process we mentioned above, but here we can dive in a bit deeper.

Exhaustive Mapping of Existing Systems:

To cover data loss, semantic risks, incomplete business information and dependency risks, exhaustive mapping of the telco’s data and existing systems is a must.

During planning, we’ll conduct deep dive sessions with domain experts to map out details and also leverage business teams to accurately map customer data on both the old and new systems.

‍
Pilot Tests, Phased Migration and Stability Periods:

We use staging environments to test the migration and rehearse roll-backs in case of emergencies. The ETL software that we’re developing supports dry-runs and rehearsals.

After the dry-runs, we can then try pilot tests. This can be done on employee data first to minimise any impacts on customers during testing. After that we can move onto phased migration, such as migrating the first 10 percent of the users.

Between phases, there will be a monitoring period of either 1 month or 1 billing cycle to iron out any issues from that batch.

Customer Communication Plan and Down Time Management:

There will either be service downtime or slower service during the actual data migration. Having the right plans in place to communicate with your customers helps them to prepare themselves in advance and limit impact on their mobile experience.

We recommend giving them at least two weeks advance notice of the migration, and to prepare for either a service down time or for slower service during off-peak times like between 2 and 4 AM.

During the downtime, they may still need to reach you so we recommend providing alternative channels for them to reach your customer service such as hotlines or customer service email addresses during the migration.

Back-Up, Rollback and Recovery Plan:

Data migration projects tend to be long and complex with many things that can go wrong.

With that in mind, we plan and rehearse roll-back plans and engage in scenario planning and testing to keep ourselves prepared.

That way, if anything happens, our teams will be ready to roll-back the database when the needed.

Performance Monitoring

The data needs to be observable, validated and usable after each extraction. 

We run multiple migration reports along with data validation checks and observability reports using various tools to ensure that nothing is missing and everything can be accessed.

We use these tools and more in this process:

• Data Bridge
• Grafana
• Apache Superset
• Metabase
• Prometheus

Entire Plan Enabled by a Robust PMO

A complex tech project needs a robust project management team. This team will handle the following:

• Alignment with Stakeholders: Ensure all key stakeholders (business owners, finance, revenue assurance, IT security) are on the same page.
• Success Metrics and Error Threshold: Define clear success metrics and acceptable error thresholds.
• Clear Roles and Responsibilities: Assign specific roles and responsibilities to each stakeholder.
• Scenario Planning: Conduct rigorous testing and rehearsals to prepare for different scenarios.
• Data Governance, Compliance, and Access Management: Ensuring adherence to regulations and proper data handling.

Data migration is a complex but essential part of digital transformation for telcos. 

By understanding the risks and implementing robust mitigation strategies, telcos can ensure a smooth transition to new systems. Circles' migration service is designed to help telcos navigate this journey efficiently, minimizing risks and maximizing benefits. 

Ready to transform your telco operations with seamless data migration? Contact us today to learn more about our migration services and how we can help you achieve your digital transformation goals.