Postgres to MongoDB Migration
Faster is better! The data migration analysis and specs that made core systems 10× faster.
TLDR;
I wrote the data migration analysis and specifications to move core systems from Postgres to MongoDB as a part of a wider legacy platform transformation project, cutting response times from 2s to 200ms.
Outcomes
- Established a migration specification: Provided technical teams with documented requirements for mapping legacy data structures to the new product's data model.
- Bridged the gap between legacy and new architecture: Translated differences between relational and document-based data structures into requirements that could guide implementation.
- Supported data continuity: Defined how existing campaign, booking and document information needed to be represented in the new product, helping preserve its business value through the transition.
- Enabled alignment between product and engineering: Created a shared reference for understanding the legacy data and the requirements of the new system, reducing ambiguity during implementation.
What I learned
- Data migration requires understanding the business, not just the database. Fields and relationships often represent important business rules and workflows. Understanding their purpose is essential when moving information into a new architecture.
- Clear specifications reduce ambiguity for technical teams. When replacing a legacy system, documenting the source data, target structures and mapping requirements creates a common reference point for product and engineering.
- A change in architecture is an opportunity to rethink how data supports the product. Moving from a relational database to a document database requires consideration of how information is organised and accessed, rather than simply replicating the legacy structure.
Context
Central was a new product being developed to bring campaign briefing, booking information and supporting documents into a centralised hub. It was built to replace an existing legacy product used by the WA and ACT federal government. As part of the rebuild, the underlying data architecture was changing from PostgreSQL to MongoDB, requiring the legacy data structures to be analysed and mapped to the new product.
Problem
The legacy product and Central used fundamentally different database structures, with no existing mapping between the two. This meant the technical teams lacked a clear specification for how legacy data should be represented in the new product. Without this analysis, there was a risk of losing important information, breaking relationships between records, or migrating data in a way that no longer supported the business workflows of the new product. The challenge was to establish a clear source-to-target mapping and define the migration requirements so that the technical teams could migrate legacy data into Central accurately and consistently.
My role
As Product Owner for Central, I was responsible for analysing the existing data structures and developing the specifications required to support the migration from PostgreSQL to MongoDB. This involved understanding the legacy data, determining how it needed to be represented in the new product, and translating those findings into documentation that technical teams could use to implement the migration.
Discovery
I needed to establish an understanding of how data was structured in the legacy product and how that information supported campaign briefing, booking and document management workflows. The analysis focused on identifying the existing data fields, their purpose, and the relationships between different records. This provided the foundation for mapping the legacy data to the new product's data model and identifying where the differences between the two systems required changes to how information was represented. The objective was to ensure that the migration specifications accounted for both the technical differences between PostgreSQL and MongoDB and the business context behind the data, rather than simply transferring fields between databases.
Solution and delivery
I developed the data migration analysis and specifications to provide technical teams with a clear reference for migrating data from the legacy product into Central. This included documenting the legacy data structures and mapping source fields to their corresponding representations in the new product. The specifications established how existing information needed to be interpreted and carried forward into the new data model, providing a shared understanding of the migration requirements between product and engineering. By translating the legacy data structure into documented requirements, I helped bridge the gap between the existing product and the new architecture, giving the technical teams a foundation for implementing the migration while preserving the information needed to support Central's functionality.