Client type
Debt Fund
Key stakeholders
Multiple business lines
Location
Global
Project outline
- Our client wanted to harness a decade’s worth of diverse historical residential mortgage-backed security (RMBS) data for monetization
- The goal was to create a configurable platform to process, clean, and transform complex data sets and allow users to manage these data sets into the future with reduced vendor dependency
- After several in-house attempts had proven unsuccessful, StructureIt was engaged to bring a new perspective to the technical solution
The Challenge
Our client, a leading data provider, held over a decade’s worth of historical raw data from multiple corporate trust data providers. This data was identified as a key asset for potential monetization, offered as strategic insights to clients, but three previous attempts at automating the creation of consolidated datasets had failed due to the following reasons:
- Varied data formats and continually evolving schemas from multiple data providers made it extremely challenging to model and build a single golden data set that had a flexible ingestion process capable of accepting a variety of changing formats.
- The above problem was compounded due to the lack of alerting from the trustees’ data providers around data changes and modifications, and revised data just arrived.
The client was manually processing incomplete, inefficient, and resource-intensive subsets of the data with Excel templates for each dataset. The goal was to create a cleansed, consolidated, and normalised view of the data sourced from multiple sources and consequently quickly and efficiently onboard new data sets into the new truth model.
How StructureIt helped
StructureIt was asked to build a solution that could rapidly deliver a clear, quick return on investment. StructureIt delivered a cloud-based data ingestion platform that:
- Centralizes data collection from multiple disparate upstream data sources
- Automatically ingests, validates, and extracts data into a cleansed layer, consolidated into a single common normalised format with additional fields derived for analytic purposes
- Provides a generic configuration-driven solution that allows complete control and versioning of historical schemas within a database, enabling the extraction process to adapt to changing data structures over time
Through the solution’s configurable design, users can now validate new schemas and data independently, streamlining adaptation to new and evolving data sets.
The platform is now a monetized asset for our customers. By providing a centralized repository for cleansed, consolidated, and normalized RMBS data, our customer has been able to deliver analytical insights via Snowflake, Databricks and other data distribution layers, offering new data products to market and strategically increasing revenue numbers.
Alongside this, the analytics teams and domain experts are enabled to independently onboard and analyze new data sets within days, rather than weeks. This has also significantly reduced the business dependency on IT resources, and the platform’s ability to validate and adapt to new data significantly shortens analytics production cycles.
With this foundation in place, our customer is now expanding the platform to cater to additional data sources, which is crucial for future growth and adaptability in the evolving data landscape.
Collaboration, on your terms
We collaborate with you to develop personalized software solutions that contribute to your business growth in the capital markets
