Technology Stack: Cloud Data Platform: Azure, Data Platform: Databricks Data Intelligence Platform, Optimized Data Storage Framework: Delta Lake, Data Visualization: PowerBI, Machine Learning Platform: MLflow
The bank aspired to deliver its customers a truly digital banking experience. However, its legacy infrastructure and data warehouses on the on-premises Hadoop system were impacting this larger business goal. Leveraging the full potential of the large volumes of data, that they generated across hundreds of disparate source systems, became a challenge. The traditional setup presented data access complications and created data silos. This resulted in siloed data insights and best practices across the global teams working in isolation, impacting innovation at scale. Overall, the inefficient data processes and workflows slowed down business operations and the go-to-market capabilities of the bank.
As a first step, we moved the traditional Hadoop system to the Azure cloud. This transition to Azure not only brought unparalleled scalability but also set the stage to harness the Databricks Data Intelligence Platform for its advanced analytics and other capabilities.
Databricks Data Intelligence Platform unified data from multiple sources into a single platform. It created a collaborative environment for data engineers, scientists, and analysts. The fast and reliable data pipelines built using Delta Lake enabled access to high-quality data critical for accurate insights and model training.