Case Study

Data Integration System Implementation

A technology solutions provider for supply chain financing in APAC overcomes data integration challenges by leveraging WSO2 and AWS solutions, streamlining onboarding, ensuring flexibility, and enhancing scalability.

Harnessing WSO2 Integration and AWS tools, a leading working capital solutions company modernized its data platform, conquering integration hurdles, and achieving efficiency, flexibility, and scalability.

Download PDF

Industry & Region: Financial Services, APAC

Technology Stack: API Integration: WSO2 Integration Platform, Bank-side Integration: AWS S3 and Lambda Functions, Connectors: WSO2 native Connectors for Flat files, AWS S3, Containerization: Docker hosted on AWS ECS (Elastic Container Service), Log Monitoring: AWS CloudWatch, Data Sources: SAP, Salesforce, MS Dynamics
Client Overview
Our client is a technology provider that helps businesses with their supply chain financing operations. They specialize in crafting dynamic, industry-agnostic solutions that accelerate financial processes within the supply chain. Their expertise spans “buy-side,” “sell-side,” and “bank solutions,” all designed to seamlessly address their clients’ financing needs.
Business Challenge
The client built a data platform as their business offering, which was turning out to be old and incompatible. This platform’s earlier success was built on its ability to seamlessly acquire data from diverse source systems through custom-built API interfaces, tailored to each source system’s unique data model. As the client’s customer base expanded and the data ecosystem grew more complex, they encountered a significant challenge.
With increasing demands to integrate multiple source and target systems, they faced inefficiencies in retrieving and ingesting data from these diverse endpoints. This posed a risk to their data flow’s smooth and uninterrupted operation. Hence, the client needed a solution that could streamline the process of connecting to various source and target systems, ensuring the efficiency and flexibility necessary to accommodate different API formats and specifications, matching the unique data models of the source systems.
Solution Offered
Our team initiated the solution by conducting a thorough assessment of the client’s existing data flow. This included gaining a deep understanding of the source and target systems, as well as the functionalities behind them. This step was critical in identifying the specific integration challenges and requirements. The solution offered to the client was a comprehensive approach to streamline data integration, enhancing their ability to efficiently retrieve and ingest data from diverse source and target systems while accommodating various API formats and specifications.
  • API Specification Design and Optimization
Once the assessment was complete, the team designed and optimized API specifications to cater to the client’s unique needs. These specifications served as the blueprint for creating the integration solutions. The goal was to ensure that these APIs seamlessly connected to the various source and target systems, facilitating data retrieval and ingestion.
  • WSO2 Integration Platform
The cornerstone of the solution was the WSO2 integration platform. This powerful platform was chosen for its robust capabilities in designing, developing, and optimizing APIs. WSO2 provided a flexible and scalable framework to create and manage the required integration components effectively.
The solution leveraged connectors and mediators to address specific challenges and requirements:
  • Easier Client On-boarding
A custom API was designed to streamline the on-boarding of new clients into the system. It enabled users to add client details and metadata information into a file, making it easy to access specific client information when needed. The “file” read connector offered options to configure local, FTP, or SFTP connections, and dynamic reading. Additionally, this API facilitated encryption/decryption of sensitive client Personally Identifiable Information (PII).
  • Efficient Validation of Invoices and Loading into Target Tables
Data from various source systems, with different file formats (.csv, .json, .xml), were efficiently processed. The “call” mediator within WSO2 allowed for easy configuration and calling of different endpoints, simplifying data retrieval. The “Data Mapper” mediator was utilized to implement business rules for every column and map them to the target columns. Data was loaded into the target database using the “DBReport” mediator. The “Iterate” mediator helped process each record, while other mediators facilitated data format conversions and specific handling based on the uploaded file type. The “Payload” mediator ensured that data was properly formatted for API responses.
  • Integration with Bank Systems
AWS connectors were used to integrate with AWS S3 and Lambda functions for interactions with bank systems. After loading data into target tables, the system fetched the required information, wrote them to files, and sent them to the bank. “DSS” mediators allowed for interactions with Data Services, configured to perform necessary database operations. The system was equipped to handle responses from the bank, with the “switch” mediator routing different logic processes based on response files. Additionally, the Integration Studio offered a “Scheduled Task” option for automatic file reading at specified time intervals.
  • Deployment and Monitoring
Deployment of the solution involved generating a “.car” file for each API, containerization using Docker, and deployment in AWS ECS. AWS CloudWatch was used to monitor system logs, ensuring the health and performance of the deployed APIs.
  • Incremental Load Testing
To guarantee that the APIs met the client’s performance requirements, incremental load testing was performed. This testing approach allowed for the assessment of the APIs’ capabilities under various loads, ensuring they could handle increased demands and maintain efficient data flow.
Value Delivered
  • Streamlined client on-boarding, saving time and effort.
  • Provided great flexibility in data handling by strengthening the system to accommodate various data formats and sources.
  • Reduced development time by using connectors and mediators.
  • Enabled the system to be scalable to meet growing demands.
  • Ensured reliable system performance through rigorous testing.
Ready to upgrade your data integration capabilities? Reach out to KANINI today for expert guidance and data solutions tailored to your specific needs!
Talk to us
© 2024 KANINI Software Solutions | All Rights Reserved | Privacy Policy

Data Integration System Implementation