End-to-end investment data solution to deliver value to investment data management.

• ChallengeInvestment data management solution using unstructured data and web app development from scratch
• SolutionData entry templates & pipeline on AWS Cloud environment for integration with a web app
• Technologies and toolsCloud environment AWS, AWS S3, AWS RDS, AWS Lambda Functions, AWS Serverless Application Model, AWS ECS, AWS ESR, Amazon Aurora PostgreSQL, Python, Flask, Github Actions

Client

The client is a group of investment specialists that approached InData Labs with a request for a data solution for their real estate investment platform.

The development of a complete data pipeline on the cloud required an end-to-end integration with the web application to process users’ inputs and enable all the investment management features that would allow general partners to thrive on fruitful investment opportunities by crafting a successful investment strategy.

Challenge: investment data management solution using unstructured data and web app development from scratch

Developing a solution from scratch is always a challenging task, as it requires constant alignment and balance between technical requirements and the customer’s vision of success.

We were challenged to develop a data solution from scratch, considering different data input formats, such as Excel files and PDFs, and their correct processing, enrichment, and integration to enable all the features planned in the web application which was being developed in parallel.

Our main objective was to develop two platforms separately and together simultaneously, ensuring that in the end, our team had a complete, integrated solution enabling functionalities, to provide users with a pleasant experience and a vision of comprehensive assessment of performance and business strategies.
https://www.youtube.com/embed/4SK1vqBq9x0?si=ras8028VOIAUFBED

Solution: data entry templates & pipeline on AWS cloud environment for integration with a web app

The InData Labs team started with analyzing the customer’s business to get complete understanding of its needs and requirements. The project was phased to deliver a minimum viable product and allow the customer to test their product in real scenarios.

• Data and performance reports from a group of potential users were gathered and analyzed in detail to map and understand the data to be treated, input formats, and indicators used, among other particularities.
• Input templates were created to enable a standardized, efficient way to feed the application with investment performance data and reports. Microsoft Excel was used to ease the process for users that are used to this format.
• Examples of structured and unstructured input file formats were gathered and processed for data extraction and integration with the cloud environment using Python, Lambda Functions, and other AWS Big data processing resources.
• The integrated data was enriched and modelled in a structured way to provide calculated KPIs, performance indicators, and strategic information. Data visualization was also developed using a Flask application to deliver instant access to investments’ metrics.
• The data solution was integrated with the web application using API to receive inputs and feed the application with investment data and metrics in a safe way.
• CI/CD was implemented using Github Actions to enable a fully integrated infrastructure.

We have completed an MVP development in 6 months by a team of 5 professionals: a Project Manager, an AWS Data Engineer, a Data Analyst, a Business Consultant, and a Technical Supervisor.

Solution Architecture

Result: streamlined processes and simplified investment management

Our team has built a reliable cloud infrastructure based on AWS that makes it possible to extract and process users’ investment data to simplify and add value to investment management.

As a result, we’ve provided the client with a managed data service solution for investment management that meets the client’s business  needs through optimizations such as decreased data crunching time, reduction of error and security risks, instant access to investment metrics; standardization and ease of processes for fund managers.

The InData Labs team connected the data solution with the client’s web application through API and provided data visualization through a Flask Application. The client can now ingest and consume data in a safe, practical, and efficient way, to craft a successful investment strategy.