alunablue - stock.adobe.com

How Endowus is leveraging Kafka and microservices

The Singapore-based financial technology company built a distributed microservices architecture to ensure its investment platform remains resilient at all times

As a financial technology company, Singapore-based Endowus has to abide by strict service standards and rules imposed by the country’s financial regulator, including allowing its customers to redeem their mutual funds at all times.

That means the startup, which has more than S$2bn of assets under management, cannot afford to build its investment platform as a single application as that could take down the whole service if any part of the application fails.

“Separating the application into multiple applications and running them in physically isolated servers is a requirement and not an option for us,” said Joo Won Lee, chief technology officer and partner at Endowus.

To ensure its platform is resilient and can recover quickly from outages, among other requirements, Endowus adopted a distributed microservices architecture where services communicate with each other through Apache Kafka, the open-source distributed event streaming platform.

“We have a redemption service that just handles redemption requests, and that service must be live all the time,” said Lee, adding that the Endowus platform also interacts with other services such as SingPass, Singapore’s national identity system, for customer onboarding.

Through a managed Kafka service from Confluent, Endowus has been collecting event streaming data on customer interactions and other insights. This could range from drop-off points during onboarding to variations in risk appetites of customers.

Endowus has already acted on the onboarding data, which is now being used by a team of engineers and product managers to reduce onboarding time and improve customer conversion rates. “What we could potentially do further is A/B testing, which we are not doing enough of in our digital marketing campaigns,” said Lee, adding that this could include testing which landing pages resonate better with customers, for instance.

Read more about microservices in ASEAN

But being a nimble, born-digital company that works with microservices has not been easy for Endowus. As it focuses on managing investment portfolios and providing advisory services, it has to rely on UOB Kay Hian, a local brokerage firm, to hold customer deposits and reconcile transactions for its customers in Singapore.

“Not all partners we interact with are as digitally savvy as us,” said Lee, adding that UOB Kay Hian, for example, provides data in the form of files rather than events, which means Endowus has had to convert the data in those files into events in a standard format.

Endowus did not always rely on a managed Kafka service. It only signed up for Confluent just over two years ago, when it was growing rapidly and decided not to manage its own Kafka infrastructure.

“Engineers are expensive, and we probably won’t be able to hire many of them,” said Lee. “Plus, our investment team is consistently coming up with new product ideas which we want to support, rather than spend time managing Kafka clusters and worrying about security patches.”

Test environments

Lee said the Confluent service has also enabled his team of almost 70 engineers to spin up multiple test environments quickly by using application programming interfaces. This helped to speed up application testing, eliminating a key bottleneck that had made the company less agile.

The experience that Endowus has gleaned from building a distributed microservices architecture will come in handy in its upcoming expansion to Hong Kong.

With the ability to accept deposits directly from Hong Kong customers through a trust bank and execute trades without relying on a broker, the company is not only replicating its platform in the Chinese territory, but also building new brokerage and liquidity management capabilities, said Lee.

“While those capabilities are new to Endowus, they are not new to our engineering team as many of us had worked in investment banks and other companies that built similar systems,” he added.

Read more on Microservices

CIO
Security
Networking
Data Center
Data Management
Close