fair image - stock.adobe.com

CW Innovation Awards: SensorFlow reaps returns with re-architecture

Smart building management specialist SensorFlow re-architected its entire platform to reduce cost, complexity and implement a real-time business intelligence dashboard for its customers

Since it was founded in 2016, Singapore-based SensorFlow has been working with hotels and other building owners to make smart buildings a reality by combining the use of the internet of things (IoT) and artificial intelligence (AI)-driven automation.

Its customers included well-known players in the hospitality industry that need smart building management services. This includes maximising their energy efficiency, optimising operational productivity, while promoting sustainability. SensorFlow is staffed with 24 employees, with a total of four in-house IT staff and has an annual IT budget of $1m.

The company initially chose to base its solution stack on Amazon Web Services’ (AWS) DynamoDB and its Lambda serverless service. While this worked well initially, SensorFlow discovered that rising costs and IT complexities began to creep in. Its monthly infrastructure and resource costs skyrocketed to more than $50,000. Managing the serverless DynamoDB service became unwieldy and cumbersome for its engineers.

Other challenges included the rising cost of data storage, inconsistent data quality extraction due to the schema-less approach to DynamoDB, and information that was not easily accessible to people without programming knowledge.

This meant SensorFlow’s non-technical decision makers were not able to receive real-time reports and answer important business questions in a timely manner. It also meant that new reports or investigations required the involvement of all engineering teams across the stack to extract, process and present the data.

Re-architecting from within

Facing these challenges, SensorFlow decided to re-architect its entire platform instead of making incremental changes. The project was split equally between internal resources and external developers. The former mostly concentrated on front-end and application programming interface (API) layers for their dashboard development while the latter handled data migration and the design of data schema architecture.

The project had three main objectives: reduce IT infrastructure cost to less than $10,000 per month, improve data quality access and decision-making agility across the company, and implement a new real-time business intelligence (BI) dashboard for its customers.

To do this effectively, the team moved the entire back-end stack from a serverless DynamoDB-based architecture to a data-centric SQL-based architecture powered by Aiven. It also redesigned the front-end user interface for both customers as well as internal users, and integrated a BI platform, Looker BI, to serve as an insight delivery platform for the company.

The entire project took six to eight months and comprised 13 people from engineering to product teams, with a project budget of between $400,000 and $550,000. The return-on-investment savings of $40,000 per month gained from this re-architecting effort was expected to be about 13 months.

Overall, SensorFlow reduced the cost required to maintain and operate IT infrastructure by 80%, the ability to implement new dashboard features within days instead of weeks, and the speeding up of decision-making based on correct data. The project also allowed departments to engage SensorFlow’s self-service business intelligence portal instead of having to rely on help from the data science team.

Other benefits gained include the optimisation of IT infrastructure headcount so that only two employees can manage the workload and the improvement of product delivery and installation due to better actionable information.

“The project saved us thousands of dollars in infrastructure expenses and at the same time dramatically improved access to business critical data across all teams,” said Max Pagel, co-founder and chief technology officer of SensorFlow. “This democratisation of data has enabled all teams to work much more efficiently and sped up decision-making significantly.”

Challenges and lessons

One of the biggest challenges of this project was to convince the company of the necessity of a full re-architecture. Initially, the real cost of the old architecture was hidden due to the availability of AWS credits. To ensure the project took off, SensorFlow built a functional prototype to demonstrate business value.

The team achieved this by building out parts of the stack on a separate analytics database in order to demonstrate the possible improved speed and decision quality, when given access to a well-structured database. SensorFlow was also able to demonstrate the value delivered from this prototype to convince management and the rest of the company of the value of such an architecture.

Another important lesson learnt was to carefully assess good vendors who can manage the change management process and to work hand-in-hand with them on the change management journey.

Lastly, to ensure long-term success, SensorFlow created a solid information model. Being thorough at the beginning of the project was also key, as any mistakes in building the model could lead to significant problems.

“Embarking on a full re-architecture is a daunting task and a decision not to be taken lightly, but when done right, it can be a massive enabler for the success of our business in the future,” said Pagel.

Read more about CW Innovation Awards APAC 2023

  • Mondelez, the company behind Oreo biscuits, is tapping low-code and no-code development, drones and blockchain, among other emerging technologies, to drive innovation and improve efficiency.
  • Zuellig Pharma’s business-intelligence-as-a-service offering is touted to plug data gaps and has helped one pharmaceutical client improve customer loyalty.
  • The state government of Andhra Pradesh has digitised information on water supply pipelines and implemented a geographical information system to help deliver clean drinking water to rural communities.
  • DBS Bank’s Technology Marketplace portal has enabled the bank to speed up provisioning of infrastructure and applications through automation and improve developer productivity.
  • Moving to a 5G network has significantly improved video quality and reduced the time needed for Trandev Sydney Ferries to scour hours of footage, enabling the ferry operator to quickly investigate customer cases.

Read more on Information technology (IT) in Australia & New Zealand

Data Center
Data Management