Scylla [pronounced: sill-la] was (and to all intents and purposes still is) a Greek god era sea monster whose mission is to haunt and torment the rocks of a narrow strait of water opposite the Charybdis [pronounced: car-ib-diss] whirlpool.
Ships who sailed too close to her (she was thought to have been created from a beautiful nymph) rocks would risk having sailors killed by the razor-sharp shards of Scylla’s darting heads.
Scylla and ScyllaDB on the other hand are neither mythological, sea-based or dangerous to your health… but this open source-centric real-time big data database does have shards.
Scylla uses a sharded design on each node, meaning each CPU core handles a different subset of data. It is fully compatible with Apache Cassandra and embraces a shared-nothing approach that increases throughput and storage capacity as much as 10X that of Cassandra.
A shared-nothing architecture (SN) is a distributed computing architecture in which each update request is satisfied by a single node (processor/memory/storage unit) — the intent is to eliminate contention among nodes… and in this way, nodes do not share memory or storage because they access them independently.
ScyllaDB has now announced that Southeast Asia’s leading ‘super app’, Grab, is using Scylla Enterprise to support real-time data processing and time-series data for the millions of transactions it processes every day.
What is a super app?
According to gojek, “A super app is many apps within an umbrella app. It’s an OS that unbundles the tyranny of apps. It’s the portal to the Internet for a mobile-first generation. More often than not, it will likely be operate at the intersection of logistics/hyper-local delivery, commerce, payments and social.”
Grab offers a wide range of everyday services such as on-demand rides, food and package delivery and mobile payments. The Grab app has been downloaded over 152 million times and its service area covers Cambodia, Indonesia, Malaysia, Myanmar, the Philippines, Singapore, Thailand and Vietnam.
Every Grab transaction needs to be processed in near-real time.
To ensure its operations stay fast, responsive and reliable, Grab developed a microservices architecture based on data streaming with Apache Kafka. These streams power Grab’s business and provide a source of intelligence.
Close to the metal
Grab’s engineering teams aggregate and republish the streams using a low-latency metadata store built on Scylla Enterprise, a massively scalable real-time NoSQL database designed for high-performance applications. Coded ‘close to the metal’ in C++, Scylla Enterprise offers high availability with improved latency, throughput, resource efficiency and fault tolerance.
Grab’s Trust and Safety Engineering team uses the metadata stored in Scylla to monitor and analyse activity streams for signs of illicit transactions.
“Scylla does a fantastic job for us in helping Grab process the massive amounts of data generated everyday, and turns it into usable information for the rest of the business. When you deal with millions and millions of events coming in everyday, being able to efficiently find and remove duplicate events, pull together aggregate data, and run joins across multiple real-time streams can be challenging,” said Brian Trenbeath, a technical program manager at Grab.
As ScyllaDB’s premium database product, Scylla Enterprise offers the highest level of QA, scale, security and performance testing, along with client-customised hot-fixes, formal EOL policies and 24/7 support.