Snowflake founders reveal ‘cuckoo cloud’ vision that disrupted big data

In 2012, with the public cloud gaining traction and the concept of big data well-established, Snowflake founders Benoit Dageville and Thierry Cruanes recognised that analysing large datasets required significant computing resources for short periods, making it ideally suited for cloud deployment.

They also realised that when “large amounts” meant petabytes, combining data warehouses and analytics capabilities was necessary to deliver answers almost immediately rather than the next day.

So, the two men left Oracle, where they were working, to develop these ideas. From the start, they settled on an as-a-service model, where the company would shield customers from the complications of provisioning, and provide users with a level of simplicity that would democratise analytics.

That might sound straightforward, but Dageville said it took a few months of “heated discussion” to develop a suitable architecture.

“We liked the idea of building something,” he told Computer Weekly in an interview at a Snowflake event in Melbourne. And if they did fail, the worst that could happen was that they would go back to being employees.

They attracted a seed investment from venture capitalist Mike Speiser of Sutter Hill Ventures, who initially served as Snowflake’s CEO.

“Big data was all the rage” in 2012, recalled Cruanes, but the prevailing view was that Hadoop was going to be the dominant platform, so investing in Snowflake was a bet on a future where that wasn’t true.

Dageville said the pair were convinced there were many reasons why Hadoop would fail. They had a plan and knew they wanted to build what would become Snowflake, but they didn’t know if it was possible.

Fortunately, Speiser was well known, and his involvement served as a magnet for other investors. By the time Snowflake had built a talented team, the worst exit strategy would have been a buyout, with very little chance of the early investors losing money. As it happened, Speiser stayed the distance, and his share of the company was worth $1.2bn when Snowflake went public in 2020.

Data sharing was part of the original concept. The idea was that “any compute can access any data”, explained Dageville, but only with the data owner’s consent. But should Snowflake support one-to-one sharing, or should the company act as a broker? While Snowflake used data from Twitter, now X, to demonstrate sharing, it realised that public data was not very interesting and that it would be better to provide a market for proprietary data.

“We knew that if people adopted the platform, it would become an ecosystem,” he recalled.

Having created a unique architecture, it was necessary to present it to potential customers in a familiar way, so the early conversations were around data warehouses and virtual racks of hardware, said Cruanes.

But once the Snowflake platform was established, it was possible to bring in real innovation, such as a marketplace for data that could be used in place, without having to copy or transfer it to another system.

One of their ideas was to combine structured and unstructured data, but as Dageville pointed out, storing unstructured data is easy, but what do you do with it after it has been stored? Now, artificial intelligence (AI) means a machine can work with structured data, and “that’s a big revolution”, he said, even though integrating AI with a data platform was a difficult job.

It has been a long time since the co-founders turned their vision into a product, but integrating new ideas with the platform requires deep knowledge. According to Dageville, he and Cruanes are still coding on a regular basis, but mostly on weekends, working on smaller projects so they can stay grounded on the product without becoming a bottleneck in the development process.

“Coding is the happy place for both of us,” admitted Cruanes.

Customers will drive Snowflake’s future direction, said Cruanes, but wherever it goes, Snowflake will continue to make technology easy, simple and available to help them solve their problems and enable them to do business.

The next innovation won’t come from Snowflake, Dageville predicted. It will instead be seen in what customers do with the platform. The company’s job is to evolve the infrastructure and incorporate new technologies that will make that innovation possible.

Snowflake’s senior vice-president for Australia and New Zealand, Theo Hourmouzis, pointed out that a big difference between Snowflake and its competitors is that it provides a complete product, not a set of parts.

While Amazon Web Services (AWS), for example, offers a range of task-specific products, such as Redshift and SageMaker, which can be connected, they are not tightly integrated, said Dageville. Snowflake, in contrast, builds a single, fully integrated cloud platform, not just a product.

Therefore, the company views AWS, Microsoft and Google Cloud as competitors. However, because it doesn’t build its own infrastructure, “we are a cuckoo cloud”, providing the “operating system” for customers to build applications, said Dageville. 

Furthermore, Snowflake is “opinionated”, preferring to offer what it considers the best approach, rather than multiple options to achieve a certain outcome.

This enables customers to build complete applications incorporating AI and streaming technologies more quickly, particularly with built-in security, he said, adding that the process is “as seamless and fast as we can make it”.

When asked for entrepreneurial advice, Cruanes initially demurred, but then suggested that everyone should embrace emerging technologies and explore their potential applications.

Everyone wants to make an impact on their organisation, suggested Dageville, and that is enabled by democratising analytics by applying AI, which Snowflake makes possible.