SaaS criteria to track for business outcomes

Evaluating small players

Many of today’s leading SaaS suppliers are small, but firms shouldn’t disregard them outright based on their size. They are also often unprofitable initially – the SaaS subscription-based pricing model contributes to this.

But buyers should be cautious and consider other key factors related to their viability: How much revenue does the provider generate annually? Is the provider particularly susceptible to currency fluctuation? If so, which currencies and why? If privately owned, who are the provider’s investors? What are their ties and backgrounds? Is the SaaS system just one part of a portfolio of software, or is the company 100% SaaS?

These questions will help you move beyond the obvious evaluation points – and also help you evaluate these suppliers fairly based on the maturity of the marketplace. In some cases, you can also ask about SaaS escrow relationships that may give you an extra level of protection in case the supplier goes out of business.

Performance and business continuity

Forrester recommends you measure actual performance and uptime, inclusive of planned maintenance windows. Increasingly, SaaS suppliers are making it easy for their customers to track these metrics by exposing data through customer portals or even public websites. You can consider third-party uptime monitoring tools as well, such as those from McAfee or HP Enterprise. Also, ask your provider about recovery time objective, which is the time to get data back after a failure, and recovery point objective (RPO), which is the time between backups – both of which translate into risk of data loss.

Typical uptime guarantees in SaaS agreements range from 99% to 99.99%, but are usually exclusive of planned maintenance. Typical RPO is 12 hours, which puts you at some risk for data loss. You can consider SaaS backup tools such as Backupify or SaaSify for additional control and redundancy over data backup.

While SaaS suppliers have made good progress on criteria that matters to business, such as access to new features and speed to deploy, SaaS leaders are pushing the envelope to drive business outcomes and to show that they drive business outcomes.

Of course, this assumes that you have clearly defined your business goals and business case for using the SaaS systems, which is not always the case – particularly when business buyers are deploying SaaS under the radar. Hint: The real value in SaaS is generally the business value, not necessarily cost savings.

Tracking and benchmarks

Few organisations track whether or not the SaaS tools they use actually improve their customer experience, customer retention or customer satisfaction. Your customers’ satisfaction may be more directly correlated with certain SaaS systems – e-commerce, for example – and less directly related to others, such as human resources applications or salesforce automation applications.

This is an important metric to track, especially if meeting the rising expectations of your customers drove you to SaaS in the first place.

SaaS suppliers are increasingly providing you with proactive health checks and dashboards for tracking use. Most of these are high level – number of logins, for example – but SaaS suppliers, such as Salesforce.com’s Cloudpulse or SuccessFactors’ Employee Central, and add-on tools such as Gainsight and Applango, are starting to offer more meaningful dashboards that show you detailed analysis of utilisation rates and patterns correlated with business results, such as login and use behaviour for top-performing sales reps.

Nearly 15 years on since modern SaaS systems such as Salesforce, Google Apps and NetSuite first emerged, we still have very limited benchmarks on business metrics. Look for this to change with the increased focus on analytics, demonstrated by the introduction of tools such as Salesforce’s Wave and Workday Big Data Analytics. Workday offers various compensation benchmarks, as well as “smart” recommendations based on machine learning and technology it acquired from startup Identified – such as who is at risk of leaving and what changes might influence them to stay. Other examples are IBM’s industry benchmark for its digital analytics customers and SuccessFactors’ Workforce Analytics product.

Along with metrics, you should assess how well the SaaS system connects to your ecosystem of customers and partners. SaaS promises to make it easier to connect with suppliers and customers since it uses the public internet and open standards‑based application programming interfaces.

Some SaaS suppliers go even further by prebuilding connections and creating communities.

For example, Ariba is not just a SaaS version of a sourcing tool, it is a fully integrated business network that you get access to as soon as you subscribe. Some SaaS suppliers also have connections to customer communities that could be valuable to you. Leading SaaS suppliers, for instance, might integrate with Facebook or Twitter for closed-loop communication with your customers on service or sales issues they are facing. Monitor the value of the community – and whether it is relevant to your business needs.

This is an extract of the Forrester report “SaaS Vendor Evaluation: Criteria To Track For Business Outcomes” (January, 2015) by Forrester Vice President and Principal Analyst Liz Herbert.