monsitj - Fotolia

Systems of insight light way ahead for data

Computer Weekly looks at why data management must become oriented towards business outcomes, not technology management outcomes

Data is only as good as the insights it produces, the actions it influences, and the results it fosters. That is the secret recipe for data management. Big data promises business-changing insights, but technology management is still the largest user and benefactor.

For enterprise architects to advance the business with data, data management must become oriented toward business outcomes, not technology management outcomes. In research conducted across 35 firms, Forrester found digital businesses, and some enterprises, reinventing their business models to create centres of excellence for systems of insight.

What makes these firms different is that they embrace an insight manifesto with five core values:

• First, the system is self-reinforcing by continuously planning, executing and learning from the results of insights introduced into business processes and decisions made.

• The second core value is hyper-collaboration. Insights teams are a cultural part of all areas of the business, and these teams create, share and refine insights in the context of business objectives.

• Third is the ability to operate at scale. Federation replaces centralisation at all data management tiers – organisation, process, architecture and performance.

• The fourth value is user empowerment. Everyone is an analyst, with access to data and tools to derive, use and share insights.

• Fifth is the ability to move quickly. The data supply chain is frictionless and allows insights to emerge when needed, to be applied in the context of action and to speed business results.

Priorities and competencies determine the enterprise architecture data management course to support systems of insight. Core technology management competencies (process, organisation and technology) emphasise the foundations of enablement with an eye towards flexibility, agility and skills. System-of-insight competencies push enterprise architects to expand their data management to emphasise the system-of-insight core values that produce insights to attain business outcomes and renew business models.

Big data investments

Business decision-makers view big data investments as mechanisms to optimise their resources – in other words, technology management objectives. Yet their expectations are that data will improve customer experience and increase revenue, which are insight objectives. Enterprise architecture professionals should transition to a system-of-insight data management model by assessing data management success against business stakeholders’ perceptions and the tangible outcomes they realise.

Data governance is a process and outcome, and is ineffective without the right balance of collaboration among enterprise architects, business data owners and subject matter experts, and executives. If you’re still selling data governance to the business, you don’t have data governance; you have a technology management process that will produce technology management outcomes.

Changes to data management as systems of insight mature with AI

• Systems of insight allow firms to work with data at scale, necessitating the rest of the organisation joining the management and governance effort as data owners and experts.

• Artificial intelligence (AI) systems rely on taxonomies and ontologies to decipher meaning as well as provide insight. This means data management technology will transition from integration architecture to insight architecture.

• With AI, data management and governance processes shift to training and educating based on the suggestions and outcomes of the data.

 Source: Forrester

Data management will become increasingly distributed as it assimilates into system-of-insight teams. While centralised data management processes for development will still exist, collaboration with analysts and data consumers will raise complex dependencies across business requirements, stretching enterprise architecture and development resources. Communication and collaboration pathways will be key, not only for the business but also internally for enterprise architecture and development teams. Also, adoption of agile data development will be crucial to scale to the demand.

Generalists with deep experience and expertise in what works and what doesn’t are needed to formally lead data management; some firms are hiring chief data officers for this. At the same time, enterprise architects need to foster data management teams that have strong specialists across core competencies for data integration and data warehousing, as well as engineering, modelling and semantics. This positions data management for modern architecture for big data, open source and cloud while delivering on advanced skills to address complex analytic and artificial intelligence technologies coming to market.

Data refinement

The days of enterprise architects building data silo fields and developers building application programming interfaces (APIs) independently to deliver data are over. A significant amount of data refinement and translation is still needed to go from a systems-of-record to a systems-of-engagement or analytic workbench.

Analysts and business data consumers with subject matter expertise demand self-service and will go around technology management if it gets in the way. Rather than fight this behaviour and push to enforce the standard, surrender. Work with the business’s best practices rather than trying to change them. Put data capabilities in place that make it easier to access and use the data. Introduce tools that help with sourcing, blending, cleansing and sharing data and insights. Be part of the insights team.

Donald Farmer, vice-president of innovation and design at Qlik, recommends failing fast and failing often. He says if you aren’t failing, you aren’t innovating – and businesses need to innovate. A system of insight thrives on a test-and-learn culture because it makes insights better, which fuels actions with better outcomes.

Measure business alignment, data governance, organisation, process, technology and delivery in the context of the influence they have on business outcomes. Then put the results back in to optimise your data management practice.

We are entering an age where algorithms aren’t just for data scientists, but are at the core of our business systems. IBM and Microsoft are leading this change by designing commercial platforms and intelligent applications to replace existing data management platforms and business workspaces. Our relationship with data will change from that of a farmer with hands in the data to that of a professor who trains the student. Putting a system-of-insight lens on data management that focuses on business outcomes and a test-and-learn paradigm prepares enterprise architecture professionals and organisations to effectively manage data in a highly ambiguous and contextually driven environment.

Systems of insight are helping enterprise architects transition from educating the organisation with insight to educating the system to deliver relevant insight. Enterprise architects must assess their data management practices with an eye on today’s data delivery as well as what is to come.

This article is based on Forrester’s “Evaluate your data management readiness for systems of insight assessment: the data management playbook” by Michele Goetz.

Read more on Database management