Veritas survey provokes Talend opinion on GDPR data developer responsibility

A new study from information management firm Veritas Technologies has suggested that 86 percent of organisations worldwide are concerned that a failure to adhere to the upcoming General Data Protection Regulation (GDPR) could have a major negative impact on their business.

Nearly 20 percent said they fear that non-compliance could put them out of business.

NOTE: This is in the face of potential fines for non-compliance as high as $21 million or four per cent of annual turnover – whichever is greater.

Getting anal over data retention

There is also widespread concern about data retention.

More than 40 percent (42%) of organizations admitted that there is no mechanism in place to determine which data should be saved or deleted based on its value.

Under GDPR, companies can retain personal data if it is still being used for the purpose that was notified to the individual concerned when the data was collected, but must delete personal data when it is no longer needed for that purpose.

“There is just over a year to go before GDPR comes into force, yet the ‘out of sight, out of mind’ mentality still exists in organizations around the world. It doesn’t matter if you’re based in the EU or not, if your organisation does business in the region, the regulation applies to you,” said Mike Palmer, executive vice president and chief product officer at Veritas. “A sensible next step would be to seek an advisory service that can check the level of readiness and build a strategy that ensures compliance. A failure to react now puts jobs, brand reputation and the livelihood of businesses in jeopardy.”

So how can developers help?

As the new breed of developers starts to encompass data developers, data scientists and data engineers, what can data-centric programmers do to help?

Jean-Michel Franco is the product marketing for data governance man at open source integration data integration software products company Talend.

Franco says that GDPR is all about accountability and that in the past, only a few stakeholders in organisations felt concerned with the need to protect data related to privacy. This may have resulted in a somewhat uncontrolled proliferation of those data across systems and files folders.

“GDPR is game changing because it elevates the challenge at the enterprise level, with high stakes due to huge potential fine. Developers now need to establish privacy by design. This is game changing for a developer, they must consider privacy from each end every activity they manage,” said Talend’s Franco.

“This is not only true for new systems, but developers also must think about the legacy system that needs to be referenced and audited, this might need to be updated for compliance. For example, Business intelligence or analytics systems might expose sensitive data to a very wide range of users in an inappropriate way,” he added.

New data management challenges?

Franco says that the first challenge is privacy by design. As mentioned above, developers must consider privacy at the very beginning of the design of an application.

He urges us to think about a clickstream analytics application to track and trace customer journeys in the website. An application designer must consider whether this application contains privacy related data. If it does, they need to make sure they have collected the consent of the web visitors.

“In some cases, they might need to do an audit of the risk associated with this application in case of a data breach. Then they need to make sure this information can be referenced centrally in the organisation and reconciled with the other PII data, define policies such as those related to retention, and finally comply with the rights for the subject for accessibility and rectification,” said Talend’s Franco.

Talend’s Franco writes…

The concluding portion of this story is wholly attributed to Jean-Michel Franco.

The second challenge is data governance. Data governance is the new challenge in the digital era. There are so many data sources, users that want to be self-sufficient with data, and use cases that a central organisation cannot manage without engaging many users within the organisation for accountability.

As it is a team effort across the organisations, data governance policies are known to be challenging to establish. A regulation such as GDPR elevate data governance as a mandate, so companies now need the establish, or re-enforce, a dedicated organisation and set of best practices to deliver on the promise of data governance.

The third challenge relates to the transparency of algorithms. With the growing popularity of machine learning and artificial intelligence, decisions tend to be more and more automated. GDPR establishes the right to explanation, whereby a user can ask for an explanation of an algorithmic decision that was made about them. Again, this will have a huge impact, because it pushes accountability on data scientists and developers to enable explanation of the insight that are given as outcomes in their models.

It also a call for code of conduct to make sure that the use of advanced analytics avoid discrimination, knowing that a big step towards countering discriminatory algorithms is to ensure we can understand them.

Concluding thoughts

It is true, the rise of the data developer and the need to understand the difference between data engineers and data scientists has now come to the fore.