Big data analytics and infrastructure deployment company BlueData has attempted to explain why developers and data scientists working on the front line of big data implementations need to consider themselves as team players.
VP of marketing Jason Schroedl spoke to the Computer Weekly Developer Blog to explain that the data science community has to play a team game to the extent that it actually adopts its own kind of DevOps model.
#1 job in the USA
The claim is that this new role of ‘data scientist’ is the #1 job in the USA for the second year in a row.
But who are these people?
Schroedl says they are highly skilled at developing advanced analytical models and prototypes and often come with a core background in software engineering as a developer with an appreciation for higher-level system architecture needs.
But there’s a problem with data science
The issue is that the siloed efforts and custom-crafted prototypes of individual data scientists can be difficult to scale, reproduce and share across multiple users.
“What works for an ad-hoc model in development may not necessarily work in production; what works as a one-off prototype on a laptop might not work as a consistent and repeatable process in a distributed computing environment,” said Schroedl.
How do we fix this issue? There’s you team sport twist.
The suggestion from BlueData’s Schroedl is that there now needs to be a coming together of involving multiple data scientists, data engineers, data analysts and developers that have different skillsets and different knowledge of specialised tools.
“What’s needed is an approach that brings the agility, automation, and collaboration of DevOps to these data science and engineering teams. They need to operationalise the data science lifecycle in a streamlined and repeatable way. They require an Agile and lean process that enables them to iterate quickly and fail fast. They need the ability to easily share data, models and code in a secure distributed environment. Finally, they need the flexibility to use their own preferred tools and try out new technologies in the rapidly changing field of data science,” said Schroedl.
Team sport DevOps in a big data-as-a-service world? That’s bdass.