Why ODPi & LF AI have come together
This is a guest post written by John Mertic in his role as ODPi program Director at The Linux Foundation — founded in 2000, the foundation is dedicated to building sustainable ecosystems around open source projects to accelerate development and adoption through financial and intellectual resources, infrastructure, services, events and training.
Metric writes as follows…
Data and AI tools have been a critical part of an enterprise’s infrastructure for decades now… and open source tooling in particular being the way these enterprises have been able to innovate and act quickly.
If we look into the history of data and AI-focused open source projects, we see a multitude of strategies for sustainability and building community. While many of these projects have achieved some degree of adoption, the key challenge we tend to see is the ‘glass ceiling’ effect of a project without sustainability in mind.
Sustainability means many things, including creating:
- A vendor-neutral home that increases the willingness of developers from various organisation to use, collaborate and contribute to the project.
- A home for the project assets and legal management (DCO, CLA, trademark, export control, licensing compliance, etc.)
- A means of open and transparent project governance that enables a ‘do-ocracy’ and community driven direction away from the influence of the project founder.
- Infrastructure that is owned and managed by the community under the guidance of a vendor-neutral entity (mailing lists, wiki, website, build systems, etc.)
A positive-sum game
The result of sustainability is the formation of an ecosystem, which brings together a broad set of stakeholders (users and contributors, who in some cases are competitors in the marketplace).
A basic premise behind the Linux Foundation and open source in general, is that most interactions are a positive-sum game (i.e. one where nobody wins at the expense of someone else’s loss). There is no fixed amount of investment, mindshare, or development contributions that is allocated to specific projects. Just as open source development is based on the idea that, collectively, we are smarter than any one of us, open source foundations work to make the entire community better off. Equally important, a neutral home for a project and community fosters this type of positive-sum thinking and drives growth and diversity that we believe are core elements of a successful open source project.
There have been multiple efforts within the Linux Foundation focusing on building community in the data and AI space.
The first such effort was ODPi, launched in 2015 to drive standardisation across the growing big data ecosystem. As this effort drove forward consolidation in the industry in its early days, it saw the larger challenge of adoption of these open source projects in enterprises. With that insight from the community, it turned its focus to being a home for these efforts. ODPi first launched Egeria in 2018 with a focus on standardising metadata repositories and enabling enterprise-wide data governance, followed by OpenDS4All in 2019 to provide open source and standardized data science education in academia.
AI & ML & DL
Coming in 2018 was LF Deep Learning, which was rebranded as LF AI in 2019, focusing on growing open source innovation in artificial intelligence, machine learning and deep learning. It has quickly grown since the first project Accumoby John s was launched in 2018, with over 20 projects now being hosted by LF AI. These projects represent some of the most innovative and fastest growing projects in Artificial Intelligence, Machine Learning and Deep Learning, becoming the focal point for these technologies.
The announcement of the coming together of both the LF AI and ODPi communities under the name of LF AI and Data is a natural step and both communities have intertwined dependencies and touchpoints and downstream users look at both AI and Data as integrated strategies.
Other key benefits include…
- Unified guidance on the use of these open source projects within organisations.
- Better interoperability, and capabilities across projects through easier collaboration and facilitated integration efforts between various project communities.
- Being able to go to market together speaks a stronger message and helps ensure long term investment in these key technologies.
You can learn more about the coming together of these communities in the announcement, and we also invite you to learn more about LF AI and Data and its hosted projects.
Open source AI & data – ecosystem challenges