ra2 studio - Fotolia

How to avoid technical debt in open source communities

Businesses that make use of open source software should dedicate resources to supporting and maintaining the projects their organisations rely on

Most businesses and government systems use some open source components. Some companies go further and run their core systems on open source technologies.

But the people who maintain these projects need more support from the user community, according to Mike McQuaid, senior software engineer at GitHub.

“We are trying to get more people involved in open source projects,” he said. “The best place for people to contribute to open source is not in their free time, but in their work time.”

It can be fun for an individual to contribute to open source, said McQuaid, but it is even more valuable to the organisation if those contributions are aligned with projects the business relies on.

“Every engineer nowadays should be spending a couple of hours a week working on open source projects that their company relies on,” he said.

If a company uses open source technology such as MySQL, someone in that organisation should be trying to figure out how to contribute to MySQL, said McQuaid.

“If you have someone who has built up expertise and knowledge, then that becomes extremely valuable when you have problems with MySQL.”

McQuaid is heading a programme at GitHub that is looking at how to make the lives of code maintainers easier.

In his experience, people can become overworked and stressed by the overwhelming responsibility of maintaining open source.

Maintainers should automate

McQuaid believes part of the problem is that maintainers often choose manual processes over automated tools, which can reduce their workload substantially. “There is a lot of stuff you an do in open source either very manually or try to use automation,” he said. “I think there are a lot of open source maintainers who are trying to do a lot of things manually that could be automated. It is like trying to cut down a tree with a blunt axe.”

For example, he said someone could submit a pull request, where a contributor submits code to the project. “But if you don’t have automated tests to run and automated checks and code coverage tools, you have to do all of this manually and it is incredibly time-consuming,” he said.

Managing open source submissions was the topic discussed by Lauri Apple, agile project manager at Zalando, during her presentation at GitHub Europe in London this week.

Apple is the open source evangelist at the online fashion retailer, which has a team of 1,700 developers.

Zalando used to run its IT in a command-and-control fashion, with teams not encouraged to get involved in open source software. This changed when the teams started using Postgre and Python, said Apple. “In 2015, we set out some open source software principles and by January 2016, we had 400 repositories on GitHub,” she said.

The company lost its way in understanding why code was being published to the open source community via GitHub, said Apple. “We were compulsively releasing work and we weren’t asking what was the value.

“There was fear that if we didn’t rush to publish, someone else would.”

Among the problems this created was huge variation in the quality of code being published on GitHub, said Apple. “Sometimes we were missing documentation or test files and we weren’t responding to people.”

Risk of open source submissions

While sharing open source code encouraged Zalando’s programmers to develop and allowed it to give back to the community, some of the code was not particularly useful, said Apple.

“We took advantage of the freeness of GitHub, but if we asked the community, we would have done it differently.”

As an example, during her presentation, Apple showed a slide of a piece of Zalando cloud infrastructure management that was published on GitHub. It relied on many additional components, specific to the way Zalando’s cloud operated, limiting its usefulness. “Less is more,” she said.

Apple urged developers submitting code to try to publish a minimum viable product, rather than a fully working system. “It’s about contributing something that is basic, which people can contribute to,” she said.

Read more about open source submissions

  • Bet365 hopes its open-source initiative will help to drive adoption of the Erlang functional programming language among enterprise developers.
  • GitHub, a bastion of software developers, also offers automation scripts and open source collaboration for datacentre administrators.

Arguably, among the key indicators of a successful open source submission is when the wider community develops source code forks, which create branches of the original project.

Apple warned delegates that far from being free, submitting open source code to a repository like GitHub could incur considerable costs. Code that gets adopted by open source developers needs regular maintenance to support queries, comments and code contributions, which costs time and money. If the code is not adopted by anyone, “craftsmanship suffers because no one edits the work, so you end up with an open source junkyard”, she said.

Apple recommended that companies looking to contribute code to the open source community should build hygiene into their project to manage technical debt.

Read more on Software development tools

Data Center
Data Management