Uber Pyro: an open source 'probabilistic' language

Online transportation company Uber has released its open sourced Pyro – a homegrown probabilistic programming language that has been developed internally.

As many readers will know, a probabilistic programming language is a high-level language (one with a high level of abstraction upwards from machine code) designed to give software programmers a way of defining probability models, which can then be solved by the power of the language model automatically.

As O’Reilly points out, these languages incorporate random events as primitives and their runtime environment handles inference

Uber is releasing Pyro with a specific hope that developers will use it to create further language advancements which will serve its own industry — and (presumably) for the benefit of other vertical industries.

Deep probabilistic modelling

The company describes Pyro as a tool for deep probabilistic modelling, unifying the best of modern deep learning and Bayesian modelling.

As clarified on probabilistic-programming.org, “Probabilistic graphical models provide a formal lingua franca for modelling and a common target for efficient inference algorithms. Their introduction gave rise to an extensive body of work in machine learning, statistics, robotics, vision, biology, neuroscience, Artificial Intelligence (AI) and cognitive science.”

Predictive models are communicated using a mix of natural language, pseudo code and mathematical formulae and solved using special purpose, one-off inference methods.

Now with an eye on AI and its use in autonomous vehicles, Uber says its believes the critical ideas to solve AI will come from a joint effort among a worldwide community of people pursuing diverse approaches.

According to Uber, “By open sourcing Pyro, we hope to encourage the scientific world to collaborate on making AI tools more flexible, open, and easy-to-use. We expect the current (alpha!) version of Pyro will be of most interest to probabilistic modellers who want to leverage large data sets and deep networks, PyTorch users who want easy-to-use Bayesian computation, and  data scientists ready to explore the ragged edge of new technology.”

Marrying probability with representational power

The company further explains that specifying probabilistic models directly can be cumbersome and implementing them can be very error-prone. Software engineers at Uber have explained that probabilistic programming languages (PPLs) solve these problems by marrying probability with the representational power of programming languages.

Uber’s Noah Goodman further clarifies that a probabilistic program is a mix of ordinary deterministic computation and randomly sampled values; this stochastic computation represents a generative story about data. The probabilities are implicit in this representation—there is no need to derive formulas—and yet this specification is also universal: any computable probabilistic model can be written this way.

Pyro builds on full Python as its base language.