概率編程編程

I recently started a new newsletter focus on AI education. TheSequence is a no-BS( meaning no hype, no news etc) AI-focused newsletter that takes 5 minutes to read. The goal is to keep you up to date with machine learning projects, research papers and concepts. Please give it a try by subscribing below:

我最近開始了一份有關AI教育的新時事通訊。 TheSequence是無BS(意味著沒有炒作，沒有新聞等)，它是專注于AI的新聞通訊，需要5分鐘的閱讀時間。 目標是讓您了解機器學習項目，研究論文和概念的最新動態。 請通過以下訂閱嘗試一下：

Probabilistic thinking is an incredibly valuable tool for decision making. From economists to poker players, people that can think in terms of probabilities tend to make better decisions when faced with uncertain situations. The fields of probabilities and game theory have been established for centuries and decades but are not experiencing a renaissance with the rapid evolution of artificial intelligence(AI). Can we incorporate probabilities as a first class citizen of software code? Welcome to the world of probabilistic programming languages(PPLs)

概率思維是決策中極為寶貴的工具。 從經濟學家到撲克玩家，那些會考慮概率的人在遇到不確定的情況時往往會做出更好的決策。 概率和博弈論領域已經建立了幾個世紀和幾十年，但隨著人工智能(AI)的快速發展，它并沒有經歷復興。 我們可以將概率作為軟件代碼的一等公民納入其中嗎？ 歡迎來到概率編程語言(PPL)的世界

The use of statistics to overcome uncertainty is one of the pillars of a large segment of the machine learning market. Probabilistic reasoning has long been considered one of the foundations of inference algorithms and is represented is all major machine learning frameworks and platforms. Recently, probabilistic reasoning has seen major adoption within tech giants like Uber, Facebook or Microsoft helping to push the research and technological agenda in the space. Specifically, PPLs have become one of the most active areas of development in machine learning sparking the release of some new and exciting technologies.

使用統計數據克服不確定性是機器學習市場很大一部分的Struts之一。 長期以來，概率推理一直被認為是推理算法的基礎之一，并且代表了所有主要的機器學習框架和平臺。 最近，概率推理已在Uber，Facebook或Microsoft等技術巨頭中得到廣泛采用，有助于推動該領域的研究和技術議程。 具體地說，PPL成為機器學習中最活躍的發展領域之一，從而激發了一些令人興奮的新技術的發布。

什么是概率編程語言？ (What are Probabilistic Programming Languages?)

Conceptually, probabilistic programming languages(PPLs) are domain-specific languages that describe probabilistic models and the mechanics to perform inference in those models. The magic of PPL relies on combining the inference capabilities of probabilistic methods with the representational power of programming languages.

從概念上講，概率編程語言(PPL)是領域特定的語言，描述了概率模型以及在這些模型中進行推理的機制。 PPL的魔力在于將概率方法的推理能力與編程語言的表示能力相結合。

In a PPL program, assumptions are encoded with prior distributions over the variables of the model. During execution, a PPL program will launch an inference procedure to automatically compute the posterior distributions of the parameters of the model based on observed data. In other words, inference adjusts the prior distribution using the observed data to give a more precise mode. The output of a PPL program is a probability distribution, which allows the programmer to explicitly visualize and manipulate the uncertainty associated with a result.

在PPL程序中，假設使用模型變量的先驗分布進行編碼。 在執行期間，PPL程序將啟動一個推理過程，以根據觀察到的數據自動計算模型參數的后驗分布。 換句話說，推論使用觀察到的數據來調整先驗分布以給出更精確的模式。 PPL程序的輸出是概率分布，它使程序員可以顯式可視化和操縱與結果相關的不確定性。

To illustrate the simplicity of PPLs, let’s use one of the most famous problems of modern statistics: a biased coin toss. The idea of this problem is to calculate the bias of a coin. Let’s assume that xi = 1 if the result of the i-th coin toss is head and xi = 0 if it is tail. Our context assumes that individual coin tosses are independent and identically distributed (IID) and that each toss follows a Bernoulli distribution with parameter θ: p(xi = 1 | θ) = θ and p(xi = 0 | θ) = 1 ? θ. The latent (i.e., unobserved) variable θ is the bias of the coin. The task is to infer θ given the results of previously observed coin tosses, that is, p(θ | x1, x2, . . . , xN ).

為了說明PPL的簡單性，讓我們使用現代統計中最著名的問題之一：偏向拋硬幣。 這個問題的想法是計算硬幣的偏差。 假設第i次拋硬幣的結果為正面時xi = 1，如果為尾部則xi = 0。 我們的上下文假設單個拋硬幣是獨立且均勻分布的(IID)，并且每個拋硬幣都遵循具有參數θ的伯努利分布：p(xi = 1 |θ)=θ和p(xi = 0 |θ)= 1-θ 。 潛變量(即未觀察到的變量)是硬幣的偏差。 任務是根據先前觀察到的拋硬幣的結果推論θ，即p(θ| x1，x2，...，xN)。

Modeling a simple program like the biased coin toss in a general-purpose programing language can result on hundreds of lines of code. However, PPLs like Edward express this problem in a few simple likes of code:

用通用編程語言對像有偏的拋硬幣之類的簡單程序進行建模可能會產生數百行代碼。 但是，像Edward這樣的PPL用一些簡單的代碼來表達這個問題：

# Model
theta = Uniform(0.0, 1.0)
x = Bernoulli(probs=theta, sample_shape=10)
Data 5 data = np.array([0, 1, 0, 0, 0, 0, 0, 0, 0, 1])
Inference
qtheta = Empirical( 8 tf.Variable(tf.ones(1000) ? 0.5))
inference = ed.HMC({theta: qtheta},
data={x: data})
inference.run()
Results 13 mean, stddev = ed.get_session().run( [qtheta.mean(),qtheta.stddev()])
print("Posterior mean:", mean)
print("Posterior stddev:", stddev)

圣杯：深入的PPL (The Holy Grail: Deep PPL)

For decades, the machine learning space was divided in two irreconcilable camps: statistics and neural networks. One camp gave birth to probabilistic programming while the other was behind transformational movements such as deep learning. Recently, the two schools of thought have come together to combine deep learning and Bayesian modeling into single programs. The ultimate expression of this effort is deep probabilistic programming languages(Deep PPLs).

幾十年來，機器學習空間被劃分為兩個不可調和的陣營：統計和神經網絡。 一個陣營催生了概率編程，而另一個陣營則產生了諸如深度學習之類的變革性運動。 最近，這兩個思想流派聚集在一起，將深度學習和貝葉斯建模結合到單個程序中。 這種努力的最終表達是深度概率編程語言(Deep PPL)。

Conceptually, Deep PPLs can express Bayesian neural networks with probabilistic weights and biases. Practically speaking, Deep PPLs have materialized as new probabilistic languages and libraries that integrate seamlessly with popular deep learning frameworks.

從概念上講，深度PPL可以表達具有概率權重和偏差的貝葉斯神經網絡。 實際上，深度PPL已實現為與流行的深度學習框架無縫集成的新概率語言和庫。

您需要了解的3個深層PPL (3 Deep PPLs You Need to Know About)

The field of probabilistic programming languages(PPLs) have been exploding with research and innovation in recent years. Most of that innovations have come from combining PPLs and deep learning methods to build neural networks that can efficiently handle uncertainty. Tech giants such as Google, Microsoft or Uber have been responsible for pushing the boundaries of Deep PPLs into large scale scenarios. Those efforts have translated into completely new Deep PPLs stacks that are becoming increasingly popular within the machine learning community. Let’s explore some of the most recent advancements in the Deep PPL space.

近年來，概率編程語言(PPL)領域一直在研究和創新中發展。 大多數創新來自將PPL和深度學習方法相結合，以構建可以有效處理不確定性的神經網絡。 諸如Google，Microsoft或Uber之類的技術巨頭一直負責將Deep PPL的邊界推向大規模方案。 這些努力已經轉化為全新的Deep PPL堆棧，這些堆棧在機器學習社區中越來越受歡迎。 讓我們探索Deep PPL空間中的一些最新進展。

愛德華 (Edward)

Edward is a Turing-complete probabilistic programming language(PPL) written in Python. Edward was originally championed by the Google Brain team but now has an extensive list of contributors. The original research paper of Edward was published in March 2017 and since then the stack has seen a lot of adoption within the machine learning community. Edward fuses three fields: Bayesian statistics and machine learning, deep learning, and probabilistic programming. The library integrates seamlessly with deep learning frameworks such as Keras and TensorFlow.

Edward是一種用Python編寫的圖靈完備的概率編程語言(PPL)。 Edward最初是Google Brain團隊的擁護者，但現在有大量的貢獻者 。 愛德華(Edward )的原始研究論文于2017年3月發表，從那時起，該堆棧在機器學習社區中得到了廣泛采用。 愛德華融合了三個領域：貝葉斯統計和機器學習，深度學習和概率編程。 該庫與Keras和TensorFlow等深度學習框架無縫集成。

1 # Model
2 theta = Uniform(0.0, 1.0)
3 x = Bernoulli(probs=theta, sample_shape=10)
4 # Data
5 data = np.array([0, 1, 0, 0, 0, 0, 0, 0, 0, 1])
6 # Inference
7 qtheta = Empirical(
8 tf.Variable(tf.ones(1000) ? 0.5))
9 inference = ed.HMC({theta: qtheta},
10 data={x: data})
11 inference.run()
12 # Results
13 mean, stddev = ed.get_session().run(
14 [qtheta.mean(),qtheta.stddev()])
15 print("Posterior mean:", mean)
16 print("Posterior stddev:", stddev)
1 # Inference Guide
2 qalpha = tf.Variable(1.0)
3 qbeta = tf.Variable(1.0)
4 qtheta = Beta(qalpha, qbeta)
5 # Inference
6 inference = ed.KLqp({theta: qtheta}, {x: data})
7 inference.run()

火焰兵 (Pyro)

Pyro is a deep probabilistic programming language(PPL) released by Uber AI Labs. Pyro is built on top of PyTorch and is based on four fundamental principles:

Pyro是由Uber AI Labs發布的一種深度概率編程語言(PPL)。 Pyro建立在PyTorch之上，并基于以下四個基本原則：

• Universal: Pyro is a universal PPL — it can represent any computable probability distribution. How? By starting from a universal language with iteration and recursion (arbitrary Python code), and then adding random sampling, observation, and inference.

  通用 ：Pyro是通用PPL-它可以表示任何可計算的概率分布。 怎么樣？ 從具有迭代和遞歸的通用語言(任意Python代碼)開始，然后添加隨機采樣，觀察和推斷。

• Scalable: Pyro scales to large data sets with little overhead above hand-written code. How? By building modern black box optimization techniques, which use mini-batches of data, to approximate inference.

  可擴展 ：Pyro可以擴展到大型數據集，而手寫代碼的開銷卻很小。 怎么樣？ 通過構建使用小批數據的現代黑盒優化技術來近似推斷。

• Minimal: Pyro is agile and maintainable. How? Pyro is implemented with a small core of powerful, composable abstractions. Wherever possible, the heavy lifting is delegated to PyTorch and other libraries.

  最小 ：Pyro是敏捷且可維護的。 怎么樣？ Pyro是由強大的可組合抽象的一小部分實現的。 盡可能將繁重的工作委托給PyTorch和其他庫。

• Flexible: Pyro aims for automation when you want it and control when you need it. How? Pyro uses high-level abstractions to express generative and inference models, while allowing experts to easily customize inference.

  靈活 ：Pyro的目標是在需要時實現自動化，并在需要時進行控制。 怎么樣？ Pyro使用高級抽象來表示生成模型和推理模型，同時使專家可以輕松自定義推理。

Just as other PPLs, Pyro combines deep learning models and statistical inference using a simple syntax as illustrated in the following code:

與其他PPL一樣，Pyro使用簡單的語法將深度學習模型和統計推斷相結合，如以下代碼所示：

1 # Model
2 def coin():
3 theta = pyro.sample("theta", Uniform(
4 Variable(torch.Tensor([0])),
5 Variable(torch.Tensor([1])))
6 pyro.sample("x", Bernoulli(
7 theta ? Variable(torch.ones(10)))
8 # Data
9 data = {"x": Variable(torch.Tensor(
10 [0, 1, 0, 0, 0, 0, 0, 0, 0, 1]))}
11 # Inference
12 cond = pyro.condition(coin, data=data)
13 sampler = pyro.infer.Importance(cond,
14 num_samples=1000)
15 post = pyro.infer.Marginal(sampler, sites=["theta"])
16 # Result
17 samples = [post()["theta"].data[0] for _ in range(1000)]
18 print("Posterior mean:", np.mean(samples))
19 print("Posterior stddev:", np.std(samples))# Inference Guide
2 def guide():
3 qalpha = pyro.param("qalpha", Variable(torch.Tensor([1.0]), requires_grad=True))
4 qbeta = pyro.param("qbeta", Variable(torch.Tensor([1.0]), requires_grad=True))
5 pyro.sample("theta", Beta(qalpha, qbeta))
6 # Inference
7 svi = SVI(cond, guide, Adam({}), loss="ELBO", num_particles=7)
8 for step in range(1000):
9 svi.step()

推斷網 (Infer.Net)

Microsoft recently open sourced Infer.Net a framework that simplifies probabilistic programming for .Net developers. Microsoft Research has been working on Infer.Net since 2004 but it has been only recently, with the emergence of deep learning, that the framework has become really popular. Infer.Net provides some strong differentiators that makes it a strong choice for developers venturing into the Deep PPL space:

微軟最近開放了Infer.Net的源代碼，該框架簡化了.Net開發人員的概率編程。 自2004年以來，Microsoft Research一直在研究Infer.Net，但是直到最近，隨著深度學習的出現，該框架才真正流行起來。 Infer.Net提供了一些強大的優勢，這使其成為進入Deep PPL空間的開發人員的強大選擇：

• Rich modelling language” Support for univariate and multivariate variables, both continuous and discrete. Models can be constructed from a broad range of factors including arithmetic operations, linear algebra, range and positivity constraints, Boolean operators, Dirichlet-Discrete, Gaussian, and many others.

  豐富的建模語言 ”支持連續和離散的單變量和多變量。 可以從多種因素構建模型，包括算術運算，線性代數，范圍和正性約束，布爾運算符，Dirichlet-Discrete，高斯等。

• Multiple inference algorithms” Built-in algorithms include Expectation Propagation, Belief Propagation (a special case of EP), Variational Message Passing and Gibbs sampling.

  多個推理算法 ”內置算法包括期望傳播，置信傳播(EP的特殊情況)，變消息傳遞和Gibbs抽樣。

• Designed for large scale inference: Infer.NET compiles models into inference source code which can be executed independently with no overhead. It can also be integrated directly into your application.

  專為大規模推理而設計 ：Infer.NET將模型編譯成推理源代碼，這些代碼可以獨立執行而不會產生開銷。 它也可以直接集成到您的應用程序中。

• User-extendable: Probability distributions, factors, message operations and inference algorithms can all be added by the user. Infer.NET uses a plug-in architecture which makes it open-ended and adaptable.

  用戶可擴展 ： 用戶可以添加概率分布，因子，消息操作和推理算法。 Infer.NET使用一種插件架構，使其具有開放性和適應性。

Lets look at our coin toss example in Infer.Net

讓我們看看Infer.Net中的拋硬幣示例

Variable<bool> firstCoin = Variable.Bernoulli(0.5);
Variable<bool> secondCoin = Variable.Bernoulli(0.5);
Variable<bool> bothHeads = firstCoin & secondCoin;
InferenceEngine engine = new InferenceEngine();
Console.WriteLine("Probability both coins are heads: "+engine.Infer(bothHeads));

The field of Deep PPL has is steadily becoming an important foundational block of the machine learning ecosystem. Pyro, Edward and Infer.Net are just three recent examples of Deep PPLs but not the only relevant ones. The intersection of deep learning frameworks and PPL offers an incredible large footprint for innovation and new use cases are likely to push the boundaries of Deep PPLs in the near future.

深度PPL領域已穩步成為機器學習生態系統的重要基礎塊。 Pyro，Edward和Infer.Net只是Deep PPL的三個最新示例，但不是唯一相關的示例。 深度學習框架與PPL的交集為創新提供了難以置信的龐大資源，新用例可能會在不久的將來推動Deep PPL的界限。

翻譯自: https://medium.com/swlh/a-gentle-introduction-to-probabilistic-programming-languages-bf1e19042ab6

概率編程編程

總結

以上是生活随笔為你收集整理的概率编程编程_概率编程语言的温和介绍的全部內容，希望文章能夠幫你解決所遇到的問題。

如果覺得生活随笔網站內容還不錯，歡迎將生活随笔推薦給好友。