opengl層次建模

介紹 (Introduction)

It is not uncommon to find samples in our datasets that are not completely independent. Samples in datasets often form clusters or groups within which some properties are shared. This often requires some special attention while modeling to build reliable models for two major reasons. First, the independence of samples is an important assumption for several statistical analysis procedures like maximum likelihood estimation. Second, we want to capture the variation of the influence of the predictors across groups in our model called contextual effect.

在我們的數據集中發現并非完全獨立的樣本并不少見。 數據集中的樣本通常形成群集或組，在其中共享某些屬性。 在建模以建立可靠的模型時，通常需要特別注意兩個方面。 首先，樣本的獨立性是一些統計分析程序(例如最大似然估計)的重要假設。 其次，我們要捕獲模型中稱為上下文效應的各組預測變量影響的變化。

A commonly used example, that I find sufficient to understand the scenario is that of the performance of students in a school. In a school with students in multiple classes, the academic performance of a student is influenced by their individual capabilities (called fixed effects) and the class they’re a part of (called random effects). Maybe the teacher assigned to a particular class teaches better than the others, or a higher proportion of intelligent students in a class creates a competitive environment for the students to perform better.

我發現足以理解這種情況的一個常用示例是學校學生的表現。 在有多個班級學生的學校中，學生的學習成績受其個人能力(稱為固定效應)及其所屬班級(稱為隨機效應)的影響。 也許分配給特定班級的老師的教學要比其他班級的老師好，或者班級中有更多聰明的學生為學生創造更好的競爭環境。

One approach to handle the case is to build multiple models for each class, referred to as pooling. But such an approach might not always produce reliable results. For example, the model corresponding to a class with very few students will be very misleading. A single unpooled model might not be able to fit sufficiently on the data. We want to find a middle ground that finds a compromise between these extremes — partial pooling. This brings us to Bayesian hierarchical modeling, also known as multilevel modeling.

處理這種情況的一種方法是為每個類構建多個模型，稱為pooling 。 但是，這種方法可能并不總是產生可靠的結果。 例如，與一個只有很少學生的班級相對應的模型將具有很大的誤導性。 單個非池化模型可能無法充分適合數據。 我們希望找到一個在這些極端之間折衷的中間立場-部分池。 這使我們進入了貝葉斯層次建模，也稱為多層次建模。

In this method, parameters are nested within one another at different levels of groups. Roughly, it gives us the weighted average of the unpooled and pooled model estimates. Hierarchical modeling is one of the most powerful, yet simple, techniques in Bayesian inference and possibly in statistical modeling. In this post, I will introduce the idea with a practical example. Note that this post does not cover the fundamentals of Bayesian analysis. The source code for the example is available as a notebook in GitHub.

在這種方法中，參數彼此嵌套在不同級別的組中。 粗略地講，它為我們提供了未合并和合并的模型估計值的加權平均值。 層次建模是貝葉斯推斷以及統計建模中最強大但最簡單的技術之一。 在這篇文章中，我將通過一個實際的例子來介紹這個想法。 請注意，本文不涵蓋貝葉斯分析的基礎知識。 該示例的源代碼可在GitHub中作為筆記本使用 。

數據 (Data)

The dataset used for illustration is similar to the students example above except we’re trying to find the math scores of students from different schools. We use the homework completion as a predictor for our example. You can find the original data here and it’s csv version here.

用于說明的數據集與上面的學生示例相似，除了我們試圖查找來自不同學校的學生的數學成績。 在本例中，我們將作業完成情況作為預測指標。 你可以找到原始數據在這里 ，它的CSV版本在這里 。

第一眼 (First look)

Plotting the math scores of the students against homework along with the unpooled OLS regression fit gives us this:

通過對學生的數學成績與作業進行比較，以及未匯總的 OLS回歸擬合，可以得出以下結論 ：

The data samples with unpooled regression fit具有非池化回歸擬合的數據樣本

Visualizing the data at the school level reveals some interesting patterns. We also plot the pooled regression lines fit on each school and the unpooled regression fit for reference. For simplicity, we use OLS regression.

在學校級別對數據進行可視化顯示了一些有趣的模式。 我們還繪制了每所學校的合并回歸線擬合值和非合并回歸擬合值以供參考。 為簡單起見，我們使用OLS回歸。

The data samples with pooled regression fit across groups具有合并回歸的數據樣本適合各組

The plot shows the variation of the relationship across groups. We also notice that the estimates are highly influenced by the few data points (possible outliers) with high homework completions in some of the groups.

該圖顯示了各組之間關系的變化。 我們還注意到，在某些組中，這些估計值受功課完成率高的幾個數據點(可能的異常值)的影響很大。

層次模型 (Hierarchical model)

We will construct our Bayesian hierarchical model using PyMC3. We will construct hyperpriors on our group-level parameters to allow the model to share the individual properties of the student among the groups.

我們將使用PyMC3構建貝葉斯分層模型。 我們將在組級別參數上構建超級優先級，以允許模型在組之間共享學生的個人屬性。

The model can be represented as y? = α?? + β??x? + ε? ,

該模型可以表示為y?=α??+β??x?+ε?，

or in probabilistic notation as y ～ N(α? + β?x, ε).

或以概率符號y?N(α?+β?x，ε)表示。

For this model, we will use a random slope β and intercept α. This means that they will vary with each group instead of a constant slope and intercept for the entire data. The graphical representation of the probabilistic model is shown below.

對于此模型，我們將使用隨機斜率β和截距α。 這意味著它們將隨每個組而變化，而不是恒定的斜率，并且會截取整個數據。 概率模型的圖形表示如下所示。

Graph representation of the hierarchical model used in this example本示例中使用的層次模型的圖形表示

While I choose my priors here by eyeballing the general distribution of the samples, using uninformative priors will lead to similar results. The code snippet below defines the PyMC3 model used.

雖然我通過關注樣本的總體分布來選擇我的先驗，但是使用無信息的先驗會導致相似的結果。 下面的代碼段定義了使用的PyMC3模型。

with pm.Model() as model:
# Hyperpriors
mu_a = pm.Normal('mu_a', mu=40, sigma=50)
sigma_a = pm.HalfNormal('sigma_a', 50)

mu_b = pm.Normal('mu_b', mu=0, sigma=10)
sigma_b = pm.HalfNormal('sigma_b', 5)

# Intercept
a = pm.Normal('a', mu=mu_a, sigma=sigma_a, shape=n_schools) # Slope
b = pm.Normal('b', mu=mu_b, sigma=sigma_b, shape=n_schools)

# Model error
eps = pm.HalfCauchy('eps', 5)

# Model
y_hat = a[school] + b[school] * homework

# Likelihood
y_like = pm.Normal('y_like', mu=y_hat, sigma=eps, observed=math)

We will use the NUTS sampler for drawing samples from the posterior distribution.

我們將使用NUTS采樣器從后驗分布中抽取樣本。

with model:
step = pm.NUTS()
trace = pm.sample(2000, tune=1000)

The trace plot and the posterior distribution of the slope and intercept corresponding to each school is visualized below.

下面顯示了與每個學校相對應的坡度和截距的軌跡圖和后驗分布。

Trace plot for the hierarchical model層次模型的跡線圖 Posterior distribution of the intercept and slope for each group每組的截距和斜率的后驗分布

We can see the variation in the estimates of our coefficients across different schools. We can also clearly interpret the uncertainty associated with our estimates from the distributions. The posterior predictive regression (gray) lines below, sampled from the posterior distribution of the estimates of each group, gives a better picture of the model with respect to the data.

我們可以看到不同學校對我們系數的估計的差異。 我們還可以從分布中清楚地解釋與我們的估計相關的不確定性。 下面的后預測回歸(灰色)線是從每組估計的后分布中采樣的，相對于數據，該模型可以提供更好的模型描述。

Posterior predictive fits of the hierarchical model層次模型的后驗預測擬合

Note the general higher uncertainty around groups that show a negative slope. The model finds a compromise between sensitivity to noise at the group level and the global estimates at the student level (apparent in IDs 7472, 7930, 25456, 25642). This implies that we must be a little warier of the decisions derived from the model on these groups. We also observe that with more data and lesser deviation, the Bayesian model converges to the OLS model of the group (ID 62821) as expected. We can also check the student level relationship by plotting the regression lines from mu_a and mu_b (which I omit here).

請注意，在顯示負斜率的組周圍，總體上不確定性較高。 該模型在小組級別的噪聲敏感性和學生級別的全局估計(在ID 7472、7930、25456、25642中明顯)之間找到了折衷方案。 這意味著我們必須對從這些群體的模型得出的決策有所警惕。 我們還觀察到，隨著數據的增加和偏差的減小，貝葉斯模型可以收斂到該組的OLS模型(ID 62821)，這與預期的一樣。 我們還可以通過繪制來自mu_a和mu_b (在此省略)來檢查學生水平的關系。

Cross-validation with the different models will show the superiority of the hierarchical modeling approach. Cross-validation can be performed at 2 levels:

與不同模型的交叉驗證將顯示分層建模方法的優越性。 交叉驗證可以在2個級別執行：

Hold out students within a group and evaluate against its prediction.

支持小組中的學生，并根據其預測進行評估。

Hold out an entire group and evaluate its prediction. Note that this is not possible with the pooling model.

支持整個小組并評估其預測。 請注意，這對于合并模型是不可能的。

I do not perform validation here as the frequentist and Bayesian models used here don’t make for a fair (or easy) comparison. But the CV can be performed by replacing the OLS regression with Bayesian linear regression and comparing their Root Mean Squared Deviation (RMSD) of the models.

我在這里不進行驗證，因為此處使用的常駐模型和貝葉斯模型無法進行公平(或簡單)的比較。 但是，可以通過用貝葉斯線性回歸代替OLS回歸并比較模型的均方根偏差(RMSD)來執行CV。

結論 (Conclusion)

Bayesian hierarchical modeling can produce robust models with naturally clustered data. They often allow us to build simple and interpretable models as opposed to the frequentist techniques like ensembling or neural networks that are commonly used for such complex data. They also prevent overfitting despite the increase in the number of parameters in the model. The post is merely an introduction to hierarchical modeling and its inherent simplicity allows us to implement different variations of the model specific to our data (eg: adding sub-groups, using more group-level predictors) and conduct different types of analysis (eg: find correlation among levels).

貝葉斯分層建模可以生成具有自然聚類數據的健壯模型。 與通常用于此類復雜數據的集成或神經網絡之類的頻頻技術相比，它們通常使我們能夠構建簡單且可解釋的模型。 盡管模型中參數的數量增加了，但它們也可以防止過度擬合。 這篇文章只是對層次建模的介紹，其固有的簡單性使我們能夠實現針對數據的模型的不同變體(例如：添加子組，使用更多的組級預測變量)并進行不同類型的分析(例如：找到各個級別之間的相關性)。

資源資源 (Resources)

A. Gelman et al, Bayesian Data Analysis (2013), Chapter 5, CRC press

A.Gelman等人，貝葉斯數據分析(2013)，第5章，CRC出版社

Thank you for reading! I would appreciate any feedback on the post.

感謝您的閱讀！ 我將不勝感激對此職位的任何反饋。

You can connect with me on Linkedin and follow me on GitHub.

您可以在Linkedin上與我聯系，并在GitHub上關注我。

翻譯自: https://towardsdatascience.com/introduction-to-hierarchical-modeling-a5c7b2ebb1ca

opengl層次建模

總結

以上是生活随笔為你收集整理的opengl层次建模_层次建模简介的全部內容，希望文章能夠幫你解決所遇到的問題。

如果覺得生活随笔網站內容還不錯，歡迎將生活随笔推薦給好友。