文章目錄

• 一、問題分析
• • 1.1 問題描述
  • 1.2 問題分析
• 二、具體代碼及注釋
• • 2.1 代碼及注釋
  • 2.2 繪圖結果

---

一、問題分析

1.1 問題描述

In this assignment you must choose one of the options presented below and submit a visual as well as your source code for peer grading. The details of how you solve the assignment are up to you, although your assignment must use matplotlib so that your peers can evaluate your work. The options differ in challenge level, but there are no grades associated with the challenge level you chose. However, your peers will be asked to ensure you at least met a minimum quality for a given technique in order to pass. Implement the technique fully (or exceed it!) and you should be able to earn full grades for the assignment.

??????Ferreira, N., Fisher, D., & Konig, A. C. (2014, April). Sample-oriented task-driven visualizations: allowing users to make better, more confident decisions.
??????In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (pp. 571 ?????In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (pp. 571-580). ACM. ([video](https://www.youtube.com/watch?v=BI7GAs-va-Q))

In this paper the authors describe the challenges users face when trying to make judgements about probabilistic data generated through samples. As an example, they look at a bar chart of four years of data (replicated below in Figure 1). Each year has a y-axis value, which is derived from a sample of a larger dataset. For instance, the first value might be the number votes in a given district or riding for 1992, with the average being around 33,000. On top of this is plotted the 95% confidence interval for the mean (see the boxplot lectures for more information, and the yerr parameter of barcharts).

A challenge that users face is that, for a given y-axis value (e.g. 42,000), it is difficult to know which x-axis values are most likely to be representative, because the confidence levels overlap and their distributions are different (the lengths of the confidence interval bars are unequal). One of the solutions the authors propose for this problem (Figure 2c) is to allow users to indicate the y-axis value of interest (e.g. 42,000) and then draw a horizontal line and color bars based on this value. So bars might be colored red if they are definitely above this value (given the confidence interval), blue if they are definitely below this value, or white if they contain this value.

Easiest option: Implement the bar coloring as described above - a color scale with only three colors, (e.g. blue, white, and red). Assume the user provides the y axis value of interest as a parameter or variable.

Harder option: Implement the bar coloring as described in the paper, where the color of the bar is actually based on the amount of data covered (e.g. a gradient ranging from dark blue for the distribution being certainly below this y-axis, to white if the value is certainly contained, to dark red if the value is certainly not contained as the distribution is above the axis).

Even Harder option: Add interactivity to the above, which allows the user to click on the y axis to set the value of interest. The bar colors should change with respect to what value the user has selected.

Hardest option: Allow the user to interactively set a range of y values they are interested in, and recolor based on this (e.g. a y-axis band, see the paper for more details).

---

Note: The data given for this assignment is not the same as the data used in the article and as a result the visualizations may look a little different.

Use the following data for this assignment:

import pandas as pd import numpy as np from scipy import stats %matplotlib notebooknp.random.seed(12345)df = pd.DataFrame([np.random.normal(32000,200000,3650), np.random.normal(43000,100000,3650), np.random.normal(43500,140000,3650), np.random.normal(48000,70000,3650)], index=[1992,1993,1994,1995])

1.2 問題分析

• 分析Even Harder option選項：

??本題給出了1992年-1995年4年間的數據集，在給定某一個特定的值的時候，判斷其屬于那個年份的概率最大。根據中心極限定理，我們假設每一年的數據分布應該是屬于正態分布的（結合核密度曲線觀察、利用Shapiro-Wilk檢驗或者Kolmogorov-Smirnov檢驗法，本題中可省略），首先我們設定$\alpha =0.05$，給定置信水平為95%，以此計算出每個數據的置信區間。根據置信區間的中值（同時也是數據集的中值）繪制柱狀圖，在根據置信區間的范圍繪制errorbar，這樣即可給出在0.95置信水平內可能屬于該數據集的觀測值的范圍。

??考慮到越靠近置信區間的均值，則隸屬于該數據集的概率越大，利用$2?(\bar{X}?Y)/(X_{max}?X_{min})$來設計函數用以計算觀測值屬于各個數據集的概率。

??選擇色階圖的時候應該選擇分散形（Diverging）的色階圖colormap，表現為中間白兩頭漸變，中間值為1，兩頭值為0，首先需要對-1到1的數據設計規范化，用plt.Normalize(-1,1)，根據已經得到的數據概率值，對觀測值屬于各個數據集的概率值進行上色，最后完善圖的細節（例如提高有效墨水比例ink-ratio和減少繪圖垃圾信息），便完成了一幀的制作，對于色階圖的運用可以查看
【DS with Python】Matplotlib入門(三)：cm模塊、colormap配色、animation動畫與canvas交互設計。

??最后根據鼠標點擊創建事件，在點擊時獲取當前的y坐標event.ydata，將其當作觀測值，帶入上面設計好的繪圖函數完成每一幀的制作中即可。

二、具體代碼及注釋

2.1 代碼及注釋

# Use the following data for this assignment:import pandas as pd import numpy as np from scipy import stats %matplotlib notebooknp.random.seed(12345)#四個數據集 df = pd.DataFrame([np.random.normal(32000,200000,3650), np.random.normal(43000,100000,3650), np.random.normal(43500,140000,3650), np.random.normal(48000,70000,3650)], index=[1992,1993,1994,1995]) import matplotlib.pyplot as plt from matplotlib import cm from scipy import stats#計算95%置信區間 intervals=[] for idx in df.index:interval=stats.norm.interval(0.95,np.mean(df.loc[idx]),stats.sem(df.loc[idx]))intervals.append(interval)#計算yerr值(本質上就是置信區間減去期望值)用于在柱狀圖上繪制errorbar err_1992=np.array(stats.norm.interval(0.95,np.mean(df.loc[1992]),stats.sem(df.loc[1992])))-np.mean(df.loc[1992]) err_1993=np.array(stats.norm.interval(0.95,np.mean(df.loc[1993]),stats.sem(df.loc[1993])))-np.mean(df.loc[1993]) err_1994=np.array(stats.norm.interval(0.95,np.mean(df.loc[1994]),stats.sem(df.loc[1994])))-np.mean(df.loc[1994]) err_1995=np.array(stats.norm.interval(0.95,np.mean(df.loc[1995]),stats.sem(df.loc[1995])))-np.mean(df.loc[1995]) err=np.array([err_1992,err_1993,err_1994,err_1995]).T## 提供另一種思路：直接在上面的95%置信區間內減掉對應的數據 # idx_2=1992 # intervals_2=[] # for interval in intervals: # interval_2=np.array(interval)-np.mean(df.loc[idx_2]) # intervals_2.append(interval_2) # idx_2+=1 # err=np.array([intervals_2[0],intervals_2[1],intervals_2[2],intervals_2[3]]).T#提取df的index屬性和均值 index=df.T.describe().loc['mean',:].index.values values=df.T.describe().loc['mean',:].values#設置虛線y的默認值為4條柱狀圖均值的均值 y=np.mean(values)#創建新圖像 plt.figure()#從colormap中選定色彩，這里選擇了'collwarm'，也可以選擇其他的發散式colormap，或自定義 cmap=cm.get_cmap('coolwarm')#計算概率，完全超過95%置信區間為0，即藍色，完全低于95%置信區間為1，即紅色 def calculate_probability(y,interval):if y<interval[0]:return 1elif y>interval[1]:return -1return 2*((interval[1]+interval[0])/2-y)/(interval[1]-interval[0])#LC表達式對各個置信區間求解 probs=[calculate_probability(y,interval) for interval in intervals]#設置各個概率對應的顏色 colors=cmap(probs)#設置ScalarMappable sm = cm.ScalarMappable(cmap=cmap,norm=plt.Normalize(-1,1)) sm.set_array([])#畫柱狀圖 bars=plt.bar(range(len(values)),values,color=sm.to_rgba(probs))#畫誤差線 plt.gca().errorbar(range(len(values)),values,yerr=abs(err),c='k',fmt=' ',capsize=15)#畫面設置 plt.xticks(range(len(values)),index) plt.ylabel('Values') plt.xlabel('Year') plt.ylim([0,60000]) plt.gca().set_title('Assignment3')#設置水平色階圖 plt.colorbar(sm,orientation='horizontal')#去掉兩兩條邊框，減少繪圖垃圾 [plt.gca().spines[loc].set_visible(False) for loc in ['top','right']] [plt.gca().spines[loc].set_alpha(0.3) for loc in ['left','bottom']]#更新虛線y的y軸坐標 yticks = plt.gca().get_yticks() new_yticks=np.append(yticks,y) plt.gca().set_yticks(new_yticks)#畫觀測值的虛線 h_line=plt.axhline(y,color='gray',linestyle='--',linewidth=1)#給每個柱添加注釋 text=plt.text(1.5,58000,'y={:5.0f}'.format(y),bbox={'fc':'w','ec':'k'},ha='center') text1=plt.text(bars[0].get_x()+bars[0].get_width()/2,bars[0].get_height()+10000,'prob={:.2f}'.format(1-abs(probs[0])),bbox={'fc':'w','ec':'k'},ha='center') text2=plt.text(bars[1].get_x()+bars[1].get_width()/2,bars[1].get_height()+10000,'prob={:.2f}'.format(1-abs(probs[1])),bbox={'fc':'w','ec':'k'},ha='center') text3=plt.text(bars[2].get_x()+bars[2].get_width()/2,bars[2].get_height()+10000,'prob={:.2f}'.format(1-abs(probs[2])),bbox={'fc':'w','ec':'k'},ha='center') text4=plt.text(bars[3].get_x()+bars[3].get_width()/2,bars[3].get_height()+10000,'prob={:.2f}'.format(1-abs(probs[3])),bbox={'fc':'w','ec':'k'},ha='center')#設置交互函數 def onclick(event):#計算概率probs=[calculate_probability(event.ydata,interval) for interval in intervals]#用cmap給數值上色colors=cmap(probs)#print(probs)plt.bar(range(len(values)),values,color=sm.to_rgba(probs))plt.gca().errorbar(range(len(values)),values,yerr=abs(err),c='k',fmt=' ',capsize=15)#更改觀測值h_line.set_ydata(event.ydata)#得到新的y刻度new_yticks=np.append(yticks,event.ydata)#更新新的y刻度plt.gca().set_yticks(new_yticks)#給每個柱添加注釋text.set_text('y={:5.0f}'.format(event.ydata))text1.set_text('prob={:.2f}'.format(1-abs(probs[0])))text2.set_text('prob={:.2f}'.format(1-abs(probs[1])))text3.set_text('prob={:.2f}'.format(1-abs(probs[2])))text4.set_text('prob={:.2f}'.format(1-abs(probs[3])))#text=plt.gca().text(1.5,55000,'y={:5.0f}'.format(event.ydata),bbox={'fc':'w','ec':'k'},ha='center')plt.gcf().canvas.mpl_connect('button_press_event', onclick)

2.2 繪圖結果

請運行腳本體驗交互過程，jupyter notebook請運行%matplotlib notebook進行交互模式

• 初始頁面
  當觀測值過高或者過低都為0，y=40603時，屬于1994年一類的概率最大。

• 交互頁面
  觀測值過低時，概率皆為0
  
  觀測值始終時屬于各個數據集的概率：
  
  觀測值過高時，概率皆為0
  

總結

以上是生活随笔為你收集整理的【DS实践 | Coursera】Assignment 3 | Applied Plotting, Charting Data Representation in Python的全部內容，希望文章能夠幫你解決所遇到的問題。

如果覺得生活随笔網站內容還不錯，歡迎將生活随笔推薦給好友。