為什么80%的碼農都做不了架構師？>>> ??

數據血緣通常是指數據產生的鏈路，其采集主要通過自動解析（存儲過程、SQL、ETL過程等文件）結合人工收集的方式實現。本文不涉及數據血緣如何獲取，只對如何通過python操作mongodb并可視化數據血緣關系提供一些思路。

首先通過pymongo連接本地數據庫，并插入測試數據

import pymongo myclient = pymongo.MongoClient("mongodb://127.0.0.1:27017/") mydb = myclient["world"] mycol = mydb["areas"] mylist = [ { "_id" : 1, "name" : "Asia" }, { "_id" : 2, "name" : "China", "belongsto" : "Asia" }, { "_id" : 3, "name" : "ZheJiang", "belongsto" : "China" }, { "_id" : 4, "name" : "HangZhou", "belongsto" : "ZheJiang" }, { "_id" : 5, "name" : "NingBo", "belongsto" : "ZheJiang" }, { "_id" : 6, "name" : "Xihu", "belongsto" : "HangZhou" } ] x = mycol.insert_many(mylist) for x in mycol.find():print(x)

輸出結果：?

{'_id': 1, 'name': 'Asia'}
{'_id': 2, 'name': 'China', 'belongsto': 'Asia'}
{'_id': 3, 'name': 'ZheJiang', 'belongsto': 'China'}
{'_id': 4, 'name': 'HangZhou', 'belongsto': 'ZheJiang'}
{'_id': 5, 'name': 'NingBo', 'belongsto': 'ZheJiang'}
{'_id': 6, 'name': 'Xihu', 'belongsto': 'HangZhou'}

遞歸查詢name='Xihu'這個節點的父節點

pipeline = [{'$graphLookup': {'from': "areas",'startWith': "$belongsto",'connectFromField': "belongsto",'connectToField': "name",'as': "belongHierarchy"}},{'$match': {'name' : 'Xihu'}}] for doc in (mycol.aggregate(pipeline)):print (doc)

輸出結果：?

{'_id': 6, 'name': 'Xihu', 'belongsto': 'HangZhou','reportingHierarchy':

[{'_id': 1, 'name': 'Asia'},

{'_id': 2, 'name': 'China', 'belongsto': 'Asia'},

{'_id': 3, 'name': 'ZheJiang', 'belongsto': 'China'},

{'_id': 4, 'name': 'HangZhou', 'belongsto': 'ZheJiang'}]}

解析輸出結果并可視化展示節點關系

import networkx as nx import matplotlib.pyplot as plt rs = list(mycol.aggregate(pipeline)) def get_relation(rs):G = nx.DiGraph()for node in rs:try:G.add_edge(node['name'], node['belongsto'])for item in node['belongHierarchy']:if 'belongsto' in item.keys():G.add_edge(item['name'], item['belongsto'])else:passexcept:passreturn G G = get_relation(rs) nx.draw(G, with_labels=True, font_weight='bold') plt.show()

展示area這個collection中所有的節點

pipeline = [{'$graphLookup': {'from': "areas",'startWith': "$belongsto",'connectFromField': "belongsto",'connectToField': "name",'as': "belongHierarchy"}}] rs = list(mycol.aggregate(pipeline)) def get_relation(rs):G = nx.DiGraph()for node in rs:try:G.add_edge(node['name'], node['belongsto'])for item in node['belongHierarchy']:if 'belongsto' in item.keys():G.add_edge(item['name'], item['belongsto'])else:passexcept:passreturn G G = get_relation(rs) nx.draw(G, with_labels=True, font_weight='bold') plt.show()

轉載于:https://my.oschina.net/aubao/blog/3035977

超強干貨來襲 云風專訪：近40年碼齡，通宵達旦的技術人生

總結

以上是生活随笔為你收集整理的python pymongo+networkx 实现mongo数据血缘关系可视化的全部內容，希望文章能夠幫你解決所遇到的問題。

如果覺得生活随笔網站內容還不錯，歡迎將生活随笔推薦給好友。