python 模糊匹配 合并_Python Pandas模糊合并/匹配重复
我目前有2個數據幀,1個用于捐贈者,1個用于籌款.理想情況下,我想要找到的是,如果有任何籌款人也捐贈,如果是的話,將一些信息復制到我的募捐人數據集(捐贈者姓名,電子郵件和他們的第一次捐贈).我的數據有問題
1)我需要通過姓名和電子郵件進行匹配,但用戶可能會略有不同的名稱(前Kat和Kathy).
2)捐贈者和籌款人的名稱重復.
2a)有了捐贈者,我可以得到唯一的姓名/電子郵件組合,因為我只關心第一個捐贈日期
2b)雖然我需要保留兩行,而不是像日期一樣丟失數據.
我現在的示例代碼:
import pandas as pd
import datetime
from fuzzywuzzy import fuzz
import difflib
donors = pd.DataFrame({"name": pd.Series(["John Doe","John Doe","Tom Smith","Jane Doe","Jane Doe","Kat test"]), "Email": pd.Series(['a@a.ca','a@a.ca','b@b.ca','c@c.ca','something@a.ca','d@d.ca']),"Date": (["27/03/2013 10:00:00 AM","1/03/2013 10:39:00 AM","2/03/2013 10:39:00 AM","3/03/2013 10:39:00 AM","4/03/2013 10:39:00 AM","27/03/2013 10:39:00 AM"])})
fundraisers = pd.DataFrame({"name": pd.Series(["John Doe","John Doe","Kathy test","Tes Ester", "Jane Doe"]),"Email": pd.Series(['a@a.ca','a@a.ca','d@d.ca','asdf@asdf.ca','something@a.ca']),"Date": pd.Series(["2/03/2013 10:39:00 AM","27/03/2013 11:39:00 AM","3/03/2013 10:39:00 AM","4/03/2013 10:40:00 AM","27/03/2013 10:39:00 AM"])})
donors["Date"] = pd.to_datetime(donors["Date"], dayfirst=True)
fundraisers["Date"] = pd.to_datetime(donors["Date"], dayfirst=True)
donors["code"] = donors.apply(lambda row: str(row['name'])+' '+str(row['Email']), axis=1)
idx = donors.groupby('code')["Date"].transform(min) == donors['Date']
donors = donors[idx].reset_index().drop('index',1)
因此,這給了我每個捐贈者的第一次捐贈(假設任何具有完全相同名稱和電子郵件的人都是同一個人).
理想情況下,我希望我的籌款人數據集看起來像:
Date Email name Donor Name Donor Email Donor Date
2013-03-27 10:00:00 a@a.ca John Doe John Doe a@a.ca 2013-03-27 10:00:00
2013-01-03 10:39:00 a@a.ca John Doe John Doe a@a.ca 2013-03-27 10:00:00
2013-02-03 10:39:00 d@d.ca Kathy test Kat test d@d.ca 2013-03-27 10:39:00
2013-03-03 10:39:00 asdf@asdf.ca Tes Ester
2013-04-03 10:39:00 something@a.ca Jane Doe Jane Doe something@a.ca 2013-04-03 10:39:00
我嘗試了這個帖子:is it possible to do fuzzy match merge with python pandas?但是不斷讓索引超出范圍錯誤(猜測它不喜歡籌款活動中的重復名稱):(那么任何想法如何匹配/合并這些數據集?
用for循環做它(它工作但速度很慢,我覺得必須有更好的方法)
fundraisers["donor name"] = ""
fundraisers["donor email"] = ""
fundraisers["donor date"] = ""
for donindex in range(len(donors.index)):
max = 75
for funindex in range(len(fundraisers.index)):
aname = donors["name"][donindex]
comp = fundraisers["name"][funindex]
ratio = fuzz.ratio(aname, comp)
if ratio > max:
if (donors["Email"][donindex] == fundraisers["Email"][funindex]):
ratio *= 2
max = ratio
fundraisers["donor name"][funindex] = aname
fundraisers["donor email"][funindex] = donors["Email"][donindex]
fundraisers["donor date"][funindex] = donors["Date"][donindex]
總結
以上是生活随笔為你收集整理的python 模糊匹配 合并_Python Pandas模糊合并/匹配重复的全部內容,希望文章能夠幫你解決所遇到的問題。
- 上一篇: spark入门_Spark技术入门——配
- 下一篇: python从ip端口 获取数据_pyt