使用python向Redis批量导入数据
生活随笔
收集整理的這篇文章主要介紹了
使用python向Redis批量导入数据
小編覺得挺不錯的,現在分享給大家,幫大家做個參考.
1.使用pipeline進行批量導入數據。包含先使用rpush插入數據,然后使用expire改動過期時間
class Redis_Handler(Handler):def connect(self):#print self.host,self.port,self.tableself.conn = Connection(self.host,self.port,self.table) def execute(self, action_name):filename = "/tmp/temp.txt"batch_size = 10000with open(filename) as file:try:count = 0pipeline_redis = self.conn.client.pipeline()for lines in file:(key,value) = lines.split(',')count = count + 1if len(key)>0:pipeline_redis.rpush(key,value.strip())if not count % batch_size:pipeline_redis.execute()count = 0#send the last batchpipeline_redis.execute()except Exception:print 'redis add error'
class Redis_Handler(Handler):def connect(self):#print self.host,self.port,self.tableself.conn = Connection(self.host,self.port,self.table) def execute(self, action_name):filename = "/tmp/temp.txt"batch_size = 10000with open(filename) as file:try:count = 0pipeline_redis = self.conn.client.pipeline()for lines in file:(key,value) = lines.split(',')count = count + 1if len(key)>0:pipeline_redis.rpush(key,value.strip())if not count % batch_size:pipeline_redis.execute()count = 0#send the last batchpipeline_redis.execute()except Exception:print 'redis add error'
總結
以上是生活随笔為你收集整理的使用python向Redis批量导入数据的全部內容,希望文章能夠幫你解決所遇到的問題。
- 上一篇: 从几个sample来学习Java堆,方法
- 下一篇: WPF学习开发客户端软件-任务助手(下