?

轉(zhuǎn)自：http://hideto.iteye.com/blog/1328921

最近一段時間在學(xué)習(xí)leveldb的源碼，找到了一些相關(guān)的資源，用于后續(xù)查考和學(xué)習(xí)使用。

leveldb介紹?
http://code.google.com/p/leveldb/?
http://en.wikipedia.org/wiki/LevelDB?
http://highscalability.com/blog/2011/8/10/leveldb-fast-and-lightweight-keyvalue-database-from-the-auth.html?
http://news.ycombinator.com/item?id=2526032?
http://basho.com/blog/technical/2011/07/01/Leveling-the-Field/?
http://blog.yufeng.info/archives/1327?
http://www.slideshare.net/sunzhidong/google-leveldb-study-discuss?

leveldb官方文檔?
http://leveldb.googlecode.com/svn/trunk/doc/index.html?
http://leveldb.googlecode.com/svn/trunk/doc/benchmark.html?
http://leveldb.googlecode.com/svn/trunk/doc/impl.html?
http://leveldb.googlecode.com/svn/trunk/doc/table_format.txt?
http://leveldb.googlecode.com/svn/trunk/doc/log_format.txt?

leveldb內(nèi)部實現(xiàn)和源碼解析?
http://blog.xiaoheshang.info/?cat=26?
http://rdc.taobao.com/blog/cs/?p=1378?
http://www.cnblogs.com/haippy/archive/2011/12/04/2276064.html?

bigtable/mapreduce/gfs/lsm-tree/skiplist論文?
http://blademaster.ixiezi.com/2010/03/27/bigtable：一個分布式的結(jié)構(gòu)化數(shù)據(jù)存儲系統(tǒng)中文版/?
http://blademaster.ixiezi.com/2010/03/27/google-mapreduce中文版/?
http://blademaster.ixiezi.com/2010/03/27/the-google-file-system中文版/?
http://staff.ustc.edu.cn/~jpq/paper/flash/1996-The%20Log-Structured%20Merge-Tree%20%28LSM-Tree%29.pdf?
http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.15.9072&rep=rep1&type=pdf?

Tair ldb?
http://rdc.taobao.com/blog/cs/?p=1394?
http://code.taobao.org/p/tair/wiki/index/?
http://code.taobao.org/p/tair/src/branches/ldb/src/storage/ldb/?

相關(guān)資料?
http://www.quora.com/What-is-an-SSTable-in-Googles-internal-infrastructure?
http://www.ningoo.net/html/tag/dynamo?
http://wiki.apache.org/cassandra/MemtableSSTable?
http://wiki.apache.org/cassandra/ArchitectureSSTable?
http://en.wikipedia.org/wiki/Queuing_theory?
http://rdc.taobao.com/team/jm/archives/1344?

Notes?
leveldb的Write/Delete:?
DB::Put/Delete(DB::Open時*dbptr = impl) => DBImpl::Write => (1) 寫log: log_->AddRecord (2) 寫memtable: WriteBatchInternal::InsertInto(updates, mem_)?

leveldb的Get:?
DBImpl::Get => (1) 查memtable: mem->Get (2) 查immutable memtable: imm->Get (3) 查文件 versions_->current() => current->Get => Version::Get?

leveldb的Compaction:?
leveldb在Open/Get/Write時都有可能做Compaction: DB::Open/DBImpl::Get/DBImpl::Write(DBImpl::MakeRoomForWrite) =>DBImpl::MaybeScheduleCompaction => env_->Schedule(&DBImpl::BGWork, this) => (1) 啟后臺線程 PosixEnv::Schedule (2) DBImpl::BGWork => DBImpl::BackgroundCall => DBImpl::BackgroundCompaction?

leveldb的多線程寫:?
DBImpl::Write的瓶頸在AcquireLoggingResponsibility，多線程寫同一個db時互相競爭logger_，性能反而沒有單寫線程快. 所以為了scale，對leveldb做sharding，將key做hash后分到多個db，這樣多線程讀寫不會相互競爭，經(jīng)測試 num_threads : num_dbs為1:1時性能最好，充分利用多核?

leveldb的性能調(diào)優(yōu):?

通過sharding/batch writes/increase block_size(size per data block, default 4KB)/increase block_cache(LRUCache, default 8MB)/increase write_buffer_size(memtable size, default 4MB)來提高性能，經(jīng)過測試，單機24-core采用16 threads/16 shards/1000 batch_sizes/block_size 8K/write_buffer_size 32MB能達到70w+ ops/sec的寫性能

總結(jié)

以上是生活随笔為你收集整理的leveldb资料整理的全部內(nèi)容，希望文章能夠幫你解決所遇到的問題。

如果覺得生活随笔網(wǎng)站內(nèi)容還不錯，歡迎將生活随笔推薦給好友。