Redis設(shè)計(jì)了多種數(shù)據(jù)結(jié)構(gòu)，并以此為基礎(chǔ)構(gòu)建了多種對(duì)象，每種對(duì)象（除了新出的 stream 以外）都有超過一種的實(shí)現(xiàn)。

redisObject 這個(gè)結(jié)構(gòu)體反應(yīng)了 Redis 對(duì)象的內(nèi)存布局

typedef struct redisObject {unsigned type:4;//對(duì)象類型 4bitunsigned encoding:4;//底層數(shù)據(jù)結(jié)構(gòu) 4 bitunsigned lru:LRU_BITS; /* LRU time (relative to global lru_clock) or* LFU data (least significant 8 bits frequency* and most significant 16 bits access time). */ // 24 bitint refcount; // 4 bytevoid *ptr;//指向數(shù)據(jù)結(jié)構(gòu)的指針 // 8 byte } robj;

可以看出，robj 用4個(gè) bit 存儲(chǔ)對(duì)象類型，4個(gè) bit 存儲(chǔ)對(duì)象的底層數(shù)據(jù)結(jié)構(gòu)

以及 robj 的固定大小為 16 byte

其中對(duì)象類型有下面幾種：

#define OBJ_STRING 0 /* String object. *///字符串類型 #define OBJ_LIST 1 /* List object. *///列表類型 #define OBJ_SET 2 /* Set object. *///集合對(duì)象 #define OBJ_ZSET 3 /* Sorted set object. *///有序集合對(duì)象 #define OBJ_HASH 4 /* Hash object. *///哈希對(duì)象 #define OBJ_MODULE 5 /* Module object. *///模塊對(duì)象 #define OBJ_STREAM 6 /* Stream object. *///流對(duì)象，redis 5中新增

數(shù)據(jù)結(jié)構(gòu)有下面幾種：

#define OBJ_ENCODING_RAW 0 /* Raw representation *///基本 sds #define OBJ_ENCODING_INT 1 /* Encoded as integer *///整數(shù)表示的字符串 #define OBJ_ENCODING_HT 2 /* Encoded as hash table *///字典 #define OBJ_ENCODING_ZIPMAP 3 /* Encoded as zipmap */ //廢棄 #define OBJ_ENCODING_LINKEDLIST 4 /* No longer used: old list encoding. *//廢棄 #define OBJ_ENCODING_ZIPLIST 5 /* Encoded as ziplist *///壓縮列表 #define OBJ_ENCODING_INTSET 6 /* Encoded as intset *///整數(shù)集合 #define OBJ_ENCODING_SKIPLIST 7 /* Encoded as skiplist *///跳躍表 #define OBJ_ENCODING_EMBSTR 8 /* Embedded sds string encoding *///embstr #define OBJ_ENCODING_QUICKLIST 9 /* Encoded as linked list of ziplists */ #define OBJ_ENCODING_STREAM 10 /* Encoded as a radix tree of listpacks */

?

其實(shí)觀察?objectComputeSize 這個(gè)方法就看出對(duì)象與數(shù)據(jù)結(jié)構(gòu)的關(guān)聯(lián)關(guān)系

OBJ_STRING =?OBJ_ENCODING_RAW +?OBJ_ENCODING_INT +?OBJ_ENCODING_EMBSTR

OBJ_LIST =?OBJ_ENCODING_QUICKLIST +?OBJ_ENCODING_ZIPLIST

OBJ_SET =?OBJ_ENCODING_INTSET +?OBJ_ENCODING_HT

OBJ_ZSET =?OBJ_ENCODING_SKIPLIST +?OBJ_ENCODING_ZIPLIST

OBJ_HASH =?OBJ_ENCODING_HT +?OBJ_ENCODING_ZIPLIST

OBJ_STREAM =?OBJ_ENCODING_STREAM

?

?

為什么要設(shè)置這么復(fù)雜的對(duì)象系統(tǒng)呢，主要還是為了壓縮內(nèi)存。

以最最常見的字符串對(duì)象為例，它對(duì)應(yīng)的數(shù)據(jù)結(jié)構(gòu)是最多的，有三種，其目的在一個(gè)名為?tryObjectEncoding 的函數(shù)中可見一斑：

//嘗試壓縮 string //1. 檢查是否可以直接用 INT 存儲(chǔ)，最好能用 shared.integers 來存 //2. 檢查是否可以用 embstr 來存儲(chǔ) //3. 如果 sds 有1/10的空間空閑，則壓縮空閑空間 /* Try to encode a string object in order to save space */ robj *tryObjectEncoding(robj *o) {long value;sds s = o->ptr;size_t len;....../* Check if we can represent this string as a long integer.* Note that we are sure that a string larger than 20 chars is not* representable as a 32 nor 64 bit integer. */len = sdslen(s);if (len <= 20 && string2l(s,len,&value)) {//檢查是否為長(zhǎng)度<=20的整數(shù)/* This object is encodable as a long. Try to use a shared object.* Note that we avoid using shared integers when maxmemory is used* because every object needs to have a private LRU field for the LRU* algorithm to work well. *///檢查 value 是否落在 [0， OBJ_SHARED_INTEGERS)這個(gè)區(qū)間里if ((server.maxmemory == 0 ||!(server.maxmemory_policy & MAXMEMORY_FLAG_NO_SHARED_INTEGERS)) &&value >= 0 &&value < OBJ_SHARED_INTEGERS){decrRefCount(o);incrRefCount(shared.integers[value]);return shared.integers[value];} else {if (o->encoding == OBJ_ENCODING_RAW) sdsfree(o->ptr);o->encoding = OBJ_ENCODING_INT;o->ptr = (void*) value;return o;}}/* If the string is small and is still RAW encoded,* try the EMBSTR encoding which is more efficient.* In this representation the object and the SDS string are allocated* in the same chunk of memory to save space and cache misses. *///是否可以用 embstr 來存儲(chǔ)：檢查string 的長(zhǎng)度是否 <= 44if (len <= OBJ_ENCODING_EMBSTR_SIZE_LIMIT) {robj *emb;if (o->encoding == OBJ_ENCODING_EMBSTR) return o;emb = createEmbeddedStringObject(s,sdslen(s));decrRefCount(o);return emb;}/* We can't encode the object...** Do the last try, and at least optimize the SDS string inside* the string object to require little space, in case there* is more than 10% of free space at the end of the SDS string.** We do that only for relatively large strings as this branch* is only entered if the length of the string is greater than* OBJ_ENCODING_EMBSTR_SIZE_LIMIT. *///嘗試壓縮 sds 的空間if (o->encoding == OBJ_ENCODING_RAW &&sdsavail(s) > len/10){o->ptr = sdsRemoveFreeSpace(o->ptr);}/* Return the original object. */return o; }

可以看出 Redis 對(duì)內(nèi)存的使用是非常克制的。

?

分析一個(gè)很有意思的細(xì)節(jié)：為什么 embstr 與 raw sds 的分界線在 44 這個(gè)長(zhǎng)度呢？
看一下sdshdr8這個(gè)結(jié)構(gòu)體

struct __attribute__ ((__packed__)) sdshdr8 {uint8_t len; /* used */ // 1 byteuint8_t alloc; /* excluding the header and null terminator */ // 1 byteunsigned char flags; /* 3 lsb of type, 5 unused bits */ // 1 bytechar buf[]; };

可以看出len + alloc + flags = 3 byte

然后Redis 會(huì)默認(rèn)在存儲(chǔ)的字符串尾部加一個(gè) '\0'，這個(gè)也會(huì)占據(jù)一個(gè)1 byte 的空間

也就是說一個(gè)?sdshdr8 除去內(nèi)容以外至少要占 4個(gè) byte 的空間

再加上 robj 頭的大小 16 byte，那就是20 byte

而 jemalloc 會(huì)固定分配8/16/32/64 等大小的內(nèi)存， 所以以 44 為embstr 與 raw sds 的分界線，是有深意的（是否可以再細(xì)一點(diǎn)，將 12 作為另外一種更小的字符串的分界線呢？）

?

更有趣的是，如果往前翻幾個(gè)版本，可以發(fā)現(xiàn)這個(gè)分界線是在 39 byte，這是因?yàn)槔习姹镜?sds 只有一種：

struct sdshdr {unsigned int len;//4 byteunsigned int free;//4 bytechar buf[]; };

可以看出sdshdr 的固定開銷是4+4+1 = 9 byte，再加上 robj 的16byte就是25byte，所以分界線就只能定為39byte 了

新版本的sdshdr8 與之相比，硬是摳出了5個(gè) byte 的空間，真的非常了不起

?

轉(zhuǎn)載于:https://www.cnblogs.com/stevenczp/p/9471793.html

總結(jié)

以上是生活随笔為你收集整理的Redis 源码走读（二）对象系统的全部?jī)?nèi)容，希望文章能夠幫你解決所遇到的問題。

如果覺得生活随笔網(wǎng)站內(nèi)容還不錯(cuò)，歡迎將生活随笔推薦給好友。