在Android中使用FlatBuffers
生活随笔
收集整理的這篇文章主要介紹了
在Android中使用FlatBuffers
小編覺得挺不錯的,現在分享給大家,幫大家做個參考.
總覽
先來看一下 FlatBuffers 項目已經為我們提供了什么，而我們在將 FlatBuffers 用到我們的項目中時又需要做什么的整體流程。如下圖：
：.jpg
在使用 FlatBuffers 時，我們需要以特殊的格式定義我們的結構化數據，保存為 .fbs 文件。FlatBuffers 項目為我們提供了編譯器，可用于將 .fbs 文件編譯為Java文件，C++文件等，以用于我們的項目。FlatBuffers 編譯器在我們的開發機，比如Ubuntu，Mac上運行。這些源代碼文件是基于 FlatBuffers 提供的Java庫生成的，同時我們也需要利用這個Java庫的一些接口來序列化或解析數據。
我們將 FlatBuffers 編譯器生成的Java文件及 FlatBuffers 的Java庫導入我們的項目，就可以用 FlatBuffers 來對我們的結構化數據執行序列化和反序列化了。盡管每次手動執行 FlatBuffers 編譯器生成Java文件非常麻煩，但不像 Protocol Buffers 那樣，當前還沒有Google官方提供的gradle插件可用。不過，我們這邊開發了一個簡單的 FlatBuffers gradle插件，后面會簡單介紹一下，歡迎大家使用。
接下來我們更詳細地看一下上面流程中的各個部分。
下載、編譯 FlatBuffers 編譯器
我們可以在如下位置：
https://github.com/google/flatbuffers/releases獲取官方發布的打包好的版本。針對Windows平臺有編譯好的可執行安裝文件，對其它平臺還是打包的源文件。我們也可以指向clone repo的代碼，進行手動編譯。這里我們從GitHub上clone代碼并手動編譯編譯器：
$ git clone https://github.com/google/flatbuffers.git Cloning into 'flatbuffers'... remote: Counting objects: 7340, done. remote: Compressing objects: 100% (46/46), done. remote: Total 7340 (delta 16), reused 0 (delta 0), pack-reused 7290 Receiving objects: 100% (7340/7340), 3.64 MiB | 115.00 KiB/s, done. Resolving deltas: 100% (4692/4692), done. Checking connectivity... done.下載代碼之后，我們需要用cmake工具來為flatbuffers生成Makefile文件并編譯：
$ cd flatbuffers/ $ cmake CMakeLists.txt -- The C compiler identification is AppleClang 7.3.0.7030031 -- The CXX compiler identification is AppleClang 7.3.0.7030031 -- Check for working C compiler: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/cc -- Check for working C compiler: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/cc -- works -- Detecting C compiler ABI info -- Detecting C compiler ABI info - done -- Detecting C compile features -- Detecting C compile features - done -- Check for working CXX compiler: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/c++ -- Check for working CXX compiler: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/c++ -- works -- Detecting CXX compiler ABI info -- Detecting CXX compiler ABI info - done -- Detecting CXX compile features -- Detecting CXX compile features - done -- Configuring done -- Generating done -- Build files have been written to: /Users/netease/Projects/OpenSource/flatbuffers $ make && make install安裝之后執行如下命令以確認已經裝好：
$ flatc --version flatc version 1.4.0 (Dec 7 2016)flatc沒有為我們提供 --help 選項，不過加了錯誤的參數時這個工具會為我們展示詳細的用法：
$ flatc --help flatc: unknown commandline argument: --help usage: flatc [OPTION]... FILE... [-- FILE...]--binary -b Generate wire format binaries for any data definitions.--json -t Generate text output for any data definitions.--cpp -c Generate C++ headers for tables/structs.--go -g Generate Go files for tables/structs.--java -j Generate Java classes for tables/structs.--js -s Generate JavaScript code for tables/structs.--csharp -n Generate C# classes for tables/structs.--python -p Generate Python files for tables/structs.--php Generate PHP files for tables/structs.-o PATH Prefix PATH to all generated files.-I PATH Search for includes in the specified path.-M Print make rules for generated files.--version Print the version number of flatc and exit.--strict-json Strict JSON: field names must be / will be quoted,no trailing commas in tables/vectors.--allow-non-utf8 Pass non-UTF-8 input through parser and emit nonstandard\x escapes in JSON. (Default is to raise parse error onnon-UTF-8 input.)--defaults-json Output fields whose value is the default whenwriting JSON--unknown-json Allow fields in JSON that are not defined in theschema. These fields will be discared when generatingbinaries.--no-prefix Don't prefix enum values with the enum type in C++.--scoped-enums Use C++11 style scoped and strongly typed enums.also implies --no-prefix.--gen-includes (deprecated), this is the default behavior.If the original behavior is required (no includestatements) use --no-includes.--no-includes Don't generate include statements for includedschemas the generated file depends on (C++).--gen-mutable Generate accessors that can mutate buffers in-place.--gen-onefile Generate single output file for C#.--gen-name-strings Generate type name functions for C++.--escape-proto-ids Disable appending '_' in namespaces names.--gen-object-api Generate an additional object-based API.--cpp-ptr-type T Set object API pointer type (default std::unique_ptr)--raw-binary Allow binaries without file_indentifier to be read.This may crash flatc given a mismatched schema.--proto Input is a .proto, translate to .fbs.--schema Serialize schemas instead of JSON (use with -b)--conform FILE Specify a schema the following schemas should bean evolution of. Gives errors if not.--conform-includes Include path for the schema given with --conformPATH FILEs may be schemas, or JSON files (conforming to preceding schema) FILEs after the -- must be binary flatbuffer format files. Output files are named using the base file name of the input, and written to the current directory or the path given by -o. example: flatc -c -b schema1.fbs schema2.fbs data.json創建 .fbs 文件
flatc支持將為 Protocol Buffers 編寫的 .proto 文件轉換為 .fbs 文件，如：
$ ls addressbook.proto $ flatc --proto addressbook.proto $ ls -l total 16 -rw-r--r-- 1 netease staff 431 12 7 17:21 addressbook.fbs -rw-r--r--@ 1 netease staff 486 12 1 15:18 addressbook.protoProtocol Buffers 消息文件中的一些寫法，FlatBuffers 編譯器還不能很好的支持，如option java_package，option java_outer_classname，和嵌套類。這里我們基于 FlatBuffers 編譯器轉換的 .proto 文件來獲得我們的 .fbs 文件：
// Generated from addressbook.protonamespace com.example.tutorial;enum PhoneType : int {MOBILE = 0,HOME = 1,WORK = 2, }namespace com.example.tutorial;table Person {name:string (required);id:int;email:string;phone:[com.example.tutorial._Person.PhoneNumber]; }namespace com.example.tutorial._Person;table PhoneNumber {number:string (required);type:int; }namespace com.example.tutorial;table AddressBook {person:[com.example.tutorial.Person]; }root_type AddressBook;可以參考 官方的文檔 來了解 .fbs 文件的詳細的寫法。
編譯 .fbs 文件
可以通過如下命令編譯 .fbs 文件：
$ flatc --java -o out addressbook.fbs--java用于指定編譯的目標編程語言。-o 參數則用于指定輸出文件的路徑，如過沒有提供則將當前目錄用作輸出目錄。FlatBuffers 編譯器按照為不同的數據結構聲明的namespace生成目錄結構。對于上面的例子，會生成如下的這些文件：
$ find out p.p1 {margin: 0.0px 0.0px 0.0px 0.0px; font: 11.0px Menlo}span.s1 {font-variant-ligatures: no-common-ligatures}$ find out/ out/ out//com out//com/example out//com/example/tutorial out//com/example/tutorial/_Person out//com/example/tutorial/_Person/PhoneNumber.java out//com/example/tutorial/AddressBook.java out//com/example/tutorial/Person.java out//com/example/tutorial/PhoneType.java在Android項目中使用 FlatBuffers
我們將前面由 .fbs 文件生成的Java文件拷貝到我們的項目中。我們前面提到的，FlatBuffers 的Java庫比較薄，當前并沒有發不到jcenter這樣的maven倉庫中，因而我們需要將這部分代碼也拷貝到我們的額項目中。FlatBuffers 的Java庫在其repo倉庫的 java 目錄下。引入這些文件之后，我們的代碼結構如下：
添加訪問 FlatBuffers 的類：
package com.netease.volleydemo;import com.example.tutorial.AddressBook; import com.example.tutorial.Person; import com.example.tutorial._Person.PhoneNumber; import com.google.flatbuffers.FlatBufferBuilder;import java.nio.ByteBuffer;/*** Created by hanpfei0306 on 16-12-5.*/public class AddressBookFlatBuffers {public static byte[] encodeTest(String[] names) {FlatBufferBuilder builder = new FlatBufferBuilder(0);int[] personOffsets = new int[names.length];for (int i = 0; i < names.length; ++ i) {int name = builder.createString(names[i]);int email = builder.createString("zhangsan@gmail.com");int number1 = builder.createString("0157-23443276");int type1 = 1;int phoneNumber1 = PhoneNumber.createPhoneNumber(builder, number1, type1);int number2 = builder.createString("136183667387");int type2 = 0;int phoneNumber2 = PhoneNumber.createPhoneNumber(builder, number2, type2);int[] phoneNubers = new int[2];phoneNubers[0] = phoneNumber1;phoneNubers[1] = phoneNumber2;int phoneNumbersPos = Person.createPhoneVector(builder, phoneNubers);int person = Person.createPerson(builder, name, 13958235, email, phoneNumbersPos);personOffsets[i] = person;}int persons = AddressBook.createPersonVector(builder, personOffsets);AddressBook.startAddressBook(builder);AddressBook.addPerson(builder, persons);int eab = AddressBook.endAddressBook(builder);builder.finish(eab);byte[] data = builder.sizedByteArray();return data;}public static byte[] encodeTest(String[] names, int times) {for (int i = 0; i < times - 1; ++ i) {encodeTest(names);}return encodeTest(names);}public static AddressBook decodeTest(byte[] data) {AddressBook addressBook = null;ByteBuffer byteBuffer = ByteBuffer.wrap(data);addressBook = AddressBook.getRootAsAddressBook(byteBuffer);return addressBook;}public static AddressBook decodeTest(byte[] data, int times) {AddressBook addressBook = null;for (int i = 0; i < times; ++ i) {addressBook = decodeTest(data);}return addressBook;} }使用 flatbuf-gradle-plugin
我們有開發一個 FlatBuffers 的gradle插件，以方便開發，項目位置。這個插件的設計有參考Google的protobuf-gradle-plugin，功能與用法也與protobuf-gradle-plugin類似。在這個項目中，我們也有為 FlatBuffers 的Java庫創建一個module。
編譯并發布flatbuf-gradle-plugin
從github上下載代碼：
$ git clone https://github.com/hanpfei/flatbuffers.git然后將代碼導入Android Studio，將看到如下的代碼結構：
app 模塊是一個demo程序，flatbuf-gradle-plugin 模塊是 FlatBuffers 的gradle插件，而flatbuffers模塊則是 FlatBuffers 的Java庫。
為了使用 flatbuf-gradle-plugin，可以將插件發布到本地文件系統。這可以通過修改flatbuf-gradle-plugin/build.gradle來完成，修改 uploadArchives task 的 repository 指向本地文件系統，如：
uploadArchives {repositories {mavenDeployer {pom.groupId = 'com.netease.hearttouch'pom.artifactId = 'ht-flatbuf-gradle-plugin'pom.version = '0.0.1-SNAPSHOT'repository(url: 'file:///Users/netease/Projects/CorpProjects/ht-flatbuffers/app/plugin')}} }執行uploadArchives task，編譯并發布flatbuf-gradle-plugin到本地文件系統。
應用flatbuf-gradle-plugin
修改應用程序的 build.gradle 以應用flatbuf-gradle-plugin。
添加flatbuf塊，對flatbuf-gradle-plugin的執行做配置：
flatbuf {flatc {path = '/usr/local/bin/flatc'}generateFlatTasks {all().each { task ->task.builtins {remove java}task.builtins {java { }}}} }flatc塊用于配置 FlatBuffers 編譯器，這里我們指定用我們之前手動編譯的編譯器。
task.builtins的塊必不可少，這個塊用于指定我們要為那些編程語言生成代碼，這里我們為Java生成代碼。
這樣我們就不用再那么麻煩每次手動執行flatc了。
FlatBuffers、Protobuf及JSON對比測試
FlatBuffers相對于Protobuf的表現又如何呢？這里我們用數據說話，對比一下FlatBuffers格式、JSON格式與Protobuf的表現。測試同樣用fastjson作為JSON的編碼解碼工具。
測試用的數據結構所有的數據結構，Protobuf相關的測試代碼，及JSON的測試代碼同 在Android中使用Protocol Buffers 一文所述，FlatBuffers的測試代碼如上面看到的 AddressBookFlatBuffers。
通過如下的這段代碼來執行測試：
private class ProtoTestTask extends AsyncTask<Void, Void, Void> {private static final int BUFFER_LEN = 8192;private void compress(InputStream is, OutputStream os)throws Exception {GZIPOutputStream gos = new GZIPOutputStream(os);int count;byte data[] = new byte[BUFFER_LEN];while ((count = is.read(data, 0, BUFFER_LEN)) != -1) {gos.write(data, 0, count);}gos.finish();gos.close();}private int getCompressedDataLength(byte[] data) {ByteArrayInputStream bais =new ByteArrayInputStream(data);ByteArrayOutputStream baos = new ByteArrayOutputStream();try {compress(bais, baos);} catch (Exception e) {}return baos.toByteArray().length;}private void dumpDataLengthInfo(byte[] protobufData, String jsonData, byte[] flatbufData) {int compressedProtobufLength = getCompressedDataLength(protobufData);int compressedJSONLength = getCompressedDataLength(jsonData.getBytes());int compressedFlatbufLength = getCompressedDataLength(flatbufData);Log.i(TAG, String.format("%-120s", "Data length"));Log.i(TAG, String.format("%-20s%-20s%-20s%-20s%-20s%-20s", "Protobuf", "Protobuf (GZIP)","JSON", "JSON (GZIP)", "Flatbuf", "Flatbuf (GZIP)"));Log.i(TAG, String.format("%-20s%-20s%-20s%-20s%-20s%-20s",String.valueOf(protobufData.length), compressedProtobufLength,String.valueOf(jsonData.getBytes().length), compressedJSONLength,String.valueOf(flatbufData.length), compressedFlatbufLength));}private void doEncodeTest(String[] names, int times) {long startTime = System.nanoTime();byte[] protobufData = AddressBookProtobuf.encodeTest(names, times);long protobufTime = System.nanoTime();protobufTime = protobufTime - startTime;startTime = System.nanoTime();String jsonData = AddressBookJson.encodeTest(names, times);long jsonTime = System.nanoTime();jsonTime = jsonTime - startTime;startTime = System.nanoTime();byte[] flatbufData = AddressBookFlatBuffers.encodeTest(names, times);long flatbufTime = System.nanoTime();flatbufTime = flatbufTime - startTime;dumpDataLengthInfo(protobufData, jsonData, flatbufData);Log.i(TAG, String.format("%-20s%-20s%-20s%-20s", "Encode Times", String.valueOf(times),"Names Length", String.valueOf(names.length)));Log.i(TAG, String.format("%-20s%-20s%-20s%-20s%-20s%-20s","ProtobufTime", String.valueOf(protobufTime),"JsonTime", String.valueOf(jsonTime),"FlatbufTime", String.valueOf(flatbufTime)));}private void doEncodeTest10(int times) {doEncodeTest(TestUtils.sTestNames10, times);}private void doEncodeTest50(int times) {doEncodeTest(TestUtils.sTestNames50, times);}private void doEncodeTest100(int times) {doEncodeTest(TestUtils.sTestNames100, times);}private void doEncodeTest(int times) {doEncodeTest10(times);doEncodeTest50(times);doEncodeTest100(times);}private void doDecodeTest(String[] names, int times) {byte[] protobufBytes = AddressBookProtobuf.encodeTest(names);ByteArrayInputStream bais = new ByteArrayInputStream(protobufBytes);long startTime = System.nanoTime();AddressBookProtobuf.decodeTest(bais, times);long protobufTime = System.nanoTime();protobufTime = protobufTime - startTime;String jsonStr = AddressBookJson.encodeTest(names);startTime = System.nanoTime();AddressBookJson.decodeTest(jsonStr, times);long jsonTime = System.nanoTime();jsonTime = jsonTime - startTime;byte[] flatbufData = AddressBookFlatBuffers.encodeTest(names);startTime = System.nanoTime();AddressBookFlatBuffers.decodeTest(flatbufData, times);long flatbufTime = System.nanoTime();flatbufTime = flatbufTime - startTime;Log.i(TAG, String.format("%-20s%-20s%-20s%-20s", "Decode Times", String.valueOf(times),"Names Length", String.valueOf(names.length)));Log.i(TAG, String.format("%-20s%-20s%-20s%-20s%-20s%-20s","ProtobufTime", String.valueOf(protobufTime),"JsonTime", String.valueOf(jsonTime),"FlatbufTime", String.valueOf(flatbufTime)));}private void doDecodeTest10(int times) {doDecodeTest(TestUtils.sTestNames10, times);}private void doDecodeTest50(int times) {doDecodeTest(TestUtils.sTestNames50, times);}private void doDecodeTest100(int times) {doDecodeTest(TestUtils.sTestNames100, times);}private void doDecodeTest(int times) {doDecodeTest10(times);doDecodeTest50(times);doDecodeTest100(times);}@Overrideprotected Void doInBackground(Void... params) {TestUtils.initTest();doEncodeTest(5000);doDecodeTest(5000);return null;}@Overrideprotected void onPostExecute(Void aVoid) {super.onPostExecute(aVoid);}}這里我們執行3組編碼測試及3組解碼測試。對于編碼測試，第一組的單個數據中包含10個Person，第二組的包含50個，第三組的包含100個，然后對每個數據分別執行5000次的編碼操作。
對于解碼測試，三組中單個數據同樣包含10個Person、50個及100個，然后對每個數據分別執行5000次的解碼碼操作。
在Galaxy Nexus的Android 4.4.4 CM平臺上執行上述測試，最終得到如下結果：
編碼后數據長度對比 (Bytes)
| 10 | 860 | 288 | 1703 | 343 | 1532 | 513 |
| 50 | 4300 | 986 | 8463 | 1048 | 7452 | 1814 |
| 100 | 8600 | 1841 | 16913 | 1918 | 14852 | 3416 |
相同的數據，經過編碼，在壓縮前JSON的數據最長，FlatBuffers的數據長度與JSON的短大概10 ％，而Protobuf的數據長度則大概只有JSON的一半。而在用GZIP壓縮后，Protobuf的數據長度與JSON的接近，FlatBuffers的數據長度則接近兩者的兩倍。
編碼性能對比 (S)
| 10 | 6.000 | 8.952 | 12.464 |
| 50 | 26.847 | 45.782 | 56.752 |
| 100 | 50.602 | 73.688 | 108.426 |
編碼性能Protobuf相對于JSON有較大幅度的提高，而FlatBuffers則有較大幅度的降低。
解碼性能對比 (S)
| 10 | 0.255 | 10.766 | 0.014 |
| 50 | 0.245 | 51.134 | 0.014 |
| 100 | 0.323 | 101.070 | 0.006 |
解碼性能方面，Protobuf相對于JSON，有著驚人的提升。Protobuf的解碼時間幾乎不隨著數據長度的增長而有太大的增長，而JSON則隨著數據長度的增加，解碼所需要的時間也越來越長。而FlatBuffers則由于無需解碼，在性能方面相對于前兩者更有著非常大的提升。
FlatBuffers 編碼原理
FlatBuffers的Java庫只提供了如下的4個類：
./com/google/flatbuffers/Constants.java ./com/google/flatbuffers/FlatBufferBuilder.java ./com/google/flatbuffers/Struct.java ./com/google/flatbuffers/Table.javaConstants 類定義FlatBuffers中可用的基本原始數據類型的長度：
public class Constants {// Java doesn't seem to have these./** The number of bytes in an `byte`. */static final int SIZEOF_BYTE = 1;/** The number of bytes in a `short`. */static final int SIZEOF_SHORT = 2;/** The number of bytes in an `int`. */static final int SIZEOF_INT = 4;/** The number of bytes in an `float`. */static final int SIZEOF_FLOAT = 4;/** The number of bytes in an `long`. */static final int SIZEOF_LONG = 8;/** The number of bytes in an `double`. */static final int SIZEOF_DOUBLE = 8;/** The number of bytes in a file identifier. */static final int FILE_IDENTIFIER_LENGTH = 4; }FlatBufferBuilder 用于FlatBuffers編碼，它會將我們的結構化數據序列化為字節數組。我們借助于 FlatBufferBuilder 在 ByteBuffer 中放置基本數據類型的數據、數組、字符串及對象。ByteBuffer 用于處理字節序，在序列化時，它將數據按適當的字節序進行序列化，在發序列化時，它將多個字節轉換為適當的數據類型。在 .fbs 文件中定義的 table 和 struct，為它們生成的Java 類會繼承 Table 和 Struct。
在反序列化時，輸入的ByteBuffer數據被當作字節數組，Table提供了針對字節數組的操作，生成的Java類負責對這些數據進行解釋。對于FlatBuffers編碼的數據，無需進行解碼，只需進行解釋。在編譯 .fbs 文件時，每個字段在這段數據中的位置將被確定。每個字段的類型及長度將被硬編碼進生成的Java類。
Struct 類的代碼也比較簡潔：
package com.google.flatbuffers;import java.nio.ByteBuffer;/// @cond FLATBUFFERS_INTERNAL/*** All structs in the generated code derive from this class, and add their own accessors.*/ public class Struct {/** Used to hold the position of the `bb` buffer. */protected int bb_pos;/** The underlying ByteBuffer to hold the data of the Struct. */protected ByteBuffer bb; }整體的結構如下圖：
在序列化結構化數據時，我們首先需要創建一個 FlatBufferBuilder ，在這個對象的創建過程中會分配或從調用者那里獲取 ByteBuffer，序列化的數據將保存在這個 ByteBuffer中：
/*** Start with a buffer of size `initial_size`, then grow as required.** @param initial_size The initial size of the internal buffer to use.*/public FlatBufferBuilder(int initial_size) {if (initial_size <= 0) initial_size = 1;space = initial_size;bb = newByteBuffer(initial_size);}/*** Start with a buffer of 1KiB, then grow as required.*/public FlatBufferBuilder() {this(1024);}/*** Alternative constructor allowing reuse of {@link ByteBuffer}s. The builder* can still grow the buffer as necessary. User classes should make sure* to call {@link #dataBuffer()} to obtain the resulting encoded message.** @param existing_bb The byte buffer to reuse.*/public FlatBufferBuilder(ByteBuffer existing_bb) {init(existing_bb);}/*** Alternative initializer that allows reusing this object on an existing* `ByteBuffer`. This method resets the builder's internal state, but keeps* objects that have been allocated for temporary storage.** @param existing_bb The byte buffer to reuse.* @return Returns `this`.*/public FlatBufferBuilder init(ByteBuffer existing_bb){bb = existing_bb;bb.clear();bb.order(ByteOrder.LITTLE_ENDIAN);minalign = 1;space = bb.capacity();vtable_in_use = 0;nested = false;finished = false;object_start = 0;num_vtables = 0;vector_num_elems = 0;return this;}static ByteBuffer newByteBuffer(int capacity) {ByteBuffer newbb = ByteBuffer.allocate(capacity);newbb.order(ByteOrder.LITTLE_ENDIAN);return newbb;}下面我們更詳細地分析基本數據類型數據、數組及對象的序列化過程。ByteBuffer 為小尾端的。
FlatBuffers編碼基本數據類型
FlatBuffer 的基本數據類型主要包括如下這些：
Boolean Byte Short Int Long Float DoubleFlatBufferBuilder 提供了三組方法用于操作這些數據：
public void putBoolean(boolean x);public void putByte (byte x);public void putShort (short x);public void putInt (int x);public void putLong (long x);public void putFloat (float x);public void putDouble (double x);public void addBoolean(boolean x);public void addByte (byte x);public void addShort (short x);public void addInt (int x);public void addLong (long x);public void addFloat (float x);public void addDouble (double x);public void addBoolean(int o, boolean x, boolean d);public void addByte(int o, byte x, int d);public void addShort(int o, short x, int d);public void addInt (int o, int x, int d);public void addLong (int o, long x, long d);public void addFloat (int o, float x, double d);public void addDouble (int o, double x, double d);putXXX 那一組，直接地將一個數據放入 ByteBuffer 中，它們的實現基本如下面這樣：
public void putBoolean(boolean x) {bb.put(space -= Constants.SIZEOF_BYTE, (byte) (x ? 1 : 0));}public void putByte(byte x) {bb.put(space -= Constants.SIZEOF_BYTE, x);}public void putShort(short x) {bb.putShort(space -= Constants.SIZEOF_SHORT, x);}Boolean值會被先轉為byte類型再放入 ByteBuffer。另外一點值得注意的是，數據是從 ByteBuffer 的結尾處開始放置的，space用于記錄最近放入的數據的位置及剩余的空間。
addXXX(XXX x) 那一組在放入數據之前會先做對齊處理，并在需要時擴展 ByteBuffer 的容量：
static ByteBuffer growByteBuffer(ByteBuffer bb) {int old_buf_size = bb.capacity();if ((old_buf_size & 0xC0000000) != 0) // Ensure we don't grow beyond what fits in an int.throw new AssertionError("FlatBuffers: cannot grow buffer beyond 2 gigabytes.");int new_buf_size = old_buf_size << 1;bb.position(0);ByteBuffer nbb = newByteBuffer(new_buf_size);nbb.position(new_buf_size - old_buf_size);nbb.put(bb);return nbb;}public void pad(int byte_size) {for (int i = 0; i < byte_size; i++) bb.put(--space, (byte) 0);}public void prep(int size, int additional_bytes) {// Track the biggest thing we've ever aligned to.if (size > minalign) minalign = size;// Find the amount of alignment needed such that `size` is properly// aligned after `additional_bytes`int align_size = ((~(bb.capacity() - space + additional_bytes)) + 1) & (size - 1);// Reallocate the buffer if needed.while (space < align_size + size + additional_bytes) {int old_buf_size = bb.capacity();bb = growByteBuffer(bb);space += bb.capacity() - old_buf_size;}pad(align_size);}public void addBoolean(boolean x) {prep(Constants.SIZEOF_BYTE, 0);putBoolean(x);}public void addInt(int x) {prep(Constants.SIZEOF_INT, 0);putInt(x);}對齊是數據存放的起始位置相對于ByteBuffer的結束位置的對齊，additional bytes被認為是不需要對齊的，且在必要的時候會在ByteBuffer可用空間的結尾處填充值為0的字節。在擴展 ByteBuffer 的空間時，老的ByteBuffer被放在新ByteBuffer的結尾處。
addXXX(int o, XXX x, YYY y) 這一組方法在放入數據之后，會將 vtable 中對應位置的值更新為最近放入的數據的offset。
public void addShort(int o, short x, int d) {if (force_defaults || x != d) {addShort(x);slot(o);}}public void slot(int voffset) {vtable[voffset] = offset();}后面我們在分析編碼對象時再來詳細地了解vtable。
基本上，在我們的應用程序代碼中不要直接調用這些方法，它們主要在構造對象時用于存儲對象的基本數據類型字段。
FlatBuffers編碼數組
編碼數組的過程如下：
Encode vector
先執行 startVector()，這個方法會記錄數組的長度，處理元素的對齊，準備足夠的空間，并設置nested，用于指示記錄的開始。
然后逐個添加元素。
最后 執行 endVector()，將nested復位，并記錄數組的長度。
我們前面的AddressBook例子中有如下這樣的生成代碼：
public static int createPersonVector(FlatBufferBuilder builder, int[] data) {builder.startVector(4, data.length, 4);for (int i = data.length - 1; i >= 0; i--) builder.addOffset(data[i]);return builder.endVector();}編碼后的數組將有如下的內存分布：
Encoded Vector
其中的Vector Length為4字節的int型值。
FlatBuffers編碼字符串
FlatBufferBuilder 創建字符串的過程如下：
public int createString(CharSequence s) {int length = s.length();int estimatedDstCapacity = (int) (length * encoder.maxBytesPerChar());if (dst == null || dst.capacity() < estimatedDstCapacity) {dst = ByteBuffer.allocate(Math.max(128, estimatedDstCapacity));}dst.clear();CharBuffer src = s instanceof CharBuffer ? (CharBuffer) s :CharBuffer.wrap(s);CoderResult result = encoder.encode(src, dst, true);if (result.isError()) {try {result.throwException();} catch (CharacterCodingException x) {throw new Error(x);}}dst.flip();return createString(dst);}public int createString(ByteBuffer s) {int length = s.remaining();addByte((byte)0);startVector(1, length, 1);bb.position(space -= length);bb.put(s);return endVector();}public int createByteVector(byte[] arr) {int length = arr.length;startVector(1, length, 1);bb.position(space -= length);bb.put(arr);return endVector();}編碼字符串的過程如下：
編碼后的字符串將有如下的內存分布：
Encoded String
FlatBuffers編碼對象
對象的編碼與數組的編碼有點類似。編碼對象的過程為：
最后執行 endObject() 結束對象的編碼。
public void startObject(int numfields) {notNested();if (vtable == null || vtable.length < numfields) vtable = new int[numfields];vtable_in_use = numfields;Arrays.fill(vtable, 0, vtable_in_use, 0);nested = true;object_start = offset();}public int endObject() {if (vtable == null || !nested)throw new AssertionError("FlatBuffers: endObject called without startObject");addInt(0);int vtableloc = offset();// Write out the current vtable.for (int i = vtable_in_use - 1; i >= 0 ; i--) {// Offset relative to the start of the table.short off = (short)(vtable[i] != 0 ? vtableloc - vtable[i] : 0);addShort(off);}final int standard_fields = 2; // The fields below:addShort((short)(vtableloc - object_start));addShort((short)((vtable_in_use + standard_fields) * SIZEOF_SHORT));// Search for an existing vtable that matches the current one.int existing_vtable = 0;outer_loop:for (int i = 0; i < num_vtables; i++) {int vt1 = bb.capacity() - vtables[i];int vt2 = space;short len = bb.getShort(vt1);if (len == bb.getShort(vt2)) {for (int j = SIZEOF_SHORT; j < len; j += SIZEOF_SHORT) {if (bb.getShort(vt1 + j) != bb.getShort(vt2 + j)) {continue outer_loop;}}existing_vtable = vtables[i];break outer_loop;}}if (existing_vtable != 0) {// Found a match:// Remove the current vtable.space = bb.capacity() - vtableloc;// Point table to existing vtable.bb.putInt(space, existing_vtable - vtableloc);} else {// No match:// Add the location of the current vtable to the list of vtables.if (num_vtables == vtables.length) vtables = Arrays.copyOf(vtables, num_vtables * 2);vtables[num_vtables++] = offset();// Point table to current vtable.bb.putInt(bb.capacity() - vtableloc, offset() - vtableloc);}nested = false;return vtableloc;}結束對象編碼的過程比較有意思：
就像C++中的vtable，這里的vtable也是針對類創建的，而不是對象。
編碼后的對象有如下的內存分布：
Encoded
圖中值為0的那個位置的值實際不是0，它指向vtable，圖中是指向在創建對象時創建的vtable，但它也可以相同類已經存在的vtable。
結束編碼
編碼數據之后，需要執行 FlatBufferBuilder 的 finish() 結束編碼：
public int offset() {return bb.capacity() - space;}public void addOffset(int off) {prep(SIZEOF_INT, 0); // Ensure alignment is already done.assert off <= offset();off = offset() - off + SIZEOF_INT;putInt(off);}public void finish(int root_table) {prep(minalign, SIZEOF_INT);addOffset(root_table);bb.position(space);finished = true;}public void finish(int root_table, String file_identifier) {prep(minalign, SIZEOF_INT + FILE_IDENTIFIER_LENGTH);if (file_identifier.length() != FILE_IDENTIFIER_LENGTH)throw new AssertionError("FlatBuffers: file identifier must be length " +FILE_IDENTIFIER_LENGTH);for (int i = FILE_IDENTIFIER_LENGTH - 1; i >= 0; i--) {addByte((byte)file_identifier.charAt(i));}finish(root_table);}這個方法主要是記錄根對象的位置。給 finish() 傳入的的根對象的位置是相對于ByteBuffer結尾處的偏移，但是在 addOffset() 中，這個偏移會被轉換為相對于整個數據塊開始處的偏移。計算off值時，最后加的SIZEOF_INT是要給后面放入的off留出空間。
整個編碼后的數據有如下的內存分布：
Encoded data
FlatBuffers 解碼原理
這里我們通過一個生成的比較簡單的類 PhoneNumber 來了解FlatBuffers的解碼。
public static PhoneNumber getRootAsPhoneNumber(ByteBuffer _bb) {return getRootAsPhoneNumber(_bb, new PhoneNumber());}public static PhoneNumber getRootAsPhoneNumber(ByteBuffer _bb, PhoneNumber obj) {_bb.order(ByteOrder.LITTLE_ENDIAN);return (obj.__assign(_bb.getInt(_bb.position()) + _bb.position(), _bb));}public void __init(int _i, ByteBuffer _bb) {bb_pos = _i;bb = _bb;}public PhoneNumber __assign(int _i, ByteBuffer _bb) {__init(_i, _bb);return this;}創建對象的時候，會初始化 bb 為保存有對象數據的ByteBuffer，bb_pos 為對象數據在ByteBuffer中的偏移。在 getRootAsPhoneNumber() 中會從 ByteBuffer的position處獲取根對象的偏移，并加上position，以計算出對象在ByteBuffer中的位置。
通過生成的PhoneNumber類中的number()、type()兩個方法來看， FlatBuffers 中是怎么訪問成員的：
public String number() {int o = __offset(4);return o != 0 ? __string(o + bb_pos) : null;}public int type() {int o = __offset(6);return o != 0 ? bb.getInt(o + bb_pos) : 0;}過程大體為：
計算字段相對于對象原點位置的偏移的方法 __offset(4) 在com.google.flatbuffers.Table中定義：
protected int __offset(int vtable_offset) {int vtable = bb_pos - bb.getInt(bb_pos);return vtable_offset < bb.getShort(vtable) ? bb.getShort(vtable + vtable_offset) : 0;}在這個方法中，先是根據對象的原點處保存的vtable的偏移得到vtable的位置，然后在從vtable中獲取對象字段相對于對象原點位置的偏移。
得到字符串字段的過程如下：
protected String __string(int offset) {CharsetDecoder decoder = UTF8_DECODER.get();decoder.reset();offset += bb.getInt(offset);ByteBuffer src = bb.duplicate().order(ByteOrder.LITTLE_ENDIAN);int length = src.getInt(offset);src.position(offset + SIZEOF_INT);src.limit(offset + SIZEOF_INT + length);int required = (int)((float)length * decoder.maxCharsPerByte());CharBuffer dst = CHAR_BUFFER.get();if (dst == null || dst.capacity() < required) {dst = CharBuffer.allocate(required);CHAR_BUFFER.set(dst);}dst.clear();try {CoderResult cr = decoder.decode(src, dst, true);if (!cr.isUnderflow()) {cr.throwException();}} catch (CharacterCodingException x) {throw new Error(x);}return dst.flip().toString();}了解了前面字符串編碼的過程之后，相信也不難了解這里解碼字符串的過程，這里完全是那個過程的相反過程。
如我們所見，FlatBuffers編碼后的數據其實無需解碼，只要通過生成的Java類對這些數據進行解釋就可以了。
FlatBuffers的原理大體如此。
Done。
總結
以上是生活随笔為你收集整理的在Android中使用FlatBuffers的全部內容，希望文章能夠幫你解決所遇到的問題。
- 上一篇： 在Android中使用Protocol
- 下一篇： Caddy Web服务器QUIC部署