WEB Service 下实现大数据量的传输
生活随笔
收集整理的這篇文章主要介紹了
WEB Service 下实现大数据量的传输
小編覺得挺不錯的,現在分享給大家,幫大家做個參考.
Vs2005里面的,查詢12000條記錄,設置RemotingFormat?= SerializationFormat.Binary;
再序列化,通過WebService傳輸,客戶端接收,再反序列化,確實效果大大的優于直接傳送DataSet,不僅網絡傳輸中如此,即使本機,性能改善也非常明顯.
下面分別是WebService里面的方法和客戶端反序列化取DataSet的方法.
1. 服務器上面取數據,填充數據集,轉換為二進制格式.
?/**////?<summary>
????///?Method?for?users?data?query?with?binaryFormatter
????///?</summary>
????///?<param?name="err"></param>
????///?<returns></returns>
????public?byte[]?BinaryUserSelect(ref?string?err)
????{
????????ClearCommand();
????????m_commandStringBuilder.Append("SELECT?*?FROM?t_Users?;");
????????DataSet?dsResult?=?new?DataSet();
????????byte[]?bArrayResult?=?null;
????????try
????????{??????????????????????????????
????????????dsResult?=?SqlHelper.ExecuteDataset(m_currentConnectionString,?CommandType.Text,?m_commandStringBuilder.ToString());
????????????//?上面都是取數據的,無需關心.二進制壓縮數據集是下面一小段
????????????dsResult.RemotingFormat?=?SerializationFormat.Binary;
????????????MemoryStream?ms?=?new?MemoryStream();
????????????IFormatter?bf?=?new?BinaryFormatter();
????????????bf.Serialize(ms,?dsResult);
????????????bArrayResult?=?ms.ToArray();
????????????ms.Close();
????????????//?
????????}
????????catch?(Exception?ee)
????????{
????????????err?=?ee.ToString();
????????}
????????return?bArrayResult;????????
????}
2. 通過WebService把byte[]格式的數據發送到客戶端,這里就是WebService自己的事情了,我們無需關心
3.客戶端接收到byte[]格式的數據,對其進行反序列化,得到數據集,進行客戶端操作.
?/**////?<summary>
????????///?Get?user?data?with?Binary?format
????????///?</summary>
????????///?<returns></returns>
????????public?DataSet?GetBinaryUserData()
????????{
????????????string?err?=?"";
????????????byte[]?bUserData?=?svc.ByteArrayUserSelect(ref?err);
????????????if?(err?!=?"")
????????????{
????????????????MessageBox.Show(err);
????????????????err?=?"";
????????????????return?null;
????????????}
????????????//?反序列化的過程
????????????MemoryStream?ms?=?new?MemoryStream(bUserData);
????????????IFormatter?bf?=?new?BinaryFormatter();
????????????object?obj?=?bf.Deserialize(ms);
????????????DataSet?dsResult?=?(DataSet)obj;
????????????//
????????????ms.Close();
????????????return?dsResult;
????????}
同樣一臺機器,手工生成12000條數據,在本地使用WebService分別讀取、傳輸并在客戶端顯示數據集和byte[]格式的數據,前者平均時間2.3秒,后者平均時間為1.7秒,之間的差別僅在傳輸過程的格式,還有后者需要的序列化和反序列化的時間.本地WebService傳輸的差別尚且如此,通過網絡傳輸的時間優化自然會更明顯..
.net1.1下面微軟提供的DataSetSurrogate開發包下載地址:http://support.microsoft.com/default.aspx?scid=kb;en-us;829740?? DataSetSurrogate 在.net 2.0里自帶
在.net1.1下的實現方式實例如下
在這里,有兩種方式:可把序列化后的數據用文件形式保存在客戶端硬盤;也可用Byte[]方式傳回客戶端,以下是代碼。
web?service?端(文件形式)?
??[WebMethod(Description="循環獲取遠程DATASET")]
??public?void?SurrogateReadTable(string?TableName)
??{
???//把DataSet通過Surrogate?Class?序列化成?Binary?Stream
??
???DataSet?ds;
???ds=SqlHelper.ExecuteDataset(cnn,CommandType.Text,"select?*?from?"+TableName);
???//實例化DataSetSurrogate,傳取出的DATASET到構造函數里
???sds?=?new?DataSetSurrogate(ds);?
???//實例化二進制流
???BinaryFormatter?bf=new?BinaryFormatter();
???StreamWriter?swDat;?
???//寫到本地一個文件里
???swDat?=?new?StreamWriter(@"c:\output_surrogate_dataset.dat");
???bf.Serialize(swDat.BaseStream,?sds);
???//這里可以知道序列化后的文件的大小
???long?size?=?swDat.BaseStream.Length;
???swDat.Close();
??
??}
客戶端?
private?void?button1_Click(object?sender,?System.EventArgs?e)
??{
???label1.Text=DateTime.Now.ToString();
???button1.Enabled=false;
???//反序列化Binary?Stream能通過Surrogate?Class轉換成?DataSet
???//從WEB?SERVICE上讀取方法
???svs.SurrogateRead("t_busdocbase");
???BinaryFormatter?bf=new?BinaryFormatter();
???StreamReader?swDat;?
???swDat?=?new?StreamReader(@"c:\output_surrogate_dataset.dat");
???object?o=bf.Deserialize(swDat.BaseStream);
??DataSet?ds;
???sds?=?(DataSetSurrogate)?o;
???ds?=?sds.ConvertToDataSet();
???dataGrid1.DataSource=ds.Tables[0];
???swDat.Close();
???}
web?service?端(Byte[]方式)
[WebMethod(Description="獲取業務資料遠程DATASET")]
??????public?byte[]?SurrogateRead1()
??????{
???????DataSet?ds;
???????ds=SqlHelper.ExecuteDataset(cnn,CommandType.Text,"select?*?from?t_busdocbase");
???????sds?=?new?DataSetSurrogate(ds);?
???????MemoryStream?s=?new?MemoryStream();
???????BinaryFormatter?bf?=?new?BinaryFormatter();
???????bf.Serialize(s,sds);
?????
???????byte[]?e?=?s.ToArray();
???????return?e;?
??
??????}
客戶端???
private?void?button3_Click(object?sender,?System.EventArgs?e)
???{
????label1.Text=DateTime.Now.ToString();
????button3.Enabled=false;
????//*反序列化Binary?Stream能通過Surrogate?Class轉換成?DataSet*/
????//從WEB?SERVICE上讀取方法
?????byte?[]?bb=svs.SurrogateRead1();
?????MemoryStream?br=new?MemoryStream(bb);
?????BinaryFormatter?bf=new?BinaryFormatter();
?????object?o=bf.Deserialize(br);
?????sds?=?(DataSetSurrogate)?o;
?????ds?=?sds.ConvertToDataSet();?
?????dataGrid1.DataSource=ds.Tables[0];
?????br.Close();
????}
我個人覺得用byte[]方式會安全些,畢竟不用在客戶端產生文件,不用擔心數據的安全。
在2.0 中對數據集序列化和反序列化的方法進行了一下簡單的封裝,使其可以得到重用的效果.見下面的類DatFormatter.
通過GetBinaryFormatData方法可以轉換數據集為二進制,在服務器端使用,轉換數據集格式。發送,客戶端接收,得到二進制格式數據,使用RetrieveDataSet方法,反序列化,得到數據集,進行客戶端操作。通過這些簡單的操作(序列化和反序列化,將數據壓縮),可以使數據集等體積龐大的對象在遠程傳遞中的時間大大減少,并且可以減少網絡中斷等問題對程序的影響。
?1using?System;
?2using?System.IO;
?3using?System.Data;
?4using?System.Runtime.Serialization;
?5using?System.Runtime.Serialization.Formatters.Binary;
?6
?7namespace?Common
?8{
?9????public?class?DataFormatter
10????{
11????????private?DataFormatter()?{?}
12????????/**////?<summary>
13????????///?Serialize?the?Data?of?dataSet?to?binary?format
14????????///?</summary>
15????????///?<param?name="dsOriginal"></param>
16????????///?<returns></returns>
17????????static?public?byte[]?GetBinaryFormatData(DataSet?dsOriginal)
18????????{
19????????????byte[]?binaryDataResult?=?null;
20????????????MemoryStream?memStream?=?new?MemoryStream();
21????????????IFormatter?brFormatter?=?new?BinaryFormatter();
22????????????dsOriginal.RemotingFormat?=?SerializationFormat.Binary;
23
24????????????brFormatter.Serialize(memStream,?dsOriginal);
25????????????binaryDataResult?=?memStream.ToArray();
26????????????memStream.Close();
27????????????memStream.Dispose();
28????????????return?binaryDataResult;
29????????}
30????????/**////?<summary>
31????????///?Retrieve?dataSet?from?data?of?binary?format
32????????///?</summary>
33????????///?<param?name="binaryData"></param>
34????????///?<returns></returns>
35????????static?public?DataSet?RetrieveDataSet(byte[]?binaryData)
36????????{?
37????????????DataSet?dataSetResult?=?null;
38????????????MemoryStream?memStream?=?new?MemoryStream(binaryData);
39????????????IFormatter?brFormatter?=?new?BinaryFormatter();
40
41????????????object?obj?=?brFormatter.Deserialize(memStream);
42????????????dataSetResult?=?(DataSet)obj;
43????????????return?dataSetResult;
44????????}
45????}
46}
47
再序列化,通過WebService傳輸,客戶端接收,再反序列化,確實效果大大的優于直接傳送DataSet,不僅網絡傳輸中如此,即使本機,性能改善也非常明顯.
下面分別是WebService里面的方法和客戶端反序列化取DataSet的方法.
1. 服務器上面取數據,填充數據集,轉換為二進制格式.
?/**////?<summary>
????///?Method?for?users?data?query?with?binaryFormatter
????///?</summary>
????///?<param?name="err"></param>
????///?<returns></returns>
????public?byte[]?BinaryUserSelect(ref?string?err)
????{
????????ClearCommand();
????????m_commandStringBuilder.Append("SELECT?*?FROM?t_Users?;");
????????DataSet?dsResult?=?new?DataSet();
????????byte[]?bArrayResult?=?null;
????????try
????????{??????????????????????????????
????????????dsResult?=?SqlHelper.ExecuteDataset(m_currentConnectionString,?CommandType.Text,?m_commandStringBuilder.ToString());
????????????//?上面都是取數據的,無需關心.二進制壓縮數據集是下面一小段
????????????dsResult.RemotingFormat?=?SerializationFormat.Binary;
????????????MemoryStream?ms?=?new?MemoryStream();
????????????IFormatter?bf?=?new?BinaryFormatter();
????????????bf.Serialize(ms,?dsResult);
????????????bArrayResult?=?ms.ToArray();
????????????ms.Close();
????????????//?
????????}
????????catch?(Exception?ee)
????????{
????????????err?=?ee.ToString();
????????}
????????return?bArrayResult;????????
????}
2. 通過WebService把byte[]格式的數據發送到客戶端,這里就是WebService自己的事情了,我們無需關心
3.客戶端接收到byte[]格式的數據,對其進行反序列化,得到數據集,進行客戶端操作.
?/**////?<summary>
????????///?Get?user?data?with?Binary?format
????????///?</summary>
????????///?<returns></returns>
????????public?DataSet?GetBinaryUserData()
????????{
????????????string?err?=?"";
????????????byte[]?bUserData?=?svc.ByteArrayUserSelect(ref?err);
????????????if?(err?!=?"")
????????????{
????????????????MessageBox.Show(err);
????????????????err?=?"";
????????????????return?null;
????????????}
????????????//?反序列化的過程
????????????MemoryStream?ms?=?new?MemoryStream(bUserData);
????????????IFormatter?bf?=?new?BinaryFormatter();
????????????object?obj?=?bf.Deserialize(ms);
????????????DataSet?dsResult?=?(DataSet)obj;
????????????//
????????????ms.Close();
????????????return?dsResult;
????????}
同樣一臺機器,手工生成12000條數據,在本地使用WebService分別讀取、傳輸并在客戶端顯示數據集和byte[]格式的數據,前者平均時間2.3秒,后者平均時間為1.7秒,之間的差別僅在傳輸過程的格式,還有后者需要的序列化和反序列化的時間.本地WebService傳輸的差別尚且如此,通過網絡傳輸的時間優化自然會更明顯..
.net1.1下面微軟提供的DataSetSurrogate開發包下載地址:http://support.microsoft.com/default.aspx?scid=kb;en-us;829740?? DataSetSurrogate 在.net 2.0里自帶
在.net1.1下的實現方式實例如下
在這里,有兩種方式:可把序列化后的數據用文件形式保存在客戶端硬盤;也可用Byte[]方式傳回客戶端,以下是代碼。
web?service?端(文件形式)?
??[WebMethod(Description="循環獲取遠程DATASET")]
??public?void?SurrogateReadTable(string?TableName)
??{
???//把DataSet通過Surrogate?Class?序列化成?Binary?Stream
??
???DataSet?ds;
???ds=SqlHelper.ExecuteDataset(cnn,CommandType.Text,"select?*?from?"+TableName);
???//實例化DataSetSurrogate,傳取出的DATASET到構造函數里
???sds?=?new?DataSetSurrogate(ds);?
???//實例化二進制流
???BinaryFormatter?bf=new?BinaryFormatter();
???StreamWriter?swDat;?
???//寫到本地一個文件里
???swDat?=?new?StreamWriter(@"c:\output_surrogate_dataset.dat");
???bf.Serialize(swDat.BaseStream,?sds);
???//這里可以知道序列化后的文件的大小
???long?size?=?swDat.BaseStream.Length;
???swDat.Close();
??
??}
客戶端?
private?void?button1_Click(object?sender,?System.EventArgs?e)
??{
???label1.Text=DateTime.Now.ToString();
???button1.Enabled=false;
???//反序列化Binary?Stream能通過Surrogate?Class轉換成?DataSet
???//從WEB?SERVICE上讀取方法
???svs.SurrogateRead("t_busdocbase");
???BinaryFormatter?bf=new?BinaryFormatter();
???StreamReader?swDat;?
???swDat?=?new?StreamReader(@"c:\output_surrogate_dataset.dat");
???object?o=bf.Deserialize(swDat.BaseStream);
??DataSet?ds;
???sds?=?(DataSetSurrogate)?o;
???ds?=?sds.ConvertToDataSet();
???dataGrid1.DataSource=ds.Tables[0];
???swDat.Close();
???}
web?service?端(Byte[]方式)
[WebMethod(Description="獲取業務資料遠程DATASET")]
??????public?byte[]?SurrogateRead1()
??????{
???????DataSet?ds;
???????ds=SqlHelper.ExecuteDataset(cnn,CommandType.Text,"select?*?from?t_busdocbase");
???????sds?=?new?DataSetSurrogate(ds);?
???????MemoryStream?s=?new?MemoryStream();
???????BinaryFormatter?bf?=?new?BinaryFormatter();
???????bf.Serialize(s,sds);
?????
???????byte[]?e?=?s.ToArray();
???????return?e;?
??
??????}
客戶端???
private?void?button3_Click(object?sender,?System.EventArgs?e)
???{
????label1.Text=DateTime.Now.ToString();
????button3.Enabled=false;
????//*反序列化Binary?Stream能通過Surrogate?Class轉換成?DataSet*/
????//從WEB?SERVICE上讀取方法
?????byte?[]?bb=svs.SurrogateRead1();
?????MemoryStream?br=new?MemoryStream(bb);
?????BinaryFormatter?bf=new?BinaryFormatter();
?????object?o=bf.Deserialize(br);
?????sds?=?(DataSetSurrogate)?o;
?????ds?=?sds.ConvertToDataSet();?
?????dataGrid1.DataSource=ds.Tables[0];
?????br.Close();
????}
我個人覺得用byte[]方式會安全些,畢竟不用在客戶端產生文件,不用擔心數據的安全。
在2.0 中對數據集序列化和反序列化的方法進行了一下簡單的封裝,使其可以得到重用的效果.見下面的類DatFormatter.
通過GetBinaryFormatData方法可以轉換數據集為二進制,在服務器端使用,轉換數據集格式。發送,客戶端接收,得到二進制格式數據,使用RetrieveDataSet方法,反序列化,得到數據集,進行客戶端操作。通過這些簡單的操作(序列化和反序列化,將數據壓縮),可以使數據集等體積龐大的對象在遠程傳遞中的時間大大減少,并且可以減少網絡中斷等問題對程序的影響。
?1using?System;
?2using?System.IO;
?3using?System.Data;
?4using?System.Runtime.Serialization;
?5using?System.Runtime.Serialization.Formatters.Binary;
?6
?7namespace?Common
?8{
?9????public?class?DataFormatter
10????{
11????????private?DataFormatter()?{?}
12????????/**////?<summary>
13????????///?Serialize?the?Data?of?dataSet?to?binary?format
14????????///?</summary>
15????????///?<param?name="dsOriginal"></param>
16????????///?<returns></returns>
17????????static?public?byte[]?GetBinaryFormatData(DataSet?dsOriginal)
18????????{
19????????????byte[]?binaryDataResult?=?null;
20????????????MemoryStream?memStream?=?new?MemoryStream();
21????????????IFormatter?brFormatter?=?new?BinaryFormatter();
22????????????dsOriginal.RemotingFormat?=?SerializationFormat.Binary;
23
24????????????brFormatter.Serialize(memStream,?dsOriginal);
25????????????binaryDataResult?=?memStream.ToArray();
26????????????memStream.Close();
27????????????memStream.Dispose();
28????????????return?binaryDataResult;
29????????}
30????????/**////?<summary>
31????????///?Retrieve?dataSet?from?data?of?binary?format
32????????///?</summary>
33????????///?<param?name="binaryData"></param>
34????????///?<returns></returns>
35????????static?public?DataSet?RetrieveDataSet(byte[]?binaryData)
36????????{?
37????????????DataSet?dataSetResult?=?null;
38????????????MemoryStream?memStream?=?new?MemoryStream(binaryData);
39????????????IFormatter?brFormatter?=?new?BinaryFormatter();
40
41????????????object?obj?=?brFormatter.Deserialize(memStream);
42????????????dataSetResult?=?(DataSet)obj;
43????????????return?dataSetResult;
44????????}
45????}
46}
47
轉載于:https://www.cnblogs.com/ghd258/archive/2005/12/06/291653.html
總結
以上是生活随笔為你收集整理的WEB Service 下实现大数据量的传输的全部內容,希望文章能夠幫你解決所遇到的問題。
- 上一篇: 现在编程语言的两大主流
- 下一篇: [导入]防止你的日志页被人iframe