當(dāng)前位置：首頁 > 编程资源 > 编程问答 >内容正文

编程问答

一次.net托管内存泄露分析

發(fā)布時間：2024/9/3 编程问答 23 豆豆

生活随笔收集整理的這篇文章主要介紹了一次.net托管内存泄露分析小編覺得挺不錯的,現(xiàn)在分享給大家,幫大家做個參考.

簡介：一次.net托管內(nèi)存泄露分析

最近協(xié)助分析了一個.net進程內(nèi)存泄露的問題，過程分享給大家。

癥狀：客戶的服務(wù)端.net進程出現(xiàn)分鐘級的cpu抖動，接近100%后落回。

圖1

分析：支持同學(xué)通過procdump.exe工具抓取了進程dump, 設(shè)定觸發(fā)dump的條件為，若進程的CPU使用量超過80%持續(xù)1秒，則開始抓取。

procdump.exe -ma -s 1 -c 80 10672 f:\aliyun

Loading Dump File [E:\temp\201127\GameServer.exe_201119_171245.dmp\GameServer.exe_201119_171245.dmp]

User Mini Dump File with Full Memory: Only application data is available

Comment: '

*** e:\soft\procdump\procdump.exe -ma -s 1 -c 80 10672 f:\aliyun

*** Process exceeded 80% CPU (system scale) for 1 second. Value: 88%. Hottest Thread: 4196 (0x1064).'

************* Path validation summary **************

Response Time (ms) Location

Deferred srv*F:\symbols*https://msdl.microsoft.com/download/symbols

Symbol search path is: srv*F:\symbols*https://msdl.microsoft.com/download/symbols

Executable search path is:

Windows 8.1 Version 9600 MP (32 procs) Free x64

Product: Server, suite: TerminalServer DataCenter SingleUserTS

6.3.9600.18217 (winblue_ltsb.160124-0053)

Machine Name:

Debug session time: Thu Nov 19 17:12:45.000 2020 (UTC + 8:00)

System Uptime: 38 days 3:36:48.460

Process Uptime: 0 days 0:33:22.000

在dump抓取時，所采樣的系統(tǒng)CPU負(fù)載高達91%。

0:059> .loadby sos clr

0:059> !threadpool

CPU utilization: 91%

Worker Thread: Total: 57 Running: 3 Idle: 49 MaxLimit: 32767 MinLimit: 32

Work Request in Queue: 0

--------------------------------------

Number of Timers: 1

--------------------------------------

Completion Port Thread:Total: 89 Free: 88 MaxFree: 64 CurrentLimit: 89 MaxLimit: 1000 MinLimit: 65

查看dump抓取瞬間，有為數(shù)不多的幾個線程在使用CPU。

1)????Thread 37

0:059> ~37s

mscorlib_ni!System.IO.FileStream.WriteFileNative(Microsoft.Win32.SafeHandles.SafeFileHandle, Byte[], Int32, Int32, System.Threading.NativeOverlapped*, Int32 ByRef)$##600184B+0x86:

00007ffb`c5923d76 48894de0 mov qword ptr [rbp-20h],rcx ss:000000e2`16b0ee20=000000e3cde2fc10

0:037> kL

# Child-SP RetAddr Call Site

00 000000e2`16b0edf0 00007ffb`c5923cc2 mscorlib_ni!System.IO.FileStream.WriteFileNative(Microsoft.Win32.SafeHandles.SafeFileHandle, Byte[], Int32, Int32, System.Threading.NativeOverlapped*, Int32 ByRef)$##600184B+0x86

01 000000e2`16b0ee50 00007ffb`c5923aa7 mscorlib_ni!System.IO.FileStream.WriteCore(Byte[], Int32, Int32)$##600183D+0x62

02 000000e2`16b0eec0 00007ffb`c5923a34 mscorlib_ni!System.IO.FileStream.FlushInternalBuffer()$##600182F+0x57

03 000000e2`16b0ef00 00007ffb`c58d4f4c mscorlib_ni!System.IO.FileStream.Flush(Boolean)$##600182E+0x24

04 000000e2`16b0ef40 00007ffb`69ccac7b mscorlib_ni!System.IO.StreamWriter.Flush(Boolean, Boolean)$##60019BE+0x8c

05 000000e2`16b0efa0 00007ffb`69ccaaae 0x00007ffb`69ccac7b

06 000000e2`16b0efe0 00007ffb`69ca103e 0x00007ffb`69ccaaae

07 000000e2`16b0f030 00007ffb`c58eca72 0x00007ffb`69ca103e

08 000000e2`16b0f070 00007ffb`c58ec904 mscorlib_ni!System.Threading.ExecutionContext.RunInternal(System.Threading.ExecutionContext, System.Threading.ContextCallback, System.Object, Boolean)$##6003A95+0x162

09 000000e2`16b0f140 00007ffb`c58ec8c2 mscorlib_ni!System.Threading.ExecutionContext.Run(System.Threading.ExecutionContext, System.Threading.ContextCallback, System.Object, Boolean)$##6003A94+0x14

0a 000000e2`16b0f170 00007ffb`c5926472 mscorlib_ni!System.Threading.ExecutionContext.Run(System.Threading.ExecutionContext, System.Threading.ContextCallback, System.Object)$##6003A93+0x52

0b 000000e2`16b0f1c0 00007ffb`c8fd6793 mscorlib_ni!System.Threading.ThreadHelper.ThreadStart()$##6003B8E+0x52

0c 000000e2`16b0f200 00007ffb`c8fd6665 clr!CallDescrWorkerInternal+0x83

0d 000000e2`16b0f240 00007ffb`c8fd736d clr!CallDescrWorkerWithHandler+0x4e

0e 000000e2`16b0f280 00007ffb`c90bbf59 clr!MethodDescCallSite::CallTargetWorker+0xf8

0f 000000e2`16b0f380 00007ffb`c8fd7ce5 clr!ThreadNative::KickOffThread_Worker+0x109

10 000000e2`16b0f5e0 00007ffb`c8fd7c60 clr!Frame::Push+0x59

11 000000e2`16b0f620 00007ffb`c8fd7b9e clr!FillInRegTypeMap+0x198

12 000000e2`16b0f720 00007ffb`c8fd7d1f clr!FillInRegTypeMap+0xc1

13 000000e2`16b0f7b0 00007ffb`c90bbe3b clr!FillInRegTypeMap+0x47

14 000000e2`16b0f810 00007ffb`c919159f clr!ThreadNative::KickOffThread+0xdb

15 000000e2`16b0f8e0 00007ffb`d90d13d2 clr!Thread::intermediateThreadProc+0x86

16 000000e2`16b0fa20 00007ffb`d92254f4 kernel32!BaseThreadInitThunk+0x22

17 000000e2`16b0fa50 00000000`00000000 ntdll!RtlUserThreadStart+0x34

0:037> ub rip

mscorlib_ni!System.IO.FileStream.WriteFileNative(Microsoft.Win32.SafeHandles.SafeFileHandle, Byte[], Int32, Int32, System.Threading.NativeOverlapped*, Int32 ByRef)$##600184B+0x6c:

00007ffb`c5923d5c c9 leave

00007ffb`c5923d5d 4903d1 add rdx,r9

00007ffb`c5923d60 4533c9 xor r9d,r9d

00007ffb`c5923d63 4c894c2420 mov qword ptr [rsp+20h],r9

00007ffb`c5923d68 4c8d4de8 lea r9,[rbp-18h]

00007ffb`c5923d6c 448bc0 mov r8d,eax

00007ffb`c5923d6f e8e40bf2ff call mscorlib_ni!System.Runtime.Remoting.Activation.ActivationServices.GetActivator()$##6005B45 (mscorlib_ni+0x434958) (00007ffb`c5844958)

00007ffb`c5923d74 33c9 xor ecx,ecx

2)????Thread 39

0:042> ~39s

mscorlib_ni!System.Text.UTF8Encoding.GetBytes(Char*, Int32, Byte*, Int32, System.Text.EncoderNLS)$##600675A+0x1df:

00007ffb`c5963cef f7c280ff80ff test edx,0FF80FF80h

0:039> kL

# Child-SP RetAddr Call Site

00 000000e2`16d0ee40 00007ffb`c58d50be mscorlib_ni!System.Text.UTF8Encoding.GetBytes(Char*, Int32, Byte*, Int32, System.Text.EncoderNLS)$##600675A+0x1df

01 000000e2`16d0eed0 00007ffb`c58d4f17 mscorlib_ni!System.Text.EncoderNLS.GetBytes(Char[], Int32, Int32, Byte[], Int32, Boolean)$##6006608+0x11e

02 000000e2`16d0ef60 00007ffb`69ccac7b mscorlib_ni!System.IO.StreamWriter.Flush(Boolean, Boolean)$##60019BE+0x57

03 000000e2`16d0efc0 00007ffb`69ccaaae 0x00007ffb`69ccac7b

04 000000e2`16d0f000 00007ffb`69ca103e 0x00007ffb`69ccaaae

05 000000e2`16d0f050 00007ffb`c58eca72 0x00007ffb`69ca103e

06 000000e2`16d0f090 00007ffb`c58ec904 mscorlib_ni!System.Threading.ExecutionContext.RunInternal(System.Threading.ExecutionContext, System.Threading.ContextCallback, System.Object, Boolean)$##6003A95+0x162

07 000000e2`16d0f160 00007ffb`c58ec8c2 mscorlib_ni!System.Threading.ExecutionContext.Run(System.Threading.ExecutionContext, System.Threading.ContextCallback, System.Object, Boolean)$##6003A94+0x14

08 000000e2`16d0f190 00007ffb`c5926472 mscorlib_ni!System.Threading.ExecutionContext.Run(System.Threading.ExecutionContext, System.Threading.ContextCallback, System.Object)$##6003A93+0x52

09 000000e2`16d0f1e0 00007ffb`c8fd6793 mscorlib_ni!System.Threading.ThreadHelper.ThreadStart()$##6003B8E+0x52

0a 000000e2`16d0f220 00007ffb`c8fd6665 clr!CallDescrWorkerInternal+0x83

0b 000000e2`16d0f260 00007ffb`c8fd736d clr!CallDescrWorkerWithHandler+0x4e

0c 000000e2`16d0f2a0 00007ffb`c90bbf59 clr!MethodDescCallSite::CallTargetWorker+0xf8

0d 000000e2`16d0f3a0 00007ffb`c8fd7ce5 clr!ThreadNative::KickOffThread_Worker+0x109

0e 000000e2`16d0f600 00007ffb`c8fd7c60 clr!Frame::Push+0x59

0f 000000e2`16d0f640 00007ffb`c8fd7b9e clr!FillInRegTypeMap+0x198

10 000000e2`16d0f740 00007ffb`c8fd7d1f clr!FillInRegTypeMap+0xc1

11 000000e2`16d0f7d0 00007ffb`c90bbe3b clr!FillInRegTypeMap+0x47

12 000000e2`16d0f830 00007ffb`c919159f clr!ThreadNative::KickOffThread+0xdb

13 000000e2`16d0f900 00007ffb`d90d13d2 clr!Thread::intermediateThreadProc+0x86

14 000000e2`16d0fb40 00007ffb`d92254f4 kernel32!BaseThreadInitThunk+0x22

15 000000e2`16d0fb70 00000000`00000000 ntdll!RtlUserThreadStart+0x34

0:039> ub rip

mscorlib_ni!System.Text.UTF8Encoding.GetBytes(Char*, Int32, Byte*, Int32, System.Text.EncoderNLS)$##600675A+0x1c0:

00007ffb`c5963cd0 4889442438 mov qword ptr [rsp+38h],rax

00007ffb`c5963cd5 e979050000 jmp mscorlib_ni!System.Text.UTF8Encoding.GetBytes(Char*, Int32, Byte*, Int32, System.Text.EncoderNLS)$##600675A+0x743 (00007ffb`c5964253)

00007ffb`c5963cda 488b4c2440 mov rcx,qword ptr [rsp+40h]

00007ffb`c5963cdf 448b19 mov r11d,dword ptr [rcx]

00007ffb`c5963ce2 488b4c2440 mov rcx,qword ptr [rsp+40h]

00007ffb`c5963ce7 8b4904 mov ecx,dword ptr [rcx+4]

00007ffb`c5963cea 418bd3 mov edx,r11d

00007ffb`c5963ced 0bd1 or edx,ecx

3)????Thread 52

0:039> ~52s

clr!SVR::gc_heap::background_mark_simple1+0x48:

00007ffb`c9162f38 488bd7 mov rdx,rdi

0:052> kL

# Child-SP RetAddr Call Site

00 000000e2`17e0f140 00007ffb`c91631ae clr!SVR::gc_heap::background_mark_simple1+0x48

01 000000e2`17e0f1b0 00007ffb`c9163f14 clr!SVR::gc_heap::background_mark_simple+0x91

02 000000e2`17e0f1e0 00007ffb`c91628b4 clr!SVR::gc_heap::background_drain_mark_list+0x50

03 000000e2`17e0f210 00007ffb`c934f660 clr!SVR::gc_heap::background_mark_phase+0x3bf

04 000000e2`17e0f2a0 00007ffb`c9162244 clr! ?? ::FNODOBFM::`string'+0x8082a

05 000000e2`17e0f2f0 00007ffb`c919159f clr!SVR::gc_heap::bgc_thread_function+0x132

06 000000e2`17e0f340 00007ffb`d90d13d2 clr!Thread::intermediateThreadProc+0x86

07 000000e2`17e0fb80 00007ffb`d92254f4 kernel32!BaseThreadInitThunk+0x22

08 000000e2`17e0fbb0 00000000`00000000 ntdll!RtlUserThreadStart+0x34

0:052> ub rip

clr!SVR::gc_heap::background_mark_simple1+0x26:

00007ffb`c9162f16 488bfa mov rdi,rdx

00007ffb`c9162f19 488bd9 mov rbx,rcx

00007ffb`c9162f1c 4c8989f01e0000 mov qword ptr [rcx+1EF0h],r9

00007ffb`c9162f23 4d8d04c1 lea r8,[r9+rax*8]

00007ffb`c9162f27 4c89442478 mov qword ptr [rsp+78h],r8

00007ffb`c9162f2c 4533db xor r11d,r11d

00007ffb`c9162f2f 4885ff test rdi,rdi

00007ffb`c9162f32 0f84e1010000 je clr!SVR::gc_heap::background_mark_simple1+0x901 (00007ffb`c9163119)

4)????Thread 113

0:052> ~113s

MSVCR120_CLR0400!memset+0x23:

00007ffb`c8f0f8b3 f3aa rep stos byte ptr [rdi]

0:113> kL

# Child-SP RetAddr Call Site

00 000000e2`1c7fd580 000000ec`7cb76810 MSVCR120_CLR0400!memset+0x23

01 000000e2`1c7fd588 00007ffb`c918f750 0x000000ec`7cb76810

02 000000e2`1c7fd590 00007ffb`c918f3d2 clr!SVR::gc_heap::adjust_limit_clr+0xe0

03 000000e2`1c7fd5e0 00007ffb`c914625f clr!SVR::gc_heap::allocate_small+0x3ae

04 000000e2`1c7fd6a0 00007ffb`c58e0e5c clr!JIT_New+0x61f

*** WARNING: Unable to verify checksum for System.Core.ni.dll

*** ERROR: Module load completed but symbols could not be loaded for System.Core.ni.dll

05 000000e2`1c7fdae0 00007ffb`c36fb3ae mscorlib_ni!System.Collections.Generic.List`1[System.__Canon].System.Collections.Generic.IEnumerable.GetEnumerator()$##60039A3+0x4c

06 000000e2`1c7fdb40 00007ffb`6ad21c2d System_Core_ni+0x2db3ae

07 000000e2`1c7fdbb0 00007ffb`6ad21544 0x00007ffb`6ad21c2d

08 000000e2`1c7fdc00 00007ffb`6ad0a240 0x00007ffb`6ad21544

09 000000e2`1c7fdc60 00007ffb`6ad09cfb 0x00007ffb`6ad0a240

0a 000000e2`1c7fdcc0 00007ffb`6ad0920c 0x00007ffb`6ad09cfb

0b 000000e2`1c7fdd00 00007ffb`6ad07790 0x00007ffb`6ad0920c

0c 000000e2`1c7fdd40 00007ffb`6ace5a30 0x00007ffb`6ad07790

0d 000000e2`1c7fdda0 00007ffb`6ace38e8 0x00007ffb`6ace5a30

0e 000000e2`1c7fde90 00007ffb`6ace1a7b 0x00007ffb`6ace38e8

0f 000000e2`1c7fdf80 00007ffb`6ace1407 0x00007ffb`6ace1a7b

10 000000e2`1c7fe080 00007ffb`6a80981e 0x00007ffb`6ace1407

11 000000e2`1c7fe0b0 00007ffb`6a8081db 0x00007ffb`6a80981e

12 000000e2`1c7fe110 00007ffb`c58eca72 0x00007ffb`6a8081db

...

20 000000e2`1c7fe710 00007ffb`c8fd6665 clr!CallDescrWorkerInternal+0x83

21 000000e2`1c7fe750 00007ffb`c8fd736d clr!CallDescrWorkerWithHandler+0x4e

22 000000e2`1c7fe790 00007ffb`c8fdaf69 clr!MethodDescCallSite::CallTargetWorker+0xf8

23 000000e2`1c7fe890 00007ffb`c8fd7ce5 clr!QueueUserWorkItemManagedCallback+0x2a

24 000000e2`1c7fe980 00007ffb`c8fd7c60 clr!Frame::Push+0x59

25 000000e2`1c7fe9c0 00007ffb`c8fd7b9e clr!FillInRegTypeMap+0x198

26 000000e2`1c7feac0 00007ffb`c8fd7d1f clr!FillInRegTypeMap+0xc1

27 000000e2`1c7feb50 00007ffb`c8fdaa70 clr!FillInRegTypeMap+0x47

28 000000e2`1c7febb0 00007ffb`c8fd82b8 clr!ManagedPerAppDomainTPCount::DispatchWorkItem+0xa0

29 000000e2`1c7fed30 00007ffb`c8fd8195 clr!ThreadpoolMgr::ExecuteWorkRequest+0x64

2a 000000e2`1c7fed60 00007ffb`c919159f clr!ThreadpoolMgr::WorkerThreadStart+0xf5

2b 000000e2`1c7fee00 00007ffb`d90d13d2 clr!Thread::intermediateThreadProc+0x86

2c 000000e2`1c7ffbc0 00007ffb`d92254f4 kernel32!BaseThreadInitThunk+0x22

2d 000000e2`1c7ffbf0 00000000`00000000 ntdll!RtlUserThreadStart+0x34

0:113> ub rip

MSVCR120_CLR0400!memset+0x6:

00007ffb`c8f0f896 4983f810 cmp r8,10h

00007ffb`c8f0f89a 0f825c010000 jb MSVCR120_CLR0400!memset+0x16c (00007ffb`c8f0f9fc)

00007ffb`c8f0f8a0 0fba25b08e0a0001 bt dword ptr [MSVCR120_CLR0400!_favor (00007ffb`c8fb8758)],1

00007ffb`c8f0f8a8 730e jae MSVCR120_CLR0400!memset+0x28 (00007ffb`c8f0f8b8)

00007ffb`c8f0f8aa 57 push rdi

00007ffb`c8f0f8ab 488bf9 mov rdi,rcx

00007ffb`c8f0f8ae 8bc2 mov eax,edx

00007ffb`c8f0f8b0 498bc8 mov rcx,r8

但是我們看到這臺機器是32核心的服務(wù)器：

0:113> !cpuid

CP F/M/S Manufacturer MHz

0 6,5,7 2500

1 6,5,7 2500

2 6,5,7 2500

3 6,5,7 2500

4 6,5,7 2500

5 6,5,7 2500

6 6,5,7 2500

7 6,5,7 2500

8 6,5,7 2500

9 6,5,7 2500

10 6,5,7 2500

11 6,5,7 2500

12 6,5,7 2500

13 6,5,7 2500

14 6,5,7 2500

15 6,5,7 2500

16 6,5,7 2500

17 6,5,7 2500

18 6,5,7 2500

19 6,5,7 2500

20 6,5,7 2500

21 6,5,7 2500

22 6,5,7 2500

23 6,5,7 2500

24 6,5,7 2500

25 6,5,7 2500

26 6,5,7 2500

27 6,5,7 2500

28 6,5,7 2500

29 6,5,7 2500

30 6,5,7 2500

31 6,5,7 2500

那么，上述幾個線程不至于將服務(wù)器的CPU飆高。 dump抓取時，實際上這個進程CPU占用并不高，因此，我們也就無法通過分析這個dump中的線程行為來直接找到high cpu的原因了。

我們留意到，這個dump本身也是很大的，dump文件本身在20G左右，而且絕大多數(shù)內(nèi)存為.net托管。

0:113> !address -summary

Mapping file section regions...

Mapping module regions...

Mapping PEB regions...

Mapping TEB and stack regions...

Mapping heap regions...

Mapping page heap regions...

Mapping other regions...

Mapping stack trace database regions...

Mapping activation context regions...

--- Usage Summary ---------------- RgnCount ----------- Total Size -------- %ofBusy %ofTotal

Free 166 7ff5`c19eb000 ( 127.960 TB) 99.97%

999 a`1af16000 ( 40.421 GB) 98.65% 0.03%

Stack 862 0`10240000 ( 258.250 MB) 0.62% 0.00%

Image 712 0`0a0b6000 ( 160.711 MB) 0.38% 0.00%

Heap 61 0`08fef000 ( 143.934 MB) 0.34% 0.00%

TEB 284 0`00238000 ( 2.219 MB) 0.01% 0.00%

Other 9 0`001d1000 ( 1.816 MB) 0.00% 0.00%

PEB 1 0`00001000 ( 4.000 kB) 0.00% 0.00%

--- Type Summary (for busy) ------ RgnCount ----------- Total Size -------- %ofBusy %ofTotal

MEM_PRIVATE 1845 a`31a9f000 ( 40.776 GB) 99.52% 0.03%

MEM_IMAGE 1048 0`0ab32000 ( 171.195 MB) 0.41% 0.00%

MEM_MAPPED 35 0`02034000 ( 32.203 MB) 0.08% 0.00%

--- State Summary ---------------- RgnCount ----------- Total Size -------- %ofBusy %ofTotal

MEM_FREE 166 7ff5`c19eb000 ( 127.960 TB) 99.97%

MEM_RESERVE 655 5`2804b000 ( 20.625 GB) 50.34% 0.02%

MEM_COMMIT 2273 5`165ba000 ( 20.349 GB) 49.66% 0.02%

--- Protect Summary (for commit) - RgnCount ----------- Total Size -------- %ofBusy %ofTotal

PAGE_READWRITE 1252 5`0a7de000 ( 20.164 GB) 49.21% 0.02%

PAGE_EXECUTE_READ 74 0`073c5000 ( 115.770 MB) 0.28% 0.00%

PAGE_READONLY 360 0`02d12000 ( 45.070 MB) 0.11% 0.00%

PAGE_WRITECOPY 175 0`0102e000 ( 16.180 MB) 0.04% 0.00%

PAGE_EXECUTE_READWRITE 79 0`005d8000 ( 5.844 MB) 0.01% 0.00%

PAGE_READWRITE|PAGE_GUARD 284 0`00582000 ( 5.508 MB) 0.01% 0.00%

PAGE_EXECUTE_WRITECOPY 36 0`0016f000 ( 1.434 MB) 0.00% 0.00%

PAGE_NOACCESS 11 0`0000b000 ( 44.000 kB) 0.00% 0.00%

PAGE_EXECUTE 2 0`00003000 ( 12.000 kB) 0.00% 0.00%

那么，很值得看一下這些.net托管對象在內(nèi)存中的行為。

我們看到，客戶自己命名空間下的對象（ShowHand，已脫敏），有些已經(jīng)達到了幾百萬的數(shù)量：

0:113> !dumpheap -stat

...

00007ffb6a1fab68 1062320 95485536 ShowHand.ConfigData.CommonPropertyInfo[]

00007ffb6ac2c4c0 1310734 104858720 ShowHand.ProjectU.Common.LBStaticActorProcessingAchievement

00007ffb6ac20db0 1633682 109318576 ShowHand.ProjectU.Common.IPropertiesProvider[]

00007ffb6ac0ad88 253100 114645888 ShowHand.ProjectU.Common.ICommonActorComp[]

00007ffbc5b16948 1809532 116227168 System.String

00007ffb6a5a84a8 1076959 120619408 ShowHand.ProjectU.Common.BattleGrid

00007ffb6b04bf40 2549394 122370912 behaviac.Action+ActionTask

00007ffb6a1f8a50 5651702 135640848 ShowHand.ConfigData.CommonPropertyInfo

00007ffb6b04ae10 1986359 143017848 behaviac.Selector+SelectorTask

00007ffb6abe65f8 31814 151783472 ShowHand.ProjectU.Common.IGameEventPipeListener[]

00007ffb6b04bdc8 3176057 152450736 behaviac.Assignment+AssignmentTask

00007ffb6ac03478 14993 155447424 System.Collections.Generic.Dictionary`2+Entry[[System.Int32, mscorlib],[ShowHand.ProjectU.Common.WayPointInfo, CommonDefine]][]

00007ffb6b0ba198 3976124 159044960 ShowHand.ProjectU.Common.BattleGrid+GridLink

00007ffb6afe0ef8 228402 164720576 ShowHand.ProjectU.Common.BattleGrid[]

00007ffb6afe0698 344120 185049680 ShowHand.ProjectU.Common.BattleGridInfo4Select[]

00007ffb6abda5f0 4078096 195748608 ShowHand.ProjectU.Common.WayPointInfo

00007ffb6b04bac0 4306198 206697504 behaviac.Condition+ConditionTask

00007ffb6b04bc88 2827987 226238960 behaviac.ReferencedBehavior+ReferencedBehaviorTask

00007ffb6aba7b88 5975977 239039080 ShowHand.ProjectU.Common.ProcessingMissionInfo

00007ffb6abebab0 6016203 240648120 ShowHand.ProjectU.Common.GameEventIdDefine[]

00007ffb6abe98e0 6050943 242037720 System.Collections.Generic.List`1[[ShowHand.ProjectU.Common.GameEventIdDefine, CommonDefine]]

00007ffb6afe0c78 515019 275971336 ShowHand.ProjectU.Common.BattleGridInfo4Attack[]

...

那么，這里是否存在托管內(nèi)存泄露的問題，就值得深究一下了。

通過查看32個gc堆，可以看到這些堆中確實十分不健康：

0:113> !eeheap -gc

Number of GC Heaps: 32

------------------------------

Heap 0 (000000e275cd3290)

generation 0 starts at 0x000000e29cfa59a8

generation 1 starts at 0x000000e29b152070

generation 2 starts at 0x000000e277b31000

ephemeral segment allocation context: none

segment begin allocated size

000000e277b30000 000000e277b31000 000000e29d2959c0 0x257649c0(628509120)

Large object heap starts at 0x000000ea77b31000

segment begin allocated size

000000ea77b30000 000000ea77b31000 000000ea7811ccc8 0x5ebcc8(6208712)

Heap Size: Size: 0x25d50688 (634717832) bytes.

------------------------------

Heap 1 (000000e275cd6620)

generation 0 starts at 0x000000e2d363e938

generation 1 starts at 0x000000e2d182a800

generation 2 starts at 0x000000e2b7b31000

ephemeral segment allocation context: none

segment begin allocated size

000000e2b7b30000 000000e2b7b31000 000000e2d5478678 0x1d947678(496268920)

Large object heap starts at 0x000000ea87b31000

segment begin allocated size

000000ea87b30000 000000ea87b31000 000000ea87fa0e68 0x46fe68(4652648)

Heap Size: Size: 0x1ddb74e0 (500921568) bytes.

------------------------------

Heap 2 (000000e275cda830)

generation 0 starts at 0x000000e31f5516c8

generation 1 starts at 0x000000e31d26b3b8

generation 2 starts at 0x000000e2f7b31000

ephemeral segment allocation context: none

segment begin allocated size

000000e2f7b30000 000000e2f7b31000 000000e321b9c7b0 0x2a06b7b0(705083312)

Large object heap starts at 0x000000ea97b31000

segment begin allocated size

000000ea97b30000 000000ea97b31000 000000ea98130d60 0x5ffd60(6290784)

Heap Size: Size: 0x2a66b510 (711374096) bytes.

------------------------------

Heap 3 (000000e275cdf480)

generation 0 starts at 0x000000e35263b398

generation 1 starts at 0x000000e350719af8

generation 2 starts at 0x000000e337b31000

ephemeral segment allocation context: none

segment begin allocated size

000000e337b30000 000000e337b31000 000000e3529b73b0 0x1ae863b0(451437488)

Large object heap starts at 0x000000eaa7b31000

segment begin allocated size

000000eaa7b30000 000000eaa7b31000 000000eaa7f90b30 0x45fb30(4586288)

Heap Size: Size: 0x1b2e5ee0 (456023776) bytes.

------------------------------

以下29個heap數(shù)據(jù)略，以gc heap0 為例：

其Gen2 中大小已經(jīng)到了593M（593629296Bytes）。

0:059> ? 0x000000e29b152070-0x000000e277b31000

Evaluate expression: 593629296 = 00000000`23621070

然而0代和1代大小才幾MB和幾十MB。這種gen0 gen1很小，gen2爆大的分布是很不正常的，說明可能存在有GC不掉的托管對象。

我們查看一下heap 0 gen2中的對象，看到一個堆中，客戶命名空間下的對象多的也達20萬之多。

0:059>!dumpheap -stat 0x000000e277b31000 0x000000e29b152070

00007ffb6a5a84a8 35146 3936352 ShowHand.ProjectU.Common.BattleGrid

00007ffb6b04ae10 62375 4491000 behaviac.Selector+SelectorTask

00007ffb6a1f8a50 190065 4561560 ShowHand.ConfigData.CommonPropertyInfo

00007ffb6b04bdc8 99780 4789440 behaviac.Assignment+AssignmentTask

00007ffb6b0ba198 129947 5197880 ShowHand.ProjectU.Common.BattleGrid+GridLink

00007ffb6ac03478 508 5266944 System.Collections.Generic.Dictionary`2+Entry[[System.Int32, mscorlib],[ShowHand.ProjectU.Common.WayPointInfo, CommonDefine]][]

00007ffb6afe0698 10768 5802880 ShowHand.ProjectU.Common.BattleGridInfo4Select[]

00007ffb6b04bac0 133954 6429792 behaviac.Condition+ConditionTask

00007ffb6abda5f0 138160 6631680 ShowHand.ProjectU.Common.WayPointInfo

00007ffb6b04bc88 87804 7024320 behaviac.ReferencedBehavior+ReferencedBehaviorTask

00007ffb6aba7b88 204523 8180920 ShowHand.ProjectU.Common.ProcessingMissionInfo

00007ffb6abebab0 205528 8221120 ShowHand.ProjectU.Common.GameEventIdDefine[]

00007ffb6abe98e0 207062 8282480 System.Collections.Generic.List`1[[ShowHand.ProjectU.Common.GameEventIdDefine, CommonDefine]]

00007ffb6afe0c78 16152 8657472 ShowHand.ProjectU.Common.BattleGridInfo4Attack[]

00007ffb6b04af38 222880 8915200 System.Collections.Generic.List`1[[behaviac.BehaviorTask, BehaviacRuntime]]

00007ffb6b0ea0c0 5248 9385248 System.Collections.Generic.Dictionary`2+Entry[[System.String, mscorlib],[FixMath.NET.Fix64, Fix64]][]

00007ffbc5aea7f0 121480 9718400 System.Collections.Generic.Dictionary`2[[System.Int32, mscorlib],[System.Int32, mscorlib]]

00007ffb6b04b730 147779 10640088 behaviac.Sequence+SequenceTask

我們已ShowHand.ProjectU.Common.ProcessingMissionInfo為例，隨機挑選一些該類的對象，查看其root行為。

0:059>!dumpheap -mt 00007ffb6aba7b88 0x000000e277b31000 0x000000e29b152070

...

000000e27c723178 00007ffb6aba7b88 40

000000e27c723348 00007ffb6aba7b88 40

000000e27c723500 00007ffb6aba7b88 40

000000e27c7236b8 00007ffb6aba7b88 40

000000e27c723888 00007ffb6aba7b88 40

000000e27c723a40 00007ffb6aba7b88 40

000000e27c723bf8 00007ffb6aba7b88 40

000000e27c723dc8 00007ffb6aba7b88 40

000000e27c723f80 00007ffb6aba7b88 40

000000e27c724138 00007ffb6aba7b88 40

000000e27c724308 00007ffb6aba7b88 40

000000e27c7244c0 00007ffb6aba7b88 40

000000e27c724678 00007ffb6aba7b88 40

000000e27c724848 00007ffb6aba7b88 40

000000e27c724a00 00007ffb6aba7b88 40

000000e27c724bb8 00007ffb6aba7b88 40

000000e27c724d88 00007ffb6aba7b88 40

000000e27c724f40 00007ffb6aba7b88 40

000000e27c7250f8 00007ffb6aba7b88 40

000000e27c7252c8 00007ffb6aba7b88 40

000000e27c725480 00007ffb6aba7b88 40

000000e27c725638 00007ffb6aba7b88 40

000000e27c725808 00007ffb6aba7b88 40

000000e27c7259c0 00007ffb6aba7b88 40

000000e27c725b78 00007ffb6aba7b88 40

000000e27c725d48 00007ffb6aba7b88 40

000000e27c725f00 00007ffb6aba7b88 40

000000e27c7260b8 00007ffb6aba7b88 40

000000e27c726288 00007ffb6aba7b88 40

000000e27c726440 00007ffb6aba7b88 40

000000e27c7265f8 00007ffb6aba7b88 40

000000e27c7267c8 00007ffb6aba7b88 40

...

隨機挑選000000e27c723348和000000e27c723f80 這兩個對象，查看其引用鏈:

0:059> !gcroot 000000e27c723348

Thread 2f78:

000000e216b0efa0 00007ffb69ccac7b log4net.Appender.FileAppender.Append(log4net.Core.LoggingEvent)

rbp+10: 000000e216b0efe0

-> 000000e377b3f738 log4net.Appender.AsyncRollingFileAppender

-> 000000e377b3f850 System.Collections.Concurrent.ConcurrentQueue`1[[log4net.Core.LoggingEvent, log4net]]

-> 000000e623ca3f08 System.Collections.Concurrent.ConcurrentQueue`1+Segment[[log4net.Core.LoggingEvent, log4net]]

-> 000000e623ca3f48 log4net.Core.LoggingEvent[]

-> 000000e9555e8218 log4net.Core.LoggingEvent

-> 000000e377b3b310 log4net.Repository.Hierarchy.Hierarchy

-> 000000e377b527a0 log4net.Repository.LoggerRepositoryShutdownEventHandler

-> 000000e377b526c8 log4net.Core.WrapperMap

-> 000000e377b526f0 System.Collections.Hashtable

-> 000000e377b52740 System.Collections.Hashtable+bucket[]

-> 000000e377b527e0 System.Collections.Hashtable

-> 000000e377b52830 System.Collections.Hashtable+bucket[]

-> 000000e377b522e8 log4net.Repository.Hierarchy.DefaultLoggerFactory+LoggerImpl

-> 000000e377b3e720 log4net.Repository.Hierarchy.RootLogger

-> 000000e377b46980 log4net.Util.AppenderAttachedImpl

-> 000000e377b469a0 log4net.Appender.AppenderCollection

-> 000000e377b51130 log4net.Appender.IAppender[]

-> 000000e377b469c0 log4net.Appender.AsyncRollingFileAppender

-> 000000e377b46b70 System.Threading.Thread

-> 000000e6b7b33cb8 System.Runtime.Remoting.Contexts.Context

-> 000000e277b31560 System.AppDomain

-> 000000e6b7b67160 System.UnhandledExceptionEventHandler

-> 000000e277b32a60 ShowHand.ProjectU.GameServer.GameServer

-> 000000e377b53750 ShowHand.ServerBase.PlayerContextManager

-> 000000e377b53a48 System.Collections.Concurrent.ConcurrentDictionary`2[[System.UInt64, mscorlib],[ShowHand.ServerBase.IManagedContext, ServerBase]]

-> 000000e9c04a04d0 System.Collections.Concurrent.ConcurrentDictionary`2+Tables[[System.UInt64, mscorlib],[ShowHand.ServerBase.IManagedContext, ServerBase]]

-> 000000ec47cb0218 System.Collections.Concurrent.ConcurrentDictionary`2+Node[[System.UInt64, mscorlib],[ShowHand.ServerBase.IManagedContext, ServerBase]][]

-> 000000e9c048e440 System.Collections.Concurrent.ConcurrentDictionary`2+Node[[System.UInt64, mscorlib],[ShowHand.ServerBase.IManagedContext, ServerBase]]

-> 000000e707424da8 ShowHand.ProjectU.GameServer.GameServerPlayerContext

-> 000000e707424cf0 ShowHand.NetSharp.Client

-> 000000e707425610 System.Threading.SemaphoreSlim

-> 000000e707425668 System.Threading.SemaphoreSlim+TaskNode

-> 000000e685d6a2e0 System.Collections.Generic.List`1[[System.Object, mscorlib]]

-> 000000e6c2a468d0 System.Object[]

-> 000000e2d362b088 System.Threading.Tasks.TaskFactory+CompleteOnInvokePromise

-> 000000e707425840 System.Action

-> 000000e707425820 System.Runtime.CompilerServices.AsyncMethodBuilderCore+MoveNextRunner

-> 000000e7074258d0 ShowHand.NetSharp.Client+d__21

-> 000000e707425880 System.Threading.Tasks.Task`1[[System.Threading.Tasks.VoidTaskResult, mscorlib]]

-> 000000e707425980 System.Action

-> 000000e707425960 System.Runtime.CompilerServices.AsyncMethodBuilderCore+MoveNextRunner

-> 000000e7074259c0 ShowHand.NetSharp.Endpoint+d__8

-> 000000e2b7b495f8 ShowHand.NetSharp.Endpoint

-> 000000e2b7b49630 System.Collections.Concurrent.ConcurrentDictionary`2[[ShowHand.NetSharp.Client, NetSharp],[System.Boolean, mscorlib]]

-> 000000e5bccc1f00 System.Collections.Concurrent.ConcurrentDictionary`2+Tables[[ShowHand.NetSharp.Client, NetSharp],[System.Boolean, mscorlib]]

-> 000000eb47bd1150 System.Collections.Concurrent.ConcurrentDictionary`2+Node[[ShowHand.NetSharp.Client, NetSharp],[System.Boolean, mscorlib]][]

-> 000000e5bccbbee8 System.Collections.Concurrent.ConcurrentDictionary`2+Node[[ShowHand.NetSharp.Client, NetSharp],[System.Boolean, mscorlib]]

-> 000000e43c0e1d48 ShowHand.NetSharp.Client

-> 000000e43c0e2cb8 ShowHand.ProjectU.GameServer.GameServerPlayerContext

-> 000000e43c0e2e50 System.Collections.Generic.List`1[[ShowHand.ProjectU.Common.Components.Server.IServerDataSection, ProjectU.LogicComp]]

-> 000000e27c6b4f48 ShowHand.ProjectU.Common.Components.Server.IServerDataSection[]

-> 000000e27c6a12f8 ShowHand.ProjectU.Common.Components.Server.MissionDataSection

-> 000000e27c6a1330 System.Collections.Generic.List`1[[ShowHand.ProjectU.Common.ProcessingMissionInfo, CommonDefine]]

-> 000000e27c71d400 ShowHand.ProjectU.Common.ProcessingMissionInfo[]

-> 000000e27c723348 ShowHand.ProjectU.Common.ProcessingMissionInfo

我們可以看到，000000e27c723348間接引用自ShowHand.NetSharp.Client，我們看到000000e27c723f80和其他所有g(shù)en2中的ShowHand.ProjectU.Common.ProcessingMissionInfo都間接引用自ShowHand.NetSharp.Client，并且都引用來自同一個ShowHand.NetSharp.Endpoint對象000000e2b7b495f8。我們進而分析這個對象。

0:059> !do 000000e2b7b495f8

Name: ShowHand.NetSharp.Endpoint

MethodTable: 00007ffb69f641d0

EEClass: 00007ffb69f53fd8

Size: 56(0x38) bytes

File: e:\ServerRelease\Server\GameServer\NetSharp.dll

Fields:

MT Field Offset Type VT Attr Value Name

00007ffbc5b19288 400002d 28 System.Int32 1 instance 65536 k__BackingField

00007ffb698a8d80 400002e 8 ...pointEventHandler 0 instance 000000e277b32a60 m_endPointEventHandler

00007ffbc4bf2088 400002f 10 ...ckets.TcpListener 0 instance 000000e2b7b4ad60 m_listener

00007ffb69f64078 4000030 2c System.Int32 1 instance 0 m_state

00007ffbc5b27b38 4000031 18 ...eading.Tasks.Task 0 instance 000000e2b7b4af70 m_mainTask

00007ffb69f64888 4000032 20 ...olean, mscorlib]] 0 instance 000000e2b7b49630 m_clientActiveness

0:059> !do 000000e2b7b49630

Name: System.Collections.Concurrent.ConcurrentDictionary`2[[ShowHand.NetSharp.Client, NetSharp],[System.Boolean, mscorlib]]

MethodTable: 00007ffb69f64888

EEClass: 00007ffb69e90240

Size: 64(0x40) bytes

File: C:\Windows\Microsoft.Net\assembly\GAC_64\mscorlib\v4.0_4.0.0.0__b77a5c561934e089\mscorlib.dll

Fields:

MT Field Offset Type VT Attr Value Name

00007ffb69d33f48 4001830 8 ...olean, mscorlib]] 0 instance 000000e5bccc1f00 m_tables

00007ffbc5b34300 4001831 10 ...Canon, mscorlib]] 0 instance 0000000000000000 m_comparer

00007ffbc5b21f28 4001832 30 System.Boolean 1 instance 1 m_growLockArray

00007ffbc5b19288 4001833 20 System.Int32 1 instance 0 m_keyRehashCount

00007ffbc5b19288 4001834 24 System.Int32 1 instance 32 m_budget

00007ffbc65fc070 4001835 18 ...ean, mscorlib]][] 0 instance 0000000000000000 m_serializationArray

00007ffbc5b19288 4001836 28 System.Int32 1 instance 0 m_serializationConcurrencyLevel

00007ffbc5b19288 4001837 2c System.Int32 1 instance 0 m_serializationCapacity

00007ffbc5b21f28 400183b 10 System.Boolean 1 static

0:059> !do 000000e5bccc1f00

Name: System.Collections.Concurrent.ConcurrentDictionary`2+Tables[[ShowHand.NetSharp.Client, NetSharp],[System.Boolean, mscorlib]]

MethodTable: 00007ffb69f65790

EEClass: 00007ffb69e90968

Size: 48(0x30) bytes

File: C:\Windows\Microsoft.Net\assembly\GAC_64\mscorlib\v4.0_4.0.0.0__b77a5c561934e089\mscorlib.dll

Fields:

MT Field Offset Type VT Attr Value Name

0000000000000000 400341d 8 SZARRAY 0 instance 000000eb47bd1150 m_buckets

00007ffbc5b16fc0 400341e 10 System.Object[] 0 instance 000000e737ef1220 m_locks

00007ffbc5b19220 400341f 18 System.Int32[] 0 instance 000000e5bcc76a00 m_countPerLock

00007ffbc5b34300 4003420 20 ...Canon, mscorlib]] 0 instance 000000e2b7b41d18 m_comparer

0:059> !do 000000eb47bd1150

Name: System.Collections.Concurrent.ConcurrentDictionary`2+Node[[ShowHand.NetSharp.Client, NetSharp],[System.Boolean, mscorlib]][]

MethodTable: 00007ffb69f65610

EEClass: 00007ffbc54daa00

Size: 266240(0x41000) bytes

Array: Rank 1, Number of elements 33277, Type CLASS (Print Array)

Fields:

None

我們可以看到，某個ShowHand.NetSharp.Endpoint對象，引用了數(shù)萬的ShowHand.NetSharp.Client,而數(shù)萬的

ShowHand.NetSharp.Client又間接引用了幾千萬的客戶自己的各種對象。最終這些對象因為存在著引用，經(jīng)歷GC回收后最終被推到了Gen2。

這是一個不健康的行為，比起gen0和gen1的垃圾回收，gen2的回收則昂貴的多。基于嚴(yán)謹(jǐn)，我們不能將上述分析作為確鑿證據(jù)和文初的cpu抖動掛鉤（其實的確是又這種可能性，大規(guī)模的gen2 GC引發(fā)高CPU）。但是，該不健康的2代對象太多的問題，的確需要解決，無論它和CPU抖動有沒有直接關(guān)系，它都會給程序的健康運行帶來巨大隱患。

我們建議客戶基于以上分析，并基于自身業(yè)務(wù)考慮該情況的發(fā)生是否合理，如不合理，應(yīng)適當(dāng)考慮對程序進行優(yōu)化。

我們是阿里云智能全球技術(shù)服務(wù)-SRE團隊，我們致力成為一個以技術(shù)為基礎(chǔ)、面向服務(wù)、保障業(yè)務(wù)系統(tǒng)高可用的工程師團隊；提供專業(yè)、體系化的SRE服務(wù)，幫助廣大客戶更好地使用云、基于云構(gòu)建更加穩(wěn)定可靠的業(yè)務(wù)系統(tǒng)，提升業(yè)務(wù)穩(wěn)定性。我們期望能夠分享更多幫助企業(yè)客戶上云、用好云，讓客戶云上業(yè)務(wù)運行更加穩(wěn)定可靠的技術(shù)，您可用釘釘掃描下方二維碼，加入阿里云SRE技術(shù)學(xué)院釘釘圈子，和更多云上人交流關(guān)于云平臺的那些事。

原文鏈接：https://developer.aliyun.com/article/784874?

版權(quán)聲明：本文內(nèi)容由阿里云實名注冊用戶自發(fā)貢獻，版權(quán)歸原作者所有，阿里云開發(fā)者社區(qū)不擁有其著作權(quán)，亦不承擔(dān)相應(yīng)法律責(zé)任。具體規(guī)則請查看《阿里云開發(fā)者社區(qū)用戶服務(wù)協(xié)議》和《阿里云開發(fā)者社區(qū)知識產(chǎn)權(quán)保護指引》。如果您發(fā)現(xiàn)本社區(qū)中有涉嫌抄襲的內(nèi)容，填寫侵權(quán)投訴表單進行舉報，一經(jīng)查實，本社區(qū)將立刻刪除涉嫌侵權(quán)內(nèi)容。與50位技術(shù)專家面對面20年技術(shù)見證，附贈技術(shù)全景圖

總結(jié)

以上是生活随笔為你收集整理的一次.net托管内存泄露分析的全部內(nèi)容，希望文章能夠幫你解決所遇到的問題。

如果覺得生活随笔網(wǎng)站內(nèi)容還不錯，歡迎將生活随笔推薦給好友。

内存
net