Add msdia140.dll to required DLLs in build dir

[k0ss]

I was having trouble where my fuzzer couldn't set breakpoints on any kernel functions to catch usermode crashes, etc on Windows. I used Process Monitor and saw that wtf.exe was trying to load msdia140.dll from all the directory entries of my %PATH% env var over and over again. I copied over the msdia140.dll from the host that took the snapshot (with DbgX WinDbg) and now it works.

Before

Command log:

..\..\src\build\wtf run --backend bochscpu --state .\state --input .\inputs --edges=1 --name my_target
Initializing the debugger instance.. (this takes a bit of time)
Setting debug register status to zero.
Setting debug register status to zero.
Could not set a breakpoint at nt!KeBugCheck2.
Failed to SetBreakpoint on KeBugCheck2
Failed to SetupUsermodeCrashDetectionHooks
Could not initialize target fuzzer.

symbol_store.json:

{"nt!KeBugCheck2":"0x0"}

After

..\..\src\build\wtf run --backend bochscpu --state .\state --input .\inputs --edges=1 --name my_target
Initializing the debugger instance.. (this takes a bit of time)
Setting debug register status to zero.
Setting debug register status to zero.
Running .\inputs\initial_input.bin
--------------------------------------------------
Run stats:
Instructions executed: 55.4k (7.9k unique)
          Dirty pages: 192.0kb
      Memory accesses: 88.7kb
       Edges executed: 4.1k (898.0 unique)
#1 cov: 7894 exec/s: 0.0 lastcov: 0.0s crash: 0 timeout: 0 cr3: 1 uptime: 3.0s

symbol_store.json:

{"nt!KeBugCheck2":"0xfffff80539316a10","nt!KiFastFailDispatch":"0xfffff80539212280","nt!KiProcessControlProtection":"0xfffff805393126a0","nt!SwapContext":"0xfffff80539205de0","ntdll!RtlDispatchException":"0x7ff8fdd02310","verifier":"0x7ff8caeb0000","verifier!VerifierStopMessage":"0x7ff8caeb6770"}

I can submit a PR for this if that helps, I just wanted to make sure I'm not the only one hitting this first.

[0vercl0k]

Hmmm in theory wtf.exe should go and find the libraries itself but I don't see this happening in your log

[0vercl0k]

Oh, actually nvm I read dbgeng.dll instead of msdia

[0vercl0k]

Can you tell which module loads it by any chance?

[0vercl0k]

Just gave it a shot; even with latest debugger, if I wipe my nt symbols for example / delete the dbgeng*.dlls in wtf folder (to force it to fetch potentially new ones), I can see it loading (at least successfully; but the symbols seems to work fine).

[k0ss]

Are you certain you got rid of your state/symbol-store.json as well as wtf/src/build/msdia140.dll? I tested it again by moving my msdia140.dll from the build folder and moving state/symbol-store.json from the state folder and was able to reproduce the issue.

Here's the stacktrace from ProcMon the first time it tries to load msdia140.dll:

"Frame","Module","Location","Address","Path"
"0","FLTMGR.SYS","FltDecodeParameters + 0x210c","0xfffff8033bc564cc","C:\Windows\System32\drivers\FLTMGR.SYS"
"1","FLTMGR.SYS","FltDecodeParameters + 0x1bba","0xfffff8033bc55f7a","C:\Windows\System32\drivers\FLTMGR.SYS"
"2","FLTMGR.SYS","FltAddOpenReparseEntry + 0x560","0xfffff8033bc89ed0","C:\Windows\System32\drivers\FLTMGR.SYS"
"3","ntoskrnl.exe","IofCallDriver + 0x55","0xfffff803400d21c5","C:\Windows\system32\ntoskrnl.exe"
"4","ntoskrnl.exe","IoGetAttachedDevice + 0x194","0xfffff803400d4084","C:\Windows\system32\ntoskrnl.exe"
"5","ntoskrnl.exe","NtDeviceIoControlFile + 0x4169","0xfffff8034044f829","C:\Windows\system32\ntoskrnl.exe"
"6","ntoskrnl.exe","ObReferenceObjectByHandle + 0x4477","0xfffff80340442757","C:\Windows\system32\ntoskrnl.exe"
"7","ntoskrnl.exe","ObOpenObjectByNameEx + 0x1fa","0xfffff803404cec8a","C:\Windows\system32\ntoskrnl.exe"
"8","ntoskrnl.exe","NtCreateFile + 0x1425","0xfffff8034040ccb5","C:\Windows\system32\ntoskrnl.exe"
"9","ntoskrnl.exe","setjmpex + 0x8625","0xfffff80340211505","C:\Windows\system32\ntoskrnl.exe"
"10","ntdll.dll","NtQueryAttributesFile + 0x14","0x7ff8863adc94","C:\Windows\System32\ntdll.dll"
"11","KernelBase.dll","GetFileAttributesW + 0x85","0x7ff883d88c45","C:\Windows\System32\KernelBase.dll"
"12","dbghelp.dll","RemoveInvalidModuleList + 0xade","0x7ff83a4d5a3e","C:\wtf\src\build\dbghelp.dll"
"13","dbghelp.dll","RemoveInvalidModuleList + 0x8bb","0x7ff83a4d581b","C:\wtf\src\build\dbghelp.dll"
"14","dbghelp.dll","RemoveInvalidModuleList + 0x90ef","0x7ff83a4de04f","C:\wtf\src\build\dbghelp.dll"
"15","dbghelp.dll","SymFindFileInPathW + 0xc0a","0x7ff83a5037da","C:\wtf\src\build\dbghelp.dll"
"16","dbghelp.dll","SymFindFileInPathW + 0x56fd","0x7ff83a5082cd","C:\wtf\src\build\dbghelp.dll"
"17","dbghelp.dll","SymFindFileInPathW + 0x5392","0x7ff83a507f62","C:\wtf\src\build\dbghelp.dll"
"18","dbghelp.dll","SymFindFileInPathW + 0x13ee","0x7ff83a503fbe","C:\wtf\src\build\dbghelp.dll"
"19","dbghelp.dll","SymLoadModuleExW + 0x3f","0x7ff83a4fe08f","C:\wtf\src\build\dbghelp.dll"
"20","dbgeng.dll","Ordinal367 + 0x10c017","0x7ff821659767","C:\wtf\src\build\dbgeng.dll"
"21","dbgeng.dll","Ordinal367 + 0x10cb4d","0x7ff82165a29d","C:\wtf\src\build\dbgeng.dll"
"22","dbgeng.dll","Ordinal367 + 0x1b882f","0x7ff821705f7f","C:\wtf\src\build\dbgeng.dll"
"23","dbgeng.dll","Ordinal367 + 0x19acf4","0x7ff8216e8444","C:\wtf\src\build\dbgeng.dll"
"24","dbgeng.dll","Ordinal367 + 0x9eb7b","0x7ff8215ec2cb","C:\wtf\src\build\dbgeng.dll"
"25","dbgeng.dll","Ordinal367 + 0x1b92ed","0x7ff821706a3d","C:\wtf\src\build\dbgeng.dll"
"26","dbgeng.dll","Ordinal367 + 0xa5cdb","0x7ff8215f342b","C:\wtf\src\build\dbgeng.dll"
"27","dbgeng.dll","Ordinal367 + 0xb1ed","0x7ff82155893d","C:\wtf\src\build\dbgeng.dll"
"28","dbgeng.dll","Ordinal367 + 0xdc08","0x7ff82155b358","C:\wtf\src\build\dbgeng.dll"
"29","wtf.exe","WindowsDebugger_t::Init + 0x8c1, C:\wtf\src\wtf\debugger.h(290)","0x7ff781e2b7f1","C:\wtf\src\build\wtf.exe"
"30","wtf.exe","main + 0x24c6, C:\wtf\src\wtf\wtf.cc(461)","0x7ff781e9b606","C:\wtf\src\build\wtf.exe"
"31","wtf.exe","__scrt_common_main_seh + 0x10c, D:\a\_work\1\s\src\vctools\crt\vcstartup\src\startup\exe_common.inl(281)","0x7ff7820e68a8","C:\wtf\src\build\wtf.exe"
"32","kernel32.dll","BaseThreadInitThunk + 0x14","0x7ff885757374","C:\Windows\System32\kernel32.dll"
"33","ntdll.dll","RtlUserThreadStart + 0x21","0x7ff88635cc91","C:\Windows\System32\ntdll.dll"

It seems like it's initiated from dbghelp.dll.

[0vercl0k]

Hmm you're right that I didn't remove my symbol_store.json but on Windows it shouldn't read from it just write to it IIRC.

I'll take a closer look at this this weekend, thank you!

Cheers

[0vercl0k]

Okay I redid the test:

• Removed all the dbg*.dll from the wtf folder
• Removed the symbol-store.json
• Removed the locally cached nt symbols
• Had ProcDump running in the background

This is what I get:

c:\work\codes\wtf\targets\hevd>..\..\src\build\RelWithDebInfo\wtf.exe run --name hevd --input inputs\ioctl.bin
Found a 'state' folder in the cwd, so using it.
Initializing the debugger instance.. (this takes a bit of time)
Copied c:/program Files (x86)/windows kits/10/debuggers/x64/dbghelp.dll into the executable directory..
Copied c:/program Files (x86)/windows kits/10/debuggers/x64/symsrv.dll into the executable directory..
Copied c:/program Files (x86)/windows kits/10/debuggers/x64/dbgeng.dll into the executable directory..
Copied c:/program Files (x86)/windows kits/10/debuggers/x64/dbgcore.dll into the executable directory..
Setting debug register status to zero.
Setting debug register status to zero.
Setting mxcsr_mask to 0xffbf.
Running inputs\ioctl.bin
Hevd: This is a breakpoint executed before the first instruction :)
Hevd: DbgPrintEx: ****** HEVD_IOCTL_INSECURE_KERNEL_FILE_ACCESS ******
Hevd: DbgPrintEx: [+] Log Path: %ws
Hevd: DbgPrintEx: [+] Log Content: %s
--------------------------------------------------
Run stats:
Instructions executed: 81.2k (26.0k unique)
          Dirty pages: 560.0kb
      Memory accesses: 225.0kb
       Edges executed: 0.0 (0.0 unique)
#1 cov: 26034 exec/s: 0.0 lastcov: 0.0s crash: 0 timeout: 0 cr3: 1 uptime: 7.0s

c:\work\codes\wtf\targets\hevd>type state\symbol-store.json
{"nt!DbgPrintEx":"0xfffff8046f122bb0","nt!ExGenRandom":"0xfffff8046f0286e0","nt!KeBugCheck2":"0xfffff8046f2a53e0","nt!SwapContext":"0xfffff8046f1c3880"}

Based on ProcMon, it seems symsrv.dll in my case is doing the communication with the Microsoft Symbol servers.

Ugh, frustrating that I can't seem to reproduce this. I also can't seem to find definitive documentation on does the dbghelp DLLs need it to be pulled in or not.

Cheers

[0vercl0k]

If I do debug the Windows debugger, I do see it getting loaded by dbghelp.dll somehow 🤔:

c:\>cdbx64 cdbx64 -z bmp.dmp
...
0:000> sxe ld:msdia140.dll
0:000> g
...
ModLoad: 00007ffc`5c4f0000 00007ffc`5c71a000   C:\Program Files\WindowsApps\Microsoft.WinDbg_1.2506.12002.0_x64__8wekyb3d8bbwe\amd64\msdia140.dll
ntdll!NtMapViewOfSection+0x14:
00007ffc`b4222db4 c3              ret

0:000> kp
Child-SP          RetAddr               Call Site
0000007a`4ff2a0e8 00007ffc`b412bf3a     ntdll!NtMapViewOfSection+0x14
0000007a`4ff2a0f0 00007ffc`b412ba6e     ntdll!LdrpMinimalMapModule+0x14a
0000007a`4ff2a1b0 00007ffc`b413068c     ntdll!LdrpMapDllWithSectionHandle+0x4e
0000007a`4ff2a460 00007ffc`b412e55d     ntdll!LdrpMapDllNtFileName+0x28c
0000007a`4ff2a550 00007ffc`b413c384     ntdll!LdrpMapDllFullPath+0xf5
0000007a`4ff2a6e0 00007ffc`b40c8860     ntdll!LdrpProcessWork+0x144
0000007a`4ff2a740 00007ffc`b40c84e0     ntdll!LdrpLoadDllInternal+0x210
0000007a`4ff2a7d0 00007ffc`b4115d80     ntdll!LdrpLoadDll+0x100
0000007a`4ff2a9a0 00007ffc`b1a62ac6     ntdll!LdrLoadDll+0x170
0000007a`4ff2aa90 00007ffc`939a29e6     KERNELBASE!LoadLibraryExW+0xe6
0000007a`4ff2ab00 00007ffc`939ae068     dbghelp+0x29e6
0000007a`4ff2ab40 00007ffc`939d37da     dbghelp!RemoveInvalidModuleList+0x9108
0000007a`4ff2ae10 00007ffc`939d82cd     dbghelp!SymFindFileInPathW+0xc0a
0000007a`4ff2b0a0 00007ffc`939d7f62     dbghelp!SymFindFileInPathW+0x56fd
0000007a`4ff2b5a0 00007ffc`939d3fbe     dbghelp!SymFindFileInPathW+0x5392
0000007a`4ff2b840 00007ffc`939ce08f     dbghelp!SymFindFileInPathW+0x13ee
0000007a`4ff2bd60 00007ffc`45b09767     dbghelp!SymLoadModuleExW+0x3f
0000007a`4ff2bdb0 00007ffc`45b0a29d     dbgeng!Ordinal367+0x10c017
0000007a`4ff2cd60 00007ffc`45bb5f7f     dbgeng!Ordinal367+0x10cb4d
0000007a`4ff2cdd0 00007ffc`45b98444     dbgeng!Ordinal367+0x1b882f
0000007a`4ff2d2d0 00007ffc`45a9c2cb     dbgeng!Ordinal367+0x19acf4
0000007a`4ff2d470 00007ffc`45bb6a3d     dbgeng!Ordinal367+0x9eb7b
0000007a`4ff2d4f0 00007ffc`45aa342b     dbgeng!Ordinal367+0x1b92ed
0000007a`4ff2d520 00007ffc`45a0893d     dbgeng!Ordinal367+0xa5cdb
0000007a`4ff2d590 00007ffc`45a0b358     dbgeng!Ordinal367+0xb1ed
0000007a`4ff2d650 00007ff7`5e206a16     dbgeng!Ordinal367+0xdc08
0000007a`4ff2d680 00007ff7`5e20ac2b     cdb+0x6a16
0000007a`4ff2f700 00007ff7`5e201324     cdb+0xac2b
0000007a`4ff2f9a0 00007ffc`b2c6e8d7     cdb+0x1324
0000007a`4ff2f9e0 00007ffc`b40fc34c     KERNEL32!BaseThreadInitThunk+0x17
0000007a`4ff2fa10 00000000`00000000     ntdll!RtlUserThreadStart+0x2c

[0vercl0k]

Okay if I do the same experiment with the cdb coming from the directory from the "old" WinDbg, which is where wtf.exe finds its DLLs (c:\Program Files (x86)\Windows Kits\10\Debuggers\x64) programmatically, then I don't see it happen:

c:\>"c:\Program Files (x86)\Windows Kits\10\Debuggers\x64\cdb.exe" "c:\Program Files (x86)\Windows Kits\10\Debuggers\x64\cdb.exe" -z bmp.dmp
...
(5d48.538): Break instruction exception - code 80000003 (first chance)
ntdll!LdrpDoDebuggerBreak+0x35:
00007ffc`b41e4199 cc              int     3
0:000> sxe ld:msdia*.dll
0:000> g
...
Comment: 'yo d00d'

************* Path validation summary **************
Response                         Time (ms)     Location
Deferred                                       srv*c:\work\symbols*https://msdl.microsoft.com/download/symbols
Symbol search path is: srv*c:\work\symbols*https://msdl.microsoft.com/download/symbols
Executable search path is:
ModLoad: 00007ffc`936a0000 00007ffc`93738000   c:\Program Files (x86)\Windows Kits\10\Debuggers\x64\symsrv.dll
ModLoad: 00007ffc`b2200000 00007ffc`b2274000   C:\WINDOWS\System32\WS2_32.dll
ModLoad: 00007ffc`b1340000 00007ffc`b14b7000   C:\WINDOWS\System32\CRYPT32.dll
ModLoad: 00007ffc`b3370000 00007ffc`b3abd000   C:\WINDOWS\System32\shell32.dll
ModLoad: 00007ffc`b1550000 00007ffc`b16c3000   C:\WINDOWS\System32\wintypes.dll
ModLoad: 00007ffc`aed90000 00007ffc`af5f0000   C:\WINDOWS\SYSTEM32\windows.storage.dll
ModLoad: 00007ffc`b1110000 00007ffc`b1139000   C:\WINDOWS\SYSTEM32\profapi.dll
ModLoad: 00007ffc`981b0000 00007ffc`98439000   C:\WINDOWS\SYSTEM32\wininet.dll
ModLoad: 00007ffc`585e0000 00007ffc`58747000   c:\Program Files (x86)\Windows Kits\10\Debuggers\x64\winext\wdfkd.dll
ModLoad: 00007ffc`a70e0000 00007ffc`a71fe000   C:\WINDOWS\SYSTEM32\WINHTTP.dll
Windows 10 Kernel Version 18362 UP Free x64
Edition build lab: 18362.1.amd64fre.19h1_release.190318-1202
Kernel base = 0xfffff805`106b3000 PsLoadedModuleList = 0xfffff805`10af62f0
System Uptime: 0 days 0:01:11.493
Loading Kernel Symbols
...............................................................
..................
Loading User Symbols

ModLoad: 00007ffc`64c90000 00007ffc`64d53000   c:\Program Files (x86)\Windows Kits\10\Debuggers\x64\WINXP\exts.dll
ModLoad: 00007ffc`95f00000 00007ffc`95f38000   c:\Program Files (x86)\Windows Kits\10\Debuggers\x64\winext\kext.dll
ModLoad: 00007ffc`582d0000 00007ffc`585ce000   c:\Program Files (x86)\Windows Kits\10\Debuggers\x64\winxp\kdexts.dll
...

So my guess is you are using the DLLs from WinDbgX's folder yeah? (Which is probably what you used for snapshotting?)

Cheers

[0vercl0k]

In the end I think it might be a variation of another class of issue related to the dbgeng dlls; if you take a snapshot in a format that isn't known by "old windbg" (because maybe it's a new format that has been introduced recently) then you end up in a similar situation where wtf fails to load the dump as its dbgeng doesn't know to load that dump.

This seems slightly different but it seems this comment #196 (comment) could address both issues. What I'm going to try is to emit extra metadata in snapshot; the path & version of dbgeng.dll / dbghelp.dll / symsrv.dll / msdia140.dll / dbgcore.dll. Then, wtf at runtime will try to copy those dlls in the CWD / can warn the user if it cannot find them / has to load an older version of what has been used to generate the dump.

If msdia140 is detected while taking a dump then it'd be added in the extra metadata & wtf would try to copy it in the CWD.

Not sure this is gonna work out but will let you know :) Would appreciate if you could test it!

Cheers

[k0ss]

Yes I agree with that comment on the version number at least. I'll be more than happy to test it out for you!

[k0ss]

So my guess is you are using the DLLs from WinDbgX's folder yeah? (Which is probably what you used for snapshotting?)

Sorry, I missed this earlier reply. Yes, I am using DbgX.Shell on whichever is the latest Windows Store version. When I initially got the error trying to wtf run, I copied over the DLLs that I saw listed in the wtf source code (and present in the build dir) from the DbgX.Shell arch folder. Then, I got the issue with msdia140 being missing. So I think you're correct that the modern WinDbg's dbghelp calls out to msdia140 whereas the classic WinDbg does not.

[0vercl0k]

Cool, everything checks out at least 🫡

Cheers