[WIP] Basic C++ support for windows by t-moe · Pull Request #918 · qilingframework/qiling

t-moe · 2021-09-16T15:06:54Z

This PR is based on #917. Complete that one first.

#377 mentions that a simple hello world from C++ on windows, causes a crash.

Demo Code + Crash

HelloWorld.Cpp build with vs2019 for x86 (32 bit)

#include <iostream>
int main()
{
    std::cout << "Hello World!\n";
}

Invocation using qiling before applying this PR (dev branch).

~/workspace/qiling$ ./qltool run -f examples/HelloWorldCpp.exe --rootfs examples/rootfs/x86_windows/
[=]	Initiate stack address at 0xfffdd000 
[=]	Loading examples/HelloWorldCpp.exe to 0x400000
[=]	PE entry point at 0x411023
[=]	TEB addr is 0x6000
[=]	PEB addr is 0x6044
[=]	Loading examples/rootfs/x86_windows/Windows/System32/ntdll.dll ...
[!]	Warnings while loading examples/rootfs/x86_windows/Windows/System32/ntdll.dll:
[!]	 - SizeOfHeaders is smaller than AddressOfEntryPoint: this file cannot run under Windows 8.
[!]	 - AddressOfEntryPoint lies outside the sections' boundaries. AddressOfEntryPoint: 0x0
[=]	Done with loading examples/rootfs/x86_windows/Windows/System32/ntdll.dll
[=]	Loading examples/rootfs/x86_windows/Windows/System32/kernel32.dll ...
[=]	Done with loading examples/rootfs/x86_windows/Windows/System32/kernel32.dll
[=]	Loading examples/rootfs/x86_windows/Windows/System32/msvcp140d.dll ...
[=]	Done with loading examples/rootfs/x86_windows/Windows/System32/msvcp140d.dll
[=]	Loading examples/rootfs/x86_windows/Windows/System32/vcruntime140d.dll ...
[=]	Done with loading examples/rootfs/x86_windows/Windows/System32/vcruntime140d.dll
[=]	Loading examples/rootfs/x86_windows/Windows/System32/ucrtbased.dll ...
[=]	Done with loading examples/rootfs/x86_windows/Windows/System32/ucrtbased.dll
[=]	GetSystemTimeAsFileTime(lpSystemTimeAsFileTime = 0xffffcfd4)
[=]	GetCurrentThreadId() = 0x0
[=]	GetCurrentProcessId() = 0x7cc
[=]	QueryPerformanceCounter(lpPerformanceCount = 0xffffcfcc) = 0x0
[=]	IsProcessorFeaturePresent(ProcessorFeature = 0xa) = 0x1
[=]	_initterm_e(pfbegin = 0x41930c, pfend = 0x419618) = 0x0
[=]	_initterm(pfbegin = 0x419000, pfend = 0x419208)
[=]	_get_initial_narrow_environment() = 0x0
[=]	__p___argv() = 0x5000643
[=]	__p___argc() = 0x5000647
[=]	strlen(str = "Hello World!\n") = 0xd
[x]	CPU Context:
[x]	ah	: 0x99
[x]	al	: 0x88
[x]	ch	: 0x0
[x]	cl	: 0x0
[x]	dh	: 0x99
[x]	dl	: 0x88
[x]	bh	: 0x0
[x]	bl	: 0x0
[x]	ax	: 0x9988
[x]	cx	: 0x0
[x]	dx	: 0x9988
[x]	bx	: 0x0
[x]	sp	: 0xcd20
[x]	bp	: 0xce94
[x]	si	: 0x0
[x]	di	: 0xce84
[x]	ip	: 0x18b1
[x]	eax	: 0x100a9988
[x]	ecx	: 0x0
[x]	edx	: 0x100a9988
[x]	ebx	: 0x0
[x]	esp	: 0xffffcd20
[x]	ebp	: 0xffffce94
[x]	esi	: 0x0
[x]	edi	: 0xffffce84
[x]	eip	: 0x4118b1
[x]	cr0	: 0x11
[x]	cr1	: 0x0
[x]	cr2	: 0x0
[x]	cr3	: 0x0
[x]	cr4	: 0x0
[x]	cr5	: 0x0
[x]	cr6	: 0x0
[x]	cr7	: 0x0
[x]	cr8	: 0x0
[x]	cr9	: 0x0
[x]	cr10	: 0x0
[x]	cr11	: 0x0
[x]	cr12	: 0x0
[x]	cr13	: 0x0
[x]	cr14	: 0x0
[x]	cr15	: 0x0
[x]	st0	: 0x0
[x]	st1	: 0x0
[x]	st2	: 0x0
[x]	st3	: 0x0
[x]	st4	: 0x0
[x]	st5	: 0x0
[x]	st6	: 0x0
[x]	st7	: 0x0
[x]	ef	: 0x44
[x]	cs	: 0x1b
[x]	ss	: 0x28
[x]	ds	: 0x28
[x]	es	: 0x28
[x]	fs	: 0x73
[x]	gs	: 0x78
[x]	Hexdump:
[x]	0351048bf48bcaff
[x]	Disassembly:
[=]	004118b1 [[PE]                 + 0x0118b1]  0351048bf48bcaff15bcd041003bf4e8c0f9ffff8985acfeffff8995b0feffff83bdb0feffff007c797f0983bdacfeffff00766e8b45088b088b55080351048badd                  edx, dword ptr [ecx + 4]
> mov                  esi, esp
> mov                  ecx, edx
> call                 dword ptr [0x41d0bc]
> cmp                  esi, esp
> call                 0x411285
> mov                  dword ptr [ebp - 0x154], eax
> mov                  dword ptr [ebp - 0x150], edx
> cmp                  dword ptr [ebp - 0x150], 0
> jl                   0x411953
> jg                   0x4118e5
> cmp                  dword ptr [ebp - 0x154], 0
> jbe                  0x411953
> mov                  eax, dword ptr [ebp + 8]
> mov                  ecx, dword ptr [eax]
> mov                  edx, dword ptr [ebp + 8]
> add                  edx, dword ptr [ecx + 4]
[x]	PC = 0x004118b1 (examples/HelloWorldCpp.exe + 0x118b1)

[=]	Memory map:
[=]	Start      End        Perm    Label          Image
[=]	00006000 - 0000c000   rwx     [FS/GS]        
[=]	00030000 - 00031000   rwx     [GDT]          
[=]	00400000 - 00422000   rwx     [PE]           examples/HelloWorldCpp.exe
[=]	05000000 - 05001000   rwx     [heap]         
[=]	06000000 - 0c000000   rwx     [FS/GS]        
[=]	10000000 - 100b3000   rwx     msvcp140d.dll   examples/rootfs/x86_windows/Windows/System32/msvcp140d.dll
[=]	100c0000 - 100de000   rwx     vcruntime140d.dll   examples/rootfs/x86_windows/Windows/System32/vcruntime140d.dll
[=]	100e0000 - 10256000   rwx     ucrtbased.dll   examples/rootfs/x86_windows/Windows/System32/ucrtbased.dll
[=]	4b280000 - 4b423000   rwx     ntdll.dll      examples/rootfs/x86_windows/Windows/System32/ntdll.dll
[=]	6b800000 - 6b8e5000   rwx     kernel32.dll   examples/rootfs/x86_windows/Windows/System32/kernel32.dll
[=]	fffdd000 - ffffe000   rwx     [stack]        
Traceback (most recent call last):
  File "./qltool", line 256, in <module>
    ql.run(timeout=options.timeout)
  File "/home/user/workspace/qiling/qiling/core.py", line 728, in run
    self.os.run()
  File "/home/user/workspace/qiling/qiling/os/windows/windows.py", line 179, in run
    self.ql.emu_start(self.ql.loader.entry_point, self.exit_point, self.ql.timeout, self.ql.count)
  File "/home/user/workspace/qiling/qiling/core.py", line 875, in emu_start
    self.uc.emu_start(begin, end, timeout, count)
  File "/usr/local/lib/python3.7/dist-packages/unicorn/unicorn.py", line 341, in emu_start
    raise UcError(status)
unicorn.unicorn.UcError: Invalid memory read (UC_ERR_READ_UNMAPPED)

The reason for this crash is, that std::cout is not initialized. std::cout is usually initialized by the dll entry point of msvcp140d.dll.

There are two ways of fixing this:

Try to reimplement all std api calls in qiling, effectively replacing half of the STL.
Try to run the existing STL implementation inside of msvcp140d.dll and hope that the implementation calls primitives that we've already wrapped/replaced ("e.g. printf")

This PR focuses on the 2nd approach. The supplied changes patch some windows API functions so that the DllMain of msvcp140d.dll runs through and does not longer crash. This will intialize std::cout and prevent the crash in main().

In the current state of the PR "hello world" does not appear on the console yet. However, the sample application does no longer crash and instead terminates successfully after running main with exit code 0 🥳 .

New output

generated by qiling after applying this PR

~/workspace/qiling$ ./qltool run -f examples/HelloWorldCpp.exe --rootfs examples/rootfs/x86_windows/
< huge output of DllMain omitted >
[=]	_get_initial_narrow_environment() = 0x5007ad1
[=]	__p___argv() = 0x5007af4
[=]	__p___argc() = 0x5007af8
[=]	strlen(str = "Hello World!\n") = 0xd
[!]	api ?width@ios_base@std@@QBE_JXZ is not implemented
[!]	api ?rdbuf@?$basic_ios@DU?$char_traits@D@std@@@std@@QBEPAV?$basic_streambuf@DU?$char_traits@D@std@@@2@XZ is not implemented
[!]	api ?good@ios_base@std@@QBE_NXZ is not implemented
[!]	api ?rdstate@ios_base@std@@QBEHXZ is not implemented
[!]	api ?tie@?$basic_ios@DU?$char_traits@D@std@@@std@@QBEPAV?$basic_ostream@DU?$char_traits@D@std@@@2@XZ is not implemented
[!]	api ?flags@ios_base@std@@QBEHXZ is not implemented
[!]	api ?rdbuf@?$basic_ios@DU?$char_traits@D@std@@@std@@QBEPAV?$basic_streambuf@DU?$char_traits@D@std@@@2@XZ is not implemented
[!]	api ?sputn@?$basic_streambuf@DU?$char_traits@D@std@@@std@@QAE_JPBD_J@Z is not implemented
[!]	api ?_Pnavail@?$basic_streambuf@DU?$char_traits@D@std@@@std@@IBE_JXZ is not implemented
[!]	api ?width@ios_base@std@@QAE_J_J@Z is not implemented
[!]	api ?setstate@?$basic_ios@DU?$char_traits@D@std@@@std@@QAEXH_N@Z is not implemented
[!]	api ?rdstate@ios_base@std@@QBEHXZ is not implemented
[!]	api ?clear@?$basic_ios@DU?$char_traits@D@std@@@std@@QAEXH_N@Z is not implemented
[!]	api ?clear@ios_base@std@@QAEXH_N@Z is not implemented
[!]	api ?uncaught_exception@std@@YA_NXZ is not implemented
[!]	api __uncaught_exception is not implemented
[=]	GetLastError() = 0x0
[=]	GetProcAddress(hModule = 0x6b800000, lpProcName = "FlsGetValue") = 0x6b81e770
[=]	FlsGetValue(dwFlsIndex = 0x3) = 0x102fa248
[=]	SetLastError(dwErrCode = 0)
[!]	api ?_Osfx@?$basic_ostream@DU?$char_traits@D@std@@@std@@QAEXXZ is not implemented
[!]	api ?good@ios_base@std@@QBE_NXZ is not implemented
[!]	api ?rdstate@ios_base@std@@QBEHXZ is not implemented
[!]	api ?rdbuf@?$basic_ios@DU?$char_traits@D@std@@@std@@QBEPAV?$basic_streambuf@DU?$char_traits@D@std@@@2@XZ is not implemented
[=]	GetModuleHandleW(lpModuleName = 0) = 0x400000
[=]	exit(status = 0)

Note that uncaught_exception is a getter. There is no exception happening.

Next Steps:

Getting the output of std::cout to the console.
Try more complex c++ programs

Checklist

Which kind of PR do you create?

This PR only contains minor fixes.
This PR contains major feature update.
This PR introduces a new function/api for Qiling Framework.

Coding convention?

The new code conforms to Qiling Framework naming convention.
The imports are arranged properly.
Essential comments are added.
The reference of the new code is pointed out.

Extra tests?

No extra tests are needed for this PR.
I have added enough tests for this PR.
Tests will be added after some discussion and review.

Changelog?

This PR doesn't need to update Changelog.
Changelog will be updated after some proper review.
Changelog has been updated in my PR.

Target branch?

The target branch is dev branch.

One last thing

I have read the contribution guide

xwings · 2021-09-17T01:03:59Z

Nice work, we needed this long ago

Enteee · 2021-10-12T14:11:33Z

@wtdcode by removing old_dev you might have accidentally closed this pr... Can this be easily re targeted to dev?

wtdcode · 2021-10-12T14:14:29Z

@wtdcode by removing old_dev you might have accidentally closed this pr... Can this be easily re targeted to dev?

Github doesn't allow to change base at this time... Need @t-moe to re-open another one. Sorry for any inconvenience.

xwings · 2021-10-19T02:27:21Z

i changed the base to dev. @t-moe can you please resolv the conflict ?

T-Moe added 2 commits September 16, 2021 13:28

Fix PE loader to recursivly import symbols and call DllMain

dfc63cd

Fix windows syscalls so that DllMain of msvcp140 does not crash.

8c7e924

wtdcode deleted the branch qilingframework:dev October 12, 2021 08:49

wtdcode closed this Oct 12, 2021

Enteee mentioned this pull request Oct 13, 2021

Fix PE loader to recursivly import symbols and call DllMain #917

Closed

15 tasks

wtdcode reopened this Oct 14, 2021

xwings changed the base branch from old_dev to dev October 19, 2021 02:26

elicn mentioned this pull request Mar 22, 2022

Improvements and fixes for Windows and PE #1118

Merged

xwings closed this Oct 6, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[WIP] Basic C++ support for windows#918

[WIP] Basic C++ support for windows#918
t-moe wants to merge 2 commits into
qilingframework:devfrom
t-moe:msvc_init

t-moe commented Sep 16, 2021 •

edited

Loading

Uh oh!

xwings commented Sep 17, 2021

Uh oh!

Enteee commented Oct 12, 2021

Uh oh!

wtdcode commented Oct 12, 2021

Uh oh!

xwings commented Oct 19, 2021

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Conversation

t-moe commented Sep 16, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Checklist

Which kind of PR do you create?

Coding convention?

Extra tests?

Changelog?

Target branch?

One last thing

Uh oh!

xwings commented Sep 17, 2021

Uh oh!

Enteee commented Oct 12, 2021

Uh oh!

wtdcode commented Oct 12, 2021

Uh oh!

xwings commented Oct 19, 2021

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

t-moe commented Sep 16, 2021 •

edited

Loading