Fix bug: DLL address still use ImageBase address in spite of changing lib_base. by nacayoshi00 · Pull Request #801 · qilingframework/qiling

nacayoshi00 · 2021-05-24T13:09:47Z

… lib_base

Checklist

Which kind of PR do you create?

[ x ] This PR only contains minor fixes.
This PR contains major feature update.
This PR introduces a new function/api for Qiling Framework.

Coding convention?

[ x ] The new code conforms to Qiling Framework naming convention.
The imports are arranged properly.
Essential comments are added.
The reference of the new code is pointed out.

Extra tests?

[ x ] No extra tests are needed for this PR.
I have added enough tests for this PR.
Tests will be added after some discussion and review.

Changelog?

This PR doesn't need to update Changelog.
[ x ] Changelog will be updated after some proper review.
Changelog has been updated in my PR.

Target branch?

[ x ] The target branch is dev branch.

One last thing

[ x ] I have read the contribution guide

… lib_base.

xwings · 2021-05-25T07:11:06Z

There are some test with fixed based DLL address. Will you be able to update the test ?

nacayoshi00 · 2021-05-26T13:57:59Z

I understood. I will add the test to test_pe.py. Please be patient.

nacayoshi00 · 2021-06-01T14:05:32Z

I updated test_pe.py to check the fix of lib address relocation.
And I also update 2 fixes as below:

Implement PE loader to load all of the DLLs recursively when PE loaded.
Implement PE loader to be able to load API DLL (like api-ms-xxxx.dll).

I added a sample PE file for the test. So Should I send a pull request to https://github.com/qilingframework/rootfs/tree/master?

xwings · 2021-06-02T01:31:35Z

I updated test_pe.py to check the fix of lib address relocation.
And I also update 2 fixes as below:

Implement PE loader to load all of the DLLs recursively when PE loaded.

Implement PE loader to be able to load API DLL (like api-ms-xxxx.dll).

I added a sample PE file for the test. So Should I send a pull request to https://github.com/qilingframework/rootfs/tree/master?

Yes. If there is a new exe, you need to update rootfs

nacayoshi00 · 2021-06-02T05:44:02Z

OK, I sent the pull request of rootfs. And fix some code to pass the test

xwings · 2021-06-03T15:14:03Z

-if exist %WINDIR%\SysWOW64\downlevel\api-ms-win-core-fibers-l1-1-1.dll xcopy /f /y %WINDIR%\SysWOW64\downlevel\api-ms-win-core-fibers-l1-1-1.dll "examples\rootfs\x86_windows\Windows\System32\"
-if exist %WINDIR%\SysWOW64\downlevel\api-ms-win-core-localization-l1-2-1.dll xcopy /f /y %WINDIR%\SysWOW64\downlevel\api-ms-win-core-localization-l1-2-1.dll "examples\rootfs\x86_windows\Windows\System32\"
-if exist %WINDIR%\SysWOW64\downlevel\api-ms-win-core-sysinfo-l1-2-1.dll xcopy /f /y %WINDIR%\SysWOW64\downlevel\api-ms-win-core-sysinfo-l1-2-1.dll "examples\rootfs\x86_windows\Windows\System32\"
+xcopy /f /y %WINDIR%\SysWOW64\downlevel\api-*.dll "examples\rootfs\x86_windows\Windows\System32\"


I think we need to limit and copy only needed DLL. It seems the test took 4 hours due to additional dlls

nacayoshi00 · 2021-06-04T14:46:49Z

OK, I updated examples/scripts/dllscollector.bat to copy only needed DLL.

xwings · 2021-06-05T14:31:45Z

Redoing the test and see how long will it take to do the test.

xwings · 2021-06-05T14:59:17Z

-
-                        if self.ql.archtype == QL_ARCH.X86:
-                            address = self.ql.pack32(addr)
+            # load all DLL files linked from the target PE file recursively and make IAT of all dlls.


Here onwards

The test before this take around 15-20 min. With the new implementation. It takes around 4 hours to complete the test.

We need to find a better way, This is way too slow.

One method is to cache like how we cache the dll address. maybe we can do the same

xwings · 2021-06-05T15:05:10Z

                    self.ql.log.warning(f' - {warning}')
+
+            # [Room for Improvement] too much time when kernelbase.dll is loaded. 
+            self.ql.log.debug('relocate {}, {:x}'.format(dll_name, self.dll_last_address))


We need to standardize all the quote. Double quote. " and not '

nacayoshi00 · 2021-06-06T12:53:54Z

I see, but the result was unexpected for me because the test took only 1638s on my laptop PC.
I think the reason why the test is so long is below:

All DLLs are loaded at first even if some DLLs may not be used.
DLL reload_image() may take quite long

Since I think these fixes are needed because emulation fails when DLL's address is not resolved, I have another solution to solve the problem.

Make resolve_dll_address function that resolves address when emulator encounters unmapped address.
Register the function to hook_address.

If u r OK, I'll try to update.

xwings · 2021-06-07T02:47:04Z

1638s is one test or all test ?

The test is being done from github CI, so i guess we need to make that as benchmark.

I guess your idea is fine. We need to fix CI result. Else gonna be bad if we need to spend 4 hours for each test. Since currently only 10min for all test.

nacayoshi00 · 2021-06-07T10:24:06Z

It took 1638s to finish all the tests.
OK, I'll try to fix it. Please be patient.

xwings · 2021-06-09T00:35:24Z

Sure. Will be waiting :)

nacayoshi00 · 2021-07-18T08:15:22Z

I'm sorry for late the response. I update these files to load dll dynamically.
But the problem still remained because the reason why take quite so long is to relocate some dlls such as shell32.dll (see below log.). So I add libcache=True in all_tests of test_pe.py.

>>> timeit.timeit("import pefile;pefile.PE('qiling/examples/rootfs/x86_windows/Windows/System32/shell32.dll').relocate_image(0x7fff00000)", number=1) 205.60229980000076

I checked the time dramatically reduced in my PC when libcache is enabled.

previous ver:1638s
new version(libcache off): 1000s
new version(libcache on): 80s)

nacayoshi00 · 2021-07-19T14:14:19Z

I resolved the conflict and re-uploaded

nacayoshi00 · 2021-08-11T14:38:44Z

Could you mind if I ask you to run the workflow? Or should I do something to proceed with the workflow?

xwings · 2021-08-12T01:06:45Z

There is still a conlifct with wscok32

nacayoshi00 · 2021-08-29T08:07:24Z

Sorry too late. I removed the conflict. So could you please run the workflow?

elicn · 2022-04-24T10:13:06Z

Hi @nacayoshi00,
Based on the recent changes made to dev branch, could you please test to see whether this PR is still relevant?

xwings · 2022-10-06T03:52:36Z

Since author did not repnse since Aug 2021. PR will be close for now.

Fix bug: DLL address still use ImageBase address in spite of changing…

500bb94

… lib_base.

Fix: Dll address realocation, Cover execution with API Set Dll

4b840c9

nacayoshi00 mentioned this pull request Jun 2, 2021

Add PE file for the test of address relocation and API Set Dll qilingframework/rootfs#3

Merged

Fix for run test error

af6bc8a

add test exe source, add dll that need for the test

5210c55

xwings reviewed Jun 3, 2021

View reviewed changes

limit copy dll file.

b50f3ec

xwings reviewed Jun 5, 2021

View reviewed changes

nacayoshi00 added 9 commits July 14, 2021 00:10

add dll resolve hook

bb1b99b

update resolve_dll

171d0c4

update test_pe to use libcache in all test

011d882

save

2ff1283

check mov opcode pointer

d149126

delete lark-parser function

1183a5b

Use entry_import_table

6e46c76

update dynamic resolve dll address

6fe5e29

update dynamic dll load

8611ffe

nacayoshi00 added 2 commits July 19, 2021 22:38

delete Unnecessary description

87229f6

merge current dev branch

4a07c51

resolve conflict

fd51e4b

t-moe mentioned this pull request Sep 16, 2021

Fix PE loader to recursivly import symbols and call DllMain #917

Closed

15 tasks

wtdcode deleted the branch qilingframework:old_dev October 12, 2021 08:49

wtdcode closed this Oct 12, 2021

wtdcode reopened this Oct 14, 2021

xwings closed this Oct 6, 2022

Conversation

nacayoshi00 commented May 24, 2021

Checklist

Which kind of PR do you create?

Coding convention?

Extra tests?

Changelog?

Target branch?

One last thing

Uh oh!

xwings commented May 25, 2021

Uh oh!

nacayoshi00 commented May 26, 2021

Uh oh!

nacayoshi00 commented Jun 1, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

xwings commented Jun 2, 2021

Uh oh!

nacayoshi00 commented Jun 2, 2021

Uh oh!

xwings Jun 3, 2021

Choose a reason for hiding this comment

Uh oh!

nacayoshi00 commented Jun 4, 2021

Uh oh!

xwings commented Jun 5, 2021

Uh oh!

xwings Jun 5, 2021

Choose a reason for hiding this comment

Uh oh!

xwings Jun 5, 2021

Choose a reason for hiding this comment

Uh oh!

nacayoshi00 commented Jun 6, 2021

Uh oh!

xwings commented Jun 7, 2021

Uh oh!

nacayoshi00 commented Jun 7, 2021

Uh oh!

xwings commented Jun 9, 2021

Uh oh!

nacayoshi00 commented Jul 18, 2021

Uh oh!

nacayoshi00 commented Jul 19, 2021

Uh oh!

nacayoshi00 commented Aug 11, 2021

Uh oh!

xwings commented Aug 12, 2021

Uh oh!

nacayoshi00 commented Aug 29, 2021

Uh oh!

elicn commented Apr 24, 2022

Uh oh!

xwings commented Oct 6, 2022

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

nacayoshi00 commented Jun 1, 2021 •

edited

Loading