Qdb error when using QlLoaderBlob

[antcpl]

Describe the bug
When trying to use Qdb with a baremetal binary emulation that uses QlLoaderBlob, an AttributeError raises.

Sample Code
I am working on the dev branch.
My setup is the following one :
I emulate a bare metal binary running on ARM cortex A7 processor. For the global emulation setup I followed this qiling/examples/hello_arm_uboot.py and defined a cortex_a.ql file to setup the memory as required by the blob loader.

with open("./baremetal_binary", "rb") as f:
        binary = f.read()

ql = Qiling(code=binary[0x00000000:],archtype=QL_ARCH.ARM, ostype=QL_OS.BLOB , verbose=QL_VERBOSE.DISABLED, cputype=ARM_CPU_MODEL.ARM_CORTEX_A7, profile="cortex_a.ql")

[...]

ql.debugger= "qdb"
ql.run()

Expected behavior
No error.

Screenshots

Traceback (most recent call last):
  File "/test_qilin/qiling/examples/mcu/fuzzing_test/cortex_A7/code_coverage/dev/general_script.py", line 60, in <module>
    ql.run()
  File "/test_qilin/qilingenv/lib/python3.12/site-packages/qiling/core.py", line 585, in run
    debugger = debugger(self)
               ^^^^^^^^^^^^^^
  File "/test_qilin/qilingenv/lib/python3.12/site-packages/qiling/debugger/qdb/qdb.py", line 73, in __init__
    self.dbg_hook(list(filter(lambda d: int(d, 0) != self.ql.loader.entry_point, init_hook)))
  File "/test_qilin/qilingenv/lib/python3.12/site-packages/qiling/debugger/qdb/qdb.py", line 105, in dbg_hook
    elif self.ql.loader.entry_point:
         ^^^^^^^^^^^^^^^^^^^^^^^^^^
AttributeError: 'QlLoaderBLOB' object has no attribute 'entry_point'

Additional context
The error is normal, indeed the QlLoaderBlob object doesn't have this attribute.
I found these lines in the qdb.py file (line 140) which seems to be here to handle the case but they are after different other access to self.ql.loader.entry_point atrribute previous them :

  if self.ql.os.type is QL_OS.BLOB:
            self.ql.loader.entry_point = self.ql.loader.load_address

   elif init_hook:
            for each_hook in init_hook:
                self.do_breakpoint(each_hook)

Maybe a fix could be to change the place of these lines.

[antcpl]

I've also spotted another problem but that happens in very rare cases.
In qdb/render/render.py : context_asm function :

 lines = {}
past_list = []
from_addr = self.cur_addr - self.disasm_num
to_addr = self.cur_addr + self.disasm_num

cur_addr = from_addr
while cur_addr <= to_addr:
    insn = self.disasm(cur_addr)
    cur_addr += insn.size
    past_list.append(insn)

If the self.curr_addr (which corresponds generally to our pc) is set at the first instruction of the mapped memory where the code lies, from_addr become an unmapped address. Thus causing an unmapped access in the while loop at this line insn = self.disasm(cur_addr)

I suggest this correction :

 lines = {}
  past_list = []
  from_addr = self.cur_addr - self.disasm_num
  to_addr = self.cur_addr + self.disasm_num


  # My correction when we are booting and our pc is the first valid address hosting instructions
  if self.cur_addr == self.ql.loader.entry_point:
      from_addr = self.cur_addr 

  cur_addr = from_addr
  while cur_addr <= to_addr:
      insn = self.disasm(cur_addr)
      cur_addr += insn.size
      past_list.append(insn)

By the way we could also imagine the other case, when executing the last instruction in the mapped memory.

[elicn]

Hi @antcpl,
Could you please share repro steps for the first issue you listed here (missing property)?
The second one is not an issue: the code on dev branch attempts to read the data and fail gracefully if the data is not available.

[antcpl]

Hi @elicn !
For the first issue, I am doing baremetal emulation using Cortex_A7 Arm CPU and qdb. The qiling script is described above, I am using QL_OS.BLOB as ostype combined with this cortex_a.ql file :

[CODE]
ram_size = 0x8000
entry_point = 0xFFFF0000
heap_size = 0x300000


[MISC]
current_path = /

With this, the entry_point is setup in the ql._profile attribute but this attribute is not used in the QlLoaderBLOB.run(). Thus, the error is triggered later in the dbg_hook function in qdb.py.

        if self.ql.entry_point:            # <- this condition is False
            self.cur_addr = self.ql.entry_point
        else:
            self.cur_addr = self.ql.loader.entry_point    # <- falling here, but QlLoaderBLOB has no entry_point attribute
                                                                                      # and this cause the AttributeError

        self.init_state = self.ql.save()

        # the interpreter has to be emulated, but this is not interesting for most of the users.
        # here we start emulating from interpreter's entry point while making sure the emulator
        # stops once it reaches the program entry point
        entry = getattr(self.ql.loader, 'elf_entry', self.ql.loader.entry_point) & ~0b1
        self.set_breakpoint(entry, is_temp=True)

        # init os for integrity of hooks and patches while temporarily suppress logging to let it
        # fast-forward
        with self.__set_temp(self.ql, 'verbose', QL_VERBOSE.DISABLED):
            self.ql.os.run()

        if self.ql.os.type is QL_OS.BLOB:  #!!!!!!!!!!!!!!!!! <- this is exactly the needed code to handle this case
            self.ql.loader.entry_point = self.ql.loader.load_address # 

        elif init_hook:
            for each_hook in init_hook:
                self.do_breakpoint(each_hook)

        if self._script:
            self.run_qdb_script(self._script)

        self.cmdloop()

Check my comments in the above code, finally it just seems that the code handling this case is not at the correct place. To test, I've just moved those line above the ones causing the error and this works (for my case of course, maybe break some others). Hope this is clear !

[antcpl]

For the second one, I've seen your modification on the dev branch which handles this. What do you mean by failing gracefully ? I've tested and got this :

Traceback (most recent call last):
  File "/home/antoine/branch_dev_qiling/qiling/examples/mcu/cortex_A7/general_script.py", line 52, in <module>
    ql.run()
  File "/home/antoine/branch_dev_qiling/qilingenv/lib/python3.12/site-packages/qiling/core.py", line 581, in run
    debugger = debugger(self)
               ^^^^^^^^^^^^^^
  File "/home/antoine/branch_dev_qiling/qilingenv/lib/python3.12/site-packages/qiling/debugger/qdb/qdb.py", line 91, in __init__
    self.dbg_hook([addr for addr in init_hook if int(addr, 0) != self.ql.loader.entry_point])
  File "/home/antoine/branch_dev_qiling/qilingenv/lib/python3.12/site-packages/qiling/debugger/qdb/qdb.py", line 153, in dbg_hook
    self.ql.os.run()
  File "/home/antoine/branch_dev_qiling/qilingenv/lib/python3.12/site-packages/qiling/os/blob/blob.py", line 49, in run
    self.ql.emu_start(self.entry_point, self.exit_point, self.ql.timeout, self.ql.count)
  File "/home/antoine/branch_dev_qiling/qilingenv/lib/python3.12/site-packages/qiling/core.py", line 774, in emu_start
    raise self.internal_exception
  File "/home/antoine/branch_dev_qiling/qilingenv/lib/python3.12/site-packages/qiling/core_hooks.py", line 141, in wrapper
    return callback(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/antoine/branch_dev_qiling/qilingenv/lib/python3.12/site-packages/qiling/core_hooks.py", line 226, in _hook_trace_cb
    ret = hook.call(ql, addr, size)
          ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/antoine/branch_dev_qiling/qilingenv/lib/python3.12/site-packages/qiling/core_hooks_types.py", line 25, in call
    return self.callback(ql, *args)
           ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/antoine/branch_dev_qiling/qilingenv/lib/python3.12/site-packages/qiling/debugger/qdb/qdb.py", line 126, in __bp_handler
    self.do_context()
  File "/home/antoine/branch_dev_qiling/qilingenv/lib/python3.12/site-packages/qiling/debugger/qdb/qdb.py", line 406, in do_context
    self.render.context_asm()
  File "/home/antoine/branch_dev_qiling/qilingenv/lib/python3.12/site-packages/qiling/debugger/qdb/render/render.py", line 73, in wrapper
    wrapped(*args, **kwargs)
  File "/home/antoine/branch_dev_qiling/qilingenv/lib/python3.12/site-packages/qiling/debugger/qdb/render/render_arm.py", line 63, in context_asm
    self.render_assembly(listing, address, prediction)
  File "/home/antoine/branch_dev_qiling/qilingenv/lib/python3.12/site-packages/qiling/debugger/qdb/render/render.py", line 183, in render_assembly
    print(__render_asm_line(insn))
          ^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/antoine/branch_dev_qiling/qilingenv/lib/python3.12/site-packages/qiling/debugger/qdb/render/render.py", line 158, in __render_asm_line
    trace_line = f"{insn.address:#010x} │ {insn.bytes.hex():18s} {insn.mnemonic:12} {insn.op_str:35s}"
                                           ^^^^^^^^^^^^^^
AttributeError: 'NoneType' object has no attribute 'hex'

Is this what you were waiting for ?

If yes, so much the better.

Otherwise, I know this is a very edge case and I don't want to bother you with this but here is the debug help if needed.
Here is a print of listing just before the call to the __render_asm_line function, maybe this will help you :

[InvalidInsn(bytes=None, address=4294901756, mnemonic='(invalid)', op_str=''), InvalidInsn(bytes=None, address=4294901757, mnemonic='(invalid)', op_str=''), InvalidInsn(bytes=None, address=4294901758, mnemonic='(invalid)', op_str=''), InvalidInsn(bytes=None, address=4294901759, mnemonic='(invalid)', op_str=''), <CsInsn 0xffff0000 [080000ea]: b #0xffff0028>, <CsInsn 0xffff0004 [060000ea]: b #0xffff0024>, <CsInsn 0xffff0008 [050000ea]: b #0xffff0024>, <CsInsn 0xffff000c [040000ea]: b #0xffff0024>, <CsInsn 0xffff0010 [030000ea]: b #0xffff0024>]

As said above, the first instruction lays at : 0xffff0000.

[elicn]

@antcpl, are you using an existing example? I can't find any.
If you have a binary you can share, that would be great.

[antcpl]

Hi @elicn,
Yes of course, here is a zip with all you need to reproduce the problem : brom is a bootrom binary developed for cortex A7 (it comes from here https://github.com/FunKey-Project/Allwinner-V3s-BROM), cortex_a.ql is the profile used to match the cortex_A7 requirements (100% inspired from qiling/examples/hello_arm_uboot.py) and general_script.py is the qiling script corresponding to the setup.

archive.zip

Let me know if you are able to reproduce this.

[elicn]

Thanks, that makes it easier for me to start working on that.

Looking at the missing entry_point property, it looks like BLOB OS and Loader are confusing load_address and entry_point all around, assuming the binary needs to be loaded at entry_point (?). Untying this and differentiating between the load address and the entry point will require a bit of work.

As for the InvalidInsn case: it is quite easy to fix. If you don't want to wait for my fix and want to apply it locally, go to qiling/debugger/qdb/context.py and patch lines 60 and 78 to appear as:

insn_bytes = self.read_insn(address) or b''

[antcpl]

No problem.
Yes I agree with you observation. To answer your (question), yes for this type of binary the load_address value is the same as the entry_point.

If I could make a proposal, maybe that would make sense to allow both of the attributes (entry_point and load_address) to be set by the user in the profile file, I don't know if you were thinking about that. I think that would guarantee the widest compatibility. I'm thinking of other archs the where the entry_point could be at load_address+x for example. Don't know if that's relevant.

Thanks a lot for the quick fix.

[elicn]

If I could make a proposal, maybe that would make sense to allow both of the attributes (entry_point and load_address) to be set by the user in the profile file, I don't know if you were thinking about that. I think that would guarantee the widest compatibility.

Yes, that is what I had in mind.

[elicn]

@antcpl, there is a PR ready with the fixes. Will appreciate if you could validate it to see it indeed fixes everything we discussed here.

It looks like it uncovered a new bug though: QDB seems to display the branch target as a signed integer, so high addresses appear as negative values.

[antcpl]

@elicn Thanks for the rapid feedback ! I've tested with my own setup and all of the things discussed above are solved. I've also tested the highest code limit in memory for the disassembly rendering, it works perfectly.

Besides, I confirm the bug you described about the branch target display.

[antcpl]

Just looked around for the bug you described : only happens with branch using immediate.
Is due to this : in captsone in the arm.py file :

class ArmOpValue(ctypes.Union):
    _fields_ = (
        ('reg', ctypes.c_uint),
        ('imm', ctypes.c_int32),
        ('fp', ctypes.c_double),
        ('mem', ArmOpMem),
        ('setend', ctypes.c_int),
    )

The imm field is c_int32 type so is returned as signed value to the branch_predictor_arm.py in the __parse_op function lines 101/102 :

elif op.type == CS_OP_IMM:
      value = op.imm

[elicn]

Well, the fix could be simpler in this case, because one can instruct Capstone to read immediate values as unsigned. However, this requires a closer look to better understand all cases and see if there are ones that actually need the value to be read as signed (relative offsets?). Since I am not an ARM expert, it will require me some time to analyze this, and I did not want to hold the other fixes in the meantime.