Implementing int3 (0xCC) and step debugging

[kyr0]

Hi @copy ,

just wanted to ask about your opinion on how to implement step debugging?

I have step debugging running locally with a rather creative hack..

1. to control that the CPU stops executing, I'm setting:

*in_hlt = true;

in call_interrupt_vector when interrupt_nr equals 3 and when it is a is_software_int.

I'm also setting the FLAG_TRAP like this:

    *flags |= FLAG_TRAP;
    let state_flags = pack_current_state_flags();
    let state_flags_u32 = state_flags.to_u32() as i32;
    let tsr_offset = *segment_offsets.offset(TR as isize);
    safe_write32(tsr_offset + TSR_EFLAGS, state_flags_u32).unwrap();

2. now when the opcode 0xCC occurs, the CPU stops execution and the TRAP flag is set.
   The user-land JS code needs to be informed "instruction precise". There is no hook implementation in cpu.js as far as I could see, that's why I'm monkey patching here:

const orig_main_run = CPU.prototype.main_run;
CPU.prototype.main_run = function()
{
    const flagsValue = this.flags.valueOf()[0]

    if (flagsValue !== prevFlags) {
        prevFlags = flagsValue
        // ... code to sync the flags in user-land, detect setting and 
        // unsetting of TRAP and activating/deactivating step debugger UI components
    }
    return orig_main_run.apply(this)
}

3. the UI components allow to jump to the next breakpoint by overriding the respective byte that indicates in_hlt in memory directly:

// reset in_hlt using the Proxy view
emulator?.v86.cpu.in_hlt.valueOf().fill(0)

4. before the next instruction is executed by the CPU, the FLAG_TRAP needs to be reset. I've implemented this as the first expression in do_many_cycles_native:

// reset trap flag
*flags &= !FLAG_TRAP;

I'm pretty sure this is not the way int3 / 0xCC should be implemented, but it "works for me".

In comparison to what actually happens in the real world (overriding the first byte of opcode with 0xCC and handling the trap in-CPU), that's probably all just totally wrong. Atm you could override code with 0xCC in-memory before execution from the outside using the memory view, but how my code handles the flags is probably wrong and also, I'm sure that the debug registers should be filled with some values etc.?

However I'm just working on a simple teaching tool to teach myself Assembler; and for that I just needed basic step debugging ... and that works... :)

I'm interested in how we would implement this the right way?

Could you maybe please explain a bit more about the concept, and what I could read or work through to find the right way maybe?

I'm willing to implement this "the right way" and PR.

Thank you and best,
Aron

Edit: Got it practically working now, rephrased the post