[NTLK] Geek Alert: NewtonOS

[Matthias Melcher]

Hi Newton fans,

Earlier this week, I had a few hours to dive into the Newton ROM. Not sure, if this leads to anything, but I thought I write it down, so it is archived here.

As some of you know, I have been trying to gradually replace the emulated ROM code with native code that would run hundreds of times faster and more efficient. This would be relatively straight forward, if it wasn't for the Memory Management Unit that can interrupt our linear code execution at any time, ruining all opportunities to optimize code.

Well, I finally managed to understand what happens when the OS tries to read or write a protected memory area: first, the dreaded MMU interrupt is triggered, changing the CPU mode and starting a subroutine. This subroutine deciphers the command that caused the interrupt and restores registers to their value *before* the faulty command was executed.

Depending on the type of violation, a different Task is prepared which will fix the problem (allocate RAM for a stack, read data from a compressed file, allocate head, etc.). Then, the new task is put as the first in line to be executed. Our current (faulty) task is suspended, just as it would be when tasks are switched by the scheduler, and the task's PC is set to the faulty instruction.

Now (and that is the thing that I previously did not understand) execution does *not* return to faulty task, but instead, the scheduler switches to the first task, which will (hopefully) fix the fault. Eventually, the task is done and the scheduler will run our original task, continuing with the same instruction that caused the violation (which should not happen again this time around).

What does that mean for development?

Now that this mechanism is understood, it is possible to implement the memory violation handler in native code. This is relatively simple because this code can not cause another violation, allowing us to optimize and write good code.

Writing this code will implement large parts of the scheduler. Once the scheduler is native, we can add code that maps NewtonOS Tasks onto the thread system of the host OS. Well, and once the scheduler and task have native support, we no longer need to worry about emulating task switches, as those are now native. Handling a memory violation is now justa C call instead of running a whole new emulator fiber.

What does that mean for Einstein?

If I manage to implement this, we have enough code and knowledge to translate the entire ROM into native code. This could even be automated, generating convoluted, but working code (speed factor of 10, I would assume). With the entire ROM in native code, we can use debuggers and all the other nice tools to optimize whatever we feel is important in the code (spped factor of 100 and more possible).

Thanks for reading my ramblings ;-)

 Matthias