[NTLK] Restarting Newton development?!

[Matthias Melcher]

> On Aug 3, 2018, at 21:11, Steven Frank <stevenf at panic.com> wrote:
> 
> 
>> On Aug 3, 2018, at 10:24 AM, Matthias Melcher <m.melcher at robowerk.de> wrote:
>> 
>> I did already ported newt/0 to 64 bit. It's a bit clunky and uses lookup tables, that fill up eventually, but it's good enoigh to use newt/0 to compile something or even run a short live app.
> 
> Oh, nice!  Is that in your GitHub fork at MatthiasWM/NEWT0?

Yes, correct. Newt/0 was integrated into my Einstein branch for a while. I was trying to merge DyneTK, newt/0, and Einstein to create an IDE with emulator. It failed because writing Objective C drove me nuts (or I should have made the effort to actually learn the language :-D :-D ).

Usually, C and C++ source code can be ported to 64 bit relatively easily. newt/0 is different, because it is, among other things, a byte code interpreter. The byte code supports pointers in 32-bit only, but newt/0 byte code may very well reference 64 bit addresses in MacOS. So using a 32-bit sandbox approach was out. Changing the byte code to allow 64 bit addresses is feasible, but it would make newt/0 incompatible to old byte code (or the old bytecode would have to be reinterpreted on load), which seemed like too much work for something that will likely never be used in serious applications. If we were redesigning the language, that would be the right approach.

So I ended up using back-and-forth lookup tables through hash codes. 64-bit pointers are replaced with a unique hash code, which is then stored in the byte code. It's easy to find the 32-bit hash code for a pointer, and it is very fast to find the 64-bit address for a 32-bit hash code. Only problem: if memory is freed, and the pointer is no longer used, the hash code lookup remains and is reused as soon as the pointer to that address is allocated again. 

In a server app, that runs for weeks and months, this would eventually fill up memory, but for apps with a limited run time of minutes or hours, this should be fine.

> Can you explain more about what were some of the issues and what the lookup tables are for?  Is that specifically to support execution of existing 32-bit compiled NewtonScript?
> 
>> Simon Bell's Framework is really fantastic. I probably read through pretty much the whole thing by now, an I believe that he can convert ARM to C++ code on the fly by now, making him the first human decompiler. He did use some Apple specifics in his implementation, some Objective C, and even some assembler, which would need to be brought into common C++, but his code has become a NewtonOS reference Manual for me.
> 
> It looks very impressive and I've barely had a chance to dig into it yet.  What's the intended usage model?  For example, if I wanted to use just the NewtonScript compiler, is it best to build the entire framework and link my app against it, or just tease out the classes related to the compiler?
> 
> What's the end-goal of the "MessagePad" target in the project?  To be a native runtime for the NewtonScript extracted from the ROM, like we've occasionally discussed with Einstein?  How far along is it, and what can someone like me do to help?  (Someone like me = extensive C/C++/Obj-C/Mac experience, minimal NewtonScript experience, some knowledge of the ROM from disassembly and Einstein tinkering.)

If I understood Simon correctly, he wanted to have a simulator that would help him to develop and test apps like Mail V entirely on MacOS without using an MP or Einstein. I believe that one of his apps or libraries benefited greatly form this approach, but I do not remember which.

> Simon, if you're still lurking, what are your goals with the project, do you want help, and if so what can we do?
> 
>> I had not seem Wlater Smith's (THE Walter Smith) code online until now. That is pretty awesome, too. I wonder if he has the source code to the entire NewtonOS 2.2 somewhere on an old Exabyte Tape ;-)
> 
> I wouldn't be sad if such a thing "accidentally" ended up online some day, but I'm not holding my breath.  :)

No, me neither. I did find a DAT tape recently from 1990 which contained my diploma thesis software that was used to generate computer graphics for 14 episodes of a German TV show. Unfortunately, the DAT drive we had back then was out of alignment, and I can't read the data back (helical scan). I would have *loved* to recompile the old code that ran on tremendously expensive SGI back then, and would easily run on a low end smartphone today :-)

 - Matthias