Re: [NTLK] ATA Linear flash speed

Hello,

I'm following your discussion about ATA vs Linear flash speed with
great interest! Especially considering that I secretly hope that
someone grabs the source code and brings it to the next level.

ATA and linear cards work very differently. Globally, ATA cards feel
slower than linear cards: ATA cards can be faster in some (very)
specific scenarios, but not the ones when you traditionally wait
(i.e. package installation).

There is something that ATA cards do very bad, partly because of the
way ATA cards work and partly because ATA driver was developed while
I was figuring out the actual interface with NewtonOS, instead of
having been architected with all information beforehand. It's the
handling of LBO, large binary objects, especially within
transactions. This is typically what happens when you install packages.

The ATA card store (TATAStore) was designed with the small objects
model in mind. Each object can be a single value such as an integer,
a string, a symbol, etc. Therefore, ATA store was divided in pages
(based on the size of sectors, i.e. 512 bytes) which can contain
several objects. Each object can be within a transaction and an
object can be at most 506 bytes (with the overhead on the sector).
Larger objects are divided into smaller objects. This is very
inefficient because of the overhead and because it does not benefit
from ATA commands to read or write multiple sectors at once. This
could be fixed by maintaining statistics about the size of objects,
when they are in transaction and committed and reimplementing the ATA
Store with this information. Instead of trying to pack as much data
as possible as I (stupidly) did 8 years ago, this could be easily
done starting with an open source B+Tree implementation with blocks
that would probably be larger than 512 bytes (e.g. 2KB or 4KB). The
ATA store implementation was built around a crash test running on the
Mac, which is included in the ATA Support source package and
therefore could be used to check the soundness of the new
implementation.

The other part where ATA Support speed can be improved is by making
writes asynchronous. Newton storage is transactional and until the
final commit, there is no reason to wait for the writes. I think I
more or less started doing that by building a driver server that gets
commands from the store.

The remarks about the power usage of ATA cards and linear cards are
very interesting. While ATA Support was built with power usage in
mind, it makes sense that linear cards use less power. The power
management has not been deeply validated and there is a known bug
that makes the eMate randomly crash when it goes to sleep. It is very
difficult to investigate because the eMate powers off the serial line
connected to the debugger :)

ATA Support was designed at a moment when linear cards were horribly
expensive, and the goal was to have something reliable and cheaper.
Nowadays, (second-hand) linear cards are cheap again, especially
considering that you (usually) don't need gigabytes of memory on the
Newton. Today, the missing feature of ATA support is to store data on
a FAT partition so you could easily get it from a desktop or laptop
computer. Alternatively, ATA cards could be read by Einstein....

Paul

====================================================================
The NewtonTalk Mailing List - http://www.newtontalk.net/
The Official Newton FAQ - http://www.splorp.com/newton/faq/
The Newton Glossary - http://www.splorp.com/newton/glossary/
WikiWikiNewt - http://tools.unna.org/wikiwikinewt/
====================================================================
Received on Tue Aug 19 03:20:37 2008