IO

When I first started thinking about Fastware!, I knew I'd have a chapter on IO, but I was concerned that it would take me a while to have much to say about the topic, and I expected it to be one of the last chapters I attacked. Since then, my view has changed, and in part due to the excitement I got from reading Tom Leighton's article in CACM (and ACM Queue -- sometimes recycling isn't so great, sigh) about improving performance using the Internet, I'm now ready to draft a chapter on IO.

It's an interesting question to try to define IO. Technically, IO is probably anything that communicates with something off-chip, so accessing main memory can be considered IO, although I don't plan to treat it that way. My current plan is to focus on disk and network issues, but even there things are beginning to get fuzzy, because solid state drives use a traditional disk API, but don't have anything akin to rotational latency. At some point I may decide to create a chapter focusing on storage (cache, memory, and "disks," regardless of technology) and another on network IO, but for now, the traditional view that IO largely means disk and network access seems reasonable to me. I welcome comments on the best way to organize these issues.

So what are the topics relevant to the creation of sizzling disk and network IO? Here are the main ones on my list:

• Prefetching, buffering, and caching (by hardware, drivers, OSes, language runtime systems, and applications)
• Asynchronous and concurrent reads/writes (including disk striping)
  
• Memory-mapping files
• Avoiding disk fragmentation
• Network protocol choices (e.g., TCP vs UDP)
• Doing IO on deltas instead of full data sets
• Reducing network distances (e.g., via CDNs)
• Data compression and "bundling" (e.g., CSS sprites)
• Latency vs. Bandwidth
  

I welcome suggestions for issues related to the design and implementation of low-latency IO. Because Fastware! is a language-independent book, I expect to make at most passing references to language-specific techniques (e.g., using C++ rdbuf or istreambuf_iterators), but I'm still interested in hearing about them, because it's not uncommon for different languages to have their own approaches to a more general issue, and I want to include discussions of as many general issues as I can.