Color, Personalization, and Content/Presentation Separation

In 2001, I published my first two-color book. In it, I used red text to focus readers' attention on particular parts of code examples. Since then I've received a steady trickle of comments about my use of color, including this from a recent missive:

Nearly 10% of the male population (and a much smaller fraction of the female population) is one form of color-blind or another. By far the most common is basic red-green color deficiency where reds and/or greens are harder to detect/less vibrant colors.

The dark red chosen as a highlight color in Effective C++ has nearly the same intrinsic darkness as the black it is meant to contrast with and is not an especially vibrant shade of red. As such it is not easily distinguishable from the main text (at least by this particular color deficient male).

Goal #1: I'd like to avoid this problem. Being a software person, I'm going to give in to my inclination to generalize and assume that any given set of color choices is going to be problematic for some portion of my readership.

Consider next syntax highlighting of code. If you use vanilla Visual Studio, you're used to your code being colored in a certain way. If you use vanilla Eclipse, you're also used to your code being colored in a certain way, but probably not the same way. If you use Emacs or some other text or programmer's editor, you're probably used to your code being colored yet another way. I assume that all these code editing environments give you the ability to customize how syntax highlighting is performed, meaning that what you're used to seeing in your editor could be unique to you. If you read a book on a device supporting color, you'd probably prefer that the colors you see match the colors you are used to seeing. Goal #2: I want you to be able to do this.

Addison-Wesley and I recently released PDF versions of my books, and, like many other publishers, we eschewed restrictive DRM in favor of personalization: when you buy a PDF, each page shows your name. The idea is that you're a lot less likely to post your PDF to the world if everybody can see your name, so we do that instead of imposing technical restrictions on what you can do with the PDF you buy.

The need to personalize an ebook for each purchaser means that some kind of time-of-sale processing has to occur, so my feeling is that we might as well go ahead and generate a custom copy of the book at that point. (This is what I think should happen for Fastware! It's not what's done for the PDF versions of my current books.) And as long as we're going to do that, we might as well let you specify how color should be used. By default you might get black text on a white background with red highlighting and Eclipse-style syntax formatting, but if you want yellow text on a black background with green highlighting and syntax highlighting that we allow you to painstakingly specify on a construct-by-construct basis, what the heck, we'll generate it for you. It's not like the software cares what the output looks like, and I'm not about to tell you what works best for you.

For this to be practical, I think there has to be a physical separation of book content from presentation information, much as web page content tends to go in .html files and presentation information tends to go in .css files. From the point of view of me choosing software in which to author Fastware!, I think this rules out programs like FrameMaker, Word, and Writer, because, as far as I know, there is no practical way to get them to automatically (i.e., without human intervention) generate PDF from a document using some custom specification of how formats/styles for paragraphs/characters should be rendered. If that's true, I think we're pretty much talking markup-based approaches, notably LaTeX and DocBook.

Is this a valid conclusion?

Content and Presentation: Not Independent

One would like to believe that content decisions and presentation decisions are independent, but they are not. If I want to get a point across, do I present it in prose, as a code example, as a diagram, as a picture, or in some other way? If I want to focus a reader's attention on a particular aspect of a code example, do I change the font style of the relevant text (e.g., to bold or italic), the font background (i.e., to make it look like it's highlighted), or the font color? Or do I leave the font alone, give each line in the code example a number, then use prose before or after the example to describe what I want my readers to see? Depending on exactly what I want to do, I make different decisions. For recent examples, check out the code listings on one of the pages of a recent C++ article I published.

Speaking of code listings, one of the most important presentation decisions I have to make involves code formatting. Decisions on indentation and line breaks are critical, and my experience over many years with a number of publishers is that letting anybody edit my code examples is a sure way to publish broken code. (When I was a columnist for C++ Report, I had to fight over every column with some ninny who had "editor" on his business card and hence felt qualified to edit what I wrote, notwithstanding the fact that he didn't know C++ and was unable to understand what I was writing about.) In retrospect, this may be why I've become so accustomed to producing camera-ready copy: it avoids my having to verify that what I wrote is what ultimately gets published. I'm not perfect, of course, and my books' errata lists provide ample testimony to that, but my experience is that I am much more careful about what goes out with my name on it than anybody else.

One of the places where authors must make critical presentation decisions involves content that is dependent on the capabilities of the output device being used to present the content. For example, here's a table I used in a recent article:



Actually, that's what was published. What I submitted looked a lot nicer, in my opinion:



But consider how this table would "appear" if rendered as speech. (For an example of how a table can sound, click on the "listen" link at the top of Mike Hendrickson's "State of the Computer Book Market" blog entry from February 22, download the mp3, and start listening about 3 minutes into the audio stream.)

As an author, I know that the point of the table is to demonstrate that the "empty" objects I'm discussing in the article may be quite large: up to several thousand bytes. The table is a good way to show that to readers, but for listeners, I'd omit the table and use a prose summary instead. In fact, I already have such a summary in the article:

If such objects are not optimized away, Table 1 demonstrates that their size could be significant (up to many thousands of bytes per feature set), an artifact of the use of virtual inheritance in the current implementation.

For an audio stream, I'd simply omit the table and modify the text above as follows:

If such objects are not optimized away, their size could be significant (up to many thousands of bytes per feature set), an artifact of the use of virtual inheritance in the current implementation.

This suggests another feature I want to have as an author: conditional content. I want to be able to say:

  if (rendering for a print device)
   produce this text and show this table
 else
   produce this other text and don't show the table

In both cases, the conceptual content is the same. It's the way that information is presented that differs, and it's a decision I, as an author, am best in a position to make. (If I could find an editor who'd make such decisions for me such that I'd be happy with the results of the decisions, I'd gladly turn such decision-making over to him or her. As they say in the old movies, alas, good help is hard to find.)

WYSIWYG in a Multi-Target World

Markup or WYSIWYG, that is the question. I've done it both ways. In my academic life, I wrote extensively using markup languages, including runoff, Scribe, and LaTeX. The editing itself was done using Emacs (the One True Text Editor). I've also done a lot of writing using WYSIWYG systems, primarily FrameMaker. WYSIWYG is better.

It's better for two reasons, but before I get to those, I have to address the markup bigots who argue that WYSIWYG editing mixes structure identification (e.g., "this is a subsection") with presentation decisions (e.g., "subsections should be in 11 point bold Palatino and numbered n.m"), a situation sometimes derided as WYSIAYG ("What You See is All You've Got"). Sure, people can do this, but paragraph and character formats (as FrameMaker calls them) or styles (as Microsoft Word and OpenOffice Writer do) also make it possible to separate structure and presentation, and anybody writing something as long as a book will soon learn to do so. There's no dichotomy between WYSIWYG editing and separating structure and presentation.

I should also mention that I generally produce camera-ready copy for my books. That means I'm making most of the decisions about presentation myself: page layout, font choices, all that fun stuff. I've observed elsewhere that preparing camera-ready copy is probably not a good use of an author's time (scroll down to "Typeset"), but I have my reasons for doing it (or at least most of it), so ignoring the details of what things look like is not a practical option for me.

WYSIWYG authoring is better than markup-based authoring for two reasons. The first is latency in error detection. Suppose I want to specify that a paragraph is a second-level bullet point. With a WYSIWYG system, if I select the wrong paragraph or I don't specify a second-level bullet, I see the result of my mistake immediately, and I can correct it without further ado. With markup, I don't notice that I've done something wrong until I submit my document to the document formatting engine and look at its output. Call me a renegade, but I don't enjoy finding out several minutes after failing to close an italicized (er, excuse me, I mean emphasized) section of text that my entire document is now in italics. I'd rather see the problem the instant I specify that something is to be italicized/emphasized.

The second reason I prefer WYSIWYG editing is that what I see on my screen matches what I see on paper. This is important when editing. Having printed a section or chapter and annotated the hardcopy with changes to be made, if I need to reword a sentence at the end of page 2, I want to go to the end of page 2 on my screen and start editing. With WYSIWYG, I can. With markup, what I see in Emacs bears no resemblance to what I see on paper, so before I can do my editing, I have to search for a phrase in the text near where I want to edit, then map what I see on the screen to what I see on the paper. Only then can I start editing. It's primitive and a waste of my time.

When looking for authoring software, then, I really want a WYSIWYG system. Unfortunately, I'm not happy with my options. I run Windows (so sue me), so I need something that runs on that platform. I've written my books since 1992 using FrameMaker, and let me just say that I hate it. I don't really know Microsoft Word, but I've heard enough bad things about it to consider it a low-percentage option, and my limited experience with it reminds me of C++: you can do pretty much anything, but almost everything is complicated by the fact that features interact in ways that make sense only if you understand absolutely everything. I wanted to use OpenOffice Writer, but I wrote an article using it, and that experience demonstrated that Writer just isn't up to what I want to do. (Cross referencing was painful, and PDF output was essentially nondeterministic when the layout included floating display elements like figures, listings, and tables.)

I've recently been looking at WYSIWYG XML editors supporting DocBook, but since one of the advantages of DocBook is that it can be transformed for display on different output devices (something I definitely want), I began to realize that the meaning of WYSIWYG is less clear when you'll be "getting" different things for different output devices. Presumably this an issue familiar to web designers who (properly) employ "liquid layout," i.e., allow line breaks to be determined dynamically, because that means that the appearance of their web pages is affected by the width of the window in which it is displayed -- something over which they have no control.

I find myself seeking a WYSIWYG editing interface for a document that may be formatted for several different output devices with varying capabilities and characteristics (e.g., font selection, support for color, physical size, etc.). What do you think I should do?

Two Projects in One

I'm working on Fastware!, a book about how to write software that runs quickly. Unlike my C++ books, Fastware! will be language-independent, and it will cover a much broader range of topics than just code. (At one extreme, for example, it will consider hardware memory hierarchies, and at the other, separate processes potentially written in different languages.) I've already got a detailed outline, a sample chapter, a growing bibliography, and a ton of material I'm looking forward to incorporating. The first part of "The Fastware Project" is the normal stuff that goes into writing a book: figuring out what to include, learning enough to explain it well, and expressing it in words, diagrams, examples, etc.

The second part is the authoring itself, and my writing has been stalled for quite some time as I've wrestled with the question of what it means to write a book these days. For conventional print books, things are easy for an author, because the game is pretty well understood: ink is black, paper is white, standard font size is around 10 point, page dimensions are generally around 9"x6" with maybe a margin of around 1" on all sides. Experience tells us that we shouldn't mix too many fonts in a book, and physical constraints tell us that if we have a spreadsheet with 100 columns in it, shrinking it down to fit on a single page will likely render its content incomprehensible. So authors avoid trying to do things like that.

But I don't think the ink-on-paper world is the one I want to write for any more. I still want to write something that is recognizably a book, but I want to think of ink on paper as but one of many possible output devices. Others include computer screens (big with color support), portable ebook readers like Kindle (smaller and currently with no color support), and portable devices that happen to support text (e.g., iPhones -- very small with color support). I also want to try to support audio as an output device, because text-to-speech systems make the automatic generation of audiobooks practical, and I think this is a form of information consumption that is likely to increase in popularity.

How does one write for such a variety of output devices, and what authoring tools exist to facilitate the task? What's the proper use of color, given that it may or may not be available? What's to be done about diagrams and tables when the output form is audio? These are the kinds of questions I'm trying to think about before I do any more writing, and they're the issues I expect to address in my initial blog entries. I hope you'll help me find my way by offering comments on the issues I raise.