Saturday, November 29, 2008

Leaning Towards LaTeX

About three weeks ago I wrote that one of my goals was to make it easy to generate versions of my book using custom colors, in part because approximately 10% of the male population suffers from some kind of color blindness. Since that time, I find that achieving this goal has become increasingly important to me. Most of the world's problems are beyond my control, but this I can do something about. I concluded before that wanting to offer color customization on a per-copy basis pretty much commits me to markup-based authoring, a commitment I don't have much enthusiasm for, but the inconveniences associated with markup-based authoring can't compare with the inconveniences associated with color blindness, so I'm in no position to whine. (Not that it means I won't.) I suspect that the use of markup will ultimately allow me to do other things I'll find useful, but at the end of the day, I find that I can't get away from the idea that if I can make it possible for color blind people to see things in a way that works for them and isn't too onerous for me, I have a moral obligation to do it. I guess it's my way of adapting the ADA's "reasonable accommodation" rule to publishing.

As I wrote before, the choices in MarkupLand seem to boil down to XML+DocBook and LaTeX. I was leaning towards DocBook, but then two things happened. The first is that I went to my local technical bookstore to get a book on DocBook, and it didn't have any. Bad sign, very bad, especially since my local technical bookstore is Powell's, which is not a small store. That Powell's didn't have any books on DocBook gave me the kind of "you're in this alone" feeling that's anything but warm and fuzzy. (Yes, there are lots of online resources, and yes, the two big books on DocBook are online, but (1) I like physical books and (2) I can't shake the feeling that the availability of physical books tends to bear a correlation to the size of the user community.)

The second thing that happened is that a friend of mine who's been pushing me to choose LaTeX agreed to act as my personal LaTeX consultant. That's key. Having a person I know agree to help me is much more reassuring than having only the faceless masses of the Internet to turn to. I have a lot of respect for those faceless masses, and I've found them tremendously helpful on many occasions, but I find that the more specialized my interests, the less likely I am to get help online. When I was working with OpenOffice Writer, for example, it didn't take much time for me to go beyond what I was able to find online, and it didn't take much longer than that for me to start stumping the consultant I was working with. "Huh, nobody has ever wanted to do that before..." is not really the kind of feedback I want to start getting on a regular basis.

And then there's this one silly thing: because DocBook is XML, every paragraph has to be started with <para> and ended with </para>. This is inhuman. Literally. It's fine for machines, but not for humans. LaTeX, in contrast, while not without its own syntactic ideosyncracies, interprets a blank line as a paragraph separator. That's vastly more natural for a writer. Working with XML requires that I be aware that I'm building a tree structure that happens to have nodes of text in it. Judging from the online demos for WYSIWYG DocBook editors like oXygen and XMLMind, this is the case even if you're not typing the markup in manually. When I write, I want to think of sequences of paragraph, not nodes in trees. LaTeX lets me. I'm concerned that DocBook won't.

My current plan, then, is to take my draft Introduction for Fastware! and translate it into LaTeX. We'll see how it goes.

EPub State of the Practice, Part 3: Special Features

[Here "EPub" refers to electronic publishing in general, not specifically to the epub format for electronic documents. At the time I wrote this blog entry, I was not aware that "epub" already had a meaning for many people in the world of, well, EPub.]

Fundamentally, live links and reasonable applications of color -- the topics of my last two posts -- are both "Well, duh" features in electronic publication. Along with full-text searching, the ability to print excerpts and copy text, and, for PDF documents, the ability to add comments and define new bookmarks, they should surprise readers only if they are absent. That they are missing from many of the books I examined is a commentary on how badly the publishing industry trails what readers have a right to expect from ebooks.

However, I found a few features in a few books that take advantage of the capabilities of epublication in interesting ways -- features that go beyond the "Well, duh" threshhold.

The first of these is one-click access to code examples. In an earlier post, I remarked that CodeProject articles allow code to be copied to the Windows clipboard in a single click (for IE users; the option is unavailable for FF users, sigh). This is one way to give readers access to code, but it's not the only way. C++ in Action offers single-click downloads of zip files that contain one or more source files, and several titles from The Pragmatic Programmers (e.g., Programming Erlang and Google Maps API V2) open web browser windows on code examples when the correspoinding download links are clicked. These are steps in the right direction, but I think what readers would really like would be single-click access to ready-to-run code examples in their preferred development environment. Selecting a code example in an ebook might, for example, open Eclipse or Visual Studio or, for throwbacks like me, Emacs (or, God forbid, vi) on the code. The "ready-to-run" proviso means that if the code requires auxillary code or other scaffolding before it's ready to execute, that code or scaffolding would automatically be provided. (For my C++ books, virtually every code example would require such scaffolding, because I almost always show code fragments that won't compile by themselves.)

An interesting feature offered by Programming in Scala is that clicking on the introduction of a term (indicated to readers both by the traditional italicization of the term as well as the application of link color to the term) whisks the reader to the definition of that term in the glossary. I think a better approach would be a pop-up window containing the definition (such as I've seen in some Windows applications), but the notion that readers of ebooks shouldn't have to waste time looking up term definitions strikes me as a good one.

Programming in Scala also has a set of navigation links at the bottom of each page, making it easy to get to the beginning of the book, a course-grained TOC, a fine-grained TOC, the glossary, and the index. This is similar to the navigation links typically found on web pages, but I didn't see a similar feature on any of the other PDFs I looked at. I might quibble with the choice of including a link to the cover of the book, because my PDF viewer (Acrobat) already gives me such a link for all PDF files, but, well, there's a reason I say it'd be a quibble.

Programming in Scala includes two other links at the bottom of each page, one of which is also essentially present in the newer titles I saw from The Pragmatic Programmers: a one-click mechanism for offering page-specific feedback. Clicking on "Suggest" (for Programming in Scala ) or "Report erratum" (for the PragProg titles) opens a web form prefilled with book information (title and version) and the page number on which you're offering feedback. Nifty! The PragProg form even lists the errata that have already been reported for that page, a feature that cuts both ways: it can save readers the trouble of reporting a problem that's already known, but it also performs a subtle shift of the burden of filtering out duplicate reports from the author/publisher to the readers. (The author(s) and publisher have to do the filtering, anyway, because there's no guarantee that readers will read the list of known errata before filing a report, but with the way the web form is currently formatted -- known errata above the fields for filing a new report -- I'm reminded of how I feel when I call tech support and have to wait on hold and be chided by a disembodied voice that I should check their online FAQ and knowledge base before trying to actually talk to them.) It's possible that Programming in Scala behaves the same way, but I don't know. I never saw that behavior, but maybe I just didn't try it on a page with known errata.

I will say that I prefer the more generic "Suggest" link from Programming in Scala to the PragProg "Report erratum" link. I've been getting feedback from my readers since 1992, and while most comments come in the form of bug reports, not all do. Making it easy for readers to offer page-specific feedback seems like a great idea, but I don't think it should be limited to errata reports.

Friday, November 28, 2008

EPub State of the Practice, Part 2: Color

[Here "EPub" refers to electronic publishing in general, not specifically to the epub format for electronic documents. At the time I wrote this blog entry, I was not aware that "epub" already had a meaning for many people in the world of, well, EPub.]

As I noted in my previous blog entry, I've been looking at electronic versions of several books, and because I'm viewing them on a device offering color (my computer monitor), I expect the books to use color where color makes sense. Such use is one of the "Well, duh" features I mentioned last time.

One can argue about when using color makes sense, but I hope we can agree that one place where it does is screen shots, and I was surprised to find that three of the nine books where I found screen shots showed them only in monochrome. I hope we can also agree that most programmers these days are used to seeing their code syntax-highlighted (i.e., in color), and of the 13 books that show code, only four show syntax highlighting. The intersection of these sets -- those books that show screen shots in color and that also use color for syntax highlighting code -- is only 3 books, all of which were published by The Pragmatic Programmers in the last two years.

I also checked for the use of color for live links. Because links on web pages are traditionally rendered in a special color (typically blue, in contrast to non-link text, which is typically black), I believe that the active behavior of link text is likely to be overlooked by many readers if it looks the same as regular text. I thus believe that link text should be distinguished in some visual way. This point could be argued, i.e., one might claim that with a little experience, readers would begin to intuit which text is "linky" (e.g., cross-references, URLs, initial definitions of terms, etc.) and which text is not, and that visually distinguishing linky text wouldn't really do anything except add visual noise. One could also argue that even if linky text is to be visually distinguished, color isn't necessarily the best way to do it. (One could use, e.g., underlining or a different font face instead.) Rather than argue the point one way or the other, I'll simply remark that among the books I looked at were several that do use a special color for link text as well as several that do not.

Mine do, and I spent a lot of time trying to come up with a color that was visually recognizable while at the same time being unobtrusive. My goal was to employ something that looked and acted more or less like a web page for people who were scanning for links, while simultaneously looking and acting more or less like a standard book for people who were reading straight through. I ultimately chose a dark blue for links, which is, I hope, different enough from the surrounding black text to be distinguishable, but similar enough to it to recede into the background. Here's a sample from Effective C++. There are three links in the text shown. (The red text is used to focus readers' attention on the topic at hand and has nothing to do with links).
A different approach is taken in Agile Web Development with Rails 2E, where link text is a shade of red or pink, depending on whether the link is an internal cross-reference or a URL:
One of my concerns is that when you combine such use of color with other uses, such as syntax-colored code, the result can be chromatically rather busy. For example, here's a page from Programming Erlang where, in addition to black, text appears in pink, grey, blue, brown, and red:
I'm not saying that's too much, but I remember the horrors that arose when multiple font faces became available, so I worry about analogous things happening with font colors.

Incidentally, both Agile Web Development with Rails 2E and Programming Erlang are published by The Pragmatic Programmers which, from what I can tell, is leading the pack in thinking about ways to adapt conventional book authoring and publishing for a mixed-delivery-mechanism world. When I discuss their use of red and pink for links and their use of many colors on a page, I'm not criticizing them, I'm taking advantage of the fact that they're doing things that nobody else seems to be. The publishing world doesn't have a lot of experience with books as electronic entities, so the fact that I chose dark blue for my links and they chose pink and red says nothing about which is better. Five years from now, maybe it will be obvious that pink and red is the better choice. Or that blue is. Or that both are inferior to something else. The only way to find out is to try various options and see which ones work best.

Another color-related aspect of the ebooks I examined was whether they use color in display elements like figures, tables, and sidebars. I found that many books use it cosmetically, but far fewer use it semantically. That is, the use of color to visually set display elements off from the primary text flow or to make the display elements more visually attractive was common, but the use of color to help readers understand the information in the display elements was a lot less common. To my surprise, some of the best examples of such use came from SOA Principles of Service Design, a book whose application of semantically-meaningful color in figures contrasts sharply with its rejection of color for links, screen shots, or code fragments. Here's a sample figure from the book:One of my goals is to learn how to take advantage of color in figures, diagrams, tables, etc., to help get my information across to my readers. Whether I do the work myself (as I've done in the past) or have a professional illustrator do it, I need to break out of a monochrome way of thinking about depicting information. A good way to do that, I hope, is to pay attention to what others are doing and then shamelessly apply the sincerest form of flattery to the matter :-)

Thursday, November 27, 2008

EPub State of the Practice, Part 1: Links

[Here "EPub" refers to electronic publishing in general, not specifically to the epub format for electronic documents. At the time I wrote this blog entry, I was not aware that "epub" already had a meaning for many people in the world of, well, EPub.]

I've spent a fair amount of time recently playing around with various electronic versions (primarily PDF) of technical books in an attempt to get a handle on the special "beyond print" features that are offered. In addition to the PDF versions of my books, I looked at the following books (listed in no particular order):
All of these are in PDF format except for C++ in Action, which is published on the web.

There was nothing scientific about this selection. Some books are available for free, some I had purchased for my own use, and some were given to me by the publisher for one reason or another. I choose to assume they are representative of what is available, though that may not be true. Publication years range from 2001-2008, with newer titles generally offering more features. If you know of other books I should take a look at, please let me know.

My primary interest in looking at these books was to get a sense for the "reader experience," with a focus on (1) things I need to keep in mind as an author in order to be able to offer the feature and (2) "obvious" features that were missing, i.e., things I want to be sure to avoid. An example of (1) would be initial definitions of terms that link to glossary entries (much easier to create when writing the book than to go back and add later). An example of (2) would be URLs in the book that aren't live links.

I generally refer to "obvious" features -- the things readers should be able to expect -- as the "Well, duh" features. At the top of the list is that "linky" things should be live links. That includes TOC and index entries, cross-references within a book, and URLs and email addresses. Readers should be able to take for granted that they can click on all these things and be magically whisked to the right place. As a general rule, most books make most of these things links, but I was surprised that only my books make the "see" and "see also" entries in the index live. Color me picky, but I can't understand why, if I see this in the index,
  transaction, see atomic transaction
I should have to manually slog my way back to the beginning of the index to find the entry for "atomic transaction". Hello, computer! You know what entry I want to look at. Freakin' take me there! (Having gone through the process of making these links live in books never written for electronic publication, I can tell you that it was a lot of work, but failing to support the "Well, duh" epub features would be embarrassing, and I hate being embarrassed.)

I was also surprised to find that only my books make email addresses live. I don't get it. What's the point of putting an email address in a book but not making it clickable to send to it?

As a general rule, page numbers in the indices were live, but they were treated in different ways. If, in the index, you see that "transaction" is discussed on page 44 and you click on the 44, where do you expect to be taken? Some books take you to the top of page 44. Some books take you to the beginning of the content of page 44, meaning that the running header on the page has just scrolled off the top of the window. Still others take you to the precise point on page 44 where you discuss "transaction." (To be more precise, they take you to the point on the page where the metadata causing the index entry to be generated is located.) From personal experience I can tell you that where you go when you click on a page number in an index can be determined by the author, the indexer, the software used to write the book, the PDF generation process, or some combination of these. What I don't know is what the "proper" behavior is. Jumping to the point on the page where the metadata is located sounds like it's the most accurate, but making sense of the text you find there generally requires backing up to at least the beginning of the paragraph to pick up the context of the use. Physical books have this problem, too: if the index refers you to page n, but the discussion on page n is at the top of the page, you often have to start reading on page n-1 in order to make sense of what you read. If you have ideas on how links from index page numbers should behave, please let me know.

I checked only four of the books (beyond mine) to see how footnotes were handled, in part because it's generally not that easy to find footnotes (especially if the book doesn't have any). Of the four, only one made the footnote number a live link, and none had a back link from the footnote text to the point of reference. (My books don't make footnotes live in either direction.) For books with static page breaks (such as PDF) non-live footnotes may be defensible, but where page breaks are dynamically determined and pages may be long (e.g., web pages), links in both directions (a la Wikipedia) seems like a more useful approach. I'm inclined to think that bidirectional footnote links should be provided. After all, if they're not intrusive, readers who don't want to use them can just ignore them. If the links are missing, of course, and a reader wants them, that reader is out of luck, and I have a strong disinclination to render my readers unlucky.

Indices in Ebooks

I've been looking through a number of ebooks recently, and one of the things I've been examining has been each book's index. Assuming the book has an index. Some don't. Which raises the question: given full-text search (e.g., via book-viewing software or desktop search tools), do indices continue to be useful? Or are they an artifact of print technology that makes little sense in an electronic environment?

I'm committed to producing indices for my books, because one of the output media I target is print. Indices in print are of proven value, so I'm on the hook for them regardless. But I think they make sense even for electronic-only publication. The reason is that good indices reference not just text, but concepts. Fastware!, for example, is about how to write software that runs quickly, but there are lots of words that correspond to that idea: speed, efficiency, performance, scalability, responsiveness, latency, etc. If you're interested in reducing memory latency, and I happen to say something important in a passage where I'm discussing how to improve the hit rate of the instruction cache, you want the index entry under "latency" to point you to that passage (or, more accurately, I want the index entry under "latency" to point you to that passage), even if I somehow never manage to use the word "latency" in the passage.

Many technical books have awful indices, a situation I attribute to the facts that (1) most indices are prepared by professional indexers, who typically have no understanding of the book's content -- they literally don't know what many of the nouns and verbs mean; (2) these indexers are paid unbelievably badly (typically only a few hundred dollars to index technical books of up to a thousand pages), so they have little incentive to do more than a cursory job; and (3) the quality of a book's index doesn't seem to affect sales, so there is no economic incentive to change the situation. My guess is that as ebooks become more common, indices will fall by the wayside, because it will be easy for authors and publishers to reason that textual searches obviate the need for separate indices, and given the sorry state of most indices, this will probably be true. I'm an old-fashioned guy, however, and I think that a good index improves the usability of a book, and I also believe that the whole point of a book is to serve the interests of its readers, so for the foreseeable future, I plan to produce indices for my books, even though index preparation is, to be honest, probably the single most unpleasant part of writing a book.

So I'm going to produce an index for Fastware!, and that takes me back to page numbers. In an earlier blog entry I worried about the problem of referring to page numbers in a book that may be published in multiple forms, hence have multiple sets of page numbers. Two people commented that the solution is simple: refer to something like section numbers or paragraph numbers instead of page numbers. This is clearly the correct approach, but think of what this means for an index. A single index entry often corresponds to multiple locations in the book, which is traditionally represented as a list of page numbers. If page numbers go away, and if we assume that locations in a book are represented in the form c.p, where c is the chapter number and p is the paragraph number within that chapter, we end with index entries that might look like this:
  containers, standard
C++ 4.3, 4.55, 5.18-22, 7.23
C# 4.4, 4.80-99, 7.65
Java 4.3, 4.60, 5.22-25
It looks a bit odd to me, but I can't think of a reason why there is anything wrong with it. Can you?

Of course, we could also use the form c:p, which would make references look somewhat biblical, hence possibly enhancing their appearance of authority :-)

Friday, November 14, 2008

Beyond Static, Passive Presentation

I use Firefox for browsing, falling back on IE only when I find a site that doesn't seem to work properly with FF. I happened to be using IE the other day when I visited Jim Crafton's CodeProject article on using DocBook on Windows, and I noticed something I'd never seen before: the ability to copy code examples from the article to the clipboard with a single mouse click:

The "Copy Code" option doesn't exist under FF, alas, but the idea of making it easy for readers to work with an electronic book in a natural fashion is a good one.

In fact, it's an example of a more general idea: presentation of a book's content should take advantage of the natural capabilities of the presentation medium. Another example is also shown in the image above: the ability to dynamically collapse and expand a code fragment. Such presentation capabilities don't affect the fundamental content of a book, but authors do need to keep them in mind, because authors can often do a better job of specifying the natural "copy to clipboard" or "collapse/expand" chunks. (At CodeProject, the chunks are presumably determined by the content in some HTML block.)

The fundamental idea is this: although the content of a book is essentially static, the presentation of that content need not be.

Sunday, November 9, 2008

What can go in a Book?

I explained in my last entry that an ink-on-paper book (i.e., a bookp) is simply a physical manifestation of the book content (i.e., bookc) an author has produced. Other manifestations offer different characteristics. Publication as PDF (to be viewed on a traditional monitor) allows the use of multiple colors with no greater cost than the use of black only. Publication as an audio stream eliminates concerns about page breaks, but makes display elements like tables, figures, and code listings problematic. Publication as a web page makes pretty much anything possible: dynamically generated content, full-motion animations and video, interactive elements, etc.

What does it means to write a book (i.e., a bookc), given that you can't assume it will be packaged as a bookp? The question is important, because I can't very well author a book if I don't know which content forms I'm permitted to include and which I'm not.

My answer to this question, perhaps counterintuitively, is based on a bookp. Whatever it means to write a "book," the result should be recognizable as what we currently understand a book to be. We could call TV "radio with pictures," but we don't, and we could call theatrical plays "live TV," but we don't do that, either. I don't see any sense in defining "book" such that somebody familiar with a bookp wouldn't be able to see the connection between what they know and what I've defined.

So here's my initial working definition of a book (i.e., a bookc): it can be reasonably represented as a bookp. That is, if I say "I'm writing a book," you can assume that whatever I produce can be reasonably represented in ink-on-paper form. So we're talking static content. All the usual book stuff is included, i.e., text, diagrams, tables, pictures, etc. Dynamically generated content is out. So are interactive elements. But audio, video, and animations may make the cut, depending on the form they take.

A video of a talking head, for example, can be represented in a bookp as a frame from the video (i.e., a photo of the speaker) accompanied by a transcript of what the speaker says. Readers lose the sound and cadence, etc., of the speaker's voice, and they're deprived of seeing how the speaker's face moves as he or she talks, but -- assuming such information was never the point of the video -- the essential content has been preserved in ink-on-paper form. Talking head videos are thus permissible in a bookc (and would be delivered as such on output devices where that's possible).

Similarly, transcripts of the audio-only equivalent of talking heads ("speaking voices?") make such audio permissible in a bookc. Many podcasts could thus be considered bookc manifestations. (If books in recorded form (i.e., audiobooks) are still books, then textual representations of speech are still speech ("textaudio"?), and since textual representations of speech can be published as recognizable bookps, recorded speech is legit in a bookc.) The way people express things in spoken versus written form typically differs qualitatively, so transcribing spoken audio and publishing it in book form is likely to yield a lousy book, but my goal here is to figure out what's in my author's toolbox and what's not. If something's in, part of my job as an author is to make sure I don't just use it willy-nilly; I'm responsible for using it well.

I find animations to be a particularly interesting case. As a book author, may I include animations? Consider, for example, David Howard's animation of the behavior of a red-black tree. Can such an animation be part of a bookc?

I'm inclined to think that it can. Books have long contained "before" and "after" diagrams to help explain how a transformation between two states occurs. You'll find several in the Wikipedia entry for red-black trees. (Well, you will if you look right now. What the page will look like if much time has elapsed between when I write this and you read it is anybody's guess.) It's easy to imagine such diagrams being frames extracted from an animation. If, as an author, I have enough conditional content control to be able to say
  if (rendering for a device that can show animation)
show this animation along with this explanatory text
else
show these animation frames along with this other explanatory text
then at least some animations are within the purview of a bookc.

The set of acceptable entities in a bookc is thus a superset of those that are directly expressible in a bookp. The information content of everything in a bookc must be representable in a bookp, but if a trivial tranformation needs to be applied (e.g., video is replaced by a selected frame, audio of speech is replaced by a transcript) or even if a nontrivial-but-straightforward transformation needs to be applied (e.g., animation + description replaced by animation frames + alternate description), such content is still valid, hence part of my toolkit as a book author. Writing a multiple-platform bookc thus gives me more choices for expressing myself than I'd have if I restricted myself to bookp publication.

Saturday, November 8, 2008

Bookp versus Bookc

Consider these two statements:
  • I wrote a book.
  • The book I wrote is on my shelf.
Book does not mean the same things here, and increasingly I feel that this ambiguity causes problems. Technological changes are causing the meanings to drift further apart, so I think it's important to clarify the two meanings that book can have.

I'll call the meanings bookp and bookc. A bookp is a physical object. It consists of pages of paper with ink on them. The pages are bound together and held between two covers. Bookps are what bookstores and libraries are filled with.

A bookc is not a physical object. The c stands for content, and if we assume that the content of a book is a simple stream of text (which is essentially true for most novels), bookc is that stream of text. The text might get printed on pages that are bound together, thus yielding a bookp, but the text might also get spoken aloud and recorded as an audiobook. It might get distributed over a set of web pages, thus forming a web site. The content of a book is independent of its packaging, and in fact packaging is what the p in bookp stands for. A bookp is simply one of many different ways of packaging a bookc.

Authors don't generally write bookps, although I suppose those who self-publish and keep boxes of books in their garage do. Rather, authors write bookcs. The semantics of the statements above, then, can be depicted this way:
  • I wrote a bookc.
  • The bookp I wrote is on my shelf.
With this distinction in mind, consider Doug McCune's statement that he doesn't read books or Joel Spolsky's thesis that programmers seem to have stopped reading books (both of which I found out about thanks to Jeff Atwood's blog.) I think these statements refer to bookps, not bookcs, and that opens the door to the possibility that people who claim to not read books or people who appear to not read books actually do read them, they just don't read them in bookp form.

Like Mulder, I want to believe, because I happen to like writing books (i.e., bookcs). Other entries in this blog make clear that I think a lot about bookps, but that's simple pragmatics. The publishing industry is changing, but it's currently set up to produce, market, distribute, and sell books in printed form. Remaining mindful of authoring constraints arising from printing considerations is no different from remaining mindful of software constraints arising from Windows considerations (assuming, in both cases, you want to maximize the number of platforms on which you can deliver what you produce).

Authors who leave all the layout decisions to their publishers (i.e., most of them) worry only about bookc considerations. Authors such as me who can't keep themselves from delving into layout matters have to keep bookp issues in mind, but it doesn't change the fact that, fundamentally, writing a book means writing a bookc.

The Post-Publication Page Break Problem

Traditionally, books are published, and that's that. As I've noted before, however, I typically modify my books for new printings, so the content changes over time. Each change is small and localized: an added, removed, or rewritten sentence here; a touched-up code example there; the addition of an occasional footnote; etc. From a software development point of view, new printings are the publishing equivalent of bug fix releases.

Sometimes even small, localized changes can have extensive implications. Adding or removing a sentence on a page can cause the page breaks for that and subsequent pages to change, and when page breaks change, TOC and index entries can change, too; content that used to be on page n might now find itself on page n-1 or n+1. (There are scenarios where it can change even more than that, but we don't need to worry about those.)

Assuming the existence of a single-source automated build process for the book, each printing should be completely consistent (i.e., all TOC and index entries should be correct), but even overlooking the fact that a fully automatic pagination process might yield unfortunate page breaks, we still have a problem.

Readers shouldn't have to worry about which printing of a book they have. If Person A has a copy of Fastware! and Person B also has a copy of Fastware!, they have the same book, as far as they're concerned. The fact that there might be minor differences between the two because Person A happens to have the first printing and Person B has the tenth shouldn't matter to them. Heck, people have enough trouble remembering that different editions look different. It's not reasonable to ask them to remember that different printings of the same edition might, too.

Given that they think they have the same book (even though they might not), I want to maximize the likelihood that if Person A says something like, "As you can see on page 44, where Meyers brilliantly demonstrates that a cache-unfriendly traversal can have a significant impact on performance...," Person B can go to page 44 and find the passage Person A is referring to. The way to maximize that likelihood is to ensure that once a book is published, page breaks change as little as possible. So if between the first and tenth printings, I removed some text from, say, page 18, it's better to let page 18 be a little short (or increase the interparagraph or interline spacing on the page) than to have some text from page 19 move onto page 18 (plus the associated potential cascading text movement on subsequent pages). Similarly, if I add text to a page, it's best to prevent any existing text from moving across a page boundary.

Practically speaking, once the first printing of a book goes out, I want the page breaks to remain static, even if I tweak the content of the book such that the page breaks would fall in different locations if I were to repaginate from scratch. Not only would this help preserve the illusion that Person A's first printing and Person B's tenth printing are the same book, it would also preserve any hand-tweaking of page breaks that had been done prior to initial publication (as I discussed in my earlier blog entry on page breaks).

Which leads to the inevitable question: how can I preserve initial-publication page breaks across multiple publication platforms (i.e., print, epub, etc.), use an automatic build process, and still preserve the ability to modify content for new printings?

Friday, November 7, 2008

XML ¬⇒ ¬LaTeX!

One of the attractions of LaTeX is that it produces wonderful output. In all likelihood, TeX employs the best line/paragraph/page layout algorithms in the business. Furthermore, LaTeX is terrifically expressive, something I know even though my familiarity with it is limited. (I did a lot of writing with LaTeX in my academic days, but I tended to learn only enough to do what I wanted to do. Unlike many Computer Science graduate students, I never really sat down and studied it.) I know, for example, that LaTeX allows authors to float displays (e.g., figures, tables, listings, etc.) to the top of a page, the bottom of a page, or both. In fact, this is the default, meaning that unless authors disable such flexibility, LaTeX will float displays to the next "good" location. (Experienced LaTeXies know I'm lying a bit, but it's close to the truth, and the actual truth doesn't add anything here.) This behavior seemed so natural and obvious to me, I took it for granted for many years. Only when I started using less capable systems (among them FrameMaker, OpenOffice Writer, and, from what I can tell from varous comments on the web, Microsoft Word) that offered much more limited support for floats, did I realize I'd been spoiled by LaTeX.

Because LaTeX is so expressive and its output is so good, I was reluctant to dismiss it in favor of an XML-based approach such as DocBook. But then, in one of those Duh! moments that happen now and again, I realized that as long as there is a way to take a DocBook document and transform it into a LaTeX document, I could use XML as my document representation and LaTeX as my print rendering engine. In fact, I'm pretty sure that this is what the Pragmatic Programmers do: write books in PML (their book schema), then transform them into LaTeX source for ink-on-paper rendering.

Assuming this line of reasoning is valid, DocBook becomes a pretty appealing option, because it means I can take advantage of all the work done by and tools developed for the DocBook and XML communities, but I can still hold out hope for the layout quality of LaTeX. Such hope can exist only if DocBook offers sufficient expressiveness, however, because there's nothing to be gained by the possibility of using LaTeX as a rendering engine if I can't express what I want it to do via DocBook. My next step, therefore, is to install DocBook and see if it will let me say what I want to say. I already have a pretty good idea of what I want Fastware! to look like, because I've done preliminary page layout in both OpenOffice and FrameMaker. If I can get DocBook to produce the output I want, I can then try translating my recent C++ article into DocBook, which will be a good way to see if its support for floating displays, cross-references, and bibliographies is up to snuff. If so, DocBook will look like a reasonable book representation format, and I can move on to seeing whether I can find a decent WYSIWYG front end for it.