The twittersphere is abuzz with the current twitterstorm about Microsoft’s plan to use the “Word HTML engine” in the next version of Outlook.  It’s a campaign that’s an organization which represents people whose living depends on their ability to make compelling HTML pages in email, so it’s not surprising that they have a beautiful site which is getting a lot of people to retweet.

There are lots of campaigns that sweep the social networks on a regular basis, and this one is somewhat noteworthy because it’s about plans for a very commonly used piece of software, coordinated by marketers, and because the twittersphere is very receptive to anti-Microsoft sentiments.  None of that is what I want to talk about.

What I want to dig into a bit is how Microsoft got there, and the implications for the Open Web.  I’m not an expert on Microsoft’s history, or Outlook.  But I can make a few guesses, based on how I’ve seen similar things evolve.

Outlook became the dominant enterprise email client during a phase of Microsoft’s life where embracing the web sometimes meant making stuff up and pretending it was a standard, or equivalent shenanigans.  This was clear in Internet Explorer’s explorations outside of the normative specs, but it seems that some of the same “we can just do our own version of HTML” affected the Word team.  This makes sense — if you’re a company with market dominance and the web is not central to your value proposition, but office productivity software is, then you’re going to do what you can to make the best user experience possible for your users, even if it means that messages sent to non-customers can’t be read with as much fidelity as those sent to customers. In fact, in a very basic way, that’s standard marketing — make using your product look better, so people want to use it.

Microsoft, again logically, invested lots and lots of millions of dollars into making design tools for Word, and HTML was thought of as an export format, where low-fidelity was almost a commercial virtue (“you don’t really want that”).  The poor folks in charge of Outlook, who are mail experts, not HTML rendering wizards, had to deal with the use case of: “I want to send rich documents by email”, which blended office concepts (rich documents) and network concepts (email).  They had to choose between a moribund IE6 engine, and the maintained, evolving HTML engine designed for use in Word.  Given that most emails read in Outlook probably are written in Outlook and that Outlook users know the Word authoring tools, it was a rational choice.  It made life hard for email marketers, and for a few people who like to use HTML to express their creative side and who do care that all their correspondents can see what they intended to send.  But compromises are inevitable in a gigantic, complicated company like Microsoft.  Had I been the manager in charge, given their constraints, I may well have made the same choice.

Now, it’s 2010 (or almost).  Outlook is due for a new revision (gotta get the upgrade revenue).  The choice is stark: adopting a more standards-compliant engine like IE8’s makes sense in the framing of “html email messages going out on the net”, but to deploy it in the reality of Outlook (mostly internal emails, lots of document ping-pong, etc.) it would require that Microsoft have a stack of design tools to offer that could realistically replace their existing stacks.  There’s the rub — good HTML engines aren’t useful in a user context like Outlook’s if the authoring tools weren’t built with real HTML/CSS in mind.  And neither Word’s venerable composition tools or  Silverlight’s new-fangled ones were.  So the Outlook team is stuck with a product that needs an upgrade and a need for both composition tools and a rendering engine, neither of which it can control.  It’s not going to end well for at least some people.

[As a side note: the pragmatist in me wonders whether Outlook could use the Word HTML engine to render emails from Outlook users, and the IE8 engine for emails not from Outlook users.  As long as no one ever edits forwarded emails it’d work!]

Now, it’s awful easy to make fun of Microsoft.  The story on the side of the Open Web is better in part, but there are areas needing improvement.  On the rendering engine side (displaying beautiful documents with fidelity and speed), the world is looking better than it has in years, with several rendering engines competing in healthy ways like standards compliance, leading-edge-but-not-stupid innovation, performance, and the like.  Life is good.  For email marketers, getting email clients to render real web content is all that matters — they pay professional designers to author their HTML content using professional web page composition tools, and the revenue associated with a successful email marketing campaign makes those investments worthwhile.  Email is just a delivery vehicle to them, and it’s a perfectly valid perspective.  They like Thunderbird a lot, because we’re really good at rendering the web, thanks to Gecko.

However, for regular folks, life is not rosy yet in the Open Web world.  Authoring beautiful HTML is, even with design and graphics talent, still way, way too hard.  I’m writing this using WordPress 2.8, which has probably some of the best user experience for simple HTML authoring.  As Matt Mullenweg (the founder of WordPress) says, it’s still not good enough.  As far as I can tell, there are currently no truly modern, easy to use, open source HTML composition tools that we could use in Thunderbird for example to give people who want to design wholly original, designed email messages.  That’s a minor problem in the world of email, which is primarily about function, not form, and I think we’ll be able to go pretty far with templates, but it’s a big problem for making design on the web more approachable.

There are some valiant efforts to clean up the old, crufty, scary composer codebase that Mozilla has relied on for years.  There are simple blog-style editors like FCKEditor and its successor CKEditor.  There are in-the-browser composition tools like Google Pages or Google Docs, but those are only for use by Google apps, and only work well when they limit the scope of the design space substantially (again, a rational choice).  None of these can provide the flexibility that Ventura Publisher or PageMaker had in the dark ages; none of them can compete from a learnability point of view with the authoring tools that rely on closed stacks; none of them allow the essential polish that hand-crafted code can yield.  That’s a gap, and an opportunity.

I think radical reinvention is needed.  Something with the chutzpah of Bespin, which simply threw away most of the stack that we all assumed was needed, but this time, aimed at the creative class (and the creative side in all of us), rather than the geeks. I know that lots of folks at Mozilla would love to help work on this, but we know we’re too small to do it alone.  We know what modern CSS can do, we just don’t know how to make it invisible to authors.

This is a hard task, because it’s about designing design tools, which combines psychological, social, product design, usability, and technical challenges. It’s a worthy task, though, and one that I’d love to see someone tackle, especially if we can get non-geeks involved.  There are tens of thousands of web designers who know the magic triad of 1) design, 2) HTML/CSS, 3) what aspects of existing tools make them productive, and what aspects fail.  If we could get them to work productively with the tens of thousands of open source developers who currently build the applications that power the net (web, email, and others), we could throw away the broken metaphors of the 20th century and come up with new ways of designing using web technologies that everyone could use.  Or maybe we just need one brilliant idea.  I’ll take either.