Open Source, Open Standards, Open Data, Open Vancouver

Exciting Vancouver news!  Mayor Robertson has put forth a motion for city council to vote on next week which is chock full of amazing words, and which passed, will direct the city to have a bias towards openness — open source software, open standards, and open data.

That’s pretty impressive!  If the motion passes (which it should, riding on a global wave of sentiment towards openness, and fitting in with the platform that got seven of the councilors elected), this could mean great things for Vancouver, especially at the intersection of software, business, and the public.

On the issue of open source, I would love to show that local governments are able to recognize the strategic and control advantages inherent in software that they can influence and modify, and help push back the fear-driven campaigns which bias towards monopolies at taxpayer expense.  Similarly, promoting the use of open standards is a no-brainer that the best technocrats realize can give them the power that befits them as customers.  These ideas have been well articulated globally over the last few years, and I would hope that all high-level government staff and officials are briefed on the topics by now.  (If any local officials want to discuss this in greater detail, there are many qualified experts in Vancouver, don’t be afraid to ask for names or opinions!).

Open data is a more recent concept, the implications of which are likely as important as the rise of the web.  With open data, governments have a unique opportunity to create economic growth, reduce operating costs, and enrich the life of their constituencies, simply by making a policy decision such as the one in tuesday’s motion, and following through.

As Sir Tim Berners-Lee (the creator of the web) discusses in this 15-minute TED talk, the simple act of releasing public data enables others to create value.  Of course, as the motion indicates, personal privacy rights trump, and we don’t want to release data on individual citizens — luckily that’s not needed in order to enable value creation.  As an example, this impressive screencast of Wolfram Alpha demonstrates the power of new computational platforms leveraging public data. Vancouver’s data belongs there.

Most government data is public data by definition.  What’s compelling about open data in the age of the web isn’t the fact that citizens have access to such data — they typically have the legal right to obtain it through administrative requests, even though those are inconvenient (and very expensive for the city).  What’s compelling is that by making what belongs to the public available via the web, the city can accomplish many laudable goals at once:

  • In many cases, simply enabling self-service on the web will reduce costs for the city and provide better service to its citizens.
  • By making data that it doesn’t have time to process and analyze available, the city allows others with time and expertise to do such analysis with no cost to the city.  This will sound unbelievable to bureaucrats unused to open source, but this kind of thing really happens.  You can’t predict who will do what with what data, but you can be sure that it can’t happen unless and until the data is available.
  • Some of those activities will just be interesting. But some will create new businesses, or allow existing businesses to become more efficient.  What if local retailers could access demographic trend data for free on the web, today?  What if companies outside of Vancouver could get a deeper understanding of Vancouver simply by looking at the data?  Everyone knows that Vancouver is a great place to live.  The city’s economic strengths are not as well advertised.  Enabling an ecosystem of people who turn data into interesting, insightful, and useful applications and sites can only help.  Think of open data as the infrastructure of a chamber of commerce 2.0.
  • The city is there to serve the citizenry.  To the extent that it is the caretaker of public data, and that the public has good ideas for using it, its job should be to get out of the way.  Part of being a transparent government is to be invisible — to not get in the way of experimentation and innovation.  Promoting open data while preserving privacy feels like a great goal for the city’s IT staff.

There are also intangible benefits that come from these kinds of attitudinal shifts in how the city relates to the internet and the software economy.  From a recruitment point of view in the software industry in particular, a city which embraced openness and the internet would be that much more attractive to the kinds of technical, creative, and public-spirited individuals that I seek.

Finally, local technology leaders are that much more likely to engage with the city and provide their help.  I know that the notion of an “Open Vancouver” makes me much more keen to engage with the city, as it would put the city on the short but growing list of governments who understand how they can leverage the web and openness to improve life for their constituencies.

Support job with Mozilla Messaging

frustration, from e-magic on Flickr
"frustration", from e-magic on Flickr

One of my first jobs in IT was as a “computer consultant” for my university.  I got to learn a lot about computers of various kinds (including currently useless but still formative bits like writing REXX programs on a CP/CMS IBM 3090 mainframe), and, more importantly, I learned a lot about what it takes to really help people solve computer problems.  We had all the kinds of users you’d expect in a decent-sized community: haughty faculty (and haughtier grad students, for some reason), who absolutely demanded that you stop whatever you were doing to help them fix their margins <em>right now</em>.  We had to mediate between first-year students lost and confused in their first exposure to the net, and technical staff who were straining under the onslaught of the first week of classes.

Still, it was a lot of fun — we had autonomy to decide what problems were worthy of paper handouts, to hand out as answers to FAQs — we had special accounts with free access to the high-resolution printers — but most of all, we had lots of interactions daily with people who were truly grateful for the help we provided.  Not all interactions were positive, but the vast majority of them were.  There was a real community between the student staff, the full-time staff, and the avid users who spent their days in the computer center working on their papers, homework, and assignments.

The job

These memories come back as we’re now looking for someone to help coordinate Thunderbird’s technical support communities.  The challenge is orders of magnitude harder, but the opportunities to help are equally huge.

Mountain Lion Safety by ekai on flickr
Mountain Lion Safety by ekai on flickr (helpful, no?)

It’s become clear to me that this job is fairly unique, both in scope, and in what we’re looking for.  The right candidate will have a rare blend of empathy & technical knowledge, clear organization skills.  She or he must have the ability to bring people off the ledge, recognize and encourage peer leaders, but also know how to deal with poisonous people.  This requires both clarity of thought and clear, efficient expression.  Our ideal candidate knows Thunderbird well, but most importantly can understand both the requirements of providing technical support for hugely diverse populations of users, who often come for help when they have critical issues that are often caused by external actors like email providers.  A hard job to fill.

A big part of the job is also to understand the existing bits of the existing world of Thunderbird support, and figure out ways of connecting the bits that work, and building new bits if needed.  From where I’m standing, this ecosystem includes forums like mozillazine, which have garnered huge numbers of posts with very valuable information, but which don’t necessarily provide the best experience for novices.  It also includes the GetSatisfaction forums, which provide a lower barrier-to-entry for many users, but which are still in their infancy, both in terms of content and in terms of people providing answers.  The ecosystem also importantly includes SUMO, which right now is focused on Firefox, but which could hopefully be deployed for Thunderbird use someday.  A comprehensive solution also likely involves figuring out how to help people over Twitter, on Facebook, or wherever Thunderbird users are likely to look for answers or express frustrations.  Whoever gets this job will end up diving deep into the world of Mozilla support overall, and learn from all of the people who’ve built those foundations — then build upon them.

As the central point of contact for support issues, this person will be incredibly helpful in working with the QA and dev teams to prioritize bugs and other issues, so that the next revisions of Thunderbird or Thunderbird websites work better for more people.  Finally, it would be great to coordinate with the support micro-communities that exist around the world.  In general, the job is clear: help people with Thunderbird, and help people help each other.

If this sounds like you, send us an email, explaining why you’re the right person for the job.

Getting insight into one’s own email

Thunderbird knows a lot about your email. After all, it’s got access to large amounts of data, and builds sophisticated databases so that it can be very responsive, even when dealing with large folders containing thousands of messages. Wouldn’t it be nice if we could use this information to extract either faster workflow, or even insight into our email habits?

On Tuesday, as he was slogging through some somewhat antique code in the front-end of Thunderbird, Andrew asked whether Thunderbird’s “start page” was what we wanted it to be, which is Andrew-speak for “this is broken, we should fix it”. We’d actually discussed what to do with that page for a long time, but had focused on other, more infrastructure bits for a while. I decided it was time to dust off those plans, and see what we could do.

First, some background. The “start page”, which makes a lot of sense in Firefox, never made a huge amount of sense to me in Thunderbird. In particular, it’s shown only when a folder is selected, and no message is selected. That’s hardly a logical time to show the (colorful, pretty, but fairly useless) page we show now. Instead, why not show information about the selected folder, and help people who clearly intended to select a folder, so most likely wanted to do something related to that folder!

We could still use a small part of the pane to display interesting news snippets, as a way of keeping users aware of new developments, such as new add-ons that we want to promote, or Mozilla events. In fact, by making that page useful, we’re more likely to get people to read a sentence or two which might change periodically, as opposed to changing a page that is so big, wordy and useless, that I suspect most users are completely blind to it.

A couple of days later, we have an early patch, which feels pretty compelling to me:

  • It summarizes critical data about the folder (name, number of messages, number of unread messages);
  • if you were recently reading that folder, it points out the last message you were reading, so you can quickly find it again;
  • it shows you a few of the messages that are most likely of interest (because they’re unread or starred);
  • and finally it gives some information about the activity in the folder: a histogram showing activity over the last 52 weeks, and most prolific authors and most active threads during that same period.

For example:

Folder summary

The icing on the cake is that if you hover over an author or a thread title, it will highlight when those messages occurred in the histogram (in this case, the “An alternate take on HTML 5” thread). Readers of m.d.platform won’t be surprised at any of the names that show up above!

There are lots of possible additions: Some of the ones we’ve thought of include:

  • showing how much disk space is used by the folder
  • showing the “largest” messages
  • showing the status of autosync/indexing jobs

Note: this hasn’t been through a visual design phase yet — the look will change, once someone who knows what they’re doing has a chance to make it nicer! There’s also a lot of work to do to make sure that it does the right thing for saved searches, smart folders, etc. And yes, we’ll make it optional, just like the start page is now.

I’ve been playing with it for only a little bit, and it’s fascinating how interesting it is to get some data on one’s email. For example, I more-or-less follow a small-inbox model, where emails don’t stay in my inbox if I can avoid it — they get deleted, or archived. As a result, the sparkline for my inbox is:

inbox sparkline

which isn’t as short as it should be, telling me pretty quickly that I have some old messages that need attention. It’ll be interesting to see what else we learn from these kinds of simple personal analytics.

What other kinds of visualizations, summaries, and analysis would you like to see in Thunderbird, or in add-ons?

[Update]: I liked the idea that a commenter suggested of showing what tags were in use in the folder, so I had some fun, first with a “tagpie”, which proved mostly useless, just like most piecharts; then with a “tagdots” small-multiples visualization:

tagpies, tagdots, sparklines oh my!

I don’t think the tagpie is worth keeping, but the dots are tempting. As an aside, the Protovis library really is as easy to use as I’d hoped.