Tangled in the Threads

Jon Udell, January 10, 2001

Jabber and XML Messaging

The endgame is "an all-pervasive message plasma"

Jabber's universal chat capability is cool, but its focus on XML-ized metadata, content, and interfaces is what really justifies the buzz.

In a column last year on computer telephony, instant messaging, and metadata, I included some newsgroup discussion about Jabber, an open-source IM gateway and messaging platform that's really gaining a lot of traction lately. What was clear, in that discussion, is that IM's innovation is as much cultural as technological.

The other day, lurking in a mailing-list discussion, I was again struck by the cultural divide that separates email from IM. A younger guy, just out of college, argued that IM is a productivity booster because:

A couple of older guys chimed in with these points:

Of course everybody's right. But the chasm separating the two camps is almost as wide as the one that divides email users from non-users. I'll admit that the older-guy arguments tend to resonate with me. If I'm reading, writing, or programming, I don't want to be interrupted even once per hour. I want to poll for messages, not be notified. And yet, I'm enormously frustrated with the limitations of email. Part of that frustration is addressed by the IM presence-management protocols, which when coupled with smart filters will become even more powerful than they already are.

Mark Wilcox:

There's really very little difference between email and IM. If you're sending email to a person via a fast gateway, the messages appear almost instantaneously.

What sets IM apart, for the user, is the concept of presence, which we've just started to touch upon. Presence is useful to learn things like "Jon is online, but is busy." But it could be modified with filters to say "But I'm available to chat with anyone on the O'Reilly Email book team." This should even be doable with current Jabber filters and presence.

Filtering technology needs to improve dramatically. I just don't think people grasp the concept of email folders, or else the technology is too hard to use. In any case, people do want to filter their messages (whether email or IM) more efficiently and naturally.

Software should be smarter. For example, it should realize that I read all email from Jon Udell more quickly than I do the latest message from HR. Thus it could bubble all of Jon's messages to the top.

Indeed. And this smarter messaging client should watch and then offer to emulate my filing habits. As an independent consultant working on a variety of projects for different clients, I spend a lot of time organizing both inbound and outbound messages by project and topic. Yet often this effort falls just below the threshold at which I'll create/maintain a filter. Communication is often just too fast and free-flowing.

Why, in the year 2001, don't we yet have messaging software that sees patterns in our communication behavior and asks (politely) if we'd like to do something about them? The answer is simple: our software doesn't allow us to express those patterns, easily and naturally, as we communicate. Even here in the future, there's no free lunch. The smart client that we all dream of can't make something out of nothing. It needs some hints -- metadata hooks that it can use to understand and work intelligently with the "five Ws" (who, what, when, where, why) of electronic communication. Email messaging deals with some of this, but it lacks the ability to represent such routine, yet complex, scenarios as:

Evolution or revolution?

I've hoped, for some years now, that email technology would evolve in the right direction. It's not hard to imagine straightforward extensions to email infrastructure that would enrich the standard suite of message metadata elements, enable users to define their own custom metadata, and enable developers to build new kinds of applications that leverage the enriched metadata.

Here's a concrete example of what I mean. Lately, for a development project I'm working on, I've been intensively using an excellent open-source app called Request Tracker. It's an issue tracker (aka trouble ticketing system) that, like others in its category, is implemented as a web/email hybrid. (It has, as well, a command-line interface.) I highly recommend Request Tracker (RT). Bugzilla, the first issue tracker considered for this project, was ruled out for the complexity of its installation and maintenance. RT was (I'm told -- I didn't do this myself) a lot easier to deal with. And it does what you need: it manages message threads in a context of categorized queues with numbered items that can be 'open' or 'resolved'.

As is typical of this class of application, the email Subject: header is a key point of integration. If you want to comment on a specific item, the format is: 

Subject: [<Site ID> #<request number>] (<queue name>) <subject>

When RT sends mail to you, replying is all that's needed to capture this metadata and ensure your reply goes to the right place, rather than -- awkwardly -- creating a new and unrelated item. In theory, you can also initiate messages using this format. And this could be really useful. Often, in an interpersonal message, you'd like to Cc: the RT app to attach the message to some item in a queue. But in practice, it's nearly impossible for a human, on the fly, to produce the exact syntax required by RT. The alternative is to go into RT's web interface, find the item in its queue, and reply from there. This works, but interrupts your flow. RT isn't, after all, your primary message client. This dilemma isn't particular to RT, it afflicts all web/email hybrid apps.

If there were a simple solution to this problem, it would already be solved. But there isn't. The root cause is email's inability to capture, transmit, and react to domain-specific metadata. And I've just about given up hoping that email infrastructure is going to suddenly, one of these days, wake up and "get it." Instead, I think the solution will arrive in the context of an entirely new messaging infrastructure. And the leading example of that is Jabber.

Jabber's one-two punch

In my groupware book, I focused heavily on the problem of web/email integration because email is "where we live" -- the application in which we spend much of our time, and through which we express much of our communication. But email's absolute dominance is threatened. For the generation now entering the work world, IM is as central as email and in some cases more so.

Given this dynamic, Jabber is ideally positioned for the kind of viral play that got the first generation of browsers adopted so quickly and so fast. The Jabber server, through gateways, unites disparate IM realms, so that a Jabber user can chat with AIM or ICQ users as easily as with users of Jabber's native IM. HotJabber.com is one of the places where you can go to download a Jabber client, and join in this fun. Add up the millions of users of the various IM services that Jabber can connect, and you can see it has the potential to get really big, really fast.

Mind you, the prospect of chatting up AIM and ICQ users doesn't excite me in the least. But here's what does: Jabber, if it succeeds in becoming pervasive, will lay the foundation for messaging applications that I yearn for in the worst way. A smoother interface between the message-based and interactive components of an app like Request Tracker would be of huge significance to me, to anybody involved with software development, and indeed to anybody involved in project management of any kind.

Another example: imagine applying Jabber's backend plug-in component architecture, and its XML messaging, to the problem of group calendars. TimeDance, sadly now departed from the scene, was a web/email app that I greatly admired for its implementation of the messaging protocol, information architecture, and application features needed to usefully support group scheduling. Because of email's limitations, TimeDance was most effective in web space. But this needn't always be the case. Application-like features are, inevitably, going to find their way into our messaging clients where they belong.

Suppose a calendaring backend were implemented as an extension to the Jabber server. Mark Wilcox sketches how that would be done:

  • The jabber server (aka jabberd) is simply a message router. It doesn't really have any smarts. In your jabber server configuration all of the plugins/modules tell jabberd which namespaces and/or server IP addresses they're interested in receiving. Jabberd simply handles routing of the messages from the clients to the modules, called xdb's in Jabber parlance.

  • An xdb can be written as either a C shareable module (basically like an Apache module), as a persistent executable that communicates via STDIN/STDIO or as a standalone server (so components can live on a server farm).

  • You configure jabberd about your component.

  • Restart jabberd and it will now route messages to your component.

What would this hypothetical component do? Perhaps everything that TimeDance did: invite people to a meeting, manage a group discussion about where and when to have the meeting, organize responses into categories ("Attending"/"Not Attending"/"Not Sure"), interface to services that deliver maps and driving instructions. This, in itself, doesn't advance the state of the art. You also have to imagine a client-side component that extends the messaging client in ways specific to the group-scheduling protocol. This won't help much if a Jabber-like messaging client doesn't become the primary communication tool for a whole lot of people. But if that does happen -- and I'll hazard a prediction that it will -- we'll finally get the kind of enrichment of messaging space that we've enjoyed in web space for years.

Here's a great closing comment from DJ Adams, who's written an excellent tutorial on using Perl's Net::Jabber library to whip up IM-and-presence-enabled applications:

The combination of (relative) ease of development for Jabber 'clients' (I use the word 'clients' in the widest possible sense, though 'participants' might be a better choice), thanks to the great thoughts that have gone into the Jabber architecture design, and the XML coursing through its veins, blurs the boundaries between generic user (that is, user-interface-based) communication tools, and connectors for services, and app-specific clients. Put another way, the Jabber glue provokes you into considering all endpoint types as equal players.

Although perhaps the much-touted all-pervasive desktop environment hasn't materialised, there's a chance that an all-pervasive message plasma will.

I just love the ring of that! Yes, it's about time for an all-pervasive message plasma, provided that it works with and for us, not against us. What will make that possible is a common way of representing message metadata, message content, and component interfaces. Jabber's universal chat capability is cool, but its focus on XML-ized metadata, content, and interfaces is what really justifies the buzz.

Jon Udell (http://udell.roninhouse.com/) was BYTE Magazine's executive editor for new media, the architect of the original www.byte.com, and author of BYTE's Web Project column. He's now an independent Web/Internet consultant, and is the author of Practical Internet Groupware, from O'Reilly and Associates. His recent BYTE.com columns are archived at http://www.byte.com/index/threads

Creative Commons License
This work is licensed under a Creative Commons License.