AMQP and XMPP

2007-12-07

A while back the geek blogosphere was briefly atwitter about comparisons between the Extensible Messaging and Presence Protocol (XMPP) and the Advanced Message Queuing Protocol (AMQP). Now that the dust has settled a bit I figured I'd provide my two cents, as a far-from-unbiased observer.

First, AMQP was specifically designed to replace technologies like MQSeries and Tibco (the investment banks don't like paying large yearly fees to middleware vendors, so they decided to roll their own). It is a special-purpose technology that may get used in the financial space, but I doubt that it will ever break out of that niche. It has not received widespread review from technologists (especially regarding security and internationalization), as XMPP did when we formalized the core Jabber protocols at the IETF (though to be fair, neither did the core Jabber technologies back in the early days). And as Bill de Hóra notes:

AMQP is a complicated, single purpose protocol, and history suggests that simple general purpose protocols get bent to fit, and win out.

Second, Stephan Tilkov seem to think that it's inherently faster to use binary (AMQP) than XML (XMPP):

One of my motivations, though, was that XMPP is based on XML, while AMQP (AFAIK) is binary. This suggests to me that AMQP will probably outperform XMPP for any given scenario — at the cost of interoperability (e.g. with regard to i18n).

This assertion is commonly made, but there is no evidence for it. In particular, XMPP uses a long-lived XML stream with a single XML parser instance, which means you don't need to instantiate a new XML parser every time you process a message. This provides a significant speed improvement over sending a large number of individual XML documents. In addition, we can apply compression to the XML streams (either at the TLS layer or the application layer), reducing bandwidth usage significantly.

Third, Tim Bray says that one of the "problems with Push that don’t arise in Pull" is: "what happens when the other end gets overrun, and you don’t want to lose things?" Well, here we get to the first "P" in XMPP: presence. If an endpoint "falls over" so hard that the subscriber process dies in a publish-subscribe scenario, then that subscriber goes offline and its presence is propagated to the pubsub service. The pubsub service can then react accordingly, for example by terminating message delivery until the subscriber is available again. Yet if the pubsub service keeps a history of the notifications it sends out, then the subscriber is able to retrieve the history if necessary when it comes online again. Thus nothing is lost. The power of presence differentiates smart push from the dumb old push of "send it out and hope it gets there".

In XMPP we have more than just presence for reliable message delivery (I avoid the term "guaranteed delivery" since that has just about as many interpretations as the term "security"). For instance, in the advanced message processing (AMP) extension to XMPP you will find methods for making sure that the message is indeed delivered. Support for that extension has just been added to the ejabberd XMPP server (written in Erlang) and is probably on the way in several other XMPP server codebases.

Fourth, Tim Bray also asks "what happens when all of a sudden you have a huge number of clients wanting to be pushed to?" That's more a matter of deployment and service provisioning than protocol, to be sure. Yet we know that some XMPP implementations will scale up to hundreds of thousands and perhaps millions of concurrent users (via clustering, high availability measures, and the like). Plus, XMPP is a decentralized technology in which instances can be federated, thus spreading the load over the entire network.

Finally, none of this means that some companies won't deploy AMQP. However, I agree with Bill de Hóra and Mike Herrick that simplicity wins out. And XMPP is a powerfully simple technology. As Bill says, "XMPP's potential is huge; I suspect it will crush everything in sight just like HTTP did."


Peter Saint-Andre > Journal