Over the years we've tried many approaches to end-to-end encryption of Jabber traffic: OpenPGP, S/MIME, straight Diffie-Hellman, even (in some quarters) XML encryption. All the while, we've had client-to-server ("c2s") and server-to-server ("s2s") encryption using SSL and (more modernly) TLS, as specified in RFC 3920. Last year, Eric Rescorla suggested that we might want to look into methods for reusing those same basic TLS semantics for end-to-end communication. After some on-and-off discussion in the XMPP developer community, we've finally released specifications for that approach, using end-to-end XML streams negotiated by means of a Jingle application type and upgraded to an encrypted stream using STARTTLS.
Think of it this way: XML streams need a reliable transport. In standard c2s and s2s XMPP, that reliable transport is TCP. But for end-to-end streams, any old reliable transport will do: it could be a direct TCP connection (unlikely given firewall madness), it could be an in-band bytestream, it could be a SOCKS5 bytestream, it could be ice-tcp, or anything else. For end-to-end streams we use Jingle to negotiate the transport, then use whatever reliable transport gets chosen to set up an end-to-end stream between two endpoints (e.g., two clients). This is similar to serverless messaging and reuses all the core XML streaming support that Jabber clients already have. If you want the stream to be encrypted, you upgrade the stream to Transport Layer Security using the same STARTTLS mechanism we already use for c2s, s2s, and serverless messaging. This means that client developers have less code to write, which makes it more likely that the technology will be implemented and deployed.
Granted there are some gotchas here. Will typical end users know what to do with end-to-end streams, especially with regard to certificate checking? Can we build in some of the cool features of encrypted sessions, such as short authentication strings? Could we potentially use OpenPGP keys with TLS so that people don't need to obtain CA-issued client certificates (although I'm sure we could convince the XMPP ICA to offer those)? And so on.
No, this is not necessarily the ideal e2e encryption technology for XMPP, but we've waited far too long to develop such a technology. The perfect is the enemy of the good. I say: let's hack up support for e2e streams in a bunch of Jabber clients, test out the technology, and iteratively add security improvements over time as we gain experience. After all, some encryption is better than no encryption...