Discovery On My Mind: New Specification Published

You can say discovery is somewhat of an obsession of mine. Ever since I started a little over a year ago to think about discovery (originally for OAuth), I found it to be more and more fascinating. For a while there it felt very lonely because no one else really wanted to talk about discovery. But things are changing fast.

At the last Internet Identity Workshop I finally found my audience and things started falling into place. I gave a 4 hours long (or so) session on discovery and resource descriptors that for the first time drew a large crowd. I have been trying to get people to listen for about 8 months and that was the first time people realized just how much they need this stuff. And yesterday.

Working mostly solo did have its benefits. It allowed me to start, then retire the XRDS-Simple project and put OAuth Discovery on hold (given its strong dependencies on XRDS-Simple). You see, the basic discovery framework present in XRDS and Yadis was just not making that much sense to me. The more time I spent thinking about its various moving parts, the bigger the issues with the current approach became.

This thinking process produced a few productive collaborations. Mark Nottingham and I submitted an IETF Internet Draft (the stuff that sometimes turn into an RFC) called Site-Meta, which is a way to provide site-wide metadata using a simple document format in a well-known-location. Yes, known-locations are evil, but we hope to get away with it by promising this will be the very last. The One (one known-location to rule them all).

Site-Meta was written as a logical extension of the reintroduced HTTP Link header, a critical new building block for discovery. The recent Link header draft is a great piece of spec-writing from Mark and I hope it will be finalized soon.

I have also packed my virtual bags and moved from the XRDS-Simple list to the OASIS XRI Technical Committee – the place where XRDS started and where it was being defined. The XRI folks have been extremely supportive of my little '-Simple' project and when I started to question some of the most basic concepts common to Yadis, XRDS, and XRDS-Simple, it turned out to sound convincing enough to get the XRI TC to agree to make some big changes.

So instead of creating a simple profile of XRDS, XRDS-Simple and XRDS are now being merged into a much simpler and better designed protocol called XRD which stands for Extensible Resource Descriptor (old name fresh ideas). Descriptor is a better name than metadata because it is more generic (and not just about "data"). After all, an XRD used for OpenID isn't about data, it is about You.

The work is in its early stages but drafts are starting to emerge. The first priority was to find a replacement for Yadis. I find Yadis to be a brilliant attempt at creating a protocol anyone can implement (no matter how crappy their web hosting is). But it breaks too many things and there are new tools around today that can make it better (such as Link header and Site-Meta).

To get all the right people talking to each other, and there are actually plenty of smart folks thinking about discovery these days, I started a coordination group which is very low volume and used to keep those interested up to date. As I was gathering information for the group reports, a more complete picture emerged.

Today I am excited to report that the first draft of my HTTP-based Resource Descriptor Discovery is now available for review. If you have been reading this blog, you can actually skip half of it as it repeats the case for why I decided to include the methods I did.

From the draft:

This memo aims to provide a uniform and easily implementable process for locating resource descriptors.  With the development of interoperability specifications comes the need to enable compliant services and resources to declare their conformance to these specifications.  There is a growing need to describe resources in a way that does not depend on their internal structure, or even the availability of an HTTP-accessible representation of these resources.

For example, while an end-user is reading a web page such as a blog article, the user-agent can discover whether the content of this page has generated from an Atom feed or Atom entry and whether that feed supports Atom authoring.  It can discover whether there is an iCalendar-formatted or CalDAV calendar associated with the page, or where other content by the same page author might be found.

In an example related to the identity space, an end-user can use a URI as an identifier for signing into web services, and in turn, the web service can discover more information about the user's resources and preferences such as who did the user delegate their identity management to, where they keep their address book or list of social network friends, where their profile information is stored to reduce signup registration requirements, and what other services they use which may enhance their interaction with the web service.

Next on my plate is to (hopefully) have an engaging discussion about the draft and quickly publish a second draft. I am also putting the final touches on the first XRD 1.0 schema draft which will replace the current XRDS format. The new proposal is extremely attractive in its simplicity and realignment with Link (headers and elements) architecture.

The new draft includes some interesting new concepts about different types of discovery, the authority of entities to provide discovery information, and the relationship between discovery information and certain HTTP response code. I want to write more on these topics, but for now, go read the spec.

Not a bad way to start the new year!