Validating microformats

I'm on record as having concerns about the microformats approach to marking up data on the web. One of those concerns is validation, without which I assert that vast quantities of the data will be invalid and hence only marginally useful.

So what can we do about validation? I believe I've learned two things since the last time I considered this problem:

The second point blows direct RELAX NG validation completely out of the water. My concerns about interleaving turn out to be unfounded and irrelevant, there's just no way to express a pattern that matches an attribute that contains some token. Direct W3C XML Schema validation is impossible too.

In fact, the only direct validation technology that I know of that might work is Schematron. But the Schematron rules would be hard to write too because you have to express both what is and what isn't allowed, I think, and because you still have to mess with all that token parsing.

So let's imagine a future in which we have a standard XML pipeline language so we can easily chain a few processes together. Then we might employ the following strategy:

For each microformat that a document contains (which I hope is identified in a profile or something, otherwise I guess you just have to test them all):

If all of the microformats validate, then the document is valid. I think it makes sense to check if the document is valid overall (according to an XHTML schema, for example), though I suppose that step is technically optional if you're willing to accept random well-formed XHTML element names.

One interesting observation is that the necessary transformation can (often? always?) be deduced from the grammar. In fact, I coded up an XSLT stylesheet that does this. It reads a RELAX NG grammar and an XML document and transforms class attribute values to element names. To test this, I constructed some toy grammars for hCal, hCard, and hReview. (These are toy grammars, I don't assert that they're complete or correct.)

Consider the first restaurant review example in the hReview specification:

<div class="hreview">
 <span><span class="rating">5</span> out of 5 stars</span>
 <h4 class="summary">Crepes on Cole is awesome</h4>
 <span class="reviewer vcard">Reviewer: <span class="fn">Tantek</span> - 
 <abbr class="dtreviewed" title="20050418T2300-0700">April 18, 2005</abbr></span>
 <div class="description item vcard"><p>
  <span class="fn org">Crepes on Cole</span> is one of the best little 
  creperies in <span class="adr"><span class="locality">San Francisco</span></span>.
  Excellent food and service. Plenty of tables in a variety of sizes 
  for parties large and small.  Window seating makes for excellent 
  people watching to/from the N-Judah which stops right outside.  
  I've had many fun social gatherings here, as well as gotten 
  plenty of work done thanks to neighborhood WiFi.
 </p></div>
 <p>Visit date: <span>April 2005</span></p>
 <p>Food eaten: <span>Florentine crepe</span></p>
</div>

If we apply the pre-validation stylesheet to this document with the hReview grammar, we get:

<reviews><hreview>
 <rating>5</rating> out of 5 stars
 <summary>Crepes on Cole is awesome</summary>
 <reviewer>Reviewer:  - 
 </reviewer><dtreviewed title="20050418T2300-0700">April 18, 2005</dtreviewed>
 <description>
   is one of the best little 
  creperies in .
  Excellent food and service. Plenty of tables in a variety of sizes 
  for parties large and small.  Window seating makes for excellent 
  people watching to/from the N-Judah which stops right outside.  
  I've had many fun social gatherings here, as well as gotten 
  plenty of work done thanks to neighborhood WiFi.
 </description><item>
  <fn>Crepes on Cole</fn> is one of the best little 
  creperies in .
  Excellent food and service. Plenty of tables in a variety of sizes 
  for parties large and small.  Window seating makes for excellent 
  people watching to/from the N-Judah which stops right outside.  
  I've had many fun social gatherings here, as well as gotten 
  plenty of work done thanks to neighborhood WiFi.
 </item>
 Visit date: April 2005
 Food eaten: Florentine crepe
</hreview></reviews>

Applying the same transformation with the hCard schema produces:

<bag-of-cards><vcard>Reviewer: <fn>Tantek</fn> - 
 </vcard><vcard>
  <fn>Crepes on Cole</fn><org>Crepes on Cole</org> is one of the best little 
  creperies in .
  Excellent food and service. Plenty of tables in a variety of sizes 
  for parties large and small.  Window seating makes for excellent 
  people watching to/from the N-Judah which stops right outside.  
  I've had many fun social gatherings here, as well as gotten 
  plenty of work done thanks to neighborhood WiFi.
 </vcard></bag-of-cards>

It's very hard to tell what text can be discarded, so we wind up with a bunch of extra text. I think that's ok; we can make all of our grammar patterns mixed content.

The output of each of the preceding transformations is valid according to my manufactured schemas for the respective microformat.

Another possibility would be to simply discard all the text:

<reviews>
   <hreview>
      <rating/>
      <summary/>
      <reviewer/>
      <dtreviewed title="20050418T2300-0700"/>
      <description/>
      <item>
         <fn/>
      </item>
   </hreview>
</reviews>

I don't know. On the whole, I'm going to guess it makes more sense to preserve the (sometimes extra) text. That way your schema could test for content too when appropriate.

There are still some hard issues though:

In short, this approach works sometimes (it does detect an “hreview” with a missing “item”, for example), but I'm still not satisfied. And I remain convinced that this problem has to be solved before microformats can be considered a reliable way to encode data.

attribute class {
    foo:class {
        name = 'vevent'
    }
}

    element * {
        attribute class {
            xsd:token { pattern = "(.+\s)?foo(\s.+)?" }
        }
    }