Warn about conformance requirement violations

[yurikhan]

I tried using html5lib for HTML validation, to make sure that my users author valid, well-defined HTML. To that end, I parsed the input HTML with html5lib and used the HTMLParser.errors property to flag errors. Ideally, I would like to flag all violations listed in HTML5 §1.10 Conformance requirements for authors.

Sample input:

<foo>Hello World!</foo>

Expected behavior: error logged for each use of a tag currently undefined in the HTML spec.

Observed behavior: undefined tags are silently accepted.

Sample input:

<ul>Hello</ul>

Expected behavior: error logged for use of PCDATA in an element whose content model permits only li elements.

Observed behavior: content model violation is silently ignored.

Sample input:

<font color="red">Test</font>

Expected behavior: error logged for use of presentational markup.

Observed behavior: accepted silently.

[yurikhan]

#167

[gsnedders]

A conformance checker requires reporting far more than just parse errors — it's a vast undertaking, and probably something that can be built on top of html5lib (there will likely have to be some changes, such as #167), and out of scope of html5lib, really. HTMLParser.errors is merely parse errors.

[yurikhan]

As an outside feature, blocked by: #169.

[gsnedders]

Bit of discussion about parser APIs for this on IRC here and continuing here.

[yurikhan]

For readability, I’d like to quote the relevant part of discussion here.

[22:48] gsnedders hsivonen, MikeSmith: what does a parser need API-wise to be used as a the basis of a conformance checker?
[23:05] MikeSmith gsnedders: a conformance checker basically just needs the parser to, well, expose the parsed document
[23:07] MikeSmith gsnedders: beyond that, for use with a conformance checker, it should be an error-reporting parser that does actually report anything that the spec defines as a "parse error"
[23:07] MikeSmith gsnedders: as Henri's parser does
[23:09] MikeSmith gsnedders: as far as the nature of the API, it doesn't matter what kind of API it is except it ideally should be a streaming API that doesn't require constructing a representation of the entire document and keeping it in memory
[23:10] MikeSmith gsnedders: e.g., a SAX API, as the vnu code uses
[23:13] MikeSmith gsnedders: so, concretely, if you're working on implementing a document parser I recommend you have it provide a SAX API or some whatever other kind of API that exposes the actual parse events/tokens

---

[00:57] gsnedders MikeSmith: a SAX API that throws a fatal parse error when it hits content that needs buffering?
[01:01] MikeSmith gsnedders: yes, a SAX API that throws a fatal parse error when it hits content that needs buffering?
[01:02] MikeSmith gsnedders: ideally with an option for buffered non-streaming parsing for those that want to opt-in to that
[01:03] gsnedders MikeSmith: people are starting to more seriously try and build on top of html5lib
[01:03] MikeSmith gsnedders: and to be clear, not a fatal parse error for the document, but just fatal for that part of the tree
[01:03] gsnedders MikeSmith: that sounds horrible to implement :(
[01:03] MikeSmith gsnedders: If you know what I mean. That's how Henri's parser handles it
[01:04] MikeSmith gsnedders: well, Henri did it. So we have an existence proof
[01:04] gsnedders MikeSmith: is that even a conforming option per spec? I thought the options were stop parsing or continue per spec?
[01:04] MikeSmith dunno
[01:04] gsnedders MikeSmith: well, I'm not questioning plausiblity, just niceness :)
[01:04] gsnedders MikeSmith: what makes a streaming parser better, though? ability to cope with larger docs?
[01:06] MikeSmith gsnedders: at least for henri's implementation it emits a messaging saying basically "I'm giving up on checking this part of the tree. [There may be errors in it but I'm not able to report them because that would require non-streaming parsing, which I'm not set to do right now.]"
[01:06] MikeSmith gsnedders: performance
[01:07] MikeSmith and the ability to hook it into a pipeline and check things in parallel
[01:07] gsnedders MikeSmith: which are the non-streaming cases? AAA, foster parenting? AAA seems to be hard to mark the right part of the tree, off-hand
[01:07] MikeSmith can run multiple checks in parallel fed from the same parse events
[01:08] MikeSmith gsnedders: yeah AAA and foster parenting
[01:08] MikeSmith gsnedders: and I don't know how Henri handled it but I think it does mark the right part of the tree
[01:09] gsnedders MikeSmith: It's Henri's code, I don't doubt it does. :)
[01:09] MikeSmith heh
[01:11] MikeSmith anyway, the model of being able to use multiple SAX contenthandlers in parallel is really powerful for implementing a checker, and just makes a lot of things much easier/doable
[01:11] gsnedders well you should always be able to implement something that multiplies events
[01:11] gsnedders given a normal singular SAX handler
[01:12] MikeSmith yeah I just mean the benefit of SAX, of exposing a SAX API
[01:14] MikeSmith but more generally exposing any kind of parse-event API at all is a win for this kind of use case (checkers)
[01:14] MikeSmith and in practice, "exposing any kind of parse-event API" pretty much just means SAX
[01:15] MikeSmith *means just use SAX, because it's sane and nobody has come up with anything that's radically better

[yurikhan]

I agree that for large documents a SAX-like parser might be nice. However, for many real-world use cases, conformance checking on a DOM tree is sufficient (if a user agent can afford building DOM for a document, a conformance checker can, too) and much easier to implement.

---

I also agree that for HTML with its foster parenting and AAA it means either throw hands up at the first point where they require a change of history, or start buffering at carefully selected points.

The only place in Chapter 8 where foster parenting is said to be enabled is when encountering “Anything else” in the “In Table” insertion mode, and it happens immediately after reporting a parse error. Therefore, stopping at this point is technically conforming; flagging the remaining conformance violations is only a quality of implementation issue.

Conformance checkers must report at least one parse error condition to the user if one or more parse error conditions exist in the document and must not report parse error conditions if none exist in the document. Conformance checkers may report more than one parse error condition if more than one parse error condition exists in the document.

The adoption agency algorithm is invoked at three points in the “In Body” insertion mode. Two (when handling <a><a> and <nobr><nobr>) are immediate parse errors, but the third (closing a formatting element) is not. However, it looks to me that the cases when AAA would require a change in the sequence of emitted SAX events, a parse error is also always present.

---

(My own stand is that HTML has become a horrible beast of backward bug-for-bug compatibility, and I wish it were obsoleted or at least deprecated in favor of XHTML. Given the current state of affairs, however, a user saying “HTML is a valid and W3C-blessed syntax and I wish to author documents in that syntax” is in their own right, and the best we can do as authoring tool implementors is to provide our users with as strict conformance and coding style checkers as humanly possible, and this requires an abundance of good quality conformance and style checker libraries.)

[gsnedders]

Also, as MikeSmith says later, he's wrong, the v.nu parser does stop parsing given AAA/foster-parenting.

(It's worth pointing out most of the complexity of conformance checking is there for XHTML as well as HTML; parse errors are relatively easily handled in both cases.)