March 27, 2006

Developers and Permissive Parsing

Don't do it. Require strictness whenever you implement a parser of ANY kind.

People always ask: why should compatibility be at odds with standards compliance? This is why.

Back in the day, when Netscape was king of the hill and the web was dawning in the consumer eye (by that I mean $ ka-ching $), permissive parsing was all the rage. Mostly, to make it easy on content creators, the browser was just supposed to “work” if it could - and this extended to mark-up as well as

Turns out this was remarkably wrong-headed thinking, though we (the industry royal “we” here) all fell victim to its lure - and suffer the consequences as a result.

Unfortunately, there are now very real places where compatibility and standards compliance are directly at odds - permissive parsing means its hard to tell what the "right" thing to do is, especially as technology and standards evolve. If strictness and unit testing were the rule of the day, that wouldn't be the case.

This is a classic example (to me) of experience vs. intellect. It seems better to make it easier for your users, but it actually just makes things awful for everybody, over time, every time.

Be sure you explicitly define what doesn't work as you define what does.

Or, as I'm fond of saying:
Never put off tomorrow what you can put off today.


Dossy Shiobara said...

Oh, come on Sree, you've been around long enough to know RFC 793 and Postel's Law, right?

"Be conservative in what you do; be liberal in what you accept from others."

The reason why stuff is *broken* is because people take the attitude you're suggesting: emit crap, parse strictly.

Sree Kotay said...

HAH - funny :)

Actually Dossy, I'd say your thinking is nearly as old as that RFC. Nobody would emit crap if nobody accepted it. :P

I'm not the first to have argued it, but the problem is that Postel's Law limits growth, innovation and even clean-up by ambigously expanding the specification (e.g. formats for one's parser in this case).

(as always, IMHO :))

riz said...

I agree that "more compatible" doesn't mean "better."

It's also hard to make this right in a world where standards compliance is a nice-to-have "feature" rather than a requirement.

This has already messed it up for everyone. Take Firefox and IE. So many people say, "well, this site works in IE, but Firefox can't display it." Of course you can give them the long answer, but they don't want to hear that it's because it's not standards compliant. They just want it to work. That's the kinda thing that puts us (developers) in a tough place. Kinda like the "C'mon man, everyone's doin' it" peer-pressure from high school!

Dossy Shiobara said...

We live in an imperfect world. Software, at least within my lifetime (I'm betting), will be written by humans, who are prone to make mistakes.

The push for XML over fixed-length record formats and other delimited file formats is an example of Postel's Law in action: it allows producers to emit the same information with flexibility but still preserve the essential information being expressed, and it allows consumers to consume what they know about and ignore the rest.

Loose coupling. Permissive parsers are required to effect looser coupling. The goal here is to have stuff that "works," not to have some things work perfectly and others be wholly broken -- where "works" obeys the "don't let the best be the enemy of the good [enough]" expression ...

Anonymous said...

Does size matter? Heck of a way to start a post, but...

If I'm a big company then I would probably be inclined to be less robust [on] principle. However, if I were a smaller company I would want the cha-ching to make sure my stuff would be accepted and adopted by as many as possible. In other words, who you are in this thing, and what you are parsing, may make a difference on how one approaches this.

Perhaps the inverse is true and a big company would have pressure to be more tolerant (i.e., legacy support), but I have to say that it may be easier to become stricter over time, and effect change, as permissiveness is phased out in lieu of potentially buggy flexibility.

I think both sides have points, but I have a bias towards stricter parsing to cut down on shenanigans as long as it is done in reasonable ways.

BTW not just that TCP RFC (1981) has points, the HTTP spec (circa 1999) addresses it too:

-- RFC 2616 --
19.3 Tolerant Applications

Although this document specifies the requirements for the generation of HTTP/1.1 messages, not all applications will be correct in their implementation. We therefore recommend that operational applications be tolerant of deviations whenever those deviations can be interpreted unambiguously.

Clients SHOULD be tolerant in parsing....

Sree Kotay said...

"We therefore recommend that operational applications be tolerant of deviations whenever those deviations can be interpreted unambiguously."

This would be the key part :)

I'm suggesting a bias of STRONG conservatism in parsing will server you MUCH better from the start, over time, every time.

Charmingirl said...

Wholesale lingerie directly from China?
As a famous brand and specialized manufacturer of sexy clothing in China. Charmingirl supply the international market with fashionable sexy lingerie and sexy costume since 2002. With advanced technology,all our products are of high quality. Now we have clients all around the world. Lingerie Wholesale and OEM are welcomed!
As a Lingerie Manufacturer, Charmingirl has standard workshop and production line, professional designers and experienced workers.
We do Wholesale Underwear,
Lingerie Wholesale, including corset and bustier,
Sexy Lingerie Wholesale, including bikini, underwear
Lingerie Wholesale, and Babydolls, Sexy Lingerie Wholesale, and
Sexy Lingerie Wholesale including sleepwear,clubwear.
Lingerie Wholesale from China: Lingerie China, you will find the
Leather Lingerie and PVC Lingerie, also you can buy
Christmas Costume and Xmas Lingerie
for your Christmas Lingerie Christmas day.
Our Wholesale center: Sexy Lingerie Wholesale can do Lingerie Wholesale online.
Halloween Costume,
also wholesale Adult Costume with fashion Babydoll Babydoll, and bra and panties Bra and Panties, Sexy Uniform Sexy Uniform is also our major products.