Hypothesis: The rise (and utility) of sites like IMDB and technorati is the beginning of crumbling of search portals as navigation - a(nother) chink in the coming Web consumption shift (I'll write about thoughts on that another time).
For a while, it really looked like generic search was going to be the answer to any manner of navigational problems - heck, I still use Google to search MSDN, because the built-in search on MSDN blows chunks. And obviously, Google, Yahoo, AOL, MSN search have all hoped that you'd type in addresses, phone numbers, ticker symbols, etc. into their one generic search box.
And in some measure, that has helped them all capture incremental traffic, but it hasn't really changed the game. As nice as Google Maps might be, Mapquest as a destination hasn't been impacted at all - its been a non-event.
My argument is really a "specialization beats generalization" along two axies:
(1) "Clean" validated metadata is more important to relevancy than purely data inferred relevancy. Google's desktop search initial launch didn't even let you specifiy file types - you don't need it, was the claim. Yet (as a very small, minor example), that type of metadata is very important to relevancy: finding what you want. Similarly, tagging and related phenomenen allow users to impute metadata, and the "wisdom of crowds" assigned it validity.
(2) Specialized presentations (i.e. task oriented) will beat generic ones for all but the most trivial interactions. That is, to put in Web two-dot-oh speak: The head of "attention" engagments will increasingly be task specific. IMDB works because the UI is well organized for cross-referencing (hyperlinking, if you will :P) amongst movie cast specific criteria.
In combination, this means that crawling alone will be an insufficient input to data and application relevancy over time. Where do you go to get stock quotes?