Google’s synonym system understood that a dog was similar to a puppy and that boiling water was hot. But it also concluded that a hot dog was the same as a boiling puppy.
Web search looks simple, but try to write your own search algorithm and you’ll quickly see it’s not simple at all.
Wired has a long writeup of how Google’s search ranking algorithm has evolved. Don’t be put off by the word “algorithm” – it’s a fun and interesting narrative.
It strikes me a little bit like learning how to play chess. You have an overall algorithm in your head, and at first it’s quite flawed. Then as you play you are exposed to new positions and you try to incorporate rules for handling each of those positions. That gives you incremental improvement, but if you hit upon a more fundamental improvement of your move-selection technique that applies across many or all positions (example: realizing you should consistently evaluate the overall position), that’s when you make big leaps forward.
In fact this reminds me of an earlier discussion about theoretical frameworks in physics with Harvey and others trying to show off how smart they are. (Joking!)