I guess I don't really see much difference between academic job hunting, and job hunting in general. Starting out with undergrad admissions, there are many more qualified people for desirable positions than available slots. Who gets those slots is a matter of hard work (to get qualified) and luck (to be one of the qualified people who is "chosen"). So how is the TT any different from grad school admissions (in ANY prestige program), law firm partnership, company CEO, professional artist/athlete/performer, attending physician, investment banking, etc? The pool of qualified applicants is many times larger than the number of slots, and there are desirable perks to success (money/prestige/fame/security/intellectual freedom) making the supply of those willing to try for the goal pretty much infinite.
Maybe I have rose colored glasses on because I have always been lucky enough to find a position in research, but there are no guarantees in life. When I was interviewing in industry, I saw many really interesting jobs available to science PhD holders that were not in research. If I hadn't gone to National Lab, I would have been happy to take on one of those instead. Sure, my life would be different, but it wouldn't make my PhD a waste of time or a failed opportunity.
I've been telling my students that loving science isn't enough--they need to think about what they want to do with their PhDs, and start preparing for that before they are looking for a job so they can tailor their PhD experience appropriately (more teaching/writing/industrial partners/whatever). My current crew is undecided about what they want to do next. I do ask them periodically, though because 5 years goes by fast.