Question

I wonder what were the main experiments that led people to develop the concept of wave function collapse? (I think I am correct in including the Born Rule within the general umbrella of the collapse paradigm.) Are there any instances where cases once thought to be examples of collapse have since been explained as the normal time-evolution of the wave function?

EDIT: I'm going to have to make an objection to Ron Maimon's very excellent answer about particle tracks as evidence of collapse. I've been waiting for someone to suggest what I personally have always considered the prototype of the wave function collapse, namely the appearance of flecks of silver on a photographic plate when exposed to the light of a distant star. This has the essential elements of collapse in a way that ordinary photographic exposures do not. The mere appearance of dots on a photographic plate does not signal the collapse of anything: it is readily explainable as a consequence of the rate of silver-bromide reduction being proportional to light intensity. It is only when the intensity becomes so very low that the time taken to accumulate enough energy for a single conversion becomes unreasonable that we must consider the explanation of wave function collapse.

The tracks in the cloud chamber do not demonstrate this phenomenon since the energy needed for the creation of the tracks is already available in the supersaturated gas. It is not necessary for the incoming particle to supply energy for the creation of the track, so there is no need to collapse its wave function. The straightness of the tracks is explained by Mott as an ordinary consequence of time-evolution of the wave function. There is no experimental proof that a single "particle" cannot be responsible for multiple tracks in the cloud chamber, because the tracks are not tagged according to which particle created them.

Answer 1

I believe the notion of collapse of the wavefunction is most explicitly derived from the resolution of the 1929 Mott paradox: http://en.wikipedia.org/wiki/Mott_problem .

The Mott problem considers an electron in a spherically symmetric wave, washing over a bunch of atoms. This electron will ionize the atoms, but not in a spherically symmetric way! We know that we will see the electron's trajectory through the atoms by looking at the ionization trail, as in a bubble chamber picture.

The Mott Heisenberg analysis showed that a spherical S-wave for a high energy electron travelling through a bunch of atoms indeed does lead the atoms to ionize along certain tracks, but only if you consider the full wavefunction of the atoms and the electron. The entanglement of the two means that once one atom is ionized, the next atom will be ionized on this Everett branch in the same general direction, although with some statistical spread.

The Mott/Heisenberg analysis makes it clear that the ionization "collapses" the wavefunction of the electron. This is then turned into a general principle, whereby any interaction which gives classical information which can be irreversibly amplified up for us to see leads to collapse of the wavefunction.

The many-worlds interpretation comes later, but the spark of the mathematical ideas (although not the philosophical leaps, nor the information theory aspects analyzed by Everett under Wheeler) are mainly contained in the Mott Heisenberg analysis.

But collapse was also evident from the formulation of the atom-radiation field theory in the late 1920s, where photons are real things, and energy conservation is maintained nonetheless. I don't know the exact history, and it might be correct to attribute this to Bohr, Heisenberg, Born, Dirac, or maybe even Pauli.