Question

How would you describe hypothesis testing to a friend outside of this statistics class ?

Answer 1

Please give a positive rating.

I'll explain this to my friend in with the help of the following example:

After me and my cousin visited you last time, one of us left a pile of books in your living room. No need to bother if these were my books, but you should say something if these were my cousin's books since he will probably miss and search them. Consider the phone call you would have to do in this case would be VERY EXPENSIVE (you would not like to call for no reason). So, what do you think who these books are? Should you call or not?

You know most of the books I have, but you don't know which books my cousin has.

You can check if these books fit into my "portfolio". If this is the case, you can't say much; could be well my books, it's also still possible that these books belong to my cousin, since he could have a similar taste (what you don't know). To avoid an expensive call that might be unnecessary, you wouldn't call in this case. But if these books do not at all fit into my library, then you will strongly suppose that these must be my cousin's books - or I would have completely changed my taste, what you consider really unlikely. In this case, you feel you should inform us about the books (my cousin would surely be very happy and pay for the call).

Statistically speaking: This scenario, that I left the books is called a "hypothesis". There is an alternative scenario or alternative hypothesis, namely that my cousin left the books. You know what books to expect when *I* leave books in your room. Therefore, this scenario is entitled the "null hypothesis" (just to give it a name). Given the particular books lying on your table, one can then use this knowledge and calculate a probability value that *I* would leave *these* books (or, better, "books like those"). This may be more or less likely. If it is quite likely, these books do not provide evidence against the null hypothesis: no call. But if this is quite unlikely, you have two options: either you believe that the null hypothesis (this scenario in which I left the books) is not true (call!), or you believe that in this stupid case it was still me who left the books, but it was a very unlucky incident that I left just those books you wouldn't expect to be part of my library (strange...) (no call). Statisticians don't like to believe in unlucky or strange incidents. They would much more believe that the scenario was not the actual true situation. And if they conclude that it wasn't me who left the books, it could have only been my cousin who forgot them. In this case, they call their finding statistically significant, which means "this is considered to be too unlikely when the null hypothesis was correct". Consequently, they "reject" the null hypothesis, and they would phone me and tell me: "Hey, there is a pile of books lying around here, and these books are statistically significantly *not* your books. Hence I suppose your cousin left them here.