Question

List the four laws of probability theory and express them mathematically.

Answer 1

Probability Basics:

• Probability is a measure of the likelihood of some event happening. (Just about anything can be an event in this sense.) We measure on a scale from 0 to 1 (or 0% to 100%), where smaller numbers indicate less likelihood and higher numbers indicate greater likelihood. This system in an example of how mathematics is used to formalize and make precise our informal notions about some things being more or less likely to occur than other things.

Two kinds of probability:

We can distinguish two kinds of probability. Mathematical probability is the measure of the relative frequency of an event occurring. In addition, we will use the term personal probability for a statement of someone's degree of belief that an event will occur.

Determining Relative Frequency (Mathematical Probability)

Empirical Method :

If the process under study can be repeated or simulated many times, we can determine the empirical probability by keeping track of the outcomes in our (large number of) trials. The probability assigened is

P(A happens) = (# times A happened) / (# trials)

If the number of trials is very large, then it is quite likely that this will give us a reliable estimate.

Theoretical Method:

Sometimes we can make mathematical assumptions about a situation and use Four Basic Properties of Probability to determine the theoretical probability of an event. The accuracy of a theoretical probability depends on the validity of the mathematical assumptions made.

The four useful rules of probability are:

1. It happens or else it doesn't. The probability of an event happening added the probability of it not happening is always 1.

   P(A happens) + P(A doesn't happen) = 1

2. Exclusivity. If A and B can't both happen at the same time (in which case we say that A and B are mutually exclusive), then

   P(either A or B happens) = P(A happens) + P(B happens)

3. Independence. If B is no more or less likely to happen when A happens than when A doesn't (in which case we say that A and B are independent), then

   P(A and B both happen) = P(A happens) * P(B happens)

4. Sub-Events. If whenever A happens B must also happen, then B must be at least as likely as A, so

   P(A happens) <= P(B happens)

Empirical probabilities will also follow these rules (for a given set of trials). Becuase people often have a poor sense of the likelihood of an event, personal probabilities often do not follow these rules. A collection of personal probabilities is called coherent if it does not violate the rules for mathematical probability.

Equally likely outcomes

• One especially important use of these probability rules is the conclusions that can be drawn if we assume that a number of events are equally likely. If there are only n such events that are possible in a given situation, and all are equally likely and pairwise mutually exclusive (no two can happen at once), then each must have probability 1/n. More complicated situations can be handled by dividing a situation into a number of equally likely outcomes and counting how many of them are "of interest" (in the event). The probabilty then is given by (number of interest)/(total number), just as in the case for empirical probability.

The Law of Probability :

Law of Large Numbers :

• you can discover the unknown probability of an event through experimentation. Using the previous example, say you do not know the probability of drawing a certain colored marble, but you know there are three marbles in the bag. You perform a trial and draw a green marble. You perform another trial and draw another green marble. At this point you might claim the bag contains only green marbles, but based on two trials, your prediction is not reliable.
• It is possible the bag contains only green marbles or it could be the other two are red and you selected the only green marble sequentially. If you perform the same trial 100 times you will probably discover you select a green marble around 66% percent of the time. This frequency mirrors the correct probability more accurately than your first experiment. This is the law of large numbers: the greater the number of trials, the more accurately the frequency of an event's outcome will mirror its actual probability.

Law of Subtraction:

• probability can only range from values 0 to 1. A probability of 0 means there are no possible outcomes for that event. In our previous example, the probability of drawing a red marble is zero. A probability of 1 means the event will occur in each and every trial. The probability of drawing either a green marble or a blue marble is 1. There are no other possible outcomes. In the bag containing one blue marble and two green ones, the probability of drawing a green marble is 2/3.
• This is an acceptable number because 2/3 is greater than 0, but less than 1--within the range of acceptable probability values. Knowing this, you can apply the law of subtraction, which states if you know the probability of an event, you can accurately state the probability of that event not occurring. Knowing the probability of drawing a green marble is 2/3, you can subtract that value from 1 and correctly determine the probability of not drawing a green marble: 1/3.

Law of Multiplication :

• If you want to find the probability of two events occurring in sequential trials, use the law of multiplication. For example, instead of the previous three-marbled bag, say there is a five-marbled bag. There is one blue marble, two green marbles, and two yellow marbles. If you want to find the probability of drawing a blue marble and a green marble, in either order (and without returning the first marble to the bag), find the probability of drawing a blue marble and the probability of drawing a green marble. The probability of drawing a blue marble from the bag of five marbles is 1/5. The probability of drawing a green marble from the remaining set is 2/4, or 1/2. Correctly applying the law of multiplication involves multiplying the two probabilities, 1/5 and 1/2, for a probability of 1/10. This expresses the likelihood of the two events occurring together.

Law of Addition:

• Applying what you know about the law of multiplication, you can determine the probability of only one of two events occurring. The law of addition states the probability of one out of two events occurring is equal to the sum of the probabilities of each event occurring individually, minus the probability of both events occurring. In the five-marbled bag, say you want to know the probability of drawing either a blue marble or a green marble. Add the probability of drawing a blue marble (1/5) to the probability of drawing a green marble (2/5). The sum is 3/5.
• In the previous example expressing the law of multiplication, we found the probability of drawing both a blue and green marble is 1/10. Subtract this from the sum of 3/5 (or 6/10 for easier subtraction) for a final probability of 1/2.