Before we start with the mathematics of probability, lets start with some intuition. We all have some notion of probability. When somebody says that there is a 50-50 chance that a coin toss will come up heads, we understand that this means that if we toss a coin 100 times, approximately 50 of those tosses will come up heads. We say that the probability of the coin toss coming up heads is 1/2, or 0.5.
Consider another example, rolling a die. When you roll a die, there are 6 possible outcomes. We call the set of all possible outcomes the sample space. So in this case, the sample space is the set of numbers from 1 to 6. If we roll a die many times, some of the results will be a 6. The number of 6's divided by the total number of rolls is a number between 0 (no 6's) and 1 (all 6's). We know from experience that this number is approximately 1/6. We call this number the probability of rolling a 6.
Loosely, we say that the probability pi of an event i in a sample space to be the proportion of outcomes of event i in a very large number of trials. (A trial is just a single coin toss, or a single die roll, etc.) In other words, if an event has probability pi and we perform n trials, then we expect that npi outcomes will be the event i. We call a set of probabilities on a sample space a probability distribution on that space. The probabilities on a sample space must all add up to 1, since for any trial the outcome will be one of the elements of the sample space.
|
A histogram is a plot that graphically presents the probabilities of a system. The plot to the right shows a histogram for the roll of a single die. It is very boring. On the "i" axis we see the elements of the sample space. For each element of the sample space we have plotted a bar. The AREA of the bar is the probability of that event. This is really important. The height of the bar is not what counts. The area of the bar is the probability of the event. Since there is an even chance for each outcome, the bars all have the same area: 1/6. |
|
|
Now consider a system with two dice. The sample space (the set of all possible outcomes) is now the set of numbers from 2 to 12. What are the probabilities of each outcome? The tool to the left will help you construct these probabilities. When you hit the Trial button, the diagram will roll two dice and update the histogram to indicate the proportion of each of the outcomes. You can also hit the 50 trials button to add the results of 50 rolls to the set of trials. (This will help you construct a large number of trials quickly). After about 500 trials, the histogram will show a definite pattern. With a large number of trials, we expect the proportions shown in the histogram to be close to the true probabilities. |
In the case of two dice you can compute the probability distribution yourself. For example, there is only one way to roll a 12 - you need to get a 6 in each die. Let us say that you roll the dice separately. There is a 1/6 chance that your first roll will be a 6. And 1/6th of the times you roll a 6 in the first die, the second die will come up 6. So the probability of rolling a '12' is 1/36, or about 0.0278. This number agrees pretty well with the area of the box for '12' in the diagram after many trials.
Question 1 (2 marks)
Thinking about area and probability is probably not natural for you at first. This question makes sure you understand the relationship. After 100 trials, what is the area of the yellow region in the diagram above. (Hint: you don't need to do any computation.)
Mean and Variance
From high school, you know how to take averages. If you roll a die
n
times, the average value is the sum of all the values you rolled
divided by n, the total number of rolls. We can
ask the question,
"what do we expect the average to be?" Well, we expect that
we will get a
1 about one sixth of the time since
; that is, we expect to a get 1 about
times, and
a 2 about
times and so on.
So we expect the sum of all the rolls to be
![\[
(\frac n6 + 2\frac n6 + 3\frac n6 + 4\frac n6 + 5\frac n6 + 6\frac
n6) = 3.5 n.
\]](gifs/index_4.gif)
To get the average we divide this number by the total number of
rolls, n, and find an expected average of 3.5.
We call this expected average the mean value of the probability
distribution and its traditional symbol is
(This is the greek
letter mu. Don't pronounce it like a cow; it rhymes with "few").
The general formula for the expected value of a probability distribution is
![\[ \mu =\frac{1}{n} \sum_{i}i n p_i =\sum_{i}i p_i \]](gifs/index_6.gif)
.|
The mean of a distribution is an indication of which outcomes we expect to see. But two probability distributions with the same mean can look very different from each another. For example, consider a die with a 3 on 3 of its faces and a 4 on the remaining 3 faces. The expected value for this die is (4+3)/2=3.5, the same as for a normal die. But all the probability for this die is clustered at 3 and 4. The histograms to the right show this difference. The distributions have the same mean, but one is more spread out than the other. |
 |
 |
We have a way of measuring how spread out a distribution is. A spread out distribution is one with significant probability far from the mean. So to measure this spread we could try to find the expected distance of an outcome from the mean. The general formula for this distance is given by
.Using the fact that
, it is easy to see that
this sum is zero. For example, for our funny die this sum is
![\[ \frac{3-3.5}2+\frac{4-3.5}2=-\frac 14 + \frac 14 = 0.
\]](gifs/index_9.gif)
The problem with the previous formula is that it distiguishes spread
to the right from spread to the left ( -1/4 and
1/4 should have the same sign
and not add to zero). To get around this problem, we could look at the
average value of
. This value gets large when
is far from
and it is always positive.
For a normal die, this value is:
![\[
\frac{(1-3.5)^2}6 +
\frac{(2-3.5)^2}6 +
\frac{(3-3.5)^2}6 +
\frac{(4-3.5)^2}6 +
\frac{(5-3.5)^2}6 +
\frac{(6-3.5)^2}6 = 2.92.
\]](gifs/index_13.gif)
We call this number the variance of the
distribution. The square root of this number, 1.72,
is called the standard deviation and its symbol is
traditionally
(this is the Greek letter
sigma). It is a measure of how spread out a distribution is.
Question 2 (2 marks)
What is the variance of the distribution for the die with only 3's and 4's on it? Since this distribution is less spread out than for a normal die, you should get an answer smaller than 2.92.
Fishy Business: Continuous Distributions
Suppose we have a species of fish that we would like to preserve and to use as a food source. To effectively control the fishery, we need to know statistical information about the fish. For example, if we want to design a gill net that only catches the largest members of the species, we need to know the expected diameter of the species. The questions that follow deal with the diameter of a fictional species of fish.
Until now, we have looked at distributions with a small sample space: 2 outcomes for a coin toss, 6 outcomes for a die roll. This time we are looking at a more realistic problem: measuring the diameter of a species of fish. In this case, the sample space is the set of all positive real numbers.
|
We can find an approximation for the probability distribution of fish diameters using the same ideas we had for finding the probability distribution for rolling two dice. In this case, to perform a trial we catch a fish and measure its diameter. Unfortunately for our analogy, it is highly unlikely that we will ever catch another fish again with exactly the same diameter. And to make matters worse, we can't even measure a fish diameter precisely anyways. To get around this problem, we could just measure ranges of diameters. For example, diameters of 1-5 cm in one range, diameters of 5-10 cm in the next, and so on. We can imagine that our boat has a number of bins on it, one for each range. When we catch a fish we put it in the appropriate bin for that range. The estimated probability for some range is just the number of fish in the bin for that range divided by the total number of fish caught. The histogram to the right shows the outcome of a set of trials like this. On the x axis we have the diameters of the fish we catch. Remember that the area of each rectangle is an estimate of the probability of catching a fish with a diameter in the corresponding range. Go ahead and catch a bunch of fish. At around 10000 trials, a pattern begins to appear. This pattern becomes sharper as you decrease the size of each range (or equivalently increase the number of bins). |
 |
You can see that area of the bars in the histogram looks more and more like the area under a function as you increase the number of trials and bins. On the left we have drawn a plot of this function. We call this function the probability density function (or p.d.f for short). You can think of the graph of a p.d.f. as a kind of histogram. If f is the p.d.f. for fish diameters, then the probability of catching a fish with diameter between d1 and d2 is just ![\[ \int_{d_1}^{d_2}f(x)dx \]](gifs/index_15.gif)
. |
|
What about the mean value
The contribution of fish of diameter xi to the mean is therefore approximated with xif(xi)dx and the mean itself is approximated with ![\[ \mu=\sum_i x_i f(x_i) dx \]](gifs/index_17.gif)
. |
|
? We can approximate the mean using
bins just like we used bins to approximate the p.d.f. For each bin
i we pick one representative fish
from that bin. It has diameter xi and
we estimate that all the fish in
that bin have diameter xi. The
probability of catching a fish in this bin is approximately the
area of a rectangle of height f(xi)
and width dx, the size of each range.
The diagram to the right shows this relationship.
Question 3 (2 marks)
As we increase the number of bins, this sum tends to which of the following integrals?.
a) ![\[ \int_{0}^{\infty} x dx \]](gifs/index_18.gif)
b) ![\[ \int_{0}^{\infty} x^2f(x) dx \]](gifs/index_19.gif)
c) ![\[ \int_{0}^{\infty} f(x) dx \]](gifs/index_20.gif)
d) ![\[ \int_{0}^{1} f(x) dx \]](gifs/index_21.gif)
e) ![\[ \int_{0}^{\infty} x f(x) dx \]](gifs/index_22.gif)
By similar reasoning, we see that the variance is given by
![\[ \sigma^2=\int_{0}^{\infty} (x-\mu)^2f(x) dx \]](gifs/index_23.gif)
.Many times we might want to ask questions like "What is the probability of catching a fish with a diameter smaller than x cm?" This probability is given by
![\[ F(x)=\int_{0}^{x} f(s) ds \]](gifs/index_24.gif)
.This is such a useful function we give it a name, the cumulative distribution function (or c.d.f. for short). In the diagram below you can see the c.d.f. for our fish. On the right is the probability density function you saw before. On the left is the cumulative distribution function for that function. You can move the dot on the left. The area on the right is the same as the value of the c.d.f. on the left.
Using the tool above you can also see the p.d.f. and c.d.f. for diameters of fish aged 1, 2, and 3 years old. Suppose we want to catch older fish, which are larger and less vital to the survival of the species. The following set of questions shows how the c.d.f. can be used to analyse problems like this.
Question 4 (4 marks)
In the appropriate area below, answer the following questions:
- What is the largest diameter of gill net that will catch 80% of all three-year old fish? (Hint: 20% of three-year old fish are smaller than what diameter?)
- What percent of two-year old fish will be caught using a net of this size?
- What percent of one-year old fish will be caught using a net of this size?
- What percent of all fish will be caught using a net of this size?
You have now completed the lab and you may save your work. If technical problems prevent your answers from being saved, you will receive a message telling you so. In that case, you will either have to submit your work from the Math labs or try again from some other place. We will NOT accept your answers by e-mail.
How are we doing? Want to complain? Pat us on the back?
Please let us know by sending us
email.