45-733 PROBABILITY AND STATISTICS I Topic #3


1 February 2000, 3 February 2000



VIII. Expectation and Variance

  1. Definition: Expected Value
    E(X) = åi=1,n xif(xi) for discrete
    E(X) = ò-¥+¥ xf(x)dx for continuous

  2. Example: Discrete Uniform Distribution: Roll one die
    
                             æ 1/6  x=1,2,3,4,5,6
                      f(x) = ç
                             è 0 otherwise
    

    E(X) = åi=1,6 xif(xi) = 1/6(1 + 2 + 3 + 4 + 5 + 6) = 21/6 = 3.5

  3. E(X) is the mean of f(x). In the simple example of one die, E(X) is quite literally the arithmetic average or center of mass or center of balance -- note that 3.5 is the "midpoint" between 1 and 6.

  4. Example: Continuous Uniform Distribution
    
                             æ 1/(b - a)  a < x < b
                      f(x) = ç
                             è 0 otherwise
    

    E(X) = ò-¥+¥ xf(x)dx = òab x[1/(b-a)]dx = [1/(b-a)](x2/2)|ab =
    [1/(b-a)][(b2 - a2)/2] = (b+a)/2

  5. Example: Bernoulli Distribution
    
                             æ  p  x = 1
                             ç 
                      f(x) = ç 1-p x = 0
                             ç
                             è  0 otherwise
    

    E(X) = åi=1,2 xif(xi) = 1*p + 0*(1 - p) = p

  6. Example:
    
                             æ 3x2  0 < x < 1
                      f(x) = ç
                             è   0 otherwise
    

    E(X) = ò-¥+¥ xf(x)dx = ò01 x[3x2]dx = [3x4/4]|01 = 3/4

  7. Expected Value of a Function of a Random Variable
    Let r(X) be any function of X. For example:
    r(X) = aX + b; where a and b are constants;
    r(X) = X1/4
    r(X) = eX; and so on.
    Then
    E(r(X)) = åi=1,n r(xi)f(xi) for discrete
    E(r(X)) = ò-¥+¥ r(x)f(x)dx for continuous

  8. Example: E(r(X)) = X2 for Discrete Uniform Distribution (roll one dice)
    E(X2) = åi=1,6 (xi2)*(1/6) = (1/6)(1 + 4 + 9 + 16 + 25 + 36) = 91/6

  9. Example: E(r(X)) = X2 for Continuous Uniform Distribution
    E(X2) = òab(x2)(1/[b-a])dx = (1/[b-a])òab(x2)dx = (1/[b-a])(x3/3)|ab =
    (1/3[b-a])(b3 - a3) = (b2 + ab + a2)/3

  10. Example: E(r(X)) = X2 for Bernoulli Distribution
    E(X2) = (12)*p + (02)*(1-p) = p

  11. Example: E(r(X)) = X2 for
    
                             æ 3x2  0 < x < 1
                      f(x) = ç
                             è   0 otherwise
    

    E(X2) = ò01(x2)3x2dx = ò013x4dx = (3x5/5)|01 = 3/5

  12. Two Important Properties of Expected Values

    1. E(aX + b) = aE(X) + b
      To see this, let r(X) = aX + b, then:
      E(r(X)) = E(aX + b) = ò-¥+¥ (ax + b)f(x)dx =
      ò-¥+¥ axf(x)dx + ò-¥+¥ bf(x)dx =
      a(ò-¥+¥ xf(x)dx) + b(ò-¥+¥ f(x)dx) = aE(X) + b

      because the the term, ò-¥+¥ xf(x)dx, is just the definition of E(X), and
      ò-¥+¥ f(x)dx = 1 by the 2nd Axiom of Probability.
      Note that this shows that the expected value of a constant is a constant!
      That is, E(b) = b, where b is a constant. Because E(X) is a constant, this means that:
      E[E(X)] = E(X)!

    2. E(X1 + X2 + ... + Xn) = E(X1) + E(X2) + ... + E(Xn)
      Proof: This will be shown in either Notes #7 or Notes #8.

  13. Variance
    Variance is a measure of the dispersion of a random variable.
    Definition: Variance: VAR(X) = s2 = E{[X - E(X)]2}
    Simplifying: VAR(X) = s2 = E{X2 -2XE(X) + [E(X)]2} =
    E(X2) -2E(X)E(X) + [E(X)]2 = E(X2) - [E(X)]2

    Note that we used (12A) to derive this.

  14. Standard Deviation
    The standard deviation of a random variable is just the square root of the variance. Namely:
    s = [VAR(X)]1/2
    The standard deviation is in the same units as the random variable! This is not true of the variance!

  15. Example: VAR(X) for Discrete Uniform Distribution (roll one dice)
    From (2) and (8), VAR(X) = s2 = 91/6 - (21/6)2

  16. Example: VAR(X) for Continuous Uniform Distribution
    From (4) and (9), VAR(X) = s2 = (b2 + ab + a2)/3 - [(b + a)/2]2 = [(b - a)2]/12

  17. Example: VAR(X) for Bernoulli Distribution
    From (5) and (10), VAR(X) = s2 = p - p2 = p(1 - p)

  18. Example: VAR(X) for:
    
                             æ 3x2  0 < x < 1
                      f(x) = ç
                             è   0 otherwise
    

    From (6) and (11), VAR(X) = 3/5 - (3/4)2 = 3/80
  19. Problem 3.18 p.87
    Let X = "daily sales in dollars". Clearly the range of X is $0, $50,000, $100,000.
    To get P(X = 100,000) note that the salesman would have to contact two customers and sell both customers the heavy-equipment.
    Hence: P(X = 100,000) = (2/3)*(1/10)*(1/10) = 2/300
    To get P(X = 0) the salesman could either contact one customer and make no sale, or contact two customers and make no sales.
    Hence: P(X = 0) = (1/3)*(9/10) + (2/3)*(9/10)*(9/10) = 252/300
    To get P(X = 50,000) the salesman could either contact one customer and make a sale, or contact two customers and make only one sale.
    Hence: P(X = 50,000) = (1/3)*(1/10)+ (2/3)*[(1/10)*(9/10) + (9/10)*(1/10)] = 46/300
    This allows us to state the distribution of X:
    
    
                              æ 252/300  x = 0  
                              ç
                              ç  46/300  x = $50,000  
                       f(x) = ç
                              ç   2/300  x = $100,000  
                              ç
                              è  0 otherwise
    

    1. To get the mean of daily sales:
      E(X) = åi=1,3 xif(xi) = 0*(252/300) + 50,000*(46/300) + 100,000*(2/300) = $8333.00

    2. To get the standard deviation, s, of daily sales, we calculate the variance and take its square root.
      E(X2) = åi=1,3 (xi)2f(xi) = (02)*(252/300) + (50,0002)*(46/300) + (100,0002)*(2/300) = 450,000,000
      VAR(X) = s2 = 450,000,000 - (83332) = 380,561,111
      So that: std. dev. = [VAR(X)1/2] = s = $19,507.98
      Recall that the standard deviation is in the original units of X and can be so interpreted! This is not true of VAR(X).

  20. Two Important Properties of Variances

    1. VAR(aX + b) = (a2)VAR(X) = (a2)s2
      To see this, let r(X) = aX + b, then:
      VAR[r(X)] = E{[aX + b - E(aX + b)]2} = E{[aX + b - aE(X) - b]2} =
      E{[a2][X - E(X)]2} = (a2)E{[X - E(X)]2} = (a2)VAR(X) = (a2)s2

      Note that (12a) of notes #5 was used to derive this.
    2. If X1, X2, ... , Xn, are independent random variables, then:
      VAR(a1X1 + a2X2 + ... + anXn + b) = åi=1,n (ai)2VAR(Xi) = åi=1,n (ai)2 si2

      See discussion in Notes #8.

  21. Problem 4.19 p.149
    By inspection, we see that this is a continuous uniform distribution with parameters:
    b = 61 and a = 59. Hence:
    E(X) = (b + a)/2 = 60 and VAR(X) = s2 = [(b - a)2]/12 = 4/12 = 1/3

  22. Problem 4.20 p.149
    
                             æ 2y  0 < y < 1
                      f(y) = ç
                             è 0 otherwise
    

    1. Find the mean and the variance:
      E(Y) = ò01 y*2ydy = (2(y3)/3)|01 = 2/3
      E(Y2) = ò01 (y2)*2ydy = (2y4/4)|01 = 1/2
      Hence: VAR(Y) = s2 = (1/2) - (2/3)2 = 1/18
    2. Let the profit be equal to: X = 200Y - 60. Find the mean and the variance of X.
      E(X) = E(200Y - 60) = 200E(Y) - 60 = 200*(2/3) - 60 = 220/3
      VAR(X) = VAR(200Y - 60) = (2002)VAR(Y) = 20,000/9
    3. To solve c) we need the Tchebysheff Inequality. Since I am not teaching it, we will skip this!