## Statistic and Probability

12 Jul

Department of
Applied Mathematics and Statistics
University of California, Santa Cruz
AMS 131: Take-Home Test 2
Target due date: Wed 18 Jul 2018 [520 total points]
1. [70 total points] (the Exchange Paradox) You’re playing the following game against an opponent,
with a referee also taking part. The referee has two envelopes (numbered 1 and 2 for the sake of
this problem, but when the game is played the envelopes have no markings on them), and (without
you or your opponent seeing what she does) she puts \$
m in envelope 1 and \$2 m in envelope 2 for
some
m > 0 (treat m as continuous in this problem even though in practice it would have to be
rounded to the nearest dollar or penny). You and your opponent each get one of the envelopes
at random. You open your envelope secretly and find \$
x (your opponent also looks secretly in his
envelope), and the referee then asks you if you want to trade envelopes with your opponent. You
reason that if you trade, you will get either \$
x2 or \$2 x, each with probability 1 2. This makes the
expected value of the amount of money you’ll get if you trade equal to
! 1 2” !

 \$2x” + ! \$5x

1 2(\$2 x) = 4,
which is greater than the \$
x you currently have, so you o↵er to trade. The paradox is that your
opponent is capable of making exactly the same calculation. How can the trade be advantageous
for both of you?
The point of this problem is to demonstrate that the above reasoning is flawed from a Bayesian
point of view; the conclusion that trading envelopes is always optimal is based on the assumption
that there’s no information obtained by observing the contents of the envelope you get, and this
assumption can be seen to be false when you reason in a Bayesian way. At a moment in time before
the game begins, let
p(m) be your prior distribution on the amount of money M the referee will
put in envelope 1, and let
X be the amount of money you’ll find in your envelope when you open
it (when the game is actually played, the observed
x, of course, will be data that can be used to
M).
(a) Explain why the setup of this problem implies that
P(X = m|M = m) = P(X = 2 m|M =
m) = 1 2, and use this to show that
P(M = x|X = x) = p(x)
p(x) + p! x2and PM = x2 \$ \$ \$ X = x= p(x) + p! x2p“! x2. (1)
Demonstrate from this that the expected value of the amount
Y of money in your opponent’s
envelope, given than you’ve found \$
x in the envelope you’ve opened, is
E(Y |X = x) = p(x)
p(x) + p! x2(2 x) + p(x) + p! x2p“! x2” ⇣x2 . (2)
[20 points]
(b) Suppose that for you in this game, money and utility coincide (or at least suppose that utility is
linear in money for you with a positive slope). Use Bayesian decision theory, through the principle
of maximizing expected utility, to show that you should o↵er to trade envelopes only if
px2 < 2 p(x) . (3)
1
error ..raaa..aa.•amm.
amazon.agqraaa.gg
If you and two friends (one of whom would serve as the referee) were to actually play this game with
real money in the envelopes, it would probably be the case that small amounts of money are more
likely to be chosen by the referee than big amounts, which makes it interesting to explore condition
(3) for prior distributions that are decreasing (that is, for which
p(m2) < p(m1) for m2 > m1). Make
a sketch of what condition (3) implies for a decreasing
p. One possible example of a continuous
decreasing family of priors on
M is the exponential distribution indexed by the parameter !, which
represents the reciprocal of the mean of the distribution. Identify the set of conditions in this family
of priors, as a function of
x and !, under which it’s optimal for you to trade. Does the inequality
you obtain in this way make good intuitive sense (in terms of both
x and !)? Explain briefly. [40
points]
(c) Looking carefully at the correct argument in paragraph 2 of this problem, identify precisely the
point at which the argument in the first paragraph breaks down, and specify what someone who
believes the argument in paragraph 1 is implicitly assuming about the prior distribution
p(m). [10
points]
2. [210 total points] (practice with joint, marginal and conditional densities) This is a toy problem
designed to give you practice in working with a number of the concepts we’ve examined; in a course
like this, every now and then you have to stop looking at real-world problems and just work on
technique (it’s similar to classical musicians needing to practice scales in addition to actual pieces
of symphonic or chamber music).
Suppose that the continuous random vector
X = (X1, X2) has PDF given by
fX(x) = 4 x0 otherwise 1 x2 for 0 < x1 < 1, 0 < x2 < 1 (4)
in which
x = (x1, x2), and define the random vector Y = (Y1, Y2) with the transformation (Y1 =
X1, Y2 = X1 X2).
(a) Are
X1 and X2 independent? Present any relevant calculations to support your answer. [10
points]
(b) Either work out the correlation (X1, X2) between X1 and X2 or explain why no calculation
is necessary in correctly identifying the value of
. [10 points]
(c) Sketch the set S of possible X values and the image T of S under the transformation from
X to Y , and show that the joint distribution of Y = (Y1, Y2) is
fY (y) = 40 otherwise y y2 1 for 0 < y1 < 1, 0 < y2 < y1 < 1 , (5)
in which
y = (y1, y2). Verify your calculation by demonstrating that RR T fY (y) dy = 1. [50
points]
(d) Work out
(i) the marginal distributions for
Y1 and Y2, sketching both distributions and checking that
they both integrate to 1;
(ii) the conditional distributions
fY1 | Y2(y1 | y2) and fY2 | Y1(y2 | y1), checking that they each
integrate to 1; and
2

(iii) the conditional expectations E(Y1 | Y2) and E(Y2 | Y1); and
(iv) the conditional variances
V (Y1 | Y2) and V (Y2 | Y1). (Hint: recall that the variance of a
random variable
W is just E (W 2) ! [E(W)]2.)
[120 points]
(e) Are Y1 and Y2 independent? Present any relevant calculations to support your answer. [10
points]
(f) Either work out the correlation (Y1, Y2) between Y1 and Y2 or explain why no calculation is
necessary in correctly identifying the value of
. [10 points]
3. [100 total points] (moment-generating functions) Distributions may in general be skewed, but
there may be conditions on their parameters that make the skewness get smaller or even disappear.
This problem uses moment-generating functions (MGFs) to explore that idea for two important
discrete distributions, the Binomial and the Poisson.
(a) We saw in class that if
X Binomial(n, p), for 0 < p < 1 and integer n # 1, then the MGF
of
X is given by
X(t) = p et + (1 ! p)n . (6)
for all real
t, and we used this to work out the first three moments of X (note that the
expression for
E (X3) is only correct for n # 3):

 E(X) = n p , E !X2” = n p[(1 + (n ! 1)p] , E !X3” = n p[1 + (n ! 2)(n ! 1)p2 + 3 (n ! 1)p] , from which we also found that V (X) = n p(1 ! p). Show that the above facts imply that skewness(X) = pn p 1 !(12!p p) . (7) (8) (9)

Under what condition on p, if any, does the skewness vanish? Under what condition on n, if
any, does the skewness tend to 0? Explain briefly.
[30 points]
(b) In our brief discussion of stochastic processes we encountered the Poisson distribution: if
Y Poisson(!), for ! > 0, then the PF of Y is
fY (y) = !yy0 otherwise e!!! for y = 0, 1, . . . . (10)
(i) Use this to show that for all real
t the MGF of Y is
Y (t) = e!(et!1) . (11)
[10 points]
(ii) Use Y (t) to compute the first three moments of Y , the variance of Y and the skewness
of
Y . Under what condition on !, if any, does the skewness either disappear or tend to
0? Explain briefly.
[60 points]
3
4. [140 total points] (archaeology) Paleobotanists estimate the moment in the remote past when a
given species became extinct by taking cylindrical, vertical core samples well below the earth’s surface and looking for the last occurrence of the species in the fossil record, measured in meters above
the point
P at which the species was known to have first emerged. Letting {yi, i = 1, . . . , n} denote a
sample of such distances above
P at a random set of locations, the model (Yi|) IID Uniform(0, ✓) ()
emerges from simple and plausible assumptions. In this model the unknown
✓ > 0 can be used,
through carbon dating, to estimate the species extinction time.
The marginal distribution of a single observation
yi in this model may be written
pYi(yi | ) = 0 otherwise 1if 0 yi = 1✓ I (0 yi ) , (12)
where
I(A) = 1 if A is true and 0 otherwise.
(a) Briefly explain why the statement
{0 yi for all i = 1, . . . , n} is equivalent to the
statement
{m = max (y1, . . . yn) }, and use this to show that the joint distribution of
Y = (Y1, . . . , Yn) in this model is
fY1,…,Yn(y1, . . . , yn) = I(m )
n . (13)
[20 points]
(b) Letting the observed values of (Y1, . . . , Yn) be y = (y1, . . . , yn), an important object in both
frequentist and Bayesian inferential statistics is the
likelihood function `(| y), which is obtained from the joint distribution of (Y1, . . . , Yn) simply by
(1) thinking of
fY1,…,Yn(y1, . . . , yn) as a function of for fixed y, and
(2) multiplying by an arbitrary positive constant
c:
`(| y) = c fY (y) . (14)
Using this terminology, in part (a) you showed that the likelihood function in this problem
is
`(| y) = !nI(# m), where m is the largest of the yi values. Both frequentists and
Bayesians are interested in something called the
maximum likelihood estimator (MLE) ˆMLE,
which is the value of
that makes `(| y) as large as possible.
(i) Make a rough sketch of the likelihood function, and use your sketch to show that the
MLE in this problem is
ˆMLE = m = max (y1, . . . yn). [20 points]
(ii) Maximization of a function is usually accomplished by setting its first derivative to 0 and
solving the resulting equation. Briefly explain why that method won’t work in finding
the MLE in this case.
[10 points]
(c) A positive quantity W follows the Pareto distribution (written W Pareto(↵, ‘)) if, for
parameters
↵, ‘ > 0, it has density
fW (w) = ↵ ‘w0 otherwise !(+1) if w # . (15)
This distribution has mean
↵\$
!1 (if ↵ > 1) and variance (!1) ↵\$ 2(2!2) (if ↵ > 2).
4

(i) For frequentists the likelihood function is just a function `(| y), but for Bayesians it
can be regarded as an un-normalized density function for
. Show that, from this point
of view, the likelihood function in this problem corresponds to the Pareto(
n ! 1, m)
distribution.
[10 points]
(ii) Bayes’s Theorem for a one-dimensional continuous unknown (such as in this situation)
says that the conditional density
f| Y (| y) for given Y = y — which is called
the
posterior distribution for given the data — is a positive (normalizing) constant c
times a PDF f() — called the prior distribution for — that captures any available
external to the data set, times the likelihood distribution `(| y):

 f⇥ | Y (✓ | y) = c · f⇥(✓) · `(✓ | y) likelihood ✓ distribution posterior ◆ = ✓ normalizing constant ◆ · ✓ distribution prior ◆ distribution ◆ · ✓

(16)
The posterior distribution is the goal of a Bayesian inferential analysis: it summarizes
all
available information, both external to and internal to your data set. Show that if the
prior distribution for
in this problem is taken to be (15), under the model () above the
posterior distribution is
f| Y (| y) = Pareto [+ n, max(‘, m)]. (Bayesian terminology:
Note that what just happened was that the product of two Pareto distributions (prior,
likelihood) is another Pareto distribution (posterior); a prior distribution that makes this
happen is called
conjugate to the likelihood in the model.) [20 points]
(d) In an experiment conducted in the Antarctic in the 1980s to study a particular species of
fossil ammonite, the following was a linearly rescaled version of the observed data:
y =
(
y1, . . . , yn) = (2.8, 1.7, 1.0, 5.1, 3.7, 1.5, 4.3, 2.0, 3.2, 2.1, 0.4). Prior information equivalent to a Pareto distribution specified by the choice (↵, ‘) = (2.5, 4) was available.
(i) Plot the prior, likelihood, and posterior distributions arising from this data set on
the same graph, explicitly identifying the three curves.
[30 points]
(ii) Work out the posterior mean and SD (square root of the posterior variance), and
use them to complete the following sentence:
On the basis of this prior and data information, the value for this species