PRML 第1章 1.12(標準)

---

$$
\begin{array}{ll}
(1.49) & \mathbb{E}[x] = {\displaystyle \int_{-\infty}^\infty} \mathcal{N}(x|\mu,\sigma^2)~xdx = \mu\\[1em]
(1.50) & \mathbb{E}[x^2] = {\displaystyle \int_{-\infty}^\infty} \mathcal{N}(x|\mu,\sigma^2)~x^2dx = \mu^2 + \sigma^2\\[1em]
(1.53) & p(\mathbf{x} | \mu,\sigma^2) = {\displaystyle \prod_{n=1}^N} \mathcal{N}(x_n|\mu,\sigma^2)\\[1em]
(1.57) & \mathbb{E}[\mu_{\scriptscriptstyle\mathrm{ML}}] = \mu\\[1em]
(1.50) & \mathbb{E}[\sigma_{\scriptscriptstyle\mathrm{ML}}^2] = \left( \dfrac{N-1}{N} \right) \sigma^2\\[1em]
(1.130) & \mathbb{E}[x_nx_m] = \mu^2 + I_{nm}\sigma^2
\end{array}
$$

---

解答

　$${m = n}$$の場合，

$$
\begin{array}{l}
\mathbb{E}[x_nx_m] = \mathbb{E}[x_n^2] = \mu^2 + \sigma^2 = \mu^2 + 1\cdot\sigma^2
\end{array}
$$

となる．$${m \neq n}$$の場合，$${x_m}$$，$${x_n}$$は独立同分布(i.i.d.)であることから，(1.53)より，

$$
\begin{array}{l}
p(x_n,x_m | \mu,\sigma^2) = \mathcal{N}(x_n|\mu,\sigma^2) \mathcal{N}(x_m|\mu,\sigma^2)
\end{array}
$$

である．したがって，

$$
\begin{array}{lll}
\mathbb{E}[x_nx_m] &=& {\displaystyle \iint} p(x_n,x_m | \mu,\sigma^2)~x_nx_m dx_n dx_m\\[1em]
&=& {\displaystyle \iint} \mathcal{N}(x_n|\mu,\sigma^2) \mathcal{N}(x_m|\mu,\sigma^2)~x_nx_m dx_n dx_m\\[1em]
&=& {\displaystyle \int} \mathcal{N}(x_n|\mu,\sigma^2)~x_n dx_n {\displaystyle \int} \mathcal{N}(x_m|\mu,\sigma^2)~x_m dx_m\\[1em]
&=& \mu^2\\[1em]
&=& \mu^2 + 0\cdot\sigma^2
\end{array}
$$

となる．以上により，(1.130)が成り立つ．

　また，$${\mathbb{E}}$$の線形性を用いることで，

$$
\begin{array}{lll}
\mathbb{E}[\mu_{\scriptscriptstyle\mathrm{ML}}]
&=& \mathbb{E} \left[ \dfrac{1}{N} {\displaystyle \sum_{n=1}^N} x_n \right]
= \dfrac{1}{N} {\displaystyle \sum_{n=1}^N} \mathbb{E}[x_n]
= \dfrac{1}{N} {\displaystyle \sum_{n=1}^N} \mu
= \mu\\[3em]
\mathbb{E}[\sigma_{\scriptscriptstyle\mathrm{ML}}^2]
&=& \mathbb{E} \left[ \dfrac{1}{N} {\displaystyle \sum_{n=1}^N} (x_n - \mu_{\scriptscriptstyle\mathrm{ML}})^2 \right]\\[2em]
&=& \dfrac{1}{N} \mathbb{E} \left[ {\displaystyle \sum_{n=1}^N} \left(x_n - \dfrac{1}{N} {\displaystyle \sum_{m=1}^N} x_m\right)^2 \right]\\[2em]
&=& \dfrac{1}{N} {\displaystyle \sum_{n=1}^N} \mathbb{E} \left[ x_n^2 - \dfrac{2}{N} {\displaystyle \sum_{m=1}^N} x_nx_m + \dfrac{1}{N^2} \left( {\displaystyle \sum_{m=1}^N} x_m\right)^2 \right]\\[2em]
&=& \dfrac{1}{N} {\displaystyle \sum_{n=1}^N} \left\{ \mathbb{E} [x_n^2] - \dfrac{2}{N} {\displaystyle \sum_{m=1}^N} \mathbb{E}[x_nx_m] + \dfrac{1}{N^2} {\displaystyle \sum_{k=1}^N \sum_{l=1}^N} \mathbb{E}[x_kx_l] \right\}\\[2em]
&=& \dfrac{1}{N} {\displaystyle \sum_{n=1}^N} \left\{ (\mu^2 + \sigma^2) - \dfrac{2}{N} {\displaystyle \sum_{m=1}^N} (\mu^2 + I_{nm}\sigma^2) + \dfrac{1}{N^2} {\displaystyle \sum_{k=1}^N \sum_{l=1}^N} (\mu^2 + I_{kl}\sigma^2) \right\}\\[2em]
&=& \dfrac{1}{N} {\displaystyle \sum_{n=1}^N} \left\{ (\mu^2 + \sigma^2) - \dfrac{2}{N}(N\mu^2 + \sigma^2) + \dfrac{1}{N^2} (N^2 \mu^2+ N\sigma^2) \right\}\\[2em]
&=& \dfrac{1}{N} {\displaystyle \sum_{n=1}^N} \dfrac{N-1}{N}\sigma^2\\[2em]
&=& \left( \dfrac{N-1}{N} \right) \sigma^2
\end{array}
$$

が成り立つ．