AVFedotovLogA002 < Main

Main Web>TWikiUsers>AlexanderFedotov>AVFedotovLogA>AVFedotovLogA002 (2010-02-01, AlexanderFedotov) (raw view)
   * Set DISABLEDPLUGINS = LatexModePlugin
   * Set LATEXFONTSIZE = footnotesize


-- Main.AlexanderFedotov - 04-Dec-2009

<center> 

              |  <h1> Sample Statistics </h1>  |

</center> 

%TOC{ title = "Sections:" }%

---++ 1 Population quantities

Let %$ x $% have  a probability density function %$ P(x) $%


---+++ 1.1 Moments = Raw Moments = Crude Moments. Mean.


A _raw moment_ %$ \mu'_n $% 
(or just a _moment_, or a _crude moment_ )
is a moment taken about 0 ![[[#ReF1][1]]] :
    <center><latex>
       \begin{array}{lr} \displaystyle
        \mu'_n = \langle x^n \rangle=  \int x^n P(x) dx & \qquad  (1.1.1) 
       \end{array}
    </latex></center>
By definition, _the first moment_ is the _mean_ of the distribution, %$ \mu $%,
    %\[  \mu = \langle x \rangle = \mu'_1 \quad . \qquad (1.1.2)  \]%



---+++ 1.2 Central moments. Variance. Relations with the raw moments

A _central moment_ %$ \mu _n $% 
is a moment taken about the _mean_ %$ \mu = \mu' _1 $%  ![[[#ReF2][2]]] :
    <center><latex>
       \begin{array}{lr} \displaystyle
        \mu_n = \langle (x - \mu)^n \rangle=  \int (x - \mu)^n P(x) \ dx \quad .
        & \qquad (1.2.1)
       \end{array}
    </latex></center>

By definition the second central moment %$ \mu_2 $% is the _variance_
which is usually denoted as %$ \sigma^2 $%  ![[[#ReF10][10]]]:
%\[  \sigma^2 \equiv \mu_2  \qquad . \qquad  (1.2.2) \]% 
The square root of the variance is called ![[[#ReF10][10]]]
the _standard deviation_ :
%\[  \sigma \equiv \sqrt{\sigma^2}  \qquad . \qquad  (1.2.3) \]% 


The _central_ moments are expressed via the _raw_ moments using
 _binomial transform_  ![[[#ReF2][2]]] :
    <center><latex>
        \begin{array}{lr} \displaystyle
        \mu_n =  \sum_{k=0}^n \binom{n}{k} (-1)^{n-k} \mu'_k \, \mu'_1{}^{n-k} \quad . 
        & \qquad(1.2.4)
       \end{array}
    </latex></center>
In particular:
    <center><latex>
       \begin{array}{lr}
       \mu_1 = 0 & (1.2.4a) \\
       \mu_2 = -\mu'_1{}^2 + \mu'_2  & (1.2.4b) \\
       \mu_3 = 2 \mu'_1{}^3 - 3 \mu'_1 \mu'_2 + \mu'_3  & (1.2.4c) \\
       \mu_4 = -3 \mu'_1{}^4 + 6 \mu'_1{}^2 \mu'_2 - 4 \mu'_1 \mu'_3 + \mu'_4 & \qquad (1.2.4d) 
       \end{array}
    </latex></center>
The _raw_ moments are expressed via the _central_ moments using
_inverse binomial transform_  ![[[#ReF1][1]]]
(note  %$ \mu_0 = 1 $% and %$ \mu_1 = 0 $% ):
    <center><latex>
        \mu'_n =  \sum_{k=0}^n \binom{n}{k}  \mu_k \, \mu'_1{}^{n-k} \quad . 
        \qquad(1.2.5)
    </latex></center>
In particular:
    <center><latex>
     \begin{array}{lr}
       \mu'_1 = \mu'_1 &\text{ (an identity)} \\
       \mu'_2 = \mu_2{}  + \mu'_1{}^2 & (1.2.5b) \\
       \mu'_3 = \mu_3 + 3 \mu_2 \mu'_1 + \mu'_1{}^3 & (1.2.5c) \\
       \mu'_4 = \mu_4 + 4 \mu_3 \mu'_1 + 6 \mu_2 \mu'_1{}^2 + \mu'_1{}^4
                                                                   &\qquad (1.2.5d)
     \end{array}
    </latex></center>




---+++ 1.3 Cumulants and their expressions via moments

The _characteristic function_  %$ \phi(t) $% associated with the 
_probability density function_ %$ P(x) $%  is defined as a _Fourier transform_ 
![[[#ReF3][3]]] :
    %\[ \phi(t) =  \int_{-\infty}^{\infty} e^{itx} P(x) dx \quad . \qquad(1.3.1) \]%
The cumulants  %$ \kappa_n $%  are then defined by 
    %\[ \ln \phi(t) \equiv  \sum_{n=1}^{\infty} \kappa_n \frac{(it)^n}{n!} \quad .    \qquad(1.3.2) \]%
They can be expressed through _raw moments_  %$ \mu'_n $%:
    <center><latex>
     \begin{array}{lr}
       \kappa_1 = \mu'_1 & (1.3.3a) \\
       \kappa_2 = \mu'_2  - \mu'_1{}^2 & (1.3.3b) \\
       \kappa_3 = 2\mu'_1{}^3 - 3 \mu'_1 \mu'_2 + \mu'_3 & (1.3.3c) \\
       \kappa_4 = -6 \mu'_1{}^4 + 12 \mu'_1{}^2 \mu'_2 - 3 \mu'_2{}^2
                          -4  \mu'_1 \mu'_3 + \mu'_4 & \qquad (1.3.3d) \\
       \ldots &  \\
     \end{array}
    </latex></center>
or in terms of _central moments_ %$ \mu_n $% :
    <center><latex>
     \begin{array}{lr}
       \kappa_1 = \mu     & (1.3.4a) \\
       \kappa_2 = \mu_2   & (1.3.4b) \\
       \kappa_3 = \mu_3   &  (1.3.4c) \\
       \kappa_4 =  \mu_4 - 3 \mu_2{}^2   & \qquad (1.3.4d) \\
       \ldots \\
     \end{array}
    </latex></center>
where %$ \mu $% is the _mean_ and %$ \sigma^2 \equiv \mu_2 $% is the _variance_.


---++ 2 Sample quantities

A _sample_ is a subset of a population ![[[#ReF4][4]]] 
          %\[  x_1, ... , x_N  \]%
where %$ N $% is the size of the sample.

---+++ 2.1 Power sums

_Power sum_ is the sum of %$p$%th powers of the sample elements:
     %\[  S_p =  \sum_{i=1}^N x_i^p \quad . \qquad (2.1.1) \]%

---+++ 2.2 Sample raw moments

The %$ r $%th _sample raw moment_ %$ m'_r $% is defined as  ![[[#ReF5][5]]] 
     %\[  m'_r =   \frac{1}{N}  \sum_{i=1}^N x_i^r \quad . \qquad (2.2.1) \]%
They are related to the _power sums_ (2.1.1) by
     %\[  m'_r =   \frac{S_r}{N}   \quad . \qquad (2.2.2) \]%
They are unbiased estimators of the population _raw moments_ (1.1.1) :
     %\[  \langle m'_r \rangle =   \mu'_r   \quad . \qquad (2.2.3) \]%

---+++ 2.3 Sample mean

The _sample mean_ ![[[#ReF6][6]]] is defined by
    %\[  m =   \frac{1}{N}  \sum_{i=1}^N x_i \quad . \qquad (2.3.1) \]%
It is equal to the sample _first moment_  (2.2.1),
    %\[  m =   m'_1 \quad . \qquad (2.3.1) \]%
It is an unbiased estimator for the population mean %$ \mu $% (1.1.2)

---+++ 2.4 Sample central moments

The %$ r $%th _sample central moment_ %$ m_r $% is defined as  ![[[#ReF7][7]]] 
     %\[  m_r =   \frac{1}{N}  \sum_{i=1}^N (x_i - m )^r \quad . \qquad (2.4.1) \]%
where %$ m = m'_1 $% is the sample _mean_ .
The first few sample central moments are related to power sums %$ S_k $% by 
    <center><latex size="normalsize">
     \begin{array}{lr}
       m_1 = 0     & (2.4.2a) \\ \\
       m_2 = -\frac{S_1^2}{N^2} + \frac{S_2}{N}  & (2.4.2b) \\ \\
       m_3 =  \frac{2 S_1^3}{N^3} - \frac{3 S_1 S_2}{N^2} + \frac{S_3}{N} &  (2.4.2c) \\ \\
       m_4 =  -\frac{3 S_1^4}{N^4} + \frac{6 S_1^2 S_2}{N^3}
                   - \frac{4 S_1 S_3}{N^2} + \frac{S_4}{N} & \qquad (2.4.2d) \\
       \ldots \\
     \end{array}
    </latex></center>

In terms of the population central moments, the expectation values  of the first few sample central moments are 
    <center><latex size="normalsize">

     \begin{array}{lr}
       \\ 
       \langle m_2 \rangle =  \frac{ (N-1) }{N} \mu_2                          & (2.4.3b) \\ \\
       \langle m_3 \rangle =  \frac{ (N-1)(N-2) }{N^2} \mu_3              &  (2.4.3c) \\ \\
       \langle m_4 \rangle =  \frac{ (N-1)}{N^3} [3(2N-3) \mu_2^2 +(N^2-3N+3) \mu_4 ] 
                                                                                                           & \qquad (2.4.3d) \\
       \ldots \\
     \end{array}

    </latex></center>

---+++ 2.5 k-Statistics

The %$ n $%th _k-statistic_ %$ k_n $% is ![[[#ReF8][8]]]
the unique symmetric unbiased estimator of the cumulant %$ \kappa_n $%
(see. e.g. eq.(1.3.4)), i.e., %$ k_n $% is defined so that
%\[ \langle k_n \rangle = \kappa_n \qquad . \qquad (2.5.1) \]%
In addition, the variance
%\[ var(k_n) = \langle (k_n - \kappa_n)^2 \rangle  \qquad (2.5.2) \]%
is a minimum compared to all other unbiased estimators of %$ \kappa_n $%.

The k-statistics can be given in terms of the power sums (2.1.1)

    <center><latex>
     \begin{array}{llr}
       k_1 = & \frac{1}{N} S_1     & (2.5.3a) \\ \\
       k_2 = & \frac{1}{N(N-1)} [NS_2 - S_1^2]  & (2.5.3b) \\ \\
       k_3 = & \frac{1}{N(N-1)(N-2)} [2S_1^3 -3 N S_1 S_2 + N^2 S_3 ] &  (2.5.3c) \\ \\
       k_4 = & \frac{1}{N(N-1)(N-2)(N-3)} [-6 N_1^4 + 12 N S_1^2 S_2^2 & \\ \\
             & \quad - 3 N(N-1)S_2^2 -4N(N+1)S_1 S_3 + N^2(N+1)S_4] & \qquad (2.5.3d) \\
       \ldots \\
     \end{array}
    </latex></center>

Alternatively, they can be expressed via the sample mean %$ m $% (2.3.1)
and central moments %$ m_i $% (2.4.1) by
    <center><latex size = "normalsize">
     \begin{array}{lr}
       k_1 = m     & (2.5.4a) \\ \\
       k_2 = \frac{N}{N-1} m_2  & (2.5.4b) \\ \\
       k_3 = \frac{N^2}{(N-1)(N-2)} m_3  &  (2.5.4c) \\ \\
       k_4 = \frac{N^2}{(N-1)(N-2)(N-3)}[(N+1) m_4 - 3(N-1) m_2^2]  & \qquad (2.5.4d) \\
       \ldots \\
     \end{array}
    </latex></center>

Since the %$ k_i $% is defined to be an unbiased estimator for the %$ \kappa_i $%, one has %$ \langle k_i \rangle  = \kappa_i $%, and then eqs.(1.3.3) give for the _expectation values_ :
    <center><latex>
     \begin{array}{lr}
       \langle k_1 \rangle  = \mu  & (2.5.5a) \\
       \langle k_2 \rangle  = \mu_2  & (2.5.5b) \\
       \langle k_3 \rangle  = \mu_3  & (2.5.5c) \\
       \langle k_4 \rangle  = \mu_4 - 3\mu_2^2 & \qquad (2.5.5d) \\
       \ldots \\
     \end{array}
    </latex></center>

The variances of the first few k-statistics are given by 
    <center><latex size = "normalsize">
     \begin{array}{lr}
       \mathrm{var}(k_1) = \frac{\kappa_1}{N}  & (2.5.6a) \\ \\
       \mathrm{var}(k_2) = \frac{\kappa_4}{N} + \frac{2\kappa_2^2}{N-1} & (2.5.6b) \\ \\
       \mathrm{var}(k_3) =  \frac{\kappa_6}{N} +\frac{9(\kappa_2\kappa_4 + \kappa_3^2) }{N-1} + \frac{6N\kappa_2^3}{(N-1)(N-2)} & \qquad  (2.5.6c) \\
       \ldots \\
     \end{array}
    </latex></center>

An _unbiased estimator_ for %$ \mathrm{var}(k_2) $% is given by
%\[ \hat{\mathrm{var}(k_2)} = \frac{2Nk_2^2 + (N-1) k_4}{N(N+1)} \qquad \qquad  (2.5.7) \]%

In the special case of a *normal* parent population,
an _unbiased estimator_ for %$ \mathrm{var}(k_3) $% is given by
%\[ \hat{ \mathrm{var}(k_3)} = \frac{6N(N-1)k_2^3}{(N-2)(N+1)(N+3)} \qquad \qquad  (2.5.8) \]%



---+++ 2.6 Sample variance 


The sample variance is the second central moment %$ m_2 $% (2.4.1):
  %\[  m_2 \equiv \frac{1}{N} \sum_{i=1}^N (x_i - m)^2  \qquad ,\qquad (2.6.1) \]%
where %$ m $% is the sample mean (2.3.1) .
It is commonly written as ![[[#ReF9][9]]]
  %\[ s^2 \equiv m_2 \qquad ,\qquad (2.6.2)  \]%
or sometimes
  %\[ s_N^2 \equiv m_2 \qquad ,\qquad (2.6.2)  \]%

The square root of the sample variance is called
the _sample standard deviation_ ![[[#ReF10][10]]]
  %\[ s_N \equiv \sqrt{ s_N^2} \qquad .\qquad (2.6.3)  \]%
Another widespread definition for the sample standard deviation is
  %\[ s_{N-1} \equiv \sqrt{ s_{N-1}^2} \qquad ,\qquad (2.6.4)  \]%
where
  %\[  s_{N-1}^2 \equiv \frac{1}{N-1} \sum_{i=1}^N (x_i - m)^2 = \frac{N}{N-1} s_N^2 =\frac{N}{N-1} m_2 \qquad .\qquad (2.6.5) \]%
By definition the %$ s_{N-1}^2 $% statistic is an alternative name
for the %$ k_2$% statistic defined above, see eq.(2.5.4b):
  %\[ s_{N-1}^2 = k_2  \qquad .\qquad (2.6.6) \]%

According to eq.(2.4.3b), the expectation values for the variance and
the %$ s_{N-1}^2 $% statistic are
  %\[ \langle s^2 \rangle = \langle m_2 \rangle  =  \frac{N-1}{N}\mu_2 \qquad ,\qquad (2.6.6) \]% 
  %\[  \langle s_{N-1}^2 \rangle =  \mu_2  \qquad .\qquad (2.6.7) \]% 
Surely, the eq.(2.6.7) agrees with the eq.(2.5.5b).

The variances are (eq.(4) in ![[[#ReF11][11]]])
  %\[ \mathbf{D}(s^2) = \frac{(N-1)^2}{N^3}\Big[\mu_4 - \frac{N-3}{N-1}\mu_2^2 \Big]  \qquad ,\qquad (2.6.8)\]%
<br>
  %\[ \mathbf{D}(s_{N-1}^2) = \frac{1}{N}\Big[\mu_4 - \frac{N-3}{N-1}\mu_2^2 \Big]  \qquad .\qquad (2.6.9)\]% 

Eq.(23) in ![[[#ReF11][11]]] gives a usefull expression for the
 %$ \langle s^4 \rangle $% :
  %\[ \langle s^4 \rangle = \langle m_2^2 \rangle  = \frac{N-1}{N^3}\Big[(N-1)\mu_4 + (N^2 - 2N + 3)\mu_2^2\Big] \qquad .\qquad (2.6.10)\]% 


---+++ 2.7 Unbiased estimators for %$ \mu_i $% and %$ \mu_2^2 $%

According to eq.(2.5.5a-c) the unbiased estimators for
%$ \mu $%, %$ \mu_2 $% and %$ \mu_3 $% are given by %$ k_1 = m $%,
%$ k_2 = s_{N-1}^2 $% and (see eq.(2.5.4c)) %$ k_3 $%,
respectively.
It is also important to have estimators for %$ \mu_4 $% and
%$ \mu_2^2 $%.

Eqs. (2.4.3d) and (2.6.10) can be combined into a matrix equation

 <center><latex>
  \left(
  \begin{array}{c}
    \langle m_4   \rangle \\ \\
    \langle m_2^2 \rangle
  \end{array}
  \right)
  =
  \left(
  \begin{array}{ll}
    \frac{N-1}{N^3} (N^2-3N+3) &
    \frac{N-1}{N^3} 3(2N-3)\\ \\
    \frac{N-1}{N^3} (N-1)&
    \frac{N-1}{N^3} (N^2-2N+3)
  \end{array}
  \right)
  \cdot
  \left(
  \begin{array}{c}
    \mu_4  \\ \\
    \mu_2^2
  \end{array}
  \right)
  \quad . \quad (2.7.1)
</latex></center>

Solving this system of equations relative to %$ \mu_4 $% and
%$ \mu_2^2 $% gives

  %\[ \mu_4 = \frac{N \big[ (N^2-2N+3) \langle m_4 \rangle - 3(2N-3) \langle m_2^2 \rangle \big] }{(N-1)(N-2)(N-3)} \qquad , \qquad (2.7.2) \]%
  %\[ \mu_2^2 = \frac{N \big[ (N^2-3N+3) \langle m_2^2 \rangle - (N-1) \langle m_4 \rangle   \big] }{(N-1)(N-2)(N-3)} \qquad . \qquad (2.7.3) \]%

Removing the averaging brackets around %$ m_4 $% and %$ m_2^2 $%
at the right hand side of the equations, one gets the estimators
at the left hand side:

  %\[ \hat{ \mu_4 } = \frac{N \big[ (N^2-2N+3)  m_4  - 3(2N-3)  m_2^2  \big] }{(N-1)(N-2)(N-3)} \qquad , \qquad (2.7.4) \]%

  %\[ \hat{ \mu_2^2 } = \frac{N \big[ (N^2-3N+3) m_2^2  - (N-1)  m_4   \big] }{(N-1)(N-2)(N-3)} \qquad . \qquad (2.7.5) \]%



---+++ 2.8 Unbiased estimator for %$ \mathbf{D}(s_{N-1}^2) $%

Replacing %$ \mu_4 $% and %$ \mu_2^2 $% in the expression (2.6.9) 
for %$ \mathbf{D}(s_{N-1}^2) $% by their estimators (2.7.4-5), gives
the estimator for %$ \mathbf{D}(s_{N-1}^2) $%:
  %\[ \mathbf{Est} \big[ \mathbf{D}(s_{N-1}^2)  \big] = \frac{N \big[ (N-1)^2 m_4- (N^2-3)m_2^2  \big] }{(N-1)^2(N-2)(N-3)} \qquad . \qquad (2.8.1) \]%

As %$ S_{N-1}^2 = k_2 $%, eq.(2.6.6), the same result (2.8.1)
can be obtained using formula (2.5.7) by plugging into it the
expressions (2.5.4b,c) for %$ k_2,\ k_4 $%.




---++ References

#ReF1
[1] Weisstein, Eric W. "Raw Moment." From *MathWorld--A Wolfram Web Resource*.
http://mathworld.wolfram.com/RawMoment.html
 , [[%ATTACHURL%/WMathWorld.RawMoment.pdf ][pdf]]

#ReF2
[2] Weisstein, Eric W. "Central Moment."
  http://mathworld.wolfram.com/CentralMoment.html
 , [[%ATTACHURL%/WMathWorld.CentralMoment.pdf ][pdf]]

#ReF3
[3] Weisstein, Eric W. "Cumulant."
  http://mathworld.wolfram.com/Cumulant.html
 , [[%ATTACHURL%/WMathWorld.Cumulant.pdf ][pdf]]

#ReF4
[4] Weisstein, Eric W.  "Sample."
 http://mathworld.wolfram.com/Sample.html
 , [[%ATTACHURL%/WMathWorld.Sample.pdf ][pdf]]

#ReF5
[5] Weisstein, Eric W.  "Sample Raw Moment."
 http://mathworld.wolfram.com/SampleRawMoment.html
 , [[%ATTACHURL%/WMathWorld.SampleRawMoment.pdf ][pdf]]

#ReF6
[6] Weisstein, Eric W.  "Sample Mean."
 http://mathworld.wolfram.com/SampleMean.html
 , [[%ATTACHURL%/WMathWorld.SampleMean.pdf ][pdf]]

#ReF7
[7] Weisstein, Eric W.  "Sample Central Moment."
  http://mathworld.wolfram.com/SampleCentralMoment.html
 , [[%ATTACHURL%/WMathWorld.SampleCentralMoment.pdf ][pdf]]

#ReF8
[8] Weisstein, Eric W.  "k-Statistic."
http://mathworld.wolfram.com/k-Statistic.html
 , [[%ATTACHURL%/WMathWorld.k-Statistic.pdf ][pdf]]

#ReF9
[9] Weisstein, Eric W.  "Sample Variance."
http://mathworld.wolfram.com/SampleVariance.html
 , [[%ATTACHURL%/WMathWorld.SampleVariance.pdf ][pdf]]

#ReF10
[10] Weisstein, Eric W.  "Standard Deviation."
http://mathworld.wolfram.com/StandardDeviation.html
 , [[%ATTACHURL%/WMathWorld.StandardDeviation.pdf ][pdf]]

#ReF11
[11] Weisstein, Eric W.  "Sample Variance Distribution."
http://mathworld.wolfram.com/SampleVarianceDistribution.html
 , [[%ATTACHURL%/WMathWorld.SampleVarianceDistribution.pdf ][pdf]]
Attachments
Topic attachments
I	Attachment	History	Action	Size	Date	Who
pdf	WMathWorld.CentralMoment.pdf	r1	manage	69.4 K	2010-02-01 - 10:28	AlexanderFedotov
pdf	WMathWorld.Cumulant.pdf	r1	manage	73.2 K	2010-02-01 - 10:29	AlexanderFedotov
pdf	WMathWorld.RawMoment.pdf	r1	manage	70.5 K	2010-02-01 - 10:31	AlexanderFedotov
pdf	WMathWorld.Sample.pdf	r1	manage	28.7 K	2010-02-01 - 10:31	AlexanderFedotov
pdf	WMathWorld.SampleCentralMoment.pdf	r1	manage	53.3 K	2010-02-01 - 10:33	AlexanderFedotov
pdf	WMathWorld.SampleMean.pdf	r1	manage	33.8 K	2010-02-01 - 10:34	AlexanderFedotov
pdf	WMathWorld.SampleRawMoment.pdf	r1	manage	33.2 K	2010-02-01 - 10:34	AlexanderFedotov
pdf	WMathWorld.SampleVariance.pdf	r1	manage	45.1 K	2010-02-01 - 10:35	AlexanderFedotov
pdf	WMathWorld.SampleVarianceDistribution.pdf	r1	manage	115.6 K	2010-02-01 - 10:36	AlexanderFedotov
pdf	WMathWorld.StandardDeviation.pdf	r1	manage	57.6 K	2010-02-01 - 10:37	AlexanderFedotov
pdf	WMathWorld.k-Statistic.pdf	r1	manage	93.6 K	2010-02-01 - 10:30	AlexanderFedotov
Topic revision: r7 - 2010-02-01 - AlexanderFedotov
Main
Webs
Public webs
Welcome Guest
- Cern Search
- TWiki Search
- Google Search
Main All webs
Copyright &© 2008-2024 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
or Ideas, requests, problems regarding TWiki? use Discourse or Send feedback