对线性回归模型系数标准差标准误的理解
1.生成数据
| y | x | e |
|---|---|---|
| 3.6 | 1 | 0.63 |
| 3.4 | 2 | -1.38 |
| 7.6 | 3 | 1.01 |
| 7.4 | 4 | -1.01 |
| 11.6 | 5 | 1.38 |
| 11.4 | 6 | -0.63 |
2.回归
y = β 0 + β 1 x + ϵ y = \beta_{0}+\beta_{1}x+\epsilon y=β0+β1x+ϵ
y i = β 0 + β 1 x i + e i y_{i}=\beta_{0}+\beta_{1} x_{i}+e_{i} yi=β0+β1xi+ei
reg y xSource | SS df MS Number of obs = 6
-------------+---------------------------------- F(1, 4) = 34.60Model | 57.422285 1 57.422285 Prob > F = 0.0042Residual | 6.63771505 4 1.65942876 R-squared = 0.8964
-------------+---------------------------------- Adj R-squared = 0.8705Total | 64.0600001 5 12.812 Root MSE = 1.2882------------------------------------------------------------------------------y | Coefficient Std. err. t P>|t| [95% conf. interval]
-------------+----------------------------------------------------------------x | 1.811429 .3079359 5.88 0.004 .9564615 2.666396_cons | 1.16 1.199238 0.97 0.388 -2.169618 4.489618
------------------------------------------------------------------------------3.计算回归的标准误差
（1）SSE\SSR\SST
S S E SSE SSE: Sum of Squares Error,
S S E = ∑ i = 1 n ( y i ^ − y i ) 2 = ∑ i = 1 n ( e i − e ˉ ) 2 SSE= \sum_{i=1}^{n}(\hat{y_{i}}-y_{i})^2 = \sum_{i=1}^{n}(e_{i}-\bar{e})^2 SSE=i=1∑n(yi^−yi)2=i=1∑n(ei−eˉ)2
在本示例中， S S E = ( 3.6 − 2.97 ) 2 + ( 3.4 − 4.78 ) 2 + ( 7.6 − 6.95 ) 2 + ( 7.4 − 8.41 ) 2 + ( 11.6 − 10.22 ) 2 + ( 11.4 − 12.03 ) 2 = 6.637713 SSE=(3.6-2.97)^2+(3.4-4.78)^2+(7.6-6.95)^2+(7.4-8.41)^2+(11.6-10.22)^2+(11.4-12.03)^2 = 6.637713 SSE=(3.6−2.97)2+(3.4−4.78)2+(7.6−6.95)2+(7.4−8.41)2+(11.6−10.22)2+(11.4−12.03)2=6.637713
S S R SSR SSR: Sum of Squares of the Regression
S S R = ∑ i = 1 n ( y i ^ − y ˉ ) 2 SSR= \sum_{i=1}^{n}(\hat{y_{i}}-\bar{y})^2 SSR=i=1∑n(yi^−yˉ)2
S S T SST SST: Total Sum of Squares
S S T = ∑ i = 1 n ( y i − y ˉ ) 2 SST= \sum_{i=1}^{n}(y_{i}-\bar{y})^2 SST=i=1∑n(yi−yˉ)2
（2）MSE
回归的标准误差为：
s 2 = M S E = S S E n − K = ∑ i = 1 n ( e i − e ˉ ) 2 n − K s^{2}=MSE=\frac{SSE}{n-K}=\frac{\sum_{i=1}^{n}(e_{i}-\bar{e})^2}{n-K} s2=MSE=n−KSSE=n−K∑i=1n(ei−eˉ)2
s = M S E s=\sqrt{MSE} s=MSE
s 2 = 6.637713 6 − 2 = 1.6594282 ; s = 1.288188 s^2 = \frac{6.637713}{6 - 2}=1.6594282; \ \ \ \ \ \ \ s=1.288188 s2=6−26.637713=1.6594282; s=1.288188
（3）SE
S β ^ = s 2 ∑ i = 1 n ( x i − x ˉ ) S_{\hat{\beta}} = \sqrt{\frac{s^2}{{\sum_{i=1}^{n}(x_{i}-\bar{x})}}} Sβ^=∑i=1n(xi−xˉ)s2
S β ^ = 1 n − 2 ∑ i = 1 n e 2 ^ ∑ i = 1 n ( x i − x ˉ ) S_{\hat{\beta}} = \sqrt{\frac{\frac{1}{n-2}\sum_{i=1}^{n} \hat{e^{2}}}{{\sum_{i=1}^{n}(x_{i}-\bar{x})}}} Sβ^=∑i=1n(xi−xˉ)n−21∑i=1ne2^
S β ^ = 1 4 × 6.637713 ( 1 − 3.5 ) 2 + ( 2 − 3.5 ) 2 + ( 3 − 3.5 ) 2 + ( 4 − 3.5 ) 2 + ( 5 − 3.5 ) 2 + ( 6 − 3.5 ) 2 S_{\hat{\beta}} = \sqrt{\frac{\frac{1}{4} \times 6.637713}{(1-3.5)^2+(2-3.5)^2+(3-3.5)^2+(4-3.5)^2+(5-3.5)^2+(6-3.5)^2}} Sβ^=(1−3.5)2+(2−3.5)2+(3−3.5)2+(4−3.5)2+(5−3.5)2+(6−3.5)241×6.637713
SE为何会很大？
- 样本少，分母可能大
- 极端值多
- X分布散（X距X均值离差太大）
Appendix
1. simulation code
clear
set obs 6
gen y = 3.6 in 1
replace y = 3.4 in 2
replace y = 7.6 in 3
replace y = 7.4 in 4
replace y = 11.6 in 5
replace y = 11.4 in 6
gen x = _nreg y x
predict xbgen e = y - xb
format %9.2f xb
format %9.2f e
egen addtext_mean = rowmean(y xb)
forv i = 1/6{su add in `i',dglobal y`i' = r(mean)su e in `i',dglobal e`i' = r(mean)
}tw (scatter y x, mlab(y) mlabp(1)) /// (lfit y x) /// (scatter xb x, mlab(xb) mlabp(1)) /// (rspike y xb x) ,legend(off) /// text($y1 0.9 "0.63",size(vsmall) color(red)) /// text($y2 1.9 "-1.38",size(vsmall) color(red)) /// text($y3 2.9 "1.01",size(vsmall) color(red)) /// text($y4 3.9 "-1.01",size(vsmall) color(red)) /// text($y5 4.9 "1.38",size(vsmall) color(red)) /// text($y6 5.9 "-0.63",size(vsmall) color(red))
2.序列相关 同方差 or 异方差
对于①参数线性②不存在“严格多重共线性”③随机抽样④严格外生性⑤“球形扰动项”(条件同方差+不存在自相关)五个假定均能够满足时
OLS估计量为BLUE，最优无偏线性估计量
此时，x的协方差矩阵为：
V a r ( β 1 ^ ∣ x ) = V a r ( β 1 + ∑ ( x i − x ˉ ) e i ∑ ( x i − x ˉ ) ∣ x ) Var(\hat{\beta_{1}}|x)=Var({\beta_{1}+\frac{\sum(x_{i}-\bar{x})e_{i}}{\sum(x_{i}-\bar{x})}}|x) Var(β1^∣x)=Var(β1+∑(xi−xˉ)∑(xi−xˉ)ei∣x)
V a r ( β 1 ^ ∣ x ) = V a r ( ∑ ( x i − x ˉ ) e i ∣ x ) [ ∑ ( x i − x ˉ ) 2 ] 2 Var(\hat{\beta_{1}}|x)=\frac{Var(\sum(x_{i}-\bar{x})e_{i}|x)}{[\sum(x_{i}-\bar{x})^2]^2} Var(β1^∣x)=[∑(xi−xˉ)2]2Var(∑(xi−xˉ)ei∣x)
- 倘若序列无关，那么和的方差即等价于方差的和，假设 V a r ( e i ∣ x ) = σ 2 Var(e_i|x)=\sigma^2 Var(ei∣x)=σ2
KaTeX parse error: Unknown column alignment: * at position 71: … \begin{array}{*̲*lr**} …
- 序列相关：
σ 2 ^ = ∑ e i 2 n − k − 1 \hat{\sigma^2}=\frac{\sum e_{i}^2}{n-k-1} σ2^=n−k−1∑ei2
3.calculate SE in matlab
sqrt(inv(X'*X)*1.6594282)
