Assumption

• 线性关系
• Residual: Independent, normal, mean is 0, variance is constant

R-square

• explained proportion of the variance for $y$.
• adjusted R-square adjusts the number of variables. So when adding the variable makes more benefits for the model (using number of variables to penalise), then the adjusted R-square will be higher.

Outlier

The outlier should be well treated. It can affect the consequences of the model. Can use mean or other statistics to replace.

Q-Q plot

quantile to quantile plot; plot the data’s quantile in corresponding two distributions (to be examined); if they follow the same distribution, the line should be a ‘y=x’ line.

F test

testing the goodness of the model. $\beta_i = 0$ or at least one $\ne0$

损失函数

为什么线性回归试用MSE

Feature selection

• forward selection
• backward selection
• To evaluate the features, we use some statistics

  • Mallows’ $C_p$ statistics

      Mallow's Cp is a statistical method used to assess the adequacy of a linear regression model. It is a criterion that compares the performance of a given regression model to that of a full model that includes all possible predictors.
      
      The Mallow's Cp statistic is based on the residual sum of squares (RSS) of a regression model, the number of estimated parameters, and the sample size. It is defined as:
      
      $Cp = (RSS_p / s^2) - n + 2p$
      
      where $RSS_p$ is the residual sum of squares of the model with p predictors, $s^2$ is the estimate of the error variance, n is the sample size, and p is the number of predictors in the model.
      
      **The Cp statistic measures the trade-off between model complexity and goodness of fit. A Cp value close to the number of predictors indicates a good balance between model complexity and goodness of fit. A Cp value greater than the number of predictors suggests that the model may be too complex, while a Cp value less than the number of predictors implies that the model may be too simple and could benefit from additional predictors.**
      
      Mallow's Cp is commonly used in regression model selection to identify the best subset of predictors to include in a model. The goal is to choose a model with the fewest predictors that still adequately explains the variation in the response variable.
    

  • AIC

    • distance between the proposed distribution and true distribution $D$
      • the distance is Kullback-Leibler distance
      • choose distribution that $D$ is minimised
      • to choose min AIC

    为什么要用OLS

    相比于最小绝对值法，最小二乘法的优点在于最优解唯一、求解方便和有好的解析性质，但缺点在于受异常值扰动影响大。

    最小二乘法不永远是最优的方法。对于不同数据形式和建模需求，需要能自行选择合适的建模方式。

    异方差

    1. 经典例子：研究收入对消费的影响时，因为富人消费比较有弹性，而穷人消费很少变动，所以扰动项方差可能随收入（解释变量）增加而增加
    2. 组间异方差：样本内包含两类数据时，可能出现组内同方差、组间异方差的情形。比如第一类是创业者，第二类是打工者，创业者的收入波动可能更大
    3. 组平均数：数据本身是组平均数时，大组平均数的方差通常要小。比如考虑各省的人均GDP，每个省一个数据，则人口较多的省份方差较小
    4. 后果：估计量无偏一致但不再有效，无法进行假设检验
       1. 无偏性 $E(\hat{\theta})=\theta$仍然成立
       2. 但是估计出来的估计量无效，因为不能保证方差最小

    检验异方差

    • 残差图
    • BP和White检验

    处理

    • 广义最小二乘法 GLS：generalised least squares

      • 加权最小二乘法 WLS，对相关矩阵加权 $1/\sqrt{\omega_i}$
      • 可行广义最小二乘法：对 $e_i^2$ 进行回归

    • OLS+white’s standard errors

        [计量经济学导论05：异方差 - 这个XD很懒 - 博客园](<https://www.cnblogs.com/lixddd/p/14367772.html>)
      

自相关 Autocorrelation

截面数据不容易出现自相关，时序数据中自相关比较常见：