Assignment 1 Why do we prefer Adjusted R-squared over Normal R-squared?

Adjusted R-squared is preferred over Normal R-squared because it takes into account the number of predictors in a regression model. Normal R-squared can be misleading when comparing models with different numbers of predictors, as it tends to increase with the addition of more predictors even if they do not significantly improve the model's fit.

Adjusted R-squared, on the other hand, penalizes the addition of unnecessary predictors by adjusting for the number of predictors in the model. This makes it a more reliable measure of the proportion of variance explained by the predictors, as it provides a more accurate reflection of the model's goodness of fit. Adjusted R-squared is therefore a more appropriate measure for comparing the performance of different regression models with varying numbers of predictors.

Adjusted R-squared (Adj. R-squared) is preferred over Normal R-squared (R-squared) for several reasons:

1. Penalty for Overfitting:

- Normal R-squared tends to increase with the number of predictor variables, even if some variables are not contributing to the model's predictive power.
- Adj. R-squared includes a penalty term that adjusts for the number of predictors in the model, preventing overfitting and providing a more accurate estimate of the model's performance on new data.

2. Comparability Across Models:

- Adj. R-squared allows for fairer comparisons between models with different numbers of predictors.
- It ensures that the model with the highest Adj. R-squared has not simply overfitted the data due to having more variables.

3. Focus on Predictive Power:

- Adj. R-squared emphasizes the model's predictive ability rather than its goodness of fit to the training data.
- It provides a better indication of how well the model will generalize to unseen datasets.

4. Statistical Significance:

- F-tests and other statistical tests used to evaluate the significance of the model are based on Adj. R-squared, ensuring that the results are adjusted for the number of predictors.

5. Interpretation:

- Adj. R-squared can be interpreted as the proportion of variance in the dependent variable that is explained by the model, adjusted for the degrees of freedom.
- This makes it easier to assess the model's predictive power and compare it to other models.

In summary, Adj. R-squared is preferred over Normal R-squared because it penalizes overfitting, enables fair comparisons across models, focuses on predictive power, provides statistical significance, and offers a meaningful interpretation of the model's performance.