If we have just one feature, the error is big, preformance is not good; but if we have too many features selected then it might be overfitting.
So we need to find a balance for how many features we want to select:
We can calculate the by Lasso Regression:
When we add more feature, we need to consider that it should have a bigger gain than the loss that I take as a result of having that additional feature in my regresssion.