Abstract
The least square method, as the core approach for parameter estimation in linear regression, has theoretical properties that lay the foundation for statistical inference. From the perspective of mathematical optimization, this paper systematically sorts out the statistical characteristics such as unbiasedness, validity and linearity of the least squares solution, and explains its optimality in combination with the Gauss-Markov theorem. By analyzing the theoretical constraints under the classical hypothesis framework, the influence of practical problems such as heteroscedasticity, autocorrelation and multicollinearity on the estimation efficiency is explored. Further, optimization paths such as parameter constraints, regularization improvement, and Bayesian expansion are proposed to reveal the continuous vitality of the least square method in machine learning and econometrics. Research shows that the theoretical rigor and practical adaptability of the least square method make it a bridge connecting statistical theory and applied science.

This work is licensed under a Creative Commons Attribution 4.0 International License.
Copyright (c) 2025 Han Qi (Author)