Mathematical optimization is the process of making something best by maximizing or minimizing an objective function, often used across various fields such as manufacturing, finance, and scheduling. It consists of variables that can be controlled, constraints that limit these variables, and both static and dynamic problems can be addressed. The least squares method is a popular technique for finding the best fit line in data, though it has limitations including sensitivity to outliers and the inability to account for other variables.