Curve Fitting Algorithm

Date: 6/20/96 at 15:36:45
From: Ramon Handel
Subject: Curve Fitting Algorithm

Dear Doctor Math:

I have seen a method called "Least squares approximation," but I don't 
understand how it works. Could you please explain it to me?

Yours truly,
   Ramon van Handel
   RVHANDEL@DDS.NL

Date: 6/20/96 at 17:39:10
From: Doctor Anthony
Subject: Re: Curve Fitting Algorithm

I will just indicate the method for 'least squares' curve fitting.  
In the case of a straight line of best fit, you assume its equation is 
of the form y = ax+b.  If you want the line of regression of y on x, 
then x is the independent variable, and the theoretical y values will 
be given by this equation.  The method of least squares is used to 
find a and b.  If (x1,y1), (x2,y2), (x3,y3) .... (xn,yn) are the 
experimentally determined points, then the RESIDUALS are (y1-ax1-b), 
(y2-ax2-b), ...(yn-axn-b), and we must now minimize the SUMS OF 
SQUARES OF THE RESIDUALS.

Let z = (y1-ax1-b)^2 + (y2-ax2-b)^2 + etc, etc

We now find partial(dz/db) and partial(dz/da) and equate each of these 
to zero.  This process leads to the 'normal' equations:

SUM(y) = aSUM(x) + nb
SUM(xy) = aSUM(x^2) + bSUM(x)

After some more algebra we obtain the equation of the line of best 
fit:

y-mean(y) = {cov(xy)/var(x)}[x-mean(x)]

The method of 'least squares' is not confined to straight lines.  You 
can also find parabolas or cubics etc of best fit, but clearly the 
algebra involved becomes progressively more difficult.

-Doctor Anthony,  The Math Forum
 Check out our web site!  http://mathforum.org/dr.math/