Maple User Group Answers

[Anfang] [Hauptseite] [Suchen] [LDFM]

non-linear least squares regression   (17.7.96)

Nicholas Horton

One of my users is interested in doing non-linear least squares regression fitting; does anyone know of packages that handle this, for either release 3 or 4?

Philip A. Viton   (22.7.96)

Are you really sure you want to do this in Maple? Especially if you have lots of data, I'd think it would be far better to use a dedicated statistics package, most of which contain built-in procedures for non-linear regression. (For example, on the PC I use LIMDEP, and am very pleased with it).

Robert Lopez   (25.7.96)

It would be useful for this discussion if terms were defined. I assumed that when "non-linear least squares fit" appeared, it meant "non-linear." I think that maybe some are using the term to mean

f(x) = a + b x + c x^2

is a non-linear fitting function. Well, technically, yes, but fitting such a curve is really an extension of the linear theory since the normal equations for such a fit are still linear.

A truly non-linear fit uses a function of the form

f(x) = a exp(b*x)

I don't think "most" statistics packages contain code for doing really "non-linear" fits. They can do the first kind above, but not the second (usually.)

Maple can deal with the non-linear fit if the user steers Maple through the requisite mathematics. (Form the sum of squares, differentiate with respect to the parameters, and then solve the resulting non-linear normal equations numerically.) But that does not mean Maple doesn't need an efficient numerical optimizer. Many of us eagerly await the day when such code will be found in Maple.

Casselman Tom

Bob was absolutely correct - there is no general routine for a non-linear least squares fit since each non-linear function is unique and the fit is totally dependent on the function, itself. Thats why everyone tries to linearize their problem before fitting. It makes life a lot easier! There are a bunch of non-linear functions for which people have developed algorithms specific to the function, such as ln(x), sin(x), etc.

You might ask Muggers if anyone has a procedure for the specific function(s) you are interested in.

Tim Howard

An alternative approach involves taking the log of the y data ( ln(y) = ln(a) + bx ), and using the linear method to find a fit of the form Y = A + bx ( here Y=ln(y) ), then setting a=exp(A).

David Holmgren

With regard to the recent discussions about nonlinear least-squares within Maple (assuming here that "nonlinear" refers to the fact that the function being fitted is such that one must use something like the Levenberg-Marquardt method), I have been able to do this in a limited sense with a specific problem, namely fitting a binary star orbit to a set of radial velocity observations.

To give you some idea of the nonlinearity, for each observation one must solve Kepler's equation M = E - e*sin(E) for the quantity E (called the eccentric anomaly) which is then used to compute both the function value (ie., the radial velocity) and its derivatives (6 or 7 of them usually). There are a few intermediate steps which also involve other transcendental functions. Anyway, the problem is to fit a two-body orbit to a set of observations. Since everything is known analytically, including the derivatives, one may develop a program to fit a set of radial velocities using a derivative-based method such as Levenberg-Marquardt.

I have run my Maple program for this problem on a data set of some 30 velocities, and I find it to be somewhat slower than a Fortran program. So, it is possible to do nonlinear least-squares in Maple, but the problem now is to generate a suitably general code for any nonlinear problem, hopefully taking advantage of Maple's ability to compute analytic derivatives. Another advantage of using Maple would reside in its linear algebra capabilities and variable precision arithmetic (which could be important for ill-conditioned problems).

If anyone would like a copy of my Maple code (it is not large) to see how it all works, I will be glad to send it (once I return from my current observing run!).

see: nonlinear least-squares for Maple R5

[Anfang] [Hauptseite] [Suchen] [LDFM]