This project has moved. For the latest updates, please go here.

Speed of MPFitLevenbergMarquardtOptimizer

Oct 8, 2012 at 6:30 PM

What I'm currently doing: 

Generating reflectance with no noise using the ROfRhoAndFt forward solver at 401 frequencies for a single distance.  Flattening the real/imaginary out into a single array.  

I then fit this array using the MPFitLevenbergMarquardtOptimizer and a modified foward model that also flattens the ROfRhoAndFt output.  

This is taking close to 30 seconds for a single fit.  I'm not sure if the vts gui also flattens the data for fitting but its about 10x faster.   

Not sure if its just something I'm doing or if this is what I should expect.  Maybe I need to adjust mpfit settings? Based on fits I've done in c in the past, I was hoping for something <1 second.  

Any thoughts? 

Developer
Oct 9, 2012 at 1:18 AM

Hi Brian,

The GUI also flattens the data for fitting.  If you have the source code, look at InverseSolverViewModel.cs in the Vts.Gui.Silverlight/ViewModel/Application/Panels/ folder.  There are several places where the code checks "IsComplexSolver" and if so, flattens the data making it twice as long.  You might try running the inversion using the source code and setting break points to view the values of the parameters that finally get passed to ComputationFactory.ConstructAndExecuteVectorizedOptimizer.

Carole..

Oct 9, 2012 at 8:02 PM

Thanks Carole.  I modified my code to look more like the vts gui and it runs very quickly.  (I wasn't using the ConstructAndExecuteVectorizedOptimizer method.  I was creating a MPFitLevenbergMarquardtOptimizer and using that with my own forward model method based on ROfRhoAndFt.)  

One other difference was that I was passing an array of 1's as the standard deviation values.   The vts gui is just using the dependent value array.  I'm curious how that is used.  I expect for mpfit to take (theory-dependentValue)/stdev so I would think that passing the dependent values as standard deviations would be weighting the fit which I'm not sure is intended.  

Developer
Oct 9, 2012 at 8:53 PM

Glad you got it to run faster.

Concerning the argument in the chi-square.  Ideally, it should be (meas_i - model_i) / SD(meas_i), where meas_i is the ith measurement and model_i is the model prediction of this measurement.  This quantity gets squared and summed over i, the number of measurements.  This provides a weighted chi-square because if there is large error in measurement i, then that measurement is weighted less in the overall sum.  If the SD(meas_i) is unknown, then in the past, we have divided by the meas_i (i.e., the dependentValues).  This is a weighting of a different sort that puts all the measurements on equal footing.  It has been done for R(rho) type measurements where the reflectance decay spans orders of magnitude with increasing rho, so that if no weighting is done, the contributions of the measurements at long rho were much smaller than those at close rho.  We did something like this with the delta-P1 inversions long ago.

Oct 9, 2012 at 11:10 PM

I definitely remember something along these lines with the delta-P1.   I'm curious/nervous about how this would affect the data I'm looking at however.  Seems like noisy points low relative to theory will be weighted higher than similar points that error on the high side.  I'd guess that the recovered optical properties end up undershooting the experimental data. (or maybe my thinking is all wrong?)  I suppose its time to either add some visualization or file exporting to my code so I can take a closer look at whats happening.  Also leads to my next post...

Coordinator
Oct 12, 2012 at 4:56 AM

I am curious about the weights as well. I would not prefer non-uniform weighting by default in the SFD, or in the time-FD with Real + Imag - values are typically on equal footing. Near-zero imaginary values, for example, should be contribute less to the fit than larger imaginary values, IMO.

Oct 23, 2012 at 7:55 PM

A couple more notes on the weighting: 

I was playing around with the inverse solver panel of the vts gui using the default settings. (Steady-state)  Here is a plot of 20%, 30% and 40% noise. http://i.imgur.com/vecH0.jpg   At least for these distances and optical properties, it looks like the weighting is causing much poorer fits than I would expect. The amplitude of the fitted curve drops as error increases since the low points seem to be weighted so heavily.  

Also in my initial fits of measured frequency-domain data, uniform weighting looks to work much better than this non-uniform weighting.  

Developer
Oct 24, 2012 at 3:53 AM

What about the following plan.  We use SD=1 (non-weighted chi-square) when the user is running inversions with the current Inverse Solver Panel regardless of what they set the noise level to.  In the future when we enable the capability for the user to import their own measured data, we also allow the option to import the measured data error and if given, use those values in a weighted chi-square.

BTW, a request for the ability to import measured data was recently reiterated.

Coordinator
Oct 24, 2012 at 9:20 PM

I would suggest that uniform weighting be the default, and that the user be able to toggle between uniform (1s), proportional (weighted by dependent value), or user-supplied (imported). The final one will require a little more engineering - I would say we should put a toggle between the three and grey out the final choice for the time being.