FD with noise

[talassio]

Enhance FD framework to handle noisy derivatives. The general idea is to dynamically set the step used to estimate the derivatives based on a measure of the local noise.

References

1. Optimization Methods and Software. Volume 38, 2023 - Issue 2. On the numerical performance of finite-difference-based methods for derivative-free optimization. Hao-Jun Michael Shi, Melody Qiming Xuan, Figen Oztoprak, and Jorge Nocedal. https://doi.org/10.1080/10556788.2022.2121832
2. SIAM Journal on Scientific Computing. Vol. 44, Iss. 4 (2022) Adaptive Finite-Difference Interval Estimation for Noisy Derivative-Free Optimization. Hao-Jun Michael Shi, Yuchen Xie, Melody Qiming Xuan, and Jorge Nocedal. https://doi.org/10.1137/21M1452470
3. ArXiv. Optimization and Control. On the Numerical Performance of Derivative-Free Optimization Methods Based on Finite-Difference Approximations. Hao-Jun Michael Shi, Melody Qiming Xuan, Figen Oztoprak, and Jorge Nocedal. https://doi.org/10.48550/arXiv.2102.09762
4. SIAM Journal on Optimization Vol. 29, Iss. 2 (2019) Derivative-Free Optimization of Noisy Functions via Quasi-Newton Methods. Albert S. Berahas, Richard H. Byrd, and Jorge Nocedal. https://doi.org/10.1137/18M1177718