Guidelines for Numerical Issues

Numerical instability is a generic label often applied to situations where solving an optimization model produces results that are erratic, inconsistent, or unexpected, or when the underlying algorithms exhibit poor performance or are unable to converge. There are many potential causes of this behavior; however, most can be grouped into four categories:

• Rounding coefficients while building the model.

• Limitations of floating-point arithmetic.

• Unrealistic expectations about achievable precision.

• Ill conditioning, or geometry-induced issues.

This section explains these issues and how they affect both performance and solution quality. We also provide some general rules and some advanced techniques to help avoid them. Although we will treat each of these four sources separately, it is important to remember that their effects often feed off of each other. We also provide tips on how to diagnose numerical instability in your models.

Finally, we discuss the Gurobi parameters that can be modified to improve solution accuracy. We should stress now, however, that the best way to improve numerical behavior and performance is to reformulate your model. Parameters can help to manage the effects of numerical issues, but there are limits to what they can do, and they typically come with a substantial performance cost.

Content

The section is structured into the following subsections. You can read them successively, or head directly to a specific topic.

• Avoid Rounding of Input discusses the issues that can arise when rounding the numbers during model building;

• Real Numbers are not Real demonstrates the limits of computer arithmetic;

• Tolerances and User-Scaling discusses Gurobi’s tolerances w.r.t. (in)feasibility, scaling, and provides recommendations for variable, constraints, and coefficient ranges;

• Does my Model have Numerical Issues? shows how to check a model for numerical issues;

• Solver Parameters to Manage Numerical Issues discusses parameters that you could try;

• Instability and the Geometry of Optimization Problems discusses the Condition Number and the geometry behind it;

• Source Code Examples: provides the sources for the scripts used in this guide.

Further Reading

• A Characterization of Stability in Linear Programming, Stephen M. Robinson, 1977, Operations Research 25-3:435-447.

• IEEE Standard for Binary Floating-Point Arithmetic (IEEE 754), IEEE Computer Society, 1985.

• What every computer scientist should know about floating-point arithmetic, David Goldberg, 1991, ACM Computing Surveys (CSUR), 23:5-48.

• Numerical Computing with IEEE Floating Point Arithmetic, Michael L. Overton, SIAM, 2001.

• Practical guidelines for solving difficult linear programs, Ed Klotz and Alexandra M. Newman, 2013, Surveys in Operations Research and Management Science, 18-1:1-17.

• Identification, Assessment, and Correction of Ill-Conditioning and Numerical Instability in Linear and Integer Programs, Ed Klotz, Bridging Data and Decisions, Chapter 3, 54-108.