linprog(method=’interior-point’)¶
- scipy.optimize.linprog(c, A_ub=None, b_ub=None, A_eq=None, b_eq=None, bounds=None, method='interior-point', callback=None, options={'maxiter': 1000, 'disp': False, 'presolve': True, 'tol': 1e-08, 'autoscale': False, 'rr': True, 'alpha0': 0.99995, 'beta': 0.1, 'sparse': False, 'lstsq': False, 'sym_pos': True, 'cholesky': True, 'pc': True, 'ip': False, 'permc_spec': 'MMD_AT_PLUS_A'}, x0=None)
Linear programming: minimize a linear objective function subject to linear equality and inequality constraints using the interior-point method of [4].
Linear programming solves problems of the following form:
\[\begin{split}\min_x \ & c^T x \\ \mbox{such that} \ & A_{ub} x \leq b_{ub},\\ & A_{eq} x = b_{eq},\\ & l \leq x \leq u ,\end{split}\]where \(x\) is a vector of decision variables; \(c\), \(b_{ub}\), \(b_{eq}\), \(l\), and \(u\) are vectors; and \(A_{ub}\) and \(A_{eq}\) are matrices.
Alternatively, that’s:
minimize:
c @ x
such that:
A_ub @ x <= b_ub A_eq @ x == b_eq lb <= x <= ub
Note that by default
lb = 0
andub = None
unless specified withbounds
.- Parameters
- c1-D array
The coefficients of the linear objective function to be minimized.
- A_ub2-D array, optional
The inequality constraint matrix. Each row of
A_ub
specifies the coefficients of a linear inequality constraint onx
.- b_ub1-D array, optional
The inequality constraint vector. Each element represents an upper bound on the corresponding value of
A_ub @ x
.- A_eq2-D array, optional
The equality constraint matrix. Each row of
A_eq
specifies the coefficients of a linear equality constraint onx
.- b_eq1-D array, optional
The equality constraint vector. Each element of
A_eq @ x
must equal the corresponding element ofb_eq
.- boundssequence, optional
A sequence of
(min, max)
pairs for each element inx
, defining the minimum and maximum values of that decision variable. UseNone
to indicate that there is no bound. By default, bounds are(0, None)
(all decision variables are non-negative). If a single tuple(min, max)
is provided, thenmin
andmax
will serve as bounds for all decision variables.- methodstr
This is the method-specific documentation for ‘interior-point’. ‘highs’, ‘highs-ds’, ‘highs-ipm’, ‘revised simplex’, and ‘simplex’ (legacy) are also available.
- callbackcallable, optional
Callback function to be executed once per iteration.
- Returns
- resOptimizeResult
A
scipy.optimize.OptimizeResult
consisting of the fields:- x1-D array
The values of the decision variables that minimizes the objective function while satisfying the constraints.
- funfloat
The optimal value of the objective function
c @ x
.- slack1-D array
The (nominally positive) values of the slack variables,
b_ub - A_ub @ x
.- con1-D array
The (nominally zero) residuals of the equality constraints,
b_eq - A_eq @ x
.- successbool
True
when the algorithm succeeds in finding an optimal solution.- statusint
An integer representing the exit status of the algorithm.
0
: Optimization terminated successfully.1
: Iteration limit reached.2
: Problem appears to be infeasible.3
: Problem appears to be unbounded.4
: Numerical difficulties encountered.- messagestr
A string descriptor of the exit status of the algorithm.
- nitint
The total number of iterations performed in all phases.
See also
For documentation for the rest of the parameters, see
scipy.optimize.linprog
- Options
- maxiterint (default: 1000)
The maximum number of iterations of the algorithm.
- dispbool (default: False)
Set to
True
if indicators of optimization status are to be printed to the console each iteration.- presolvebool (default: True)
Presolve attempts to identify trivial infeasibilities, identify trivial unboundedness, and simplify the problem before sending it to the main solver. It is generally recommended to keep the default setting
True
; set toFalse
if presolve is to be disabled.- tolfloat (default: 1e-8)
Termination tolerance to be used for all termination criteria; see [4] Section 4.5.
- autoscalebool (default: False)
Set to
True
to automatically perform equilibration. Consider using this option if the numerical values in the constraints are separated by several orders of magnitude.- rrbool (default: True)
Set to
False
to disable automatic redundancy removal.- alpha0float (default: 0.99995)
The maximal step size for Mehrota’s predictor-corrector search direction; see \(\beta_{3}\) of [4] Table 8.1.
- betafloat (default: 0.1)
The desired reduction of the path parameter \(\mu\) (see [6]) when Mehrota’s predictor-corrector is not in use (uncommon).
- sparsebool (default: False)
Set to
True
if the problem is to be treated as sparse after presolve. If eitherA_eq
orA_ub
is a sparse matrix, this option will automatically be setTrue
, and the problem will be treated as sparse even during presolve. If your constraint matrices contain mostly zeros and the problem is not very small (less than about 100 constraints or variables), consider settingTrue
or providingA_eq
andA_ub
as sparse matrices.- lstsqbool (default:
False
) Set to
True
if the problem is expected to be very poorly conditioned. This should always be leftFalse
unless severe numerical difficulties are encountered. Leave this at the default unless you receive a warning message suggesting otherwise.- sym_posbool (default: True)
Leave
True
if the problem is expected to yield a well conditioned symmetric positive definite normal equation matrix (almost always). Leave this at the default unless you receive a warning message suggesting otherwise.- choleskybool (default: True)
Set to
True
if the normal equations are to be solved by explicit Cholesky decomposition followed by explicit forward/backward substitution. This is typically faster for problems that are numerically well-behaved.- pcbool (default: True)
Leave
True
if the predictor-corrector method of Mehrota is to be used. This is almost always (if not always) beneficial.- ipbool (default: False)
Set to
True
if the improved initial point suggestion due to [4] Section 4.3 is desired. Whether this is beneficial or not depends on the problem.- permc_specstr (default: ‘MMD_AT_PLUS_A’)
(Has effect only with
sparse = True
,lstsq = False
,sym_pos = True
, and no SuiteSparse.) A matrix is factorized in each iteration of the algorithm. This option specifies how to permute the columns of the matrix for sparsity preservation. Acceptable values are:NATURAL
: natural ordering.MMD_ATA
: minimum degree ordering on the structure of A^T A.MMD_AT_PLUS_A
: minimum degree ordering on the structure of A^T+A.COLAMD
: approximate minimum degree column ordering.
This option can impact the convergence of the interior point algorithm; test different values to determine which performs best for your problem. For more information, refer to
scipy.sparse.linalg.splu
.- unknown_optionsdict
Optional arguments not used by this particular solver. If unknown_options is non-empty a warning is issued listing all unused options.
Notes
This method implements the algorithm outlined in [4] with ideas from [8] and a structure inspired by the simpler methods of [6].
The primal-dual path following method begins with initial ‘guesses’ of the primal and dual variables of the standard form problem and iteratively attempts to solve the (nonlinear) Karush-Kuhn-Tucker conditions for the problem with a gradually reduced logarithmic barrier term added to the objective. This particular implementation uses a homogeneous self-dual formulation, which provides certificates of infeasibility or unboundedness where applicable.
The default initial point for the primal and dual variables is that defined in [4] Section 4.4 Equation 8.22. Optionally (by setting initial point option
ip=True
), an alternate (potentially improved) starting point can be calculated according to the additional recommendations of [4] Section 4.4.A search direction is calculated using the predictor-corrector method (single correction) proposed by Mehrota and detailed in [4] Section 4.1. (A potential improvement would be to implement the method of multiple corrections described in [4] Section 4.2.) In practice, this is accomplished by solving the normal equations, [4] Section 5.1 Equations 8.31 and 8.32, derived from the Newton equations [4] Section 5 Equations 8.25 (compare to [4] Section 4 Equations 8.6-8.8). The advantage of solving the normal equations rather than 8.25 directly is that the matrices involved are symmetric positive definite, so Cholesky decomposition can be used rather than the more expensive LU factorization.
With default options, the solver used to perform the factorization depends on third-party software availability and the conditioning of the problem.
For dense problems, solvers are tried in the following order:
scipy.linalg.cho_factor
scipy.linalg.solve
with optionsym_pos=True
scipy.linalg.solve
with optionsym_pos=False
scipy.linalg.lstsq
For sparse problems:
sksparse.cholmod.cholesky
(if scikit-sparse and SuiteSparse are installed)scipy.sparse.linalg.factorized
(if scikit-umfpack and SuiteSparse are installed)scipy.sparse.linalg.splu
(which uses SuperLU distributed with SciPy)scipy.sparse.linalg.lsqr
If the solver fails for any reason, successively more robust (but slower) solvers are attempted in the order indicated. Attempting, failing, and re-starting factorization can be time consuming, so if the problem is numerically challenging, options can be set to bypass solvers that are failing. Setting
cholesky=False
skips to solver 2,sym_pos=False
skips to solver 3, andlstsq=True
skips to solver 4 for both sparse and dense problems.Potential improvements for combatting issues associated with dense columns in otherwise sparse problems are outlined in [4] Section 5.3 and [10] Section 4.1-4.2; the latter also discusses the alleviation of accuracy issues associated with the substitution approach to free variables.
After calculating the search direction, the maximum possible step size that does not activate the non-negativity constraints is calculated, and the smaller of this step size and unity is applied (as in [4] Section 4.1.) [4] Section 4.3 suggests improvements for choosing the step size.
The new point is tested according to the termination conditions of [4] Section 4.5. The same tolerance, which can be set using the
tol
option, is used for all checks. (A potential improvement would be to expose the different tolerances to be set independently.) If optimality, unboundedness, or infeasibility is detected, the solve procedure terminates; otherwise it repeats.Whereas the top level
linprog
module expects a problem of form:Minimize:
c @ x
Subject to:
A_ub @ x <= b_ub A_eq @ x == b_eq lb <= x <= ub
where
lb = 0
andub = None
unless set inbounds
. The problem is automatically converted to the form:Minimize:
c @ x
Subject to:
A @ x == b x >= 0
for solution. That is, the original problem contains equality, upper-bound and variable constraints whereas the method specific solver requires equality constraints and variable non-negativity.
linprog
converts the original problem to standard form by converting the simple bounds to upper bound constraints, introducing non-negative slack variables for inequality constraints, and expressing unbounded variables as the difference between two non-negative variables. The problem is converted back to the original form before results are reported.References
- 4(1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16)
Andersen, Erling D., and Knud D. Andersen. “The MOSEK interior point optimizer for linear programming: an implementation of the homogeneous algorithm.” High performance optimization. Springer US, 2000. 197-232.
- 6(1,2)
Freund, Robert M. “Primal-Dual Interior-Point Methods for Linear Programming based on Newton’s Method.” Unpublished Course Notes, March 2004. Available 2/25/2017 at https://ocw.mit.edu/courses/sloan-school-of-management/15-084j-nonlinear-programming-spring-2004/lecture-notes/lec14_int_pt_mthd.pdf
- 8
Andersen, Erling D., and Knud D. Andersen. “Presolving in linear programming.” Mathematical Programming 71.2 (1995): 221-245.
- 9
Bertsimas, Dimitris, and J. Tsitsiklis. “Introduction to linear programming.” Athena Scientific 1 (1997): 997.
- 10
Andersen, Erling D., et al. Implementation of interior point methods for large scale linear programming. HEC/Universite de Geneve, 1996.