Matrix Form of Input for Optimization Package Commands
This help page describes the Matrix form of input for commands in the Optimization package. For general information on all the input forms accepted by the Optimization package commands, see the Optimization/InputForms help page.
Using Matrix Form
The solvers perform most efficiently when given problems in Matrix form, with the objective function and constraints specified as Vectors, Matrices, or procedures with Vector and Matrix parameters. Matrix form requires the least amount of preprocessing because it is most similar to the format used by the internal solvers. This leads to reduced storage needs and faster computation.
Consider an optimization problem in the form:
v⁡x≤0 (nonlinear inequality constraints)
w⁡x=0 (nonlinear equality constraints)
A·x≤b (linear inequality constraints)
Aeq·x=beq (linear equality constraints)
where x is the vector of problem variables; f⁡x is the real-valued objective function; v⁡x and w⁡x are vector-valued functions of x; b, beq, bl, and bu are vectors; and A and Aeq are matrices. The dimension of x is n. The relations involving matrices and vectors are element-wise. The specifications of each component are given in the following sections.
The LPSolve command accepts a linear objective function f⁡x specified by a Vector parameter c having dimension n and containing the coefficients of the objective function.
The QPSolve command accepts a quadratic objective function. Suppose the function has the form c'⁢x+12⁢x'⁢H⁢x where c is a vector and H is the symmetric Hessian matrix. Specify the function as [c, H] when the linear part exists, or H otherwise. H must be an n by n symmetric Matrix and c must be a Vector of dimension n. In the defined function, c' and x' refer to the vector transpose.
The NLPSolve command accepts a general nonlinear objective function f(x) specified by a procedure p of type procedure[float](Vector). The procedure takes one input Vector parameter of size n representing x and returns the value of f⁡x. The value of n is passed separately to the NLPSolve command because it cannot be determined from the procedure p.
For best performance, provide the gradient of the objective function using the objectivegradient=objgrd option, where objgrd is of type procedure(Vector, Vector). The procedure takes an input Vector parameter representing x, computes the values of the gradient of f at x, and returns the results as the second Vector parameter. The dimension of each Vector is n.
Linear Least-Squares Objective
The LSSolve command accepts a linear least-squares objective function. Suppose f⁡x is the linear least-squares objective function 12c−G⁢x2. Specify f⁡x as a list [c, G] where c is a Vector of dimension q and G is a q by n Matrix.
Nonlinear Least-Squares Objective
The LSSolve command accepts a nonlinear least-squares objective function. Suppose f⁡x has the form 12⁢f1⁡x2+f2⁡x2+...+fq⁡x2. Specify f⁡x as a procedure p having type procedure(Vector, Vector). The procedure takes an input Vector parameter of dimension n representing x, computes the residuals f1, f2, ..., fq at x, and returns these residuals as the second Vector parameter. The values of n and q are passed separately to the LSSolve command because they cannot be determined from the procedure p.
Procedure p may have a third parameter needfi having type float. In this case, the body of p should be constructed so that if needfi < 0.0, all the residuals are computed; otherwise, only the ith residual need be computed where i=trunc⁡needfi. This leads to more efficient computation as residuals are computed only when necessary. Such a procedure might have the form:
p := proc(x, y, needfi)
if needfi < 0.0 or trunc(needfi) = 1 then
y := ...;
if needfi < 0.0 or trunc(needfi) = 2 then
y := ...;
For best performance, provide the Jacobian matrix of the objective function using the objectivejacobian=objjac option, where objjac has type procedure(Vector, Matrix). This procedure takes an input Vector parameter representing x, computes the values of the Jacobian of f at x, and returns these values as the Matrix parameter of dimension q by n.
Linear constraints are accepted by all the Optimization solvers that take Matrix-form input. They can take one of the following forms.
-- no linear constraints
A,b -- inequality constraints only
A,b,Aeq,beq -- inequality and equality constraints
NoUserValue,NoUserValue,Aeq,beq -- equality constraints only
A and b represent the components of the linear inequality constraints A·x≤b, while Aeq and beq represent the components of the linear equality constraints Aeq·x=beq.
b and beq can have type Vector or numeric. When either constraint is a single numeric value, it is interpreted as a Vector in which each element is equal to that value.
A and Aeq must each be a Matrix with column dimension equal to n. The row dimension of A must be equal to that of b when b is a Vector, and the row dimension of Aeq must be equal to that of beq when beq is a Vector.
Nonlinear constraints are accepted by the NLPSolve, and LSSolve commands. Consider nonlinear inequality constraints v⁡x≤0 and nonlinear equality constraints w⁡x=0. Let y⁡x be the vector consisting of v⁡x followed by w⁡x. Then y⁡x is represented by a procedure pcons of type procedure(Vector, Vector), where the first parameter is the current point x and the second is an output parameter containing the values of y at the point x. Because the Optimization routine cannot determine the number of inequality and equality constraints, this information must be specified in the calling sequence.
Procedure pcons may have a third parameter needc, which is an input Vector of floating-point values with dimension equal to the total number of constraints. In this case, the body of p should be constructed so that it checks the value of needc[i] and if this is greater than 0.0, the left-hand side of the ith constraint is computed. This leads to more efficient computation because constraint values are computed only when necessary. Such a procedure might have the form:
pcons := proc(x, y, needc)
if needc > 0.0 then
y := ...;
if needc > 0.0 then
y := ...;
For best performance, provide the Jacobian matrix of y⁡x using the constraintjacobian=conjac option, where conjac has type procedure(Vector, Matrix). This procedure takes an input Vector parameter representing x, computes the values of the Jacobian of y at x, and returns these values as the Matrix parameter. Like procedure pcons described above, procedure conjac can take a third parameter needc. Again, needc is a floating-point input Vector with 0<needci indicating that the ith row of the Jacobian Matrix should be computed.
Bounds on the problem variables are accepted by all the Optimization solvers that take Matrix-form input. They are provided as a list bl,bu containing lower bounds followed by upper bounds. If the list is empty, the bounds are assumed to be −∞ and ∞ respectively.
The bounds bl and bu can be specified as Vectors of dimension n or as numeric values. If a numeric value is provided, it is interpreted as a Vector of dimension n in which each element is equal to that value.
If there are upper bounds but no lower bounds, bl must be set to -Float(infinity). If there are lower bounds, but no upper bounds, bu must be set to Float(infinity).
The assume=nonnegative option can be used to specify the constraint that all variables must be non-negative. For more information, see the Optimization/Options help page.
Linear programs using the interior point method require finite lower bounds to be specified for each variable.
Initial values are accepted by all the Optimization solvers that take Matrix-form input and are specified using the option initialpoint=p where p is a Vector of dimension n. For more information about the initialpoint option, see the Optimization/Options help page.
The solution is a list objval,solpt where objval is a floating-point number representing the final minimum (or maximum) value and solpt is a Vector representing a point (the computed extremum).
For best performance, the components described previously must satisfy the following additional conditions.
All Matrix and Vector arguments must be constructed with datatype=float. Otherwise, the solvers must perform conversions.
A symmetric Matrix argument, such as the Hessian matrix for the QPSolve command, must be created with shape=symmetric and storage=rectangular. Otherwise, the solver must perform this conversion.
When the interior point method for linear programs is used, the constraint matrices should be specified with the option storage=sparse. Otherwise, any constraint matrices with non-sparse storage will be converted to sparse storage.
When possible, input procedures must be created so that they can be evaluated by the evalhf command. Otherwise, the solvers cannot take advantage of the more efficient evalhf mode of computation.
Procedures returning a scalar result must return a floating-point value in all cases, not simply an expression that evaluates to a float.
The following additional condition must be met. If a procedure returns a Vector or Matrix result using an output parameter, only floating-point values can be assigned to the parameter. Otherwise, an error is issued.
Express a linear program in Matrix form and solve it using the Optimization[LPSolve] command.
c ≔ Vector⁡−1,−1,datatype=float:
A ≔ Matrix⁡−3,1,5,1,datatype=float:
b ≔ Vector⁡0.5,2,datatype=float:
Express a quadratic program in Matrix form and solve it using the Optimization[QPSolve] command.
c ≔ Vector⁡2,5,datatype=float:
H ≔ Matrix⁡6,3,3,4,datatype=float:
A ≔ Matrix⁡−1,1,datatype=float:
b ≔ Vector⁡−2,datatype=float:
Consider the nonlinear objective function w3⁢v−w2+w−x−12+x−y−22+y−z−32 with a nonlinear constraint w2+x2+y+z≤5 and a linear constraint z=v. Let V represent the Vector of variables v,x,w,y,z and write the objective function as a procedure having V as an input parameter and returning a floating-point value.
p := proc (V)
Write the nonlinear constraint w2+x2+y+z≤5 as a procedure having V as an input parameter and W as a one-dimensional Vector output parameter. The needc parameter is optional.
nlc := proc(V, W, needc)
if needc > 0.0 then
W := V^2+V^2+V+V-5
Construct the list representing linear constraints. In this case, there is a single equality constraint z−u=0.
Aeq ≔ Matrix⁡−1,0,0,0,1,datatype=float:
beq ≔ Vector⁡0,datatype=float:
lc ≔ NoUserValue,NoUserValue,Aeq,beq:
For better performance, provide procedures representing the gradient of the objective function and the Jacobian of the constraints. The objgra procedure takes V as an input parameter and returns the gradient values at V in the output Vector parameter W.
objgra := proc (V, W)
W := 2*V^3*(V-V);
W := 3*V^2*(V-V)^2-2*V^3*(V-V)+
The conjac procedure takes V as an input parameter and returns the Jacobian in the output Matrix parameter M. The needc parameter is optional.
conjac := proc(V, M, needc)
if needc > 0.0 then
M[1,1] := 0;
M[1,2] := 2*V;
M[1,3] := 2*V;
M[1,4] := 1;
M[1,5] := 1;
Solve the nonlinear program with the Optimization[NLPSolve] command by providing the number of variables, 5, as the first argument. Use the assume=nonnegative option to specify that all variables must take non-negative values.
Download Help Document