Germán Ferrari Instituto de Computación, Universidad de la República, Uruguay An embedded DSL to manipulate MathProg Mixed Integer Programming models within Scala An embedded DSL to manipulate MathProg Mixed Integer Programming models within Scala Germán Ferrari Instituto de Computación, Universidad de la República, Uruguay
GNU MathProg MathProg = Mathematical Programming Has nothing to do with computer programming It’s a synonym of “mathematical optimization” A “program” here refers to a plan or schedule
GNU MathProg It’s a modeling language Optimization problem Mathematical modeling GNU MathProg Computational experiments Supported by the GNU Linear Programming Kit
GNU MathProg Supports linear models, with either real or integer decision variables This is called “Mixed Integer Programming” (MIP) because it mixes integer and real variables
Generic MIP optimization problem minimize f: sum{j in J} c[j] * x[j] + sum{k in K} d[k] * y[k]; subject to g{i in I}: sum{j in J} a[i,j] * x[j] + sum{k in K} e[i,k] * y[k] <= b[i];
Generic MIP optimization problem Find x[j] and y[k], that minimize a function f, satisfying constraints g[i] minimize f: sum{j in J} c[j] * x[j] + sum{k in K} d[k] * y[k]; subject to g{i in I}: sum{j in J} a[i,j] * x[j] + sum{k in K} e[i,k] * y[k] <= b[i];
Generic MIP optimization problem Find x[j] and y[k], that minimize a function f, satisfying constraints g[i] minimize f: sum{j in J} c[j] * x[j] + sum{k in K} d[k] * y[k]; subject to g{i in I}: sum{j in J} a[i,j] * x[j] + sum{k in K} e[i,k] * y[k] <= b[i]; Decision variables (real or integer)
Generic MIP optimization problem Find x[j] and y[k], that minimize a function f, satisfying constraints g[i] minimize f: sum{j in J} c[j] * x[j] + sum{k in K} d[k] * y[k]; subject to g{i in I}: sum{j in J} a[i,j] * x[j] + sum{k in K} e[i,k] * y[k] <= b[i]; Decision variables (real or integer) Objective function
Generic MIP optimization problem Find x[j] and y[k], that minimize a function f, satisfying constraints g[i] minimize f: sum{j in J} c[j] * x[j] + sum{k in K} d[k] * y[k]; subject to g{i in I}: sum{j in J} a[i,j] * x[j] + sum{k in K} e[i,k] * y[k] <= b[i]; Decision variables (real or integer) Objective function Constraints
Generic MIP optimization problem Find x[j] and y[k], that minimize a function f, satisfying constraints g[i] minimize f: sum{j in J} c[j] * x[j] + sum{k in K} d[k] * y[k]; subject to g{i in I}: sum{j in J} a[i,j] * x[j] + sum{k in K} e[i,k] * y[k] <= b[i]; Decision variables (real or integer) Objective function Constraints Linear
Generic MIP optimization problem Full model param m; param n; param l; set I := 1 .. m; set J := 1 .. n; set K := 1 .. l; param c{J}; param d{K}; param a{I, J}; param e{I, K}; param b{I}; var x{J} integer, >= 0; var y{K} >= 0; minimize f: sum{j in J} c[j] * x[j] + sum{k in K} d[k] * y[k]; subject to g{i in I}: sum{j in J} a[i,j] * x[j] + sum{k in K} e[i,k] * y[k] <= b[i];
param m; param n; param l; set I := 1 .. m; set J := 1 .. n; set K := 1 .. l; param c{J}; param d{K}; param a{I, J}; param e{I, K}; param b{I}; var x{J} integer, >= 0; var y{K} >= 0; minimize f: sum{j in J} c[j] * x[j] + sum{k in K} d[k] * y[k]; subject to g{i in I}: sum{j in J} a[i,j] * x[j] + sum{k in K} e[i,k] * y[k] <= b[i]; Generic MIP optimization problem Declaration of variables
param m; param n; param l; set I := 1 .. m; set J := 1 .. n; set K := 1 .. l; param c{J}; param d{K}; param a{I, J}; param e{I, K}; param b{I}; var x{J} integer, >= 0; var y{K} >= 0; minimize f: sum{j in J} c[j] * x[j] + sum{k in K} d[k] * y[k]; subject to g{i in I}: sum{j in J} a[i,j] * x[j] + sum{k in K} e[i,k] * y[k] <= b[i]; Generic MIP optimization problem Parameters
Generic MIP optimization problem param m; param n; param l; set I := 1 .. m; set J := 1 .. n; set K := 1 .. l; param c{J}; param d{K}; param a{I, J}; param e{I, K}; param b{I}; var x{J} integer, >= 0; var y{K} >= 0; minimize f: sum{j in J} c[j] * x[j] + sum{k in K} d[k] * y[k]; subject to g{i in I}: sum{j in J} a[i,j] * x[j] + sum{k in K} e[i,k] * y[k] <= b[i]; Indexing sets
Why MathProg in Scala
Why MathProg in Scala Generate constraints Create derived models Develop custom resolution methods at high level ...
MathProg in Scala (deep embedding) Syntax Represent MathProg models with Scala data types Mimic MathProg syntax with Scala functions Semantics Generate MathProg code, others ...
MathProg in Scala (deep embedding) Syntax Represent MathProg models with Scala data types Mimic MathProg syntax with Scala functions Semantics Generate MathProg code, others ... Abstract syntax tree
MathProg in Scala (deep embedding) Syntax Represent MathProg models with Scala data types Mimic MathProg syntax with Scala functions Semantics Generate MathProg code, others ... Smart constructors and combinators Abstract syntax tree
MathProg EDSL param m; param n; param l; set I := 1 .. m; set J := 1 .. n; set K := 1 .. l; param c{J}; param d{K}; param a{I, J}; // ... var x{J} integer, >= 0; var y{K} >= 0; val m = param("m") val n = param("n") val l = param("l") val I = set("I") := 1 to m val J = set("J") := 1 to n val K = set("K") := 1 to l val c = param("c", J) val d = param("d", K) val a = param("a", I, J) // ... val x = xvar("x", J).integer >= 0 val y = xvar("y", K) >= 0
MathProg EDSL param m; param n; param l; set I := 1 .. m; set J := 1 .. n; set K := 1 .. l; param c{J}; param d{K}; param a{I, J}; // ... var x{J} integer, >= 0; var y{K} >= 0; val m = param("m") val n = param("n") val l = param("l") val I = set("I") := 1 to m val J = set("J") := 1 to n val K = set("K") := 1 to l val c = param("c", J) val d = param("d", K) val a = param("a", I, J) // ... val x = xvar("x", J).integer >= 0 val y = xvar("y", K) >= 0 set param xvar ...
MathProg EDSL param m; param n; param l; set I := 1 .. m; set J := 1 .. n; set K := 1 .. l; param c{J}; param d{K}; param a{I, J}; // ... var x{J} integer, >= 0; var y{K} >= 0; val m = param("m") val n = param("n") val l = param("l") val I = set("I") := 1 to m val J = set("J") := 1 to n val K = set("K") := 1 to l val c = param("c", J) val d = param("d", K) val a = param("a", I, J) // ... val x = xvar("x", J).integer >= 0 val y = xvar("y", K) >= 0 set param xvar ... ParamStat( "c", domain = Some( IndExpr( List( IndEntry( Nil, SetRef(J) ) ))))
val m = param("m") val n = param("n") val l = param("l") val I = set("I") := 1 to m val J = set("J") := 1 to n val K = set("K") := 1 to l val c = param("c", J) val d = param("d", K) val a = param("a", I, J) // ... val x = xvar("x", J).integer >= 0 val y = xvar("y", K) >= 0 MathProg EDSL Scala range notation for arithmetic sets param m; param n; param l; set I := 1 .. m; set J := 1 .. n; set K := 1 .. l; param c{J}; param d{K}; param a{I, J}; // ... var x{J} integer, >= 0; var y{K} >= 0;
val m = param("m") val n = param("n") val l = param("l") val I = set("I") := 1 to m val J = set("J") := 1 to n val K = set("K") := 1 to l val c = param("c", J) val d = param("d", K) val a = param("a", I, J) // ... val x = xvar("x", J).integer >= 0 val y = xvar("y", K) >= 0 param m; param n; param l; set I := 1 .. m; set J := 1 .. n; set K := 1 .. l; param c{J}; param d{K}; param a{I, J}; // ... var x{J} integer, >= 0; var y{K} >= 0; MathProg EDSL Varargs for simple indexing expressions
val m = param("m") val n = param("n") val l = param("l") val I = set("I") := 1 to m val J = set("J") := 1 to n val K = set("K") := 1 to l val c = param("c", J) val d = param("d", K) val a = param("a", I, J) // ... val x = xvar("x", J).integer >= 0 val y = xvar("y", K) >= 0 param m; param n; param l; set I := 1 .. m; set J := 1 .. n; set K := 1 .. l; param c{J}; param d{K}; param a{I, J}; // ... var x{J} integer, >= 0; var y{K} >= 0; MathProg EDSL Returns a new variable with the `integer’ attribute
val m = param("m") val n = param("n") val l = param("l") val I = set("I") := 1 to m val J = set("J") := 1 to n val K = set("K") := 1 to l val c = param("c", J) val d = param("d", K) val a = param("a", I, J) // ... val x = xvar("x", J).integer >= 0 val y = xvar("y", K) >= 0 MathProg EDSL Another variable, now adding the lower bound attribute param m; param n; param l; set I := 1 .. m; set J := 1 .. n; set K := 1 .. l; param c{J}; param d{K}; param a{I, J}; // ... var x{J} integer, >= 0; var y{K} >= 0;
val m = param("m") val n = param("n") val l = param("l") val I = set("I") := 1 to m val J = set("J") := 1 to n val K = set("K") := 1 to l val c = param("c", J) val d = param("d", K) val a = param("a", I, J) // ... val x = xvar("x", J).integer >= 0 val y = xvar("y", K) >= 0 param m; param n; param l; set I := 1 .. m; set J := 1 .. n; set K := 1 .. l; param c{J}; param d{K}; param a{I, J}; // ... var x{J} integer, >= 0; var y{K} >= 0; MathProg EDSL Everything is immutable
MathProg EDSL minimize f: sum{j in J} c[j] * x[j] + sum{k in K} d[k] * y[k]; s.t. g{i in I}: sum{j in J} a[i,j] * x[j] + sum{k in K} e[i,k] * y[k] <= b[i]; val f = minimize("f") { sum(j in J)(c(j) * x(j)) + sum(k in K)(d(k) * y(k)) } val g = st("g", i in I) { sum(j in J)(a(i, j) * x(j)) + sum(k in K)(e(i, k) * y(k)) <= b(i) }
MathProg EDSL minimize st ... minimize f: sum{j in J} c[j] * x[j] + sum{k in K} d[k] * y[k]; s.t. g{i in I}: sum{j in J} a[i,j] * x[j] + sum{k in K} e[i,k] * y[k] <= b[i]; val f = minimize("f") { sum(j in J)(c(j) * x(j)) + sum(k in K)(d(k) * y(k)) } val g = st("g", i in I) { sum(j in J)(a(i, j) * x(j)) + sum(k in K)(e(i, k) * y(k)) <= b(i) }
minimize f: sum{j in J} c[j] * x[j] + sum{k in K} d[k] * y[k]; s.t. g{i in I}: sum{j in J} a[i,j] * x[j] + sum{k in K} e[i,k] * y[k] <= b[i]; val f = minimize("f") { sum(j in J)(c(j) * x(j)) + sum(k in K)(d(k) * y(k)) } val g = st("g", i in I) { sum(j in J)(a(i, j) * x(j)) + sum(k in K)(e(i, k) * y(k)) <= b(i) } MathProg EDSL Multiple parameters lists
minimize f: sum{j in J} c[j] * x[j] + sum{k in K} d[k] * y[k]; s.t. g{i in I}: sum{j in J} a[i,j] * x[j] + sum{k in K} e[i,k] * y[k] <= b[i]; val f = minimize("f") { sum(j in J)(c(j) * x(j)) + sum(k in K)(d(k) * y(k)) } val g = st("g", i in I) { sum(j in J)(a(i, j) * x(j)) + sum(k in K)(e(i, k) * y(k)) <= b(i) } MathProg EDSL Multiple parameters lists
minimize f: sum{j in J} c[j] * x[j] + sum{k in K} d[k] * y[k]; s.t. g{i in I}: sum{j in J} a[i,j] * x[j] + sum{k in K} e[i,k] * y[k] <= b[i]; val f = minimize("f") { sum(j in J)(c(j) * x(j)) + sum(k in K)(d(k) * y(k)) } val g = st("g", i in I) { sum(j in J)(a(i, j) * x(j)) + sum(k in K)(e(i, k) * y(k)) <= b(i) } MathProg EDSL addition, multiplication, summation ...
minimize f: sum{j in J} c[j] * x[j] + sum{k in K} d[k] * y[k]; s.t. g{i in I}: sum{j in J} a[i,j] * x[j] + sum{k in K} e[i,k] * y[k] <= b[i]; val f = minimize("f") { sum(j in J)(c(j) * x(j)) + sum(k in K)(d(k) * y(k)) } val g = st("g", i in I) { sum(j in J)(a(i, j) * x(j)) + sum(k in K)(e(i, k) * y(k)) <= b(i) } MathProg EDSL References to individual parameters and variables by application
minimize f: sum{j in J} c[j] * x[j] + sum{k in K} d[k] * y[k]; s.t. g{i in I}: sum{j in J} a[i,j] * x[j] + sum{k in K} e[i,k] * y[k] <= b[i]; val f = minimize("f") { sum(j in J)(c(j) * x(j)) + sum(k in K)(d(k) * y(k)) } val g = st("g", i in I) { sum(j in J)(a(i, j) * x(j)) + sum(k in K)(e(i, k) * y(k)) <= b(i) } MathProg EDSL Constraint
minimize f: sum{j in J} c[j] * x[j] + sum{k in K} d[k] * y[k]; s.t. g{i in I}: sum{j in J} a[i,j] * x[j] + sum{k in K} e[i,k] * y[k] <= b[i]; val f = minimize("f") { sum(j in J)(c(j) * x(j)) + sum(k in K)(d(k) * y(k)) } val g = st("g", i in I) { sum(j in J)(a(i, j) * x(j)) + sum(k in K)(e(i, k) * y(k)) <= b(i) } MathProg EDSL Creates a new constraint with the specified name and indexing
Implementing the syntax
Implementing the syntax Many of the syntactic constructs require: A broad (open?) number of overloadings Be used with infix notation
Many of the syntactic constructs require: A broad (open?) number of overloadings Be used with infix notation Implementing the syntax type classes and object-oriented forwarders
Many of the syntactic constructs require: A broad (open?) number of overloadings Be used with infix notation Implementing the syntax type classes and object-oriented forwarders implicits prioritization
Implementing the syntax: “>=” param("p", J) >= 0
Implementing the syntax: “>=” param("p", J) >= 0 ParamStat
Implementing the syntax: “>=” param("p", J) >= 0 ParamStat ParamStat doesn’t have a `>=’ method
Implementing the syntax: “>=” param("p", J) >= 0 implicit conversion to something that provides the method ParamStat doesn’t have a `>=’ method ParamStat
Implementing the syntax: “>=” param("p", J) >= 0 // ParamStat param("p", J) >= p2 // ParamStat xvar("x", I) >= 0 // VarStat i >= j // LogicExpr p(j1) >= p(j2) // LogicExpr x(i) >= 0 // ConstraintStat 10 >= x(i) // ConstraintStat Many overloadings … more
Implementing the syntax: “>=” implicit class GTESyntax[A](lhe: A) { def >=[B, C](rhe: B)(implicit GTEOp: GTEOp[A,B,C]): C = GTEOp.gte(lhe, rhe) }
Implementing the syntax: “>=” implicit class GTESyntax[A](lhe: A) { def >=[B, C](rhe: B)(implicit GTEOp: GTEOp[A,B,C]): C = GTEOp.gte(lhe, rhe) } Fully generic
Implementing the syntax: “>=” implicit class GTESyntax[A](lhe: A) { def >=[B, C](rhe: B)(implicit GTEOp: GTEOp[A,B,C]): C = GTEOp.gte(lhe, rhe) } trait GTEOp[A,B,C] { def gte(lhe: A, rhe: B): C } Type class
Implementing the syntax: “>=” implicit class GTESyntax[A](lhe: A) { def >=[B, C](rhe: B)(implicit GTEOp: GTEOp[A,B,C]): C = GTEOp.gte(lhe, rhe) } trait GTEOp[A,B,C] { def gte(lhe: A, rhe: B): C } Type class The implementation is forwarded to the implicit instance
Implementing the syntax: “>=” implicit class GTESyntax[A](lhe: A) { def >=[B, C](rhe: B)(implicit GTEOp: GTEOp[A,B,C]): C = GTEOp.gte(lhe, rhe) } The available implicit instances encode the rules by which the operators can be used
Implementing the syntax: “>=” implicit class GTESyntax[A](lhe: A) { def >=[B, C](rhe: B)(implicit GTEOp: GTEOp[A,B,C]): C = GTEOp.gte(lhe, rhe) } The instance of the type class is required at the method level, so both A and B can be used in the implicits search
Implementing the syntax: “>=” instances implicit def ParamStatGTEOp[A]( implicit conv: A => NumExpr): GTEOp[ParamStat, A, ParamStat] = /* ... */ >= for parameters and “numbers” param(“p”) >= m
Implementing the syntax: “>=” instances implicit def ParamStatGTEOp[A]( implicit conv: A => NumExpr): GTEOp[ParamStat, A, ParamStat] = /* ... */ >= for parameters and “numbers” param(“p”) >= m
Implementing the syntax: “>=” instances implicit def ParamStatGTEOp[A]( implicit conv: A => NumExpr): GTEOp[ParamStat, A, ParamStat] = /* ... */ >= for parameters and “numbers” param(“p”) >= m View bound
Implementing the syntax: “>=” instances implicit def VarStatGTEOp[A]( implicit conv: A => NumExpr): GTEOp[VarStat, A, VarStat] = /* ... */ >= for variables and “numbers” xvar(“x”) >= m Analogous to the parameter case
Implementing the syntax: “>=” instances implicit def NumExprGTEOp[A, B]( implicit convA: A => NumExpr, convB: B => NumExpr): GTEOp[A, B, LogicExpr] = /* ... */ >= for two “numbers” i >= j
Implementing the syntax: “>=” instances implicit def NumExprGTEOp[A, B]( implicit convA: A => NumExpr, convB: B => NumExpr): GTEOp[A, B, LogicExpr] = /* ... */ >= for two “numbers” i >= j
Implementing the syntax: “>=” instances implicit def NumExprGTEOp[A, B]( implicit convA: A => NumExpr, convB: B => NumExpr): GTEOp[A, B, LogicExpr] = /* ... */ >= for two “numbers” i >= j
Implementing the syntax: “>=” instances implicit def LinExprGTEOp[A, B]( implicit convA: A => LinExpr, convB: B => LinExpr): GTEOp[A, B, ConstraintStat] = /* ... */ >= for two “linear expressions” x(i) >= 0 and 10 >= x(i) Analogous to the numerical expression case
implicit def NumExprGTEOp[A, B]( implicit convA: A => NumExpr, convB: B => NumExpr) conflicts with: implicit def ParamStatGTEOp[A]( implicit conv: A => NumExpr) implicit def LinExprGTEOp[A, B]( implicit convA: A => LinExpr, convB: B => LinExpr) Instances are ambiguous
implicit def NumExprGTEOp[A, B]( implicit convA: A => NumExpr, convB: B => NumExpr) conflicts with: implicit def ParamStatGTEOp[A]( implicit conv: A => NumExpr) implicit def LinExprGTEOp[A, B]( implicit convA: A => LinExpr, convB: B => LinExpr) Instances are ambiguous `ParamStat <% NumExpr‘ to allow `param(“p2”) >= p1’
Instances are ambiguous implicit def NumExprGTEOp[A, B]( implicit convA: A => NumExpr, convB: B => NumExpr) conflicts with: implicit def ParamStatGTEOp[A]( implicit conv: A => NumExpr) implicit def LinExprGTEOp[A, B]( implicit convA: A => LinExpr, convB: B => LinExpr) NumExpr <: LinExpr => NumExpr <% LinExpr
Implicits prioritization trait GTEInstances extends GTEInstancesLowPriority1 { implicit def ParamStatGTEOp[A](implicit conv: A => NumExpr) /* */ implicit def VarStatGTEOp[A](implicit conv: A => NumExpr) /* */ } trait GTEInstancesLowPriority1 extends GTEInstancesLowPriority2 { implicit def NumExprGTEOp[A, B]( implicit convA: A => NumExpr, convB: B => NumExpr) /* */ } trait GTEInstancesLowPriority2 { implicit def LinExprGTEOp[A, B]( implicit convA: A => LinExpr, convB: B => LinExpr) /* */ }
Implicits prioritization trait GTEInstances extends GTEInstancesLowPriority1 { implicit def ParamStatGTEOp[A](implicit conv: A => NumExpr) /* */ implicit def VarStatGTEOp[A](implicit conv: A => NumExpr) /* */ } trait GTEInstancesLowPriority1 extends GTEInstancesLowPriority2 { implicit def NumExprGTEOp[A, B]( implicit convA: A => NumExpr, convB: B => NumExpr) /* */ } trait GTEInstancesLowPriority2 { implicit def LinExprGTEOp[A, B]( implicit convA: A => LinExpr, convB: B => LinExpr) /* */ } > priority > priority
Next steps New semantics More static controls Support other kind of problems beyond MIP Talk directly to solvers Arity, scope, ... Multi-stage stochastic programming
import amphip.dsl._import amphip.dsl._ amphip is a collection of experiments around manipulating MathProg models within Scala github.com/gerferra/amphip amphip is a collection of experiments around manipulating MathProg models within Scala github.com/gerferra/amphip
FIN github.com/gerferra/amphip

An embedded DSL to manipulate MathProg Mixed Integer Programming models within Scala

  • 1.
    Germán Ferrari Instituto deComputación, Universidad de la República, Uruguay An embedded DSL to manipulate MathProg Mixed Integer Programming models within Scala An embedded DSL to manipulate MathProg Mixed Integer Programming models within Scala Germán Ferrari Instituto de Computación, Universidad de la República, Uruguay
  • 2.
    GNU MathProg MathProg =Mathematical Programming Has nothing to do with computer programming It’s a synonym of “mathematical optimization” A “program” here refers to a plan or schedule
  • 3.
    GNU MathProg It’s amodeling language Optimization problem Mathematical modeling GNU MathProg Computational experiments Supported by the GNU Linear Programming Kit
  • 4.
    GNU MathProg Supports linearmodels, with either real or integer decision variables This is called “Mixed Integer Programming” (MIP) because it mixes integer and real variables
  • 5.
    Generic MIP optimizationproblem minimize f: sum{j in J} c[j] * x[j] + sum{k in K} d[k] * y[k]; subject to g{i in I}: sum{j in J} a[i,j] * x[j] + sum{k in K} e[i,k] * y[k] <= b[i];
  • 6.
    Generic MIP optimizationproblem Find x[j] and y[k], that minimize a function f, satisfying constraints g[i] minimize f: sum{j in J} c[j] * x[j] + sum{k in K} d[k] * y[k]; subject to g{i in I}: sum{j in J} a[i,j] * x[j] + sum{k in K} e[i,k] * y[k] <= b[i];
  • 7.
    Generic MIP optimizationproblem Find x[j] and y[k], that minimize a function f, satisfying constraints g[i] minimize f: sum{j in J} c[j] * x[j] + sum{k in K} d[k] * y[k]; subject to g{i in I}: sum{j in J} a[i,j] * x[j] + sum{k in K} e[i,k] * y[k] <= b[i]; Decision variables (real or integer)
  • 8.
    Generic MIP optimizationproblem Find x[j] and y[k], that minimize a function f, satisfying constraints g[i] minimize f: sum{j in J} c[j] * x[j] + sum{k in K} d[k] * y[k]; subject to g{i in I}: sum{j in J} a[i,j] * x[j] + sum{k in K} e[i,k] * y[k] <= b[i]; Decision variables (real or integer) Objective function
  • 9.
    Generic MIP optimizationproblem Find x[j] and y[k], that minimize a function f, satisfying constraints g[i] minimize f: sum{j in J} c[j] * x[j] + sum{k in K} d[k] * y[k]; subject to g{i in I}: sum{j in J} a[i,j] * x[j] + sum{k in K} e[i,k] * y[k] <= b[i]; Decision variables (real or integer) Objective function Constraints
  • 10.
    Generic MIP optimizationproblem Find x[j] and y[k], that minimize a function f, satisfying constraints g[i] minimize f: sum{j in J} c[j] * x[j] + sum{k in K} d[k] * y[k]; subject to g{i in I}: sum{j in J} a[i,j] * x[j] + sum{k in K} e[i,k] * y[k] <= b[i]; Decision variables (real or integer) Objective function Constraints Linear
  • 11.
    Generic MIP optimizationproblem Full model param m; param n; param l; set I := 1 .. m; set J := 1 .. n; set K := 1 .. l; param c{J}; param d{K}; param a{I, J}; param e{I, K}; param b{I}; var x{J} integer, >= 0; var y{K} >= 0; minimize f: sum{j in J} c[j] * x[j] + sum{k in K} d[k] * y[k]; subject to g{i in I}: sum{j in J} a[i,j] * x[j] + sum{k in K} e[i,k] * y[k] <= b[i];
  • 12.
    param m; paramn; param l; set I := 1 .. m; set J := 1 .. n; set K := 1 .. l; param c{J}; param d{K}; param a{I, J}; param e{I, K}; param b{I}; var x{J} integer, >= 0; var y{K} >= 0; minimize f: sum{j in J} c[j] * x[j] + sum{k in K} d[k] * y[k]; subject to g{i in I}: sum{j in J} a[i,j] * x[j] + sum{k in K} e[i,k] * y[k] <= b[i]; Generic MIP optimization problem Declaration of variables
  • 13.
    param m; paramn; param l; set I := 1 .. m; set J := 1 .. n; set K := 1 .. l; param c{J}; param d{K}; param a{I, J}; param e{I, K}; param b{I}; var x{J} integer, >= 0; var y{K} >= 0; minimize f: sum{j in J} c[j] * x[j] + sum{k in K} d[k] * y[k]; subject to g{i in I}: sum{j in J} a[i,j] * x[j] + sum{k in K} e[i,k] * y[k] <= b[i]; Generic MIP optimization problem Parameters
  • 14.
    Generic MIP optimizationproblem param m; param n; param l; set I := 1 .. m; set J := 1 .. n; set K := 1 .. l; param c{J}; param d{K}; param a{I, J}; param e{I, K}; param b{I}; var x{J} integer, >= 0; var y{K} >= 0; minimize f: sum{j in J} c[j] * x[j] + sum{k in K} d[k] * y[k]; subject to g{i in I}: sum{j in J} a[i,j] * x[j] + sum{k in K} e[i,k] * y[k] <= b[i]; Indexing sets
  • 15.
  • 16.
    Why MathProg inScala Generate constraints Create derived models Develop custom resolution methods at high level ...
  • 17.
    MathProg in Scala(deep embedding) Syntax Represent MathProg models with Scala data types Mimic MathProg syntax with Scala functions Semantics Generate MathProg code, others ...
  • 18.
    MathProg in Scala(deep embedding) Syntax Represent MathProg models with Scala data types Mimic MathProg syntax with Scala functions Semantics Generate MathProg code, others ... Abstract syntax tree
  • 19.
    MathProg in Scala(deep embedding) Syntax Represent MathProg models with Scala data types Mimic MathProg syntax with Scala functions Semantics Generate MathProg code, others ... Smart constructors and combinators Abstract syntax tree
  • 20.
    MathProg EDSL param m; paramn; param l; set I := 1 .. m; set J := 1 .. n; set K := 1 .. l; param c{J}; param d{K}; param a{I, J}; // ... var x{J} integer, >= 0; var y{K} >= 0; val m = param("m") val n = param("n") val l = param("l") val I = set("I") := 1 to m val J = set("J") := 1 to n val K = set("K") := 1 to l val c = param("c", J) val d = param("d", K) val a = param("a", I, J) // ... val x = xvar("x", J).integer >= 0 val y = xvar("y", K) >= 0
  • 21.
    MathProg EDSL param m; paramn; param l; set I := 1 .. m; set J := 1 .. n; set K := 1 .. l; param c{J}; param d{K}; param a{I, J}; // ... var x{J} integer, >= 0; var y{K} >= 0; val m = param("m") val n = param("n") val l = param("l") val I = set("I") := 1 to m val J = set("J") := 1 to n val K = set("K") := 1 to l val c = param("c", J) val d = param("d", K) val a = param("a", I, J) // ... val x = xvar("x", J).integer >= 0 val y = xvar("y", K) >= 0 set param xvar ...
  • 22.
    MathProg EDSL param m; paramn; param l; set I := 1 .. m; set J := 1 .. n; set K := 1 .. l; param c{J}; param d{K}; param a{I, J}; // ... var x{J} integer, >= 0; var y{K} >= 0; val m = param("m") val n = param("n") val l = param("l") val I = set("I") := 1 to m val J = set("J") := 1 to n val K = set("K") := 1 to l val c = param("c", J) val d = param("d", K) val a = param("a", I, J) // ... val x = xvar("x", J).integer >= 0 val y = xvar("y", K) >= 0 set param xvar ... ParamStat( "c", domain = Some( IndExpr( List( IndEntry( Nil, SetRef(J) ) ))))
  • 23.
    val m =param("m") val n = param("n") val l = param("l") val I = set("I") := 1 to m val J = set("J") := 1 to n val K = set("K") := 1 to l val c = param("c", J) val d = param("d", K) val a = param("a", I, J) // ... val x = xvar("x", J).integer >= 0 val y = xvar("y", K) >= 0 MathProg EDSL Scala range notation for arithmetic sets param m; param n; param l; set I := 1 .. m; set J := 1 .. n; set K := 1 .. l; param c{J}; param d{K}; param a{I, J}; // ... var x{J} integer, >= 0; var y{K} >= 0;
  • 24.
    val m =param("m") val n = param("n") val l = param("l") val I = set("I") := 1 to m val J = set("J") := 1 to n val K = set("K") := 1 to l val c = param("c", J) val d = param("d", K) val a = param("a", I, J) // ... val x = xvar("x", J).integer >= 0 val y = xvar("y", K) >= 0 param m; param n; param l; set I := 1 .. m; set J := 1 .. n; set K := 1 .. l; param c{J}; param d{K}; param a{I, J}; // ... var x{J} integer, >= 0; var y{K} >= 0; MathProg EDSL Varargs for simple indexing expressions
  • 25.
    val m =param("m") val n = param("n") val l = param("l") val I = set("I") := 1 to m val J = set("J") := 1 to n val K = set("K") := 1 to l val c = param("c", J) val d = param("d", K) val a = param("a", I, J) // ... val x = xvar("x", J).integer >= 0 val y = xvar("y", K) >= 0 param m; param n; param l; set I := 1 .. m; set J := 1 .. n; set K := 1 .. l; param c{J}; param d{K}; param a{I, J}; // ... var x{J} integer, >= 0; var y{K} >= 0; MathProg EDSL Returns a new variable with the `integer’ attribute
  • 26.
    val m =param("m") val n = param("n") val l = param("l") val I = set("I") := 1 to m val J = set("J") := 1 to n val K = set("K") := 1 to l val c = param("c", J) val d = param("d", K) val a = param("a", I, J) // ... val x = xvar("x", J).integer >= 0 val y = xvar("y", K) >= 0 MathProg EDSL Another variable, now adding the lower bound attribute param m; param n; param l; set I := 1 .. m; set J := 1 .. n; set K := 1 .. l; param c{J}; param d{K}; param a{I, J}; // ... var x{J} integer, >= 0; var y{K} >= 0;
  • 27.
    val m =param("m") val n = param("n") val l = param("l") val I = set("I") := 1 to m val J = set("J") := 1 to n val K = set("K") := 1 to l val c = param("c", J) val d = param("d", K) val a = param("a", I, J) // ... val x = xvar("x", J).integer >= 0 val y = xvar("y", K) >= 0 param m; param n; param l; set I := 1 .. m; set J := 1 .. n; set K := 1 .. l; param c{J}; param d{K}; param a{I, J}; // ... var x{J} integer, >= 0; var y{K} >= 0; MathProg EDSL Everything is immutable
  • 28.
    MathProg EDSL minimize f: sum{jin J} c[j] * x[j] + sum{k in K} d[k] * y[k]; s.t. g{i in I}: sum{j in J} a[i,j] * x[j] + sum{k in K} e[i,k] * y[k] <= b[i]; val f = minimize("f") { sum(j in J)(c(j) * x(j)) + sum(k in K)(d(k) * y(k)) } val g = st("g", i in I) { sum(j in J)(a(i, j) * x(j)) + sum(k in K)(e(i, k) * y(k)) <= b(i) }
  • 29.
    MathProg EDSL minimize st... minimize f: sum{j in J} c[j] * x[j] + sum{k in K} d[k] * y[k]; s.t. g{i in I}: sum{j in J} a[i,j] * x[j] + sum{k in K} e[i,k] * y[k] <= b[i]; val f = minimize("f") { sum(j in J)(c(j) * x(j)) + sum(k in K)(d(k) * y(k)) } val g = st("g", i in I) { sum(j in J)(a(i, j) * x(j)) + sum(k in K)(e(i, k) * y(k)) <= b(i) }
  • 30.
    minimize f: sum{j inJ} c[j] * x[j] + sum{k in K} d[k] * y[k]; s.t. g{i in I}: sum{j in J} a[i,j] * x[j] + sum{k in K} e[i,k] * y[k] <= b[i]; val f = minimize("f") { sum(j in J)(c(j) * x(j)) + sum(k in K)(d(k) * y(k)) } val g = st("g", i in I) { sum(j in J)(a(i, j) * x(j)) + sum(k in K)(e(i, k) * y(k)) <= b(i) } MathProg EDSL Multiple parameters lists
  • 31.
    minimize f: sum{j inJ} c[j] * x[j] + sum{k in K} d[k] * y[k]; s.t. g{i in I}: sum{j in J} a[i,j] * x[j] + sum{k in K} e[i,k] * y[k] <= b[i]; val f = minimize("f") { sum(j in J)(c(j) * x(j)) + sum(k in K)(d(k) * y(k)) } val g = st("g", i in I) { sum(j in J)(a(i, j) * x(j)) + sum(k in K)(e(i, k) * y(k)) <= b(i) } MathProg EDSL Multiple parameters lists
  • 32.
    minimize f: sum{j inJ} c[j] * x[j] + sum{k in K} d[k] * y[k]; s.t. g{i in I}: sum{j in J} a[i,j] * x[j] + sum{k in K} e[i,k] * y[k] <= b[i]; val f = minimize("f") { sum(j in J)(c(j) * x(j)) + sum(k in K)(d(k) * y(k)) } val g = st("g", i in I) { sum(j in J)(a(i, j) * x(j)) + sum(k in K)(e(i, k) * y(k)) <= b(i) } MathProg EDSL addition, multiplication, summation ...
  • 33.
    minimize f: sum{j inJ} c[j] * x[j] + sum{k in K} d[k] * y[k]; s.t. g{i in I}: sum{j in J} a[i,j] * x[j] + sum{k in K} e[i,k] * y[k] <= b[i]; val f = minimize("f") { sum(j in J)(c(j) * x(j)) + sum(k in K)(d(k) * y(k)) } val g = st("g", i in I) { sum(j in J)(a(i, j) * x(j)) + sum(k in K)(e(i, k) * y(k)) <= b(i) } MathProg EDSL References to individual parameters and variables by application
  • 34.
    minimize f: sum{j inJ} c[j] * x[j] + sum{k in K} d[k] * y[k]; s.t. g{i in I}: sum{j in J} a[i,j] * x[j] + sum{k in K} e[i,k] * y[k] <= b[i]; val f = minimize("f") { sum(j in J)(c(j) * x(j)) + sum(k in K)(d(k) * y(k)) } val g = st("g", i in I) { sum(j in J)(a(i, j) * x(j)) + sum(k in K)(e(i, k) * y(k)) <= b(i) } MathProg EDSL Constraint
  • 35.
    minimize f: sum{j inJ} c[j] * x[j] + sum{k in K} d[k] * y[k]; s.t. g{i in I}: sum{j in J} a[i,j] * x[j] + sum{k in K} e[i,k] * y[k] <= b[i]; val f = minimize("f") { sum(j in J)(c(j) * x(j)) + sum(k in K)(d(k) * y(k)) } val g = st("g", i in I) { sum(j in J)(a(i, j) * x(j)) + sum(k in K)(e(i, k) * y(k)) <= b(i) } MathProg EDSL Creates a new constraint with the specified name and indexing
  • 36.
  • 37.
    Implementing the syntax Manyof the syntactic constructs require: A broad (open?) number of overloadings Be used with infix notation
  • 38.
    Many of thesyntactic constructs require: A broad (open?) number of overloadings Be used with infix notation Implementing the syntax type classes and object-oriented forwarders
  • 39.
    Many of thesyntactic constructs require: A broad (open?) number of overloadings Be used with infix notation Implementing the syntax type classes and object-oriented forwarders implicits prioritization
  • 40.
    Implementing the syntax:“>=” param("p", J) >= 0
  • 41.
    Implementing the syntax:“>=” param("p", J) >= 0 ParamStat
  • 42.
    Implementing the syntax:“>=” param("p", J) >= 0 ParamStat ParamStat doesn’t have a `>=’ method
  • 43.
    Implementing the syntax:“>=” param("p", J) >= 0 implicit conversion to something that provides the method ParamStat doesn’t have a `>=’ method ParamStat
  • 44.
    Implementing the syntax:“>=” param("p", J) >= 0 // ParamStat param("p", J) >= p2 // ParamStat xvar("x", I) >= 0 // VarStat i >= j // LogicExpr p(j1) >= p(j2) // LogicExpr x(i) >= 0 // ConstraintStat 10 >= x(i) // ConstraintStat Many overloadings … more
  • 45.
    Implementing the syntax:“>=” implicit class GTESyntax[A](lhe: A) { def >=[B, C](rhe: B)(implicit GTEOp: GTEOp[A,B,C]): C = GTEOp.gte(lhe, rhe) }
  • 46.
    Implementing the syntax:“>=” implicit class GTESyntax[A](lhe: A) { def >=[B, C](rhe: B)(implicit GTEOp: GTEOp[A,B,C]): C = GTEOp.gte(lhe, rhe) } Fully generic
  • 47.
    Implementing the syntax:“>=” implicit class GTESyntax[A](lhe: A) { def >=[B, C](rhe: B)(implicit GTEOp: GTEOp[A,B,C]): C = GTEOp.gte(lhe, rhe) } trait GTEOp[A,B,C] { def gte(lhe: A, rhe: B): C } Type class
  • 48.
    Implementing the syntax:“>=” implicit class GTESyntax[A](lhe: A) { def >=[B, C](rhe: B)(implicit GTEOp: GTEOp[A,B,C]): C = GTEOp.gte(lhe, rhe) } trait GTEOp[A,B,C] { def gte(lhe: A, rhe: B): C } Type class The implementation is forwarded to the implicit instance
  • 49.
    Implementing the syntax:“>=” implicit class GTESyntax[A](lhe: A) { def >=[B, C](rhe: B)(implicit GTEOp: GTEOp[A,B,C]): C = GTEOp.gte(lhe, rhe) } The available implicit instances encode the rules by which the operators can be used
  • 50.
    Implementing the syntax:“>=” implicit class GTESyntax[A](lhe: A) { def >=[B, C](rhe: B)(implicit GTEOp: GTEOp[A,B,C]): C = GTEOp.gte(lhe, rhe) } The instance of the type class is required at the method level, so both A and B can be used in the implicits search
  • 51.
    Implementing the syntax:“>=” instances implicit def ParamStatGTEOp[A]( implicit conv: A => NumExpr): GTEOp[ParamStat, A, ParamStat] = /* ... */ >= for parameters and “numbers” param(“p”) >= m
  • 52.
    Implementing the syntax:“>=” instances implicit def ParamStatGTEOp[A]( implicit conv: A => NumExpr): GTEOp[ParamStat, A, ParamStat] = /* ... */ >= for parameters and “numbers” param(“p”) >= m
  • 53.
    Implementing the syntax:“>=” instances implicit def ParamStatGTEOp[A]( implicit conv: A => NumExpr): GTEOp[ParamStat, A, ParamStat] = /* ... */ >= for parameters and “numbers” param(“p”) >= m View bound
  • 54.
    Implementing the syntax:“>=” instances implicit def VarStatGTEOp[A]( implicit conv: A => NumExpr): GTEOp[VarStat, A, VarStat] = /* ... */ >= for variables and “numbers” xvar(“x”) >= m Analogous to the parameter case
  • 55.
    Implementing the syntax:“>=” instances implicit def NumExprGTEOp[A, B]( implicit convA: A => NumExpr, convB: B => NumExpr): GTEOp[A, B, LogicExpr] = /* ... */ >= for two “numbers” i >= j
  • 56.
    Implementing the syntax:“>=” instances implicit def NumExprGTEOp[A, B]( implicit convA: A => NumExpr, convB: B => NumExpr): GTEOp[A, B, LogicExpr] = /* ... */ >= for two “numbers” i >= j
  • 57.
    Implementing the syntax:“>=” instances implicit def NumExprGTEOp[A, B]( implicit convA: A => NumExpr, convB: B => NumExpr): GTEOp[A, B, LogicExpr] = /* ... */ >= for two “numbers” i >= j
  • 58.
    Implementing the syntax:“>=” instances implicit def LinExprGTEOp[A, B]( implicit convA: A => LinExpr, convB: B => LinExpr): GTEOp[A, B, ConstraintStat] = /* ... */ >= for two “linear expressions” x(i) >= 0 and 10 >= x(i) Analogous to the numerical expression case
  • 59.
    implicit def NumExprGTEOp[A,B]( implicit convA: A => NumExpr, convB: B => NumExpr) conflicts with: implicit def ParamStatGTEOp[A]( implicit conv: A => NumExpr) implicit def LinExprGTEOp[A, B]( implicit convA: A => LinExpr, convB: B => LinExpr) Instances are ambiguous
  • 60.
    implicit def NumExprGTEOp[A,B]( implicit convA: A => NumExpr, convB: B => NumExpr) conflicts with: implicit def ParamStatGTEOp[A]( implicit conv: A => NumExpr) implicit def LinExprGTEOp[A, B]( implicit convA: A => LinExpr, convB: B => LinExpr) Instances are ambiguous `ParamStat <% NumExpr‘ to allow `param(“p2”) >= p1’
  • 61.
    Instances are ambiguous implicitdef NumExprGTEOp[A, B]( implicit convA: A => NumExpr, convB: B => NumExpr) conflicts with: implicit def ParamStatGTEOp[A]( implicit conv: A => NumExpr) implicit def LinExprGTEOp[A, B]( implicit convA: A => LinExpr, convB: B => LinExpr) NumExpr <: LinExpr => NumExpr <% LinExpr
  • 62.
    Implicits prioritization trait GTEInstancesextends GTEInstancesLowPriority1 { implicit def ParamStatGTEOp[A](implicit conv: A => NumExpr) /* */ implicit def VarStatGTEOp[A](implicit conv: A => NumExpr) /* */ } trait GTEInstancesLowPriority1 extends GTEInstancesLowPriority2 { implicit def NumExprGTEOp[A, B]( implicit convA: A => NumExpr, convB: B => NumExpr) /* */ } trait GTEInstancesLowPriority2 { implicit def LinExprGTEOp[A, B]( implicit convA: A => LinExpr, convB: B => LinExpr) /* */ }
  • 63.
    Implicits prioritization trait GTEInstancesextends GTEInstancesLowPriority1 { implicit def ParamStatGTEOp[A](implicit conv: A => NumExpr) /* */ implicit def VarStatGTEOp[A](implicit conv: A => NumExpr) /* */ } trait GTEInstancesLowPriority1 extends GTEInstancesLowPriority2 { implicit def NumExprGTEOp[A, B]( implicit convA: A => NumExpr, convB: B => NumExpr) /* */ } trait GTEInstancesLowPriority2 { implicit def LinExprGTEOp[A, B]( implicit convA: A => LinExpr, convB: B => LinExpr) /* */ } > priority > priority
  • 64.
    Next steps New semantics Morestatic controls Support other kind of problems beyond MIP Talk directly to solvers Arity, scope, ... Multi-stage stochastic programming
  • 65.
    import amphip.dsl._import amphip.dsl._ amphipis a collection of experiments around manipulating MathProg models within Scala github.com/gerferra/amphip amphip is a collection of experiments around manipulating MathProg models within Scala github.com/gerferra/amphip
  • 66.