Skip to content

Commit

Permalink
removed dependencies to specific solvers and MPI.jl
Browse files Browse the repository at this point in the history
  • Loading branch information
kibaekkim authored and blegat committed Oct 29, 2016
1 parent 928c1f6 commit b34cd38
Show file tree
Hide file tree
Showing 4 changed files with 95 additions and 85 deletions.
54 changes: 18 additions & 36 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,34 +1,14 @@
# StructJuMP
The StructJuMP package provides a parallel algebraic modeling framework for block structured optimization models in Julia. StructJuMP, originally known as StochJuMP, is tailored to two-stage stochastic optimization problems and uses MPI to enable a parallel, distributed memory instantiation of the problem. StructJuMP.jl is an extension of the [JuMP.jl](https://github.com/JuliaOpt/JuMP.jl) package, which is as fast as [AMPL](http://ampl.com) and faster than any other modeling tools such as [GAMS](http://www.gams.com) and [Pyomo](http://www.pyomo.org) (see [this](http://arxiv.org/pdf/1312.1431.pdf)).

## Installation
```julia
Pkg.clone("https://github.com/StructJuMP/StructJuMP.jl.git")
```

##Nonlinear solvers

Problems modeled in StructJuMP models can be solved in parallel using the [PIPS-NLP](https://github.com/Argonne-National-Laboratory/PIPS) parallel optimization solver. In addition, StructJuMP models can be solved (in serial only) using [Ipopt](https://projects.coin-or.org/Ipopt). The solver interface and glue code for PIPS-NLP and Ipopt are located at [StructJuMPSolverInterface](https://github.com/Argonne-National-Laboratory/StructJuMPSolverInterface.jl).


##Installation

Prior its installation, StructJuMP requires both PIPS-NLP and Ipopt solvers to be installed.

1. Install and build the PIPS-NLP solver using the installation instructions found [here](https://github.com/Argonne-National-Laboratory/PIPS). Then set the environment variables `PIPS_NLP_PAR_SHARED_LIB` and `PIPS_NLP_SHARED_LIB` to the location of the PIPS-NLP parallel and serial shared (.so|.dylib) libraries. Note that these shared libraries are named `libparpipsnlp.so|.dylib` and `libpipsnlp.so|.dylib`.

2. Install the Ipopt solver, if not already installed:

`julia> Pkg.add("Ipopt")`

3. Install StructJuMP (this package):

`julia> Pkg.clone("https://github.com/joehuchette/StructJuMP.jl")

4. Install StructJuMPSolverInterface:

`julia> Pkg.clone("https://github.com/Argonne-National-Laboratory/StructJuMPSolverInterface.jl")`


##An example
## An example
```julia
using StructJuMP, JuMP
using StructJuMP

numScen = 2
m = StructuredModel(num_scenarios=numScen)
Expand All @@ -39,21 +19,23 @@ m = StructuredModel(num_scenarios=numScen)
for i in 1:numScen
bl = StructuredModel(parent=m, id=i)
@variable(bl, y[1:2])
@NLconstraint(bl, x[1] + y[1]+y[2] 0)
@NLconstraint(bl, x[2] + y[1]+y[2] 50)
@NLconstraint(bl, x[1] + y[1] + y[2] 0)
@NLconstraint(bl, x[2] + y[1] + y[2] 50)
@NLobjective(bl, Min, y[1]^2 + y[2]^2 + y[1]*y[2])
end
```
The above example builds a two level structured model `m` with 2 scenarios. The model can be solved by calling solve function with a solver that implements the StructJuMPSolverInterface.
```julia
solve(m, solver="PipsNlp") #solving using parallel PIPS-NLP solver
solve(m, solver="PipsNlpSerial") #solving using serial PIPS-NLP solver, mostly for debug and maintenance purpose
solve(m, solver="Ipopt") #solving using (serial) Ipopt solver
```
The above example builds a two level structured model `m` with 2 scenarios.

## Available Solvers for StructJuMP

### Nonlinear Solvers
Problems modeled in StructJuMP models can be solved in parallel using the [PIPS-NLP](https://github.com/Argonne-National-Laboratory/PIPS) parallel optimization solver. In addition, StructJuMP models can be solved (in serial only) using [Ipopt](https://projects.coin-or.org/Ipopt). The SturctJuMP models interface with the solvers via [StructJuMPSolverInterface.jl](https://github.com/Argonne-National-Laboratory/StructJuMPSolverInterface.jl).

## Known Limitation
* If a constraint declared at the sub-problem uses variable from the parent level, it has to be add using @NLconstraint (instead of @constraint).
### Mixed-Integer Solvers
[DSP](https://github.com/Argonne-National-Laboratory/DSP.git) can read models from StructJuMP via [DSPsolver.jl](https://github.com/Argonne-National-Laboratory/DSPsolver.jl.git). In particular, ``DSP`` can solver problems with integer variables in parallel.

### Stochastic Dual Dynamic Programming
[Stochastic Dual Dynamic Programming](https://github.com/blegat/StochasticDualDynamicProgramming.jl) can read multi-stage models from StructJuMP.

## Acknowledgements
StructJuMP has been developed under the financial support of Department of Energy (DOE), Office of Advanced Scientific Computing Research, Office of Electricity Delivery and Energy Reliability, and Grid Modernization Laboratory Consortium (GMLC) (PIs: Cosmin G. Petra and Mihai Anitescu, Argonne National Laboratory).
1 change: 0 additions & 1 deletion REQUIRE
Original file line number Diff line number Diff line change
@@ -1,3 +1,2 @@
julia 0.3-
MPI
JuMP
6 changes: 6 additions & 0 deletions UPDATE.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
# 08-03-2016

- Removed unnecessary dependencies to ``StructJuMPSolverInterface``, ``PIPS-NLP`` and ``Ipopt``. The current version of ``StructJuMPSolverInterface`` is specific to PIPS and Ipopt. (needs to be generalized)
- Removed requirement of ``MPI.jl``. This makes the ``StructJuMP`` package general enough for any user.
- Removed dependency to ``JuMP`` by exporting all JuMP functions. So, user does not need to say ``using JuMP``.
- Updated README.md. All solver specific statements are removed. They should be moved to each solver interface. Instead, three solvers that can read models from ``StructJuMP`` are briefly introduced.
119 changes: 71 additions & 48 deletions src/StructJuMP.jl
Original file line number Diff line number Diff line change
@@ -1,64 +1,79 @@
module StructJuMP

import MPI
import JuMP # To reexport, should be using (not import)
using JuMP # To reexport, should be using (not import)
import MathProgBase
import MathProgBase.MathProgSolverInterface
import ReverseDiffSparse
import StructJuMPSolverInterface

# These modules could be optional.
# import StructJuMPSolverInterface
# import MPI

export StructuredModel, getStructure, getparent, getchildren, getProcIdxSet,
num_scenarios, @second_stage, getprobability, getMyRank

# Macro to exportall
macro exportall(pkg)
Expr(:export, names(JuMP)...)
end
@exportall JuMP


# ---------------
# StructureData
# ---------------
type MPIWrapper
comm::MPI.Comm
init::Function

function MPIWrapper()
instance = new(MPI.Comm(-1))
finalizer(instance, freeMPIWrapper)

instance.init = function(ucomm::MPI.Comm)
if isdefined(:MPI) && MPI.Initialized() && ucomm.val == -1
instance.comm = MPI.COMM_WORLD
elseif isdefined(:MPI) && !MPI.Initialized()
MPI.Init()
instance.comm = MPI.COMM_WORLD
elseif isdefined(:MPI) && MPI.Initialized() && ucomm.val != -1
instance.comm = ucomm
elseif isdefined(:MPI) && MPI.Finalized()
error("MPI is already finalized!")
else
#doing nothing
end
end

return instance
end
if isdefined(:MPI)
type MPIWrapper
comm::MPI.Comm
init::Function

function MPIWrapper()
instance = new(MPI.Comm(-1))
finalizer(instance, freeMPIWrapper)

instance.init = function(ucomm::MPI.Comm)
if isdefined(:MPI) && MPI.Initialized() && ucomm.val == -1
instance.comm = MPI.COMM_WORLD
elseif isdefined(:MPI) && !MPI.Initialized()
MPI.Init()
instance.comm = MPI.COMM_WORLD
elseif isdefined(:MPI) && MPI.Initialized() && ucomm.val != -1
instance.comm = ucomm
elseif isdefined(:MPI) && MPI.Finalized()
error("MPI is already finalized!")
else
#doing nothing
end
end

return instance
end

end
function freeMPIWrapper(instance::MPIWrapper)
if isdefined(:MPI) && MPI.Initialized() && !MPI.Finalized()
MPI.Finalize()
end
end

const mpiWrapper = MPIWrapper();
function freeMPIWrapper(instance::MPIWrapper)
if isdefined(:MPI) && MPI.Initialized() && !MPI.Finalized()
MPI.Finalize()
end
end

const mpiWrapper = MPIWrapper();

type StructureData
probability::Vector{Float64}
children::Dict{Int,JuMP.Model}
parent
num_scen::Int
othermap::Dict{JuMP.Variable,JuMP.Variable}
mpiWrapper::MPIWrapper
type StructureData
probability::Vector{Float64}
children::Dict{Int,JuMP.Model}
parent
num_scen::Int
othermap::Dict{JuMP.Variable,JuMP.Variable}
mpiWrapper::MPIWrapper
end
else
type StructureData
probability::Vector{Float64}
children::Dict{Int,JuMP.Model}
parent
num_scen::Int
othermap::Dict{JuMP.Variable,JuMP.Variable}
end
end

default_probability(m::JuMP.Model) = 1 / num_scenarios(m)
default_probability(::Void) = 1.0

Expand All @@ -71,8 +86,12 @@ function StructuredModel(;solver=JuMP.UnsetSolver(), parent=nothing, same_childr
m = JuMP.Model(solver=solver)
if parent === nothing
id = 0
mpiWrapper.init(comm)
JuMP.setsolvehook(m,StructJuMPSolverInterface.sj_solve)
if isdefined(:MPI)
mpiWrapper.init(comm)
end
if isdefined(:StructJuMPSolverInterface)
JuMP.setsolvehook(m,StructJuMPSolverInterface.sj_solve)
end
else
@assert id != 0
stoch = getStructure(parent)
Expand All @@ -90,15 +109,19 @@ function StructuredModel(;solver=JuMP.UnsetSolver(), parent=nothing, same_childr
probability = Float64[]
children = Dict{Int, JuMP.Model}()
end
m.ext[:Stochastic] = StructureData(probability, children, parent, num_scenarios, Dict{JuMP.Variable,JuMP.Variable}(), mpiWrapper)
if isdefined(:MPI)
m.ext[:Stochastic] = StructureData(probability, children, parent, num_scenarios, Dict{JuMP.Variable,JuMP.Variable}(), mpiWrapper)
else
m.ext[:Stochastic] = StructureData(probability, children, parent, num_scenarios, Dict{JuMP.Variable,JuMP.Variable}())
end
m
end

# -------------
# Get functions
# -------------

getStructure(m::JuMP.Model) = m.ext[:Stochastic]::StructureData
getStructure(m::JuMP.Model) = m.ext[:Stochastic]::StructureData
getparent(m::JuMP.Model) = getStructure(m).parent
getchildren(m::JuMP.Model) = getStructure(m).children::Dict{Int,JuMP.Model}
getprobability(m::JuMP.Model) = getStructure(m).probability::Vector{Float64}
Expand Down

0 comments on commit b34cd38

Please sign in to comment.