Skip to content

Commit

Permalink
Add benchmark against linopy and cvxpy
Browse files Browse the repository at this point in the history
  • Loading branch information
metab0t committed Jun 10, 2024
1 parent 2b7f928 commit c009cfc
Show file tree
Hide file tree
Showing 2 changed files with 126 additions and 20 deletions.
88 changes: 88 additions & 0 deletions bench/bench_linopy_cvxpy.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,88 @@
import time
from numpy import arange
import numpy as np
import pandas as pd

from linopy import Model

import pyoptinterface as poi
from pyoptinterface import gurobi

import cvxpy


def create_linopy_model(N):
m = Model()
x = m.add_variables(coords=[arange(N), arange(N)])
y = m.add_variables(coords=[arange(N), arange(N)])
m.add_constraints(x - y >= arange(N))
m.add_constraints(x + y >= 0)
m.add_objective((2 * x).sum() + y.sum())
return m


def create_cvxpy_model(N):
x = cvxpy.Variable((N, N))
y = cvxpy.Variable((N, N))
constraints = []
for i in range(N):
constraints.append(x[:, i] - y[:, i] >= np.arange(N))
constraints.append(x + y >= 0)
objective = cvxpy.Minimize(2 * cvxpy.sum(x) + cvxpy.sum(y))
return cvxpy.Problem(objective, constraints)


def create_poi_model(N):
m = gurobi.Model()
x = m.add_variables(range(N), range(N))
y = m.add_variables(range(N), range(N))
for i in range(N):
for j in range(N):
m.add_linear_constraint(x[i, j] - y[i, j], poi.Geq, i)
m.add_linear_constraint(x[i, j] + y[i, j], poi.Geq, 0)
expr = poi.ExprBuilder()
poi.quicksum_(expr, x, lambda x: 2 * x)
poi.quicksum_(expr, y)
m.set_objective(expr)
return m


def bench(N):
results = {}

t0 = time.time()
model = create_poi_model(N)
model.optimize()
t1 = time.time()
results["n_variables"] = 2 * N * N
results["poi"] = t1 - t0

t0 = time.time()
model = create_linopy_model(N)
model.solve(solver_name="gurobi", io_api="direct")
t1 = time.time()
results["linopy"] = t1 - t0

t0 = time.time()
model = create_cvxpy_model(N)
model.solve(solver=cvxpy.GUROBI)
t1 = time.time()
results["cvxpy"] = t1 - t0

return results


def main():
Ns = range(100, 501, 100)
results = []
for N in Ns:
results.append(bench(N))
# create a DataFrame
df = pd.DataFrame(results, index=Ns)

# show result
print(df)


if __name__ == "__main__":
main()
58 changes: 38 additions & 20 deletions docs/source/benchmark.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,32 +8,50 @@ All code to run the benchmarks is available at [https://github.com/metab0t/PyOpt
:widths: auto
:align: center

| Model | Variables | C++ | PyOptInterface | JuMP | gurobipy | Pyomo |
|----------|-----------|-----|----------------|------|----------|-------|
| fac-25 | 67651 | 0.2 | 0.2 | 0.2 | 1.2 | 4.1 |
| fac-50 | 520301 | 0.8 | 1.2 | 1.8 | 9.7 | 32.7 |
| fac-75 | 1732951 | 2.7 | 4.1 | 6.6 | 32.5 | 119.3 |
| fac-100 | 4080601 | 6.3 | 10.0 | 17.8 | 79.1 | 286.3 |
| lqcp-500 | 251501 | 0.9 | 1.5 | 1.3 | 6.3 | 23.8 |
| lqcp-1000| 1003001 | 3.7 | 6.0 | 6.1 | 26.7 | 106.6 |
| lqcp-1500| 2254501 | 8.3 | 14.0 | 17.7 | 61.8 | 234.0 |
| lqcp-2000| 4006001 | 14.5| 24.9 | 38.3 | 106.9 | 444.1 |
| Model | Variables | C++ | PyOptInterface | JuMP | gurobipy | Pyomo |
| --------- | --------- | ---- | -------------- | ---- | -------- | ----- |
| fac-25 | 67651 | 0.2 | 0.2 | 0.2 | 1.2 | 4.1 |
| fac-50 | 520301 | 0.8 | 1.2 | 1.8 | 9.7 | 32.7 |
| fac-75 | 1732951 | 2.7 | 4.1 | 6.6 | 32.5 | 119.3 |
| fac-100 | 4080601 | 6.3 | 10.0 | 17.8 | 79.1 | 286.3 |
| lqcp-500 | 251501 | 0.9 | 1.5 | 1.3 | 6.3 | 23.8 |
| lqcp-1000 | 1003001 | 3.7 | 6.0 | 6.1 | 26.7 | 106.6 |
| lqcp-1500 | 2254501 | 8.3 | 14.0 | 17.7 | 61.8 | 234.0 |
| lqcp-2000 | 4006001 | 14.5 | 24.9 | 38.3 | 106.9 | 444.1 |

:::

:::{table} Time (second) to generate model and pass it to COPT optimizer.
:widths: auto
:align: center

| Model | Variables | C++ | PyOptInterface | JuMP | coptpy | Pyomo |
|----------|-----------|-----|----------------|------|--------|-------|
| fac-25 | 67651 | 0.3 | 0.2 | 0.3 | 0.6 | 4.1 |
| fac-50 | 520301 | 2.2 | 1.5 | 2.7 | 5.4 | 32.8 |
| fac-75 | 1732951 | 8.1 | 6.6 | 10.2 | 20.3 | 117.4 |
| fac-100 | 4080601 | 22.4| 23.4 | 30.3 | 58.0 | 284.0 |
| lqcp-500 | 251501 | 3.8 | 3.1 | 3.0 | 6.6 | 26.4 |
| lqcp-1000| 1003001 | 16.0| 15.5 | 13.9 | 28.1 | 112.1 |
| lqcp-1500| 2254501 | 37.6| 32.4 | 33.7 | 64.6 | 249.3 |
| lqcp-2000| 4006001 | 68.2| 60.3 | 66.2 | 118.4 | 502.4 |
| Model | Variables | C++ | PyOptInterface | JuMP | coptpy | Pyomo |
| --------- | --------- | ---- | -------------- | ---- | ------ | ----- |
| fac-25 | 67651 | 0.3 | 0.2 | 0.3 | 0.6 | 4.1 |
| fac-50 | 520301 | 2.2 | 1.5 | 2.7 | 5.4 | 32.8 |
| fac-75 | 1732951 | 8.1 | 6.6 | 10.2 | 20.3 | 117.4 |
| fac-100 | 4080601 | 22.4 | 23.4 | 30.3 | 58.0 | 284.0 |
| lqcp-500 | 251501 | 3.8 | 3.1 | 3.0 | 6.6 | 26.4 |
| lqcp-1000 | 1003001 | 16.0 | 15.5 | 13.9 | 28.1 | 112.1 |
| lqcp-1500 | 2254501 | 37.6 | 32.4 | 33.7 | 64.6 | 249.3 |
| lqcp-2000 | 4006001 | 68.2 | 60.3 | 66.2 | 118.4 | 502.4 |

:::

Recently, there are a lot of requests to test the performance of PyOptInterface compared with [linopy]([https://gith](https://github.com/PyPSA/linopy) and [cvxpy](https://github.com/cvxpy/cvxpy), so we prepare a [benchmark](https://github.com/metab0t/PyOptInterface/blob/master/bench/bench_linopy_cvxpy.py).

This is the result of benchmark, where the performance of PyOptInterface exceeds linopy and cvxpy significantly.

:::{table} Time (second) to generate and solve a linear programming model with Gurobi optimizer.
:widths: auto
:align: center

| N | Variables | PyOptInterface | linopy | cvxpy |
| --- | --------- | -------------- | -------- | --------- |
| 100 | 20000 | 0.112849 | 0.422408 | 0.373407 |
| 200 | 80000 | 0.294830 | 1.118702 | 1.575949 |
| 300 | 180000 | 0.710237 | 2.462809 | 4.038862 |
| 400 | 320000 | 1.256276 | 4.535225 | 8.687895 |
| 500 | 500000 | 2.189127 | 8.243707 | 18.941519 |

:::

0 comments on commit c009cfc

Please sign in to comment.