diff --git a/.pre-commit-config.yaml b/.pre-commit-config.yaml index 267e31da..a69ddae7 100644 --- a/.pre-commit-config.yaml +++ b/.pre-commit-config.yaml @@ -1,12 +1,12 @@ repos: - repo: https://github.com/pre-commit/mirrors-mypy - rev: v1.8.0 + rev: v1.12.0 hooks: - id: mypy entry: bash -c "poetry run mypy ." language: system - repo: https://github.com/astral-sh/ruff-pre-commit - rev: v0.3.3 + rev: v0.6.9 hooks: - id: ruff - id: ruff-format diff --git a/ixmp4/data/backend/db.py b/ixmp4/data/backend/db.py index 67cecf40..25f2981c 100644 --- a/ixmp4/data/backend/db.py +++ b/ixmp4/data/backend/db.py @@ -43,7 +43,7 @@ @lru_cache() def cached_create_engine(dsn: str) -> Engine: - return create_engine(dsn, pool_pre_ping=True) + return create_engine(dsn, poolclass=NullPool) class IamcSubobject(BaseIamcSubobject): diff --git a/tutorial/transport/py_transport.ipynb b/tutorial/transport/py_transport.ipynb deleted file mode 100644 index abe96991..00000000 --- a/tutorial/transport/py_transport.ipynb +++ /dev/null @@ -1,469 +0,0 @@ -{ - "cells": [ - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "# Tutorial 1

Solve Dantzig's Transport Problem using the *ix modeling platform* (ixmp4)\n", - "\n", - "\n", - "\n", - "## Aim and scope of the tutorial\n", - "\n", - "This tutorial takes you through the steps to import the data for a very simple optimization model\n", - "and solve it using the **ixmp4**-GAMS interface.\n", - "\n", - "We use Dantzig's transport problem, which is also used as the standard GAMS tutorial.\n", - "This problem finds a least cost shipping schedule that meets requirements at markets and supplies at factories.\n", - "\n", - "If you are not familiar with GAMS, please take a minute to look at the [transport.gms](transport.gms) code.\n", - "\n", - "For reference of the transport problem, see:\n", - "> Dantzig, G B, Chapter 3.3. In Linear Programming and Extensions. \n", - "> Princeton University Press, Princeton, New Jersey, 1963.\n", - "\n", - "> This formulation is described in detail in: \n", - "> Rosenthal, R E, Chapter 2: A GAMS Tutorial. \n", - "> In GAMS: A User's Guide. The Scientific Press, Redwood City, California, 1988.\n", - "\n", - "> see http://www.gams.com/mccarl/trnsport.gms\n", - "\n", - "## Tutorial outline\n", - "\n", - "The steps in the tutorial are the following:\n", - "\n", - "0. Launch an **ixmp4.Platform** instance and initialize a new **ixmp4.Run**\n", - "0. Define the **sets and parameters** in the scenario and save the data to the platform\n", - "0. Initialize **variables and equations** to import the solution from GAMS\n", - "0. Call GAMS to **solve the scenario** (export to GAMS input gdx, execute, read solution from output gdx)\n", - "0. Display the **solution** (variables and equation)" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "## Launching the *Platform* and initializing a new *Run*\n", - "\n", - "A **Platform** is the connection to the database that holds all data and relevant additional information.\n", - "\n", - "A **Run** is an object that holds all relevant information for one quantification of a scenario. \n", - "A run is identified by a model name, a scenario name and a version number (assigned automatically)." - ] - }, - { - "cell_type": "code", - "execution_count": 1, - "metadata": {}, - "outputs": [], - "source": [ - "import pandas as pd\n", - "import ixmp4" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "You have to *register a new local database* before you can run the tutorial. \n", - "\n", - "Run the following in the command-line:\n", - "```\n", - "ixmp4 platforms add sqlite-test\n", - "```\n", - "\n", - "You can then check if the database was successfully created by running\n", - "```\n", - "ixmp4 platforms list\n", - "```\n", - "\n", - "After creating the database, you can connect to it via an **ixmp4.Platform** instance." - ] - }, - { - "cell_type": "code", - "execution_count": 2, - "metadata": {}, - "outputs": [], - "source": [ - "mp = ixmp4.Platform(\"sqlite-test\")" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "Now, we initialize a new **ixmp4.Run** in the database. This is done by using the argument *version=\"new\"*." - ] - }, - { - "cell_type": "code", - "execution_count": 3, - "metadata": {}, - "outputs": [], - "source": [ - "run = mp.runs.create(model=\"transport problem\", scenario=\"standard\")" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "## Defining the *IndexSets*\n", - "\n", - "An **IndexSet** defines a named list of elements. These IndexSets can be used for \"indexed assignment\" of parameters, variables and equations. \n", - "In database-lingo, a column of a parameter can be \"foreign-keyed\" onto an IndexSet.\n", - "\n", - "Below, we first show the data as they would be written in the GAMS tutorial ([transport.gms](transport.gms) in this folder). " - ] - }, - { - "cell_type": "raw", - "metadata": {}, - "source": [ - "Sets\n", - " i canning plants / seattle, san-diego /\n", - " j markets / new-york, chicago, topeka / ;" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "We now initialize these sets and assign the elements." - ] - }, - { - "cell_type": "code", - "execution_count": 4, - "metadata": {}, - "outputs": [], - "source": [ - "i = run.optimization.IndexSet(\"i\")" - ] - }, - { - "cell_type": "code", - "execution_count": 5, - "metadata": {}, - "outputs": [], - "source": [ - "i.add([\"seattle\", \"san-diego\"])" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "We can display the elements of **IndexSet i** as a Python list." - ] - }, - { - "cell_type": "code", - "execution_count": 6, - "metadata": {}, - "outputs": [ - { - "data": { - "text/plain": [ - "['seattle', 'san-diego']" - ] - }, - "execution_count": 6, - "metadata": {}, - "output_type": "execute_result" - } - ], - "source": [ - "i.elements" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "For simplicity, the steps of creating an **IndexSet** and assigning elements can be done in one line." - ] - }, - { - "cell_type": "code", - "execution_count": 7, - "metadata": {}, - "outputs": [], - "source": [ - "run.optimization.IndexSet(\"j\").add([\"new-york\", \"chicago\", \"topeka\"])" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "## Assigning the *Parameters*\n", - "\n", - "Next, we define the parameters *capacity* and *demand*. The parameters are assigned on the IndexSets *i* and *j*, respectively." - ] - }, - { - "cell_type": "raw", - "metadata": {}, - "source": [ - "Parameters\n", - " a(i) capacity of plant i in cases\n", - " / seattle 350\n", - " san-diego 600 /\n", - " b(j) demand at market j in cases\n", - " / new-york 325\n", - " chicago 300\n", - " topeka 275 / ;" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "# capacity of plant i in cases \n", - "# add parameter elements one-by-one (string and value) \n", - "scen.init_par(\"a\", idx_sets=\"i\")\n", - "scen.add_par(\"a\", \"seattle\", 350, \"cases\")\n", - "scen.add_par(\"a\", \"san-diego\", 600, \"cases\")\n", - "\n", - "# demand at market j in cases \n", - "# add parameter elements as dataframe (with index names) \n", - "scen.init_par(\"b\", idx_sets=\"j\")\n", - "b_data = [\n", - " {'j': \"new-york\", 'value': 325, 'unit': \"cases\"},\n", - " {'j': \"chicago\", 'value': 300, 'unit': \"cases\"},\n", - " {'j': \"topeka\", 'value': 275, 'unit': \"cases\"}\n", - "]\n", - "b = pd.DataFrame(b_data)\n", - "scen.add_par(\"b\", b)" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "scen.par('b')" - ] - }, - { - "cell_type": "raw", - "metadata": {}, - "source": [ - "Table d(i,j) distance in thousands of miles\n", - " new-york chicago topeka\n", - " seattle 2.5 1.7 1.8\n", - " san-diego 2.5 1.8 1.4 ;" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "# distance in thousands of miles \n", - "scen.init_par(\"d\", idx_sets=[\"i\", \"j\"])\n", - "# add more parameter elements as dataframe by index names \n", - "d_data = [\n", - " {'i': \"seattle\", 'j': \"new-york\", 'value': 2.5, 'unit': \"km\"},\n", - " {'i': \"seattle\", 'j': \"chicago\", 'value': 1.7, 'unit': \"km\"},\n", - " {'i': \"seattle\", 'j': \"topeka\", 'value': 1.8, 'unit': \"km\"},\n", - " {'i': \"san-diego\", 'j': \"new-york\", 'value': 2.5, 'unit': \"km\"},\n", - "]\n", - "d = pd.DataFrame(d_data)\n", - "scen.add_par(\"d\", d)\n", - "\n", - "# add other parameter elements as key list, value, unit\n", - "scen.add_par(\"d\", [\"san-diego\", \"chicago\"], 1.8, \"km\")\n", - "scen.add_par(\"d\", [\"san-diego\", \"topeka\"], 1.4, \"km\")" - ] - }, - { - "cell_type": "raw", - "metadata": {}, - "source": [ - "Scalar f freight in dollars per case per thousand miles /90/ ; " - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "# cost per case per 1000 miles \n", - "# initialize scalar with a value and a unit (and optionally a comment) \n", - "scen.init_scalar(\"f\", 90.0, \"USD/km\")" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "### Committing the scenario to the ixmp database instance" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "# commit new scenario to the database\n", - "# no changes can then be made to the scenario data until a check-out is performed\n", - "comment = \"importing Dantzig's transport problem for illustration\"\n", - "comment += \" and testing of the Python interface using a generic datastructure\" \n", - "scen.commit(comment) \n", - "\n", - "# set this new scenario as the default version for the model/scenario name\n", - "scen.set_as_default()" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "### Defining variables and equations in the scenario\n", - "\n", - "The levels and marginals of these variables and equations will be imported to the scenario when reading the gdx solution file." - ] - }, - { - "cell_type": "raw", - "metadata": {}, - "source": [ - "Variables\n", - " x(i,j) shipment quantities in cases\n", - " z total transportation costs in thousands of dollars ;\n", - " \n", - "Equations\n", - " cost define objective function\n", - " supply(i) observe supply limit at plant i\n", - " demand(j) satisfy demand at market j ;" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "# perform a check_out to make further changes\n", - "scen.check_out()\n", - "\n", - "# initialize the decision variables and equations\n", - "scen.init_var(\"z\", None, None)\n", - "scen.init_var(\"x\", idx_sets=[\"i\", \"j\"])\n", - "scen.init_equ(\"demand\", idx_sets=[\"j\"])\n", - "\n", - "# commit changes to the scenario (save changes in ixmp database instance)\n", - "change_comment = \"initialize the model variables and equations\"\n", - "scen.commit(change_comment)" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "### Solve the scenario\n", - "\n", - "The ``solve()`` function exports the scenario to a GAMS gdx file, executes GAMS, and then imports the solution from an output GAMS gdx file to the database.\n", - "\n", - "For the model equations and the GAMS workflow (reading the data from gdx, solving the model, writing the results to gdx), see ``transport_ixmp.gms``." - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "scen.solve(model='dantzig')" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "### Display and analyze the results" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "# display the objective value of the solution\n", - "scen.var(\"z\")" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "# display the quantities transported from canning plants to demand locations\n", - "scen.var(\"x\")" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "# display the quantities and marginals (=shadow prices) of the demand balance constraints\n", - "scen.equ(\"demand\")" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "### Close the database connection of the ix modeling platform\n", - "\n", - "Closing the database connection is recommended when working with the local file-based database, i.e., ``dbtype='HSQLDB'``.\n", - "This command closes the database files and removes temporary data. This is necessary so that other notebooks or ``ixmp`` instances can access the database file, or so that the database files can be copied to a different folder or drive." - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "# close the connection of the platform instance to the local ixmp database files\n", - "mp.close_db()" - ] - } - ], - "metadata": { - "anaconda-cloud": {}, - "kernelspec": { - "display_name": "Python 3 (ipykernel)", - "language": "python", - "name": "python3" - }, - "language_info": { - "codemirror_mode": { - "name": "ipython", - "version": 3 - }, - "file_extension": ".py", - "mimetype": "text/x-python", - "name": "python", - "nbconvert_exporter": "python", - "pygments_lexer": "ipython3", - "version": "3.10.0" - } - }, - "nbformat": 4, - "nbformat_minor": 1 -}