Python (ixmp package)

The ix modeling platform application progamming interface (API) is organized around three classes:

Platform(*args[, name, backend]) Instance of the modeling platform.
TimeSeries(mp, model, scenario[, version, …]) Collection of data in time series format.
Scenario(mp, model, scenario[, version, …]) Collection of model-related data.

Platform

class ixmp.Platform(*args, name=None, backend=None, **backend_args)

Instance of the modeling platform.

A Platform connects two key components:

  1. A back end for storing data such as model inputs and outputs.
  2. One or more model(s); codes in Python or other languages or frameworks that run, via Scenario.solve(), on the data stored in the Platform.

The Platform parameters control these components. TimeSeries and Scenario objects tied to a single Platform; to move data between platforms, see Scenario.clone().

Parameters:
  • backend ('jdbc') – Storage backend type. ‘jdbc’ corresponds to the built-in JDBCBackend; see BACKENDS.
  • backend_args – Keyword arguments to specific to the backend. The “Other Parameters” shown below are specific to JDBCBackend.
Other Parameters:
 

  • dbtype (‘HSQLDB’, optional) – Database type to use. If None, a remote database is accessed. If ‘HSQLDB’, a local database is created and used at the path given by dbprops.

  • dbprops (path-like, optional) – If dbtype is None, the name of a database properties file (default: default.properties) in the properties file directory (see Config) or a path to a properties file.

    If dbtype is ‘HSQLDB’, the path of a local database, (default: ``$HOME/.local/ixmp/localdb/default`) or name of a database file in the local database directory (default: $HOME/.local/ixmp/localdb/).

  • jvmargs (str, optional) – Java Virtual Machine arguments. See start_jvm().

Platforms have the following methods:

add_region(region, hierarchy[, parent]) Define a region including a hierarchy level and a ‘parent’ region.
add_region_synonym(region, mapped_to) Define a synonym for a region.
add_unit(unit[, comment]) Define a unit.
check_access(user, models[, access]) Check access to specific models.
regions() Return all regions defined for the IAMC-style timeseries format including known synonyms.
scenario_list([default, model, scen]) Return information about TimeSeries and Scenarios on the Platform.
set_log_level(level) Set global logger level.
Scenario(model, scen, version=None, scheme=None, annotation=None, cache=False)

Initialize a new Scenario.

Deprecated since version 1.1.0: Instead, use:

>>> mp = ixmp.Platform(…)
>>> ixmp.Scenario(mp, …)
add_region(region, hierarchy, parent='World')

Define a region including a hierarchy level and a ‘parent’ region.

Tip

On a Platform backed by a shared database, a region may already exist with a different spelling. Use regions() first to check, and consider calling add_region_synonym() instead.

Parameters:
  • region (str) – Name of the region.
  • parent (str, default 'World') – Assign a ‘parent’ region.
  • hierarchy (str) – Hierarchy level of the region (e.g., country, R11, basin)
add_region_synonym(region, mapped_to)

Define a synonym for a region.

When adding timeseries data using the synonym in the region column, it will be converted to mapped_to.

Parameters:
  • region (str) – Name of the region synonym.
  • mapped_to (str) – Name of the region to which the synonym should be mapped.
add_unit(unit, comment='None')

Define a unit.

Parameters:
  • unit (str) – Name of the unit.
  • comment (str, optional) – Annotation describing the unit or why it was added. The current database user and timestamp are appended automatically.
check_access(user, models, access='view')

Check access to specific models.

Parameters:
  • user (str) – Registered user name
  • models (str or list of str) – Model(s) name
  • access (str, optional) – Access type - view or edit
Returns:
Return type:

bool or dict of bool
regions()

Return all regions defined for the IAMC-style timeseries format including known synonyms.

Returns:
Return type:pandas.DataFrame
scenario_list(default=True, model=None, scen=None)

Return information about TimeSeries and Scenarios on the Platform.

Parameters:
  • default (bool, optional) – Return only the default version of each TimeSeries/Scenario (see TimeSeries.set_as_default()). Any (model, scenario) without a default version is omitted. If False, return all versions.
  • model (str, optional) – A model name. If given, only return information for model.
  • scen (str, optional) – A scenario name. If given, only return information for scen.
Returns:

Scenario information, with the columns:

  • model, scenario, version, and scheme—Scenario identifiers; see Scenario.
  • is_default—True if the version is the default version for the (model, scenario).
  • is_locked—True if the Scenario has been locked for use.
  • cre_user and cre_date—database user that created the Scenario, and creation time.
  • upd_user and upd_date—user and time for last modification of the Scenario.
  • lock_user and lock_date—user that locked the Scenario and lock time.
  • annotation: description of the Scenario or changelog.
Return type:

pandas.DataFrame
set_log_level(level)

Set global logger level.

Parameters:level (str) – set the logger level if specified, see https://docs.python.org/3/library/logging.html#logging-levels

TimeSeries

class ixmp.TimeSeries(mp, model, scenario, version=None, annotation=None)

Collection of data in time series format.

TimeSeries is the parent/super-class of Scenario.

Parameters:
  • mp (Platform) – ixmp instance in which to store data.
  • model (str) – Model name.
  • scenario (str) – Scenario name.
  • version (int or str, optional) – If omitted and a default version of the (model, scenario) has been designated (see set_as_default()), load that version. If int, load a specific version. If 'new', create a new TimeSeries.
  • annotation (str, optional) – A short annotation/comment used when version='new'.

A TimeSeries is uniquely identified on its Platform by three values:

  1. model: the name of a model used to perform calculations between input and output data.

  2. scenario: the name of a specific, coherent description of the real- world system being modeled. Any model may be used to represent mutiple alternate, or ‘counter-factual’, scenarios.

  3. version: an integer identifying a specific iteration of a (model, scenario). A new version is created by:

    • Instantiating a new TimeSeries with the same model and scenario as an existing TimeSeries.
    • Calling Scenario.clone().

    Optionally, one version may be set as a default version. See set_as_default().

TimeSeries objects have the following methods:

add_geodata(df) Add geodata (layers) to the TimeSeries.
add_timeseries(df[, meta]) Add data to the TimeSeries.
check_out([timeseries_only]) Check out the TimeSeries for modification.
commit(comment) Commit all changed data to the database.
discard_changes() Discard all changes and reload from the database.
get_geodata() Fetch geodata and return it as dataframe.
is_default() Return True if the version is the default version.
last_update() get the timestamp of the last update/edit of this TimeSeries
preload_timeseries() Preload timeseries data to in-memory cache.
remove_geodata(df) Remove geodata from the TimeSeries instance.
remove_timeseries(df) Remove timeseries data from the TimeSeries instance.
run_id() get the run id of this TimeSeries
set_as_default() Set the current version as the default.
timeseries([region, variable, unit, year, iamc]) Retrieve TimeSeries data.
add_geodata(df)

Add geodata (layers) to the TimeSeries.

Parameters:df (pandas.DataFrame) –

Data to add. df must have the following columns:

  • region
  • variable
  • time
  • unit
  • year
  • value
  • meta
add_timeseries(df, meta=False)

Add data to the TimeSeries.

Parameters:
  • df (pandas.DataFrame) –

    Data to add. df must have the following columns:

    • region or node
    • variable
    • unit

    Additional column names may be either of:

    • year and value—long, or ‘tabular’, format.
    • one or more specific years—wide, or ‘IAMC’ format.
  • meta (bool, optional) – If True, store df as metadata. Metadata is treated specially when Scenario.clone() is called for Scenarios created with scheme='MESSAGE'.
check_out(timeseries_only=False)

Check out the TimeSeries for modification.

commit(comment)

Commit all changed data to the database.

If the TimeSeries was newly created (with version='new'), version is updated with a new version number assigned by the backend. Otherwise, commit() does not change the version.

Parameters:comment (str) – Description of the changes being committed.
discard_changes()

Discard all changes and reload from the database.

get_geodata()

Fetch geodata and return it as dataframe.

Returns:Specified data.
Return type:pandas.DataFrame
is_default()

Return True if the version is the default version.

last_update()

get the timestamp of the last update/edit of this TimeSeries

model = None

Name of the model associated with the TimeSeries

preload_timeseries()

Preload timeseries data to in-memory cache. Useful for bulk updates.

remove_geodata(df)

Remove geodata from the TimeSeries instance.

Parameters:df (pandas.DataFrame) –

Data to remove. df must have the following columns:

  • region
  • variable
  • unit
  • time
  • year
remove_timeseries(df)

Remove timeseries data from the TimeSeries instance.

Parameters:df (pandas.DataFrame) –

Data to remove. df must have the following columns:

  • region or node
  • variable
  • unit
  • year
run_id()

get the run id of this TimeSeries

scenario = None

Name of the scenario associated with the TimeSeries

set_as_default()

Set the current version as the default.

timeseries(region=None, variable=None, unit=None, year=None, iamc=False)

Retrieve TimeSeries data.

Parameters:
  • iamc (bool, default: False) – Return data in wide/’IAMC’ format. If False, return data in long/’tabular’ format; see add_timeseries().
  • region (str or list of strings) – Regions to include in returned data.
  • variable (str or list of strings) – Variables to include in returned data.
  • unit (str or list of strings) – Units to include in returned data.
  • year (str, int or list of strings or integers) – Years to include in returned data.
Returns:

Specified data.
Return type:

pandas.DataFrame
version = None

Version of the TimeSeries. Immutable for a specific instance.

Scenario

class ixmp.Scenario(mp, model, scenario, version=None, scheme=None, annotation=None, cache=False)

Bases: ixmp.core.TimeSeries

Collection of model-related data.

See TimeSeries for the meaning of parameters mp, model, scenario, version, and annotation.

Parameters:
  • scheme (str, optional) – Use an explicit scheme for initializing a new scenario.
  • cache (bool, optional) – Store data in memory and return cached values instead of repeatedly querying the backend.

A Scenario is a TimeSeries associated with a particular model that can be run on the current Platform by calling solve(). The Scenario also stores the output, or ‘solution’ of a model run; this includes the ‘level’ and ‘marginal’ values of GAMS equations and variables.

Data in a Scenario are closely related to different types in the GAMS data model:

  • A set is a named collection of labels. See init_set(), add_set(), and set(). There are two types of sets:
    1. Sets that are lists of labels.
    2. Sets that are ‘indexed’ by one or more other set(s). For this type of set, each member is an ordered tuple of the labels in the index sets.
  • A scalar is a named, single, numerical value. See init_scalar(), change_scalar(), and scalar().
  • Parameters, variables, and equations are multi-dimensional arrays of values that are indexed by one or more sets (i.e. with dimension 1 or greater). The Scenario methods for handling these types are very similar; they mainly differ in how they are used within GAMS models registered with ixmp:
    • Parameters are generic data that can be defined before a model run. They may be altered by the model solution. See init_par(), remove_par(), par_list(), add_par(), and par().
    • Variables are calculated during or after a model run by GAMS code, so they cannot be modified by a Scenario. See init_var(), var_list(), and var().
    • Equations describe fundamental relationships between other types (parameters, variables, and scalars) in a model. They are defined in GAMS code, so cannot be modified by a Scenario. See init_equ(), equ_list(), and equ().
add_par(name[, key_or_data, value, unit, …]) Set the values of a parameter.
add_set(name, key[, comment]) Add elements to an existing set.
change_scalar(name, val, unit[, comment]) Set the value and unit of a scalar.
clear_cache([name, ix_type]) clear the Python cache of item elements
clone([model, scenario, annotation, …]) Clone the current scenario and return the clone.
equ(name[, filters]) return a dataframe of (filtered) elements for a specific equation
equ_list() List all defined equations.
get_meta([name]) get scenario metadata
has_equ(name) check whether the scenario has an equation with that name
has_par(name) check whether the scenario has a parameter with that name
has_set(name) Check whether the scenario has a set name.
has_solution() Return True if the Scenario has been solved.
has_var(name) check whether the scenario has a variable with that name
idx_names(name) return the list of index names for an item (set, par, var, equ)
idx_sets(name) Return the list of index sets for an item (set, par, var, equ)
init_equ(name[, idx_sets, idx_names]) Initialize a new equation.
init_par(name, idx_sets[, idx_names]) Initialize a new parameter.
init_scalar(name, val, unit[, comment]) Initialize a new scalar.
init_set(name[, idx_sets, idx_names]) Initialize a new set.
init_var(name[, idx_sets, idx_names]) initialize a new variable in the scenario
load_scenario_data() Load all Scenario data into memory.
par(name[, filters]) return a dataframe of (filtered) elements for a specific parameter
par_list() List all defined parameters.
remove_par(name[, key]) Remove parameter values or an entire parameter.
remove_set(name[, key]) delete a set from the scenario or remove an element from a set (if key is specified)
remove_solution([first_model_year]) Remove the solution from the scenario
scalar(name) Return the value and unit of a scalar.
set(name[, filters]) Return the (filtered) elements of a set.
set_list() List all defined sets.
set_meta(name, value) set scenario metadata
solve([model, callback, cb_kwargs]) Solve the model and store output.
var(name[, filters]) return a dataframe of (filtered) elements for a specific variable
var_list() List all defined variables.
add_par(name, key_or_data=None, value=None, unit=None, comment=None, key=None, val=None)

Set the values of a parameter.

Parameters:
  • name (str) – Name of the parameter.
  • key_or_data (str or iterable of str or range or dict or pandas.DataFrame) – Element(s) to be added.
  • value (numeric or iterable of numeric, optional) – Values.
  • unit (str or iterable of str, optional) – Unit symbols.
  • comment (str or iterable of str, optional) – Comment(s) for the added values.
add_set(name, key, comment=None)

Add elements to an existing set.

Parameters:
  • name (str) – Name of the set.
  • key (str or iterable of str or dict or pandas.DataFrame) – Element(s) to be added. If name exists, the elements are appended to existing elements.
  • comment (str or iterable of str, optional) – Comment describing the element(s). If given, there must be the same number of comments as elements.
Raises:
change_scalar(name, val, unit, comment=None)

Set the value and unit of a scalar.

Parameters:
  • name (str) – Name of the scalar.
  • val (number) – New value of the scalar.
  • unit (str) – New unit of the scalar.
  • comment (str, optional) – Description of the change.
clear_cache(name=None, ix_type=None)

clear the Python cache of item elements

Parameters:
  • name (str, optional) – item name (None clears entire Python cache)
  • ix_type (str, optional) – type of item (if provided, cache clearing is faster)
clone(model=None, scenario=None, annotation=None, keep_solution=True, shift_first_model_year=None, platform=None, **kwargs)

Clone the current scenario and return the clone.

If the (model, scenario) given already exist on the Platform, the version for the cloned Scenario follows the last existing version. Otherwise, the version for the cloned Scenario is 1.

Note

clone() does not set or alter default versions. This means that a clone to new (model, scenario) names has no default version, and will not be returned by Platform.scenario_list() unless default=False is given.

Parameters:
  • model (str, optional) – New model name. If not given, use the existing model name.
  • scenario (str, optional) – New scenario name. If not given, use the existing scenario name.
  • annotation (str, optional) – Explanatory comment for the clone commit message to the database.
  • keep_solution (bool, optional) – If True, include all timeseries data and the solution (vars and equs) from the source scenario in the clone. If False, only include timeseries data marked meta=True (see TimeSeries.add_timeseries()).
  • first_model_year (int, optional) – If given, all timeseries data in the Scenario is omitted from the clone for years from first_model_year onwards. Timeseries data with the meta flag (see TimeSeries.add_timeseries()) are cloned for all years.
  • platform (Platform, optional) – Platform to clone to (default: current platform)
equ(name, filters=None, **kwargs)

return a dataframe of (filtered) elements for a specific equation

Parameters:
  • name (str) – name of the equation
  • filters (dict) – index names mapped list of index set elements
equ_list()

List all defined equations.

get_meta(name=None)

get scenario metadata

Parameters:name (str, optional) – metadata attribute name
has_equ(name)

check whether the scenario has an equation with that name

has_par(name)

check whether the scenario has a parameter with that name

has_set(name)

Check whether the scenario has a set name.

has_solution()

Return True if the Scenario has been solved.

If has_solution() == True, model solution data exists in the db.

has_var(name)

check whether the scenario has a variable with that name

idx_names(name)

return the list of index names for an item (set, par, var, equ)

Parameters:name (str) – name of the item
idx_sets(name)

Return the list of index sets for an item (set, par, var, equ)

Parameters:name (str) – name of the item
init_equ(name, idx_sets=None, idx_names=None)

Initialize a new equation.

Parameters:
  • name (str) – name of the item
  • idx_sets (list of str) – index set list
  • idx_names (list of str, optional) – index name list
init_par(name, idx_sets, idx_names=None)

Initialize a new parameter.

Parameters:
  • name (str) – Name of the parameter.
  • idx_sets (list of str) – Names of sets that index this parameter.
  • idx_names (list of str, optional) – Names of the dimensions indexed by idx_sets.
init_scalar(name, val, unit, comment=None)

Initialize a new scalar.

Parameters:
  • name (str) – Name of the scalar
  • val (number) – Initial value of the scalar.
  • unit (str) – Unit of the scalar.
  • comment (str, optional) – Description of the scalar.
init_set(name, idx_sets=None, idx_names=None)

Initialize a new set.

Parameters:
  • name (str) – Name of the set.
  • idx_sets (list of str, optional) – Names of other sets that index this set.
  • idx_names (list of str, optional) – Names of the dimensions indexed by idx_sets.
Raises:
init_var(name, idx_sets=None, idx_names=None)

initialize a new variable in the scenario

Parameters:
  • name (str) – name of the item
  • idx_sets (list of str) – index set list
  • idx_names (list of str, optional) – index name list
load_scenario_data()

Load all Scenario data into memory.

Raises:ValueError – If the Scenario was instantiated with cache=False.
par(name, filters=None, **kwargs)

return a dataframe of (filtered) elements for a specific parameter

Parameters:
  • name (str) – name of the parameter
  • filters (dict) – index names mapped list of index set elements
par_list()

List all defined parameters.

remove_par(name, key=None)

Remove parameter values or an entire parameter.

Parameters:
  • name (str) – Name of the parameter.
  • key (dataframe or key list or concatenated string, optional) – elements to be removed
remove_set(name, key=None)

delete a set from the scenario or remove an element from a set (if key is specified)

Parameters:
  • name (str) – name of the set
  • key (dataframe or key list or concatenated string) – elements to be removed
remove_solution(first_model_year=None)

Remove the solution from the scenario

This function removes the solution (variables and equations) and timeseries data marked as meta=False from the scenario (see TimeSeries.add_timeseries()).

Parameters:first_model_year (int, optional) – If given, timeseries data marked as meta=False is removed only for years from first_model_year onwards.
Raises:ValueError – If Scenario has no solution or if first_model_year is not int.
scalar(name)

Return the value and unit of a scalar.

Parameters:name (str) – Name of the scalar.
Returns:{‘value’
Return type:value, ‘unit’: unit}
scheme = None

Scheme of the Scenario.

set(name, filters=None, **kwargs)

Return the (filtered) elements of a set.

Parameters:
  • name (str) – Name of the set.
  • filters (dict) – Mapping of dimension_name → elements, where dimension_name is one of the idx_names given when the set was initialized (see init_set()), and elements is an iterable of labels to include in the return value.
Returns:
Return type:

pandas.DataFrame
set_list()

List all defined sets.

set_meta(name, value)

set scenario metadata

Parameters:
  • name (str) – metadata attribute name
  • value (str or number or bool) – metadata attribute value
solve(model=None, callback=None, cb_kwargs={}, **model_options)

Solve the model and store output.

ixmp ‘solves’ a model by invoking the run() method of a Model subclass—for instance, GAMSModel.run(). Depending on the underlying model code, different steps are taken; see each model class for details. In general:

  1. Data from the Scenario are written to a model input file.
  2. Code or an external program is invoked to perform calculations or optimizations, solving the model.
  3. Data representing the model outputs or solution are read from a model output file and stored in the Scenario.

If the optional argument callback is given, then additional steps are performed:

  1. Execute the callback with the Scenario as an argument. The Scenario has an iteration attribute that stores the number of times the underlying model has been solved (#2).
  2. If the callback returns False or similar, iterate by repeating from step #1. Otherwise, exit.
Parameters:
  • model (str) – model (e.g., MESSAGE) or GAMS file name (excluding ‘.gms’)
  • callback (callable, optional) – Method to execute arbitrary non-model code. Must accept a single argument: the Scenario. Must return a non-False value to indicate convergence.
  • cb_kwargs (dict, optional) – Keyword arguments to pass to callback.
  • model_options – Keyword arguments specific to the model. See GAMSModel.
Warns:

UserWarning – If callback is given and returns None. This may indicate that the user has forgotten a return statement, in which case the iteration will continue indefinitely.
Raises:

ValueError – If the Scenario has already been solved.
var(name, filters=None, **kwargs)

return a dataframe of (filtered) elements for a specific variable

Parameters:
  • name (str) – name of the variable
  • filters (dict) – index names mapped list of index set elements
var_list()

List all defined variables.

Configuration

When imported, ixmp reads configuration from the first file named config.json found in one of the following directories:

  1. The directory given by the environment variable IXMP_DATA, if defined,
  2. ${XDG_DATA_HOME}/ixmp, if the environment variable is defined,
  3. $HOME/.local/share/ixmp, or
  4. $HOME/.local/ixmp (deprecated; retained for compatibility with ixmp <= 1.1).

Tip

For most users, #2 or #3 is a sensible default; platform information for many local and remote databases can be stored in config.json and retrieved by name.

Advanced users wishing to use a project-specific config.json can set IXMP_DATA to the directory containing this file.

To manipulate the configuration file, use the platform command in the ixmp command-line interface:

# Add a platform named 'p1' backed by a local HSQL database
$ ixmp platform add p1 jdbc hsqldb /path/to/database/files

# Add a platform named 'p2' backed by a remote Oracle database
$ ixmp platform add p2 jdbc oracle \
       database.server.example.com:PORT:SCHEMA username password

# Make 'p2' the default Platform
$ ixmp platform add default p2

…or, use the methods of ixmp.config.

ixmp.config

An instance of Config.

class ixmp._config.Config(read=True)

Configuration for ixmp.

Config stores two kinds of data: simple keys with a single value, and structured Platform information.

ixmp has no built-in simple keys; however, it can store keys for other packages that build on ixmp, such as message_ix.

Parameters:read (bool) – Read config.json on startup.
add_platform(name, *args)

Add or overwrite information about a platform.

Parameters:
  • name (str) – New or existing platform name.
  • args – Positional arguments. If name is ‘default’, args must be a single string: the name of an existing configured Platform. Otherwise, the first of args specifies one of the BACKENDS, and the remaining args are specific to that backend.
clear()

Clear all configuration keys by setting empty or default values.

clear() also sets the default local platform:

{
  "platform": {
    "default": "local",
    "local": {
      "class": "jdbc",
      "driver": "hsqldb",
      "path": "~/.local/share/ixmp/localdb/default"
    },
}
get(key)

Return the value of a configuration key.

get_platform_info(name)

Return information on configured Platform name.

Parameters:name (str) – Existing platform. If name is ‘default’, then the information for the default platform is returned.
Returns:The ‘class’ key specifies one of the BACKENDS. Other keys vary by backend class.
Return type:dict
Raises:KeyError – If name is not configured as a platform.
path = None

Full-resolved path of the config.json file.

read()

Try to read configuration keys from file.

If successful, the attribute path is set to the path of the file.

register(name, type, default=None)

Register a new configuration key.

Parameters:
  • name (str) – Name of the new key; must not already exist.
  • type (object) – Type of the key’s value, such as str or pathlib.Path.
  • default (any, optional) – Default value for the key. If not supplied, the type is called to supply the default, value, e.g. str().
remove_platform(name)

Remove the configuration for platform name.

save()

Write configuration keys to file.

config.json is created in the first of the ixmp configuration directories that exists. Only non-null values are written.

set(name, value)

Set configuration key to value.

Testing utilities

testing.make_dantzig(solve=False)

Return ixmp.Scenario of Dantzig’s canning/transport problem.

Parameters:
  • mp (ixmp.Platform) – Platform on which to create the scenario.
  • solve (bool or os.PathLike) – If not False, then solve is interpreted as a path to a directory, and the model transport_ixmp.gms in the directory is run for the scenario.