SpineOpt major_upgrade_1 migration reference

Scope and interpretation

This document records the migrations declared and performed by major_upgrade_1 in the supplied Julia script. It distinguishes simple renaming (same class/dimensions, parameter or class/list name changes) from restructuring (dimension reordering, moving parameters to new classes, superclass creation, and changes to value-list semantics). It is based on the script, not on a comparison of a particular database before and after migration.

Migration order matters: the script creates superclasses/subclasses, reorders classes, scales selected values, renames parameters/classes, moves multidimensional parameters, removes obsolete classes/parameters, renames lists and list values, merges variable-type lists, moves node physics settings to grid physics, then checks values against the latest template lists.

1. Simple renaming

1.1 Parameter renames

Unless noted, each mapping renames a parameter definition and its values in the same entity class. If the target parameter already exists, the script merges the old definition into it; migrated attributes and values take precedence, while other existing values are retained. Values in the same alternative are overwritten by migrated values.

Class unit

Class node

Class connection

Class commodity

Class investment_group

Class model

Class node__node

Class node__user_constraint

Class unit__to_node

Class node__to_unit

Class unit__user_constraint

Class unit_flow__user_constraint

Class connection__from_node

Class connection__to_node

Class connection__from_node__user_constraint

Class connection__to_node__user_constraint

Class connection__user_constraint

Class temporal_block

1.2 Class rename

1.3 Parameter value-list renames

1.4 Value renames inside lists

grid_physics_list (after commodity_physics_list is renamed):

balance_type_list:

Variable-type list migration: unit_online_variable_type_list becomes variable_type_list, with values renamed as follows:

That list migration also consolidates investment-variable type handling for unit, connection, and node storage investments; see Variable-type restructuring.

2. Restructuring

2.1 Superclasses and subclasses

The migration ensures user_constraint and investment_group classes exist (creating empty classes if missing), then ensures the following classes exist:

Both unit__to_node and node__to_unit are made subclasses of unit_flow.

2.2 Multidimensional class dimension reordering / migration

The script creates destination classes, copies parameter definitions and values, and also copies entity alternatives and group memberships. The dimension mappings below specify the source dimension order used to construct the destination entity names.

Notes:

• unit__from_node is converted to node__to_unit by swapping dimensions.
• unit__to_node_ is converted to unit__to_node; the trailing underscore is an intentional staging step so unit__to_node can be recreated as a subclass of unit_flow.
• unit__from_node__investment_group and unit__from_node__user_constraint are converted to their unit_flow__... classes with the unit/node dimensions reordered.
• The unit__to_node__investment_group and unit__to_node__user_constraint classes are also migrated to the same destination classes. These source classes do not print errors for missing parameter definitions during this operation (print_errors=false).
• Where a destination entity or parameter definition already exists, the migration generally preserves it but lets migrated definitions/values take precedence.

2.3 Parameters moved from unit__node__node to unit_flow__unit_flow

The old unit__node__node class models two node dimensions associated with a unit. The migration maps its parameter values into unit_flow__unit_flow, whose dimensions are two unit-flow entities. The old parameter definitions are removed after each move, and the old class is removed later.

Flow-ratio limits

The four direction suffixes (out_in, in_out, in_in, out_out) map to dimension patterns [1, 2, 3, 1], [2, 1, 1, 3], [2, 1, 3, 1], and [1, 2, 1, 3], respectively. For each direction pattern:

• fix_ratio_* becomes flow_ratio_equality_coefficient.
• min_ratio_* becomes flow_ratio_greater_than_coefficient.
• max_ratio_* becomes flow_ratio_less_than_coefficient.

Unit-online coefficient limits

The same four direction patterns apply. fix_units_on_coefficient_*, min_units_on_coefficient_*, and max_units_on_coefficient_* become, respectively, flow_ratio_equality_online_coefficient, flow_ratio_greater_than_online_coefficient, and flow_ratio_less_than_online_coefficient.

Startup flow

The script contains three additional unit_start_flow mappings in a commented-out block; they are not executed. The comment says enabling the full migration could result in duplicate values.

2.4 Flow parameter renames on flow-direction classes

These parameter renames apply to both unit__to_node and node__to_unit, unless noted.

2.5 Variable-type restructuring

The old unit_online_variable_type_list is renamed to variable_type_list, with its values normalised to binary, integer, linear, and none. The script then creates/uses investment_variable_type2 as an intermediate parameter linked to variable_type_list for:

• unit.investment_count_max_cumulative / unit.investment_variable_type
• connection.investment_count_max_cumulative / connection.investment_variable_type
• node.storage_investment_count_max_cumulative / node.storage_investment_variable_type

If a cumulative investment-count value is nothing, the new type is none; otherwise the class-specific legacy default is used (unit: linear; connection and node storage: integer). An explicitly set old investment-variable type overrides that default: values containing integer become integer, otherwise they become linear. The script also merges legacy variable-type lists into the shared list and updates investment parameter definitions. The detailed list names and temporary definitions are implementation details in the script; the final shared list is variable_type_list.

2.6 Move node physics flags into grid physics

The script migrates boolean node parameters into values of grid_physics_list, creating corresponding grid and node__grid entities as needed:

The new list values are inserted at indices 3 and 4, respectively. This is a semantic move, not merely a parameter rename.

2.7 Connection-flow parameters and user constraints

Parameters on both connection__from_node and connection__to_node are renamed in place as follows:

On connection__from_node__user_constraint and connection__to_node__user_constraint, connection_flow_coefficient becomes coefficient_for_connection_flow.

2.8 Additional parameter renames on multidimensional classes

2.9 Scaling and unit conversion

node.storage_fom_cost is scaled before renaming from a cost per model duration unit to an annual cost; it is subsequently renamed to storage_fixed_annual_cost. The factor is 365 days expressed in the model's duration unit (hours or minutes). If models use mixed duration units, the script defaults to hourly conversion and logs a warning. Unknown or missing duration units are treated as hours with a warning. The parameter default value is scaled as well as all stored values.

3. Removals

Classes removed

• unit__commodity
• unit__node__node (after its listed parameters are migrated)

Parameter removed

• user_constraint.include_in_non_representative_periods

4. Important migration behaviour and caveats

• Summation exception: unit.units_unavailable is merged into unit.out_of_service_count_fix using sum; values are summed into the target rather than simply overwriting it.
• Existing target definitions: for ordinary parameter renames, if the new name already exists, the old definition's attributes/defaults/value list take precedence and old values overwrite same-alternative target values; other target values are retained.
• Dimension migration copies more than values: parameter definitions, entity records, parameter values, entity alternatives, and entity group memberships are copied/re-mapped. Migrated values overwrite existing values at matching destination keys.
• Potential database inconsistencies: the script explicitly warns that some older databases may have unit__node__node entities without matching unit__to_node or unit__from_node entities. It recommends fixing those entities manually based on the migration error messages and rerunning.
• Template list validation: at the end, values are checked against the latest template lists. Values not in a target list may be dropped during list changes, with warnings. Defaults not valid in the new list can also be dropped.
• Best-effort operations: some class/entity/list operations catch errors and log warnings rather than aborting the entire migration. Review migration logs, especially around class conversion and value-list changes.

5. Summary by migration type

Generated from the supplied major_upgrade_1.jl source. This is a documentation of the migration logic declared in that file; it does not claim that every operation succeeds for every database.