Orchestration Custom Functions🔗

Some advanced Custom Functions make use of algorithms where many simulations need to rerun multiple times. Furthermore, these algorithms might need the results from some simulations to feed into other simulations.

These kind of workflows and algorithms are implemented in Modelon Impact using the Orchestration feature in Custom Functions. For an example of this, see Calibration.

Note

This is an advanced topic. It is recommended to have gone through Creating Custom Functions first and have a good understanding of it.

Creating Orchestration Custom Functions🔗

A Custom Function can be made an Orchestrator by adding kind as ORCHESTRATOR in its signature. Once this is done the run method can define the argument case_manager which gives access to an API for performing the orchestration. Here is a minimal example of an Orchestration Custom Function that does nothing:

def signature():
    return {
        "name": "test-orchestrator",
        "kind": "ORCHESTRATOR",
    }

def run(case_manager):
    pass

Case Manager API🔗

The Case Manager uses and builds on objects from the Python Client and uses these as input arguments and as return type in many cases. The case manager has the methods:

create() creates a case.
run() runs or reruns a case.
create_case_config() creates a case configuration.
delete() deletes a case.

Case Manager create🔗

Before a case can be simulated as part of a workflow or algorithm it needs to be created. You can do this by calling case_manager.create. It takes the arguments:

Required parameters

custom_function - Specifies the Custom Function that the case will run with. The type of the object is a custom function as returned by the Python client

Optional named parameters

solver_options - Specifies the solver options the case will run with. If not given the default options for the Custom Function argument will be used. It should be given as a dictionary where the key is the name of the option, and the value is the choice for that option (string, bool, and numbers are allowed).
simulation_options - Specifies the simulation options the case will run with. If not given the default options for the Custom Function argument will be used. It should be given as a dictionary where the key is the name of the option, and the value is the choice for that option (string, bool, and numbers are allowed).
fmu - The FMU the case will run with. The type of the object is a model-executable. If not given, the same FMU as compiled for the current orchestrator case is used.

The method returns a case. Note that you can use this object to make further changes to the case before running it.

Tip

Set a label on your case so it is easy to find in the result browser after execution is finished.

Here is an example of creating a case

from modelon.impact.client import Client

def signature():
    return {
        "name": "test-orchestrator",
        "kind": "ORCHESTRATOR",
    }

def run(environment, case_manager):
    client = Client()
    workspace = client.get_workspace(environment["workspace_id"])
    dynamic = workspace.get_custom_function("dynamic")
    case = case_manager.create(dynamic)

    # Set label
    case.label = 'dynamic simulation'
    case.sync()

Note that if you run the above Custom Function the result will contain two cases. The created case and the case the Orchestrator Custom Function itself is running for. This case is called a Summary case. Any results saved by the Orchestrator Custom Function will exist for this case. The same is true for writing to the log or saving Custom Artifacts.

The Summary case have the label 'Summary' by default so it can be easily identified in the UI. It is possible to change this default for the case using the Python Client if needed.

Case Manager run🔗

To run or rerun cases call the case_manager.run method. It takes a list, where each element can either be a case or a Case Configuration (see below).

It will return an object that you can wait on for the case to complete. After completion you get back a list of results with the boolean attribute 'successful'.

Here is an example of creating and running a case:

from modelon.impact.client import Client

def signature():
    return {
        "name": "test-orchestrator",
        "kind": "ORCHESTRATOR",
    }

def run(environment, case_manager):
    client = Client()
    workspace = client.get_workspace(environment["workspace_id"])
    dynamic = workspace.get_custom_function("dynamic")
    case = case_manager.create(dynamic)
    results = case_manager.run([case]).wait()
    print(f"Case is successful: {results[0].successful}")

If run is called multiple times for a case it will rerun the case and overwrite the previous results. This is useful when many different parameter values must be tested to get some desired result. See create_case_config below for how this can be done efficiently.

Case Manager create_case_config🔗

For many workflows, the cases must run repeatedly with different parameter input. This could be done by setting the parameters on the case, call sync on the case, and then run it with the Case Manager. However, the overhead of doing this can be significant. To improve on this the parameters can be passed in directly when running a case as part of a Case Configuration.

To group a case with some parametrization when running the case there is the case_manager.create_case_config, which takes a case and a parametrization and returns a Case Configuration. The parametrization is a dictionary where the keys are the variable names and the values are the value of the parameter. Only strings, booleans and numbers are currently allowed for the values. The keys must be strings and the variable it represents must be a settable parameter for the case.

Here is an example that runs using a case configuration:

from modelon.impact.client import Client

def signature():
    return {
        "name": "test-orchestrator",
        "kind": "ORCHESTRATOR",
    }

def run(environment, case_manager):
    client = Client()
    workspace = client.get_workspace(environment["workspace_id"])
    dynamic = workspace.get_custom_function("dynamic")
    case = case_manager.create(dynamic)

    # Is expected to run with 'Modelica.Blocks.Examples.PID_Controller'
    case_config = case_manager.create_case_config(case, {'inertia1.J': 3})
    case_manager.run([case_config]).wait()
    print('Done!')

Case Manager delete🔗

Sometimes a case is just needed to make some computations but should not be part of the end-result. For these cases, the case_manager.delete method can be used to remove cases that are no longer needed. It takes a list, where each element is a case to be deleted.

Here is an example of creating and deleting a case (effectiely it does nothing):

from modelon.impact.client import Client

def signature():
    return {
        "name": "test-orchestrator",
        "kind": "ORCHESTRATOR",
    }

def run(environment, case_manager):
    client = Client()
    workspace = client.get_workspace(environment["workspace_id"])
    dynamic = workspace.get_custom_function("dynamic")
    case = case_manager.create(dynamic)
    case_manager.delete([case])

Reading results🔗

The Python Client can be used to read results after running some cases. For many use cases it is common to only care about the final point of simulated cases. These can be fetched by using the get_last_point method for the Python Client. Here is a code snippet for reading the final points of all cases and then ignoring the Summary case:

...

client = Client()
workspace = client.get_workspace(environment["workspace_id"])
experiment = workspace.get_experiment(environment["experiment_id"])
outputs = [...]
result = experiment.get_last_point(outputs)

# Ignoring the first Summary case, only intressted in cases created:
result_values = result.as_lists()[1:]
cases = result.cases[1:]