Project Structure #197

prasadtalasila · 2024-11-23T09:09:21Z

prasadtalasila
Nov 23, 2024
Maintainer

We are brainstorming the following project structure for the Co-Simulation Studio and the INTO-CPS Application.

Broader objective is to have the same directory structure for both the projects while providing the required environment-specific configuration in a yaml file.

The suggested structure is:

FMUs
  FMU1
  FMU2
  ...

cosimulation (one directory for each cosimulation scenario; this directory is a replacement for multi-model)
   default/
      cosim1.json
      cosim2.json
      ...
   faults/
      cosim1.json
      cosim2.json
      ...
   model-swap/
      cosim1.json
      cosim2.json
      ...

build/ (directory for storing simulation results and temporary files)
  cosimulation (same structure as top-level cosimulation)

cosim-studio.yml (or) into-cps.yml

Questions:

Is is appropriate to rename multi-model directory to cosimulation?
The suggested directory structure has one-level categorization at cosimulation while some the existing tutorials seem to have many levels of directory structure. Is this ok?
Is there any disadvantage to pushing all the logs and cosimulation outputs to build/ directory?

@kgpierce @lausdahl @CThuleHansen @pglvdm @clagms @MarkusEllyton @PandoraQS

CThuleHansen · 2024-11-23T20:34:03Z

CThuleHansen
Nov 23, 2024
Maintainer

On my phone, will respond more in full Monday.
We discussed the wording quite a lot back in the days and I cannot remember the details, I will try to look through some into-cps documents next week.

My initial thoughts: A simulation is an activity. Thus it has inputs, executes something, and generates outputs. As such the cosim.json-files are not simulations, they are input to the simulation activity. To perform the actual simulation you still need the tool, the FMUs and so forth.
But, I am all for the pragmatism, so if it is more meaningful to users to call it co-simulations, then that is the value-driver.
A small thing, on multiple occasions we lacked the capability to perform simulation of a single fmu, so does it have to be co-simulations? You might need to asses e.g. the fidelity of a single FMU in isolation.

3 replies

prasadtalasila Nov 25, 2024
Maintainer Author

@CThuleHansen

On my phone, will respond more in full Monday. We discussed the wording quite a lot back in the days and I cannot remember the details, I will try to look through some into-cps documents next week.

That would be nice. Please point me towards the documents if you find them.

My initial thoughts: A simulation is an activity. Thus it has inputs, executes something, and generates outputs. As such the cosim.json-files are not simulations, they are input to the simulation activity. To perform the actual simulation you still need the tool, the FMUs and so forth.

I agree and this is how we are structuring digital twins as well in DTaaS software. The files in cosimulation/ directory are pure configuration files. In this case, they are co-simulation configuration files. They could be some other kind of configuration files needed for co-simulation.
The only question is about the scope of the applications. Are there ever going to be non-FMU kind of simulations? I am unaware of the internal understanding of the FMI standard. @clagms can provide some insight there.
Are there existing use cases where the FMU-based simulations are ever called anything other than co-simulations?
We are in a way coupling the Maestro tightly with both the new INTO-CPS Application and CoSimulation Studio applications. If we are not going to use any other simulators inside these two applications, does it make sense to call the context anything other than cosimulation?

But, I am all for the pragmatism, so if it is more meaningful to users to call it co-simulations, then that is the value-driver. A small thing, on multiple occasions we lacked the capability to perform simulation of a single fmu, so does it have to be co-simulations? You might need to asses e.g. the fidelity of a single FMU in isolation.
I will add this to the feature backlog of both the applications. A quick question: does Maestro support this kind of scenario? If not, we might need to integrate FMpy into both the applications.

Overall, thanks for the suggestions and look forward to more feedback.

clagms Nov 25, 2024
Maintainer

Is is appropriate to rename multi-model directory to cosimulation?

Is there a strong reason to depart from the nomenclature in the into-cps application? There we have the multi-model (which is the FMUs and their interconnections), and co-simulation configuration (step size, plotted outputs, etc...). We should keep this separation, because it's likely we have multiple co-simulation configurations for a single multi-model.

Is there any disadvantage to pushing all the logs and cosimulation outputs to build/ directory?

It may sound strange to simulation practioners who are not aware of what the term "build" means. Only software people are aware of that, i think.

Are there ever going to be non-FMU kind of simulations? I am unaware of the internal understanding of the FMI standard. @clagms can provide some insight there.

Not sure I understand the question, but if you mean that you wish to run other types of simulation then I recommend you don't because it's a rabbit hole. I think the co-sim studio should stay within FMI based co-simulation.

A small thing, on multiple occasions we lacked the capability to perform simulation of a single fmu, so does it have to be co-simulations? You might need to asses e.g. the fidelity of a single FMU in isolation.

Here we abuse the word co-simulation indeed, and calling it simulation would be better, but no strong feelings otherwise.

prasadtalasila Nov 25, 2024
Maintainer Author

Is is appropriate to rename multi-model directory to cosimulation?

Is there a strong reason to depart from the nomenclature in the into-cps application? There we have the multi-model (which is the FMUs and their interconnections), and co-simulation configuration (step size, plotted outputs, etc...). We should keep this separation, because it's likely we have multiple co-simulation configurations for a single multi-model.

Agree. The question is also about the current practice among our users. Is it a hard separation (between multi-model and co-sim config) or do we want to provide flexibility to users? Maestro seems to be ok either way.
That also brings a related question. What if we have more configuration files like:

model swap
RabbitMQ config
UniFMU config

How to deal with them?

Is there any disadvantage to pushing all the logs and cosimulation outputs to build/ directory?

It may sound strange to simulation practitioners who are not aware of what the term "build" means. Only software people are aware of that, i think.

The intention is to make INTO-CPS projects git friendly. One way to achieve it would be by separating the co-simulation results. I do agree that build/ is not the best name for it and happy to change it. Perhaps results/cosimulation would do.

Are there ever going to be non-FMU kind of simulations? I am unaware of the internal understanding of the FMI standard. @clagms can provide some insight there.

Not sure I understand the question, but if you mean that you wish to run other types of simulation then I recommend you don't because it's a rabbit hole. I think the co-sim studio should stay within FMI based co-simulation.

A small thing, on multiple occasions we lacked the capability to perform simulation of a single fmu, so does it have to be co-simulations? You might need to asses e.g. the fidelity of a single FMU in isolation.

Here we abuse the word co-simulation indeed, and calling it simulation would be better, but no strong feelings otherwise.

@clagms is going to check the capability of Maestro to do single FMU simulation.

prasadtalasila · 2024-11-25T12:22:54Z

prasadtalasila
Nov 25, 2024
Maintainer Author

After an off-line discussion with @clagms , I am revising the directory structure to the following.

FMUs
  FMU1
  FMU2
  ...

cosimulation (one directory for each cosimulation scenario; this directory is a replacement for multi-model)
   default/ (plain co-simulation config with no concern for separate multi-models and experiment configurations
      cosim1.json
      cosim2.json
      ...
   multi-model-1/
      multi-model.json
      experiment-1
          experiment-1.json
      experiment-2/
          experiment-1.json
      ...
   multi-model-2/
      multi-model.json
      experiment-1
          experiment-1.json
      experiment-2/
          experiment-1.json
      ...

results/ (directory for storing simulation results and temporary files)
  cosimulation (same structure as top-level cosimulation; also remember to copy config files in the interest of traceability)

cosim-studio.yml (or) into-cps.yml

please let me know your thoughts regarding this proposal. Thanks.

0 replies

CThuleHansen · 2024-11-25T21:09:33Z

CThuleHansen
Nov 25, 2024
Maintainer

Yes, simulations some times require other artifacts performing simulations than FMUs. Yes, it is a rabbit hole :D You can create a resources folder like the FMUs :)
I agree with build being developer-specific.
cosimulation or simulation. I still think it is important to useful to simulate a single FMU. But cosimulation is fine, as this is the many purpose.
Mentions of multi-model
https://into-cps.org/fileadmin/into-cps.org/Filer/[INTO-CPS-Manifesto.pdf](https://into-cps.org/fileadmin/into-cps.org/Filer/INTO-CPS-Manifesto.pdf):
"In the INTO-CPS Application, a project comprises: a number of FMUs, optional source models (from which FMUs are exported); a collection of multi-models; and an optional SysML architectural model. A multi-model includes a list of FMUs, defined instances of those FMUs, specified connections between the inputs/outputs of the FMU instances, and defined values for design parameters of the FMU instances. For each multi-model a co-simulation configuration defines the step size configuration, start and end time for the co-simulation of that multi-model. Several configurations can be defined for each multi-model."
from https://github.com/INTO-CPS-Association/Documentation/blob/master/method_guidelines/INTO-CPS_Method_Guidelines.pdf
This also has a fair amount of mentions, you can go through them fairly quickly. https://into-cps.org/fileadmin/into-cps.org/Filer/[INTO-CPS-Manifesto.pdf](https://into-cps.org/fileadmin/into-cps.org/Filer/INTO-CPS-Manifesto.pdf

At the end of the day, the most important thing is to do something, and then change it, if it does not work! :D

2 replies

clagms Nov 26, 2024
Maintainer

@clagms is going to check the capability of Maestro to do single FMU simulation.

Checked. It is possible. See the example here: https://github.com/modelica/fmi-beginners-tutorial-2024/blob/main/part3/tutorial_multiple_FMUs/interaction_with_multiple_fmus.ipynb

It runs the following scenario.json: https://github.com/modelica/fmi-beginners-tutorial-2024/blob/main/part3/tutorial_multiple_FMUs/single_fmu_scenario.json

CThuleHansen Nov 26, 2024
Maintainer

Nice. Wonder if it breaks if there are no log variables, because then there is nothing to define the instance. @lausdahl ?

but then again, what is the purpose of a simulation if you cannot observe anything. Unless you are cold of course 😀

prasadtalasila · 2024-11-26T11:17:42Z

prasadtalasila
Nov 26, 2024
Maintainer Author

There are two different ways of providing configuration I see:

Single cosim config
multimodel with cosim config

Both go to Maestro. The problem is with separation and specification of these config files at the JSON schema level. The current implementation inside CoSimulation Studio is for single cosim config
Is there place where complete, mandatory and optional parts of schema can be downloaded from (for both single and separate configs)?

1 reply

CThuleHansen Nov 27, 2024
Maintainer

I do not think there are actual schema files. But there is a swagger api somewhere

prasadtalasila · 2024-11-27T13:45:11Z

prasadtalasila
Nov 27, 2024
Maintainer Author

@CThuleHansen @lausdahl

Please see Maestro API

Within the context of CoSimulation Studio, the following API end points are used:

From Maestro API docs

cosim config provided via /initialize/{sessionId}

This config seems to be a template for multi-model config and hints at the possibility of simulation config as well.

{
  "fmus": {
    "additionalProp1": "string",
    "additionalProp2": "string",
    "additionalProp3": "string"
  },
  "connections": {
    "additionalProp1": [
      "string"
    ],
    "additionalProp2": [
      "string"
    ],
    "additionalProp3": [
      "string"
    ]
  },
  "parameters": {},
  "logVariables": {
    "additionalProp1": [
      "string"
    ],
    "additionalProp2": [
      "string"
    ],
    "additionalProp3": [
      "string"
    ]
  },
  "parallelSimulation": true,
  "stabalizationEnabled": true,
  "global_absolute_tolerance": 0,
  "global_relative_tolerance": 0,
  "loggingOn": true,
  "visible": true,
  "simulationProgramDelay": true,
  "algorithm": {},
  "overrideLogLevel": "OFF",
  "environmentParameters": [
    "string"
  ],
  "logLevels": {
    "additionalProp1": [
      "string"
    ],
    "additionalProp2": [
      "string"
    ],
    "additionalProp3": [
      "string"
    ]
  },
  "livestream": {
    "additionalProp1": [
      "string"
    ],
    "additionalProp2": [
      "string"
    ],
    "additionalProp3": [
      "string"
    ]
  },
  "hasExternalSignals": true,
  "faultInjectConfigurationPath": "string",
  "faultInjectInstances": {
    "additionalProp1": "string",
    "additionalProp2": "string",
    "additionalProp3": "string"
  },
  "convergenceAttempts": 0,
  "externalSpecs": [
    {
      "absolute": true,
      "absolutePath": "string",
      "canonicalPath": "string",
      "directory": true,
      "executable": true,
      "file": true,
      "freeSpace": 0,
      "hidden": true,
      "lastModified": 0,
      "name": "string",
      "parent": "string",
      "path": "string",
      "readable": true,
      "totalSpace": 0,
      "usableSpace": 0,
      "writable": true
    }
  ],
  "modelTransfers": {
    "additionalProp1": "string",
    "additionalProp2": "string",
    "additionalProp3": "string"
  },
  "modelSwaps": {
    "additionalProp1": {
      "stepCondition": "string",
      "swapCondition": "string",
      "swapConnections": {
        "additionalProp1": [
          "string"
        ],
        "additionalProp2": [
          "string"
        ],
        "additionalProp3": [
          "string"
        ]
      },
      "swapInstance": "string"
    },
    "additionalProp2": {
      "stepCondition": "string",
      "swapCondition": "string",
      "swapConnections": {
        "additionalProp1": [
          "string"
        ],
        "additionalProp2": [
          "string"
        ],
        "additionalProp3": [
          "string"
        ]
      },
      "swapInstance": "string"
    },
    "additionalProp3": {
      "stepCondition": "string",
      "swapCondition": "string",
      "swapConnections": {
        "additionalProp1": [
          "string"
        ],
        "additionalProp2": [
          "string"
        ],
        "additionalProp3": [
          "string"
        ]
      },
      "swapInstance": "string"
    }
  }
}

Simulation / Experiment config via /simulate/${sessionId}

{
"startTime": 0,
"endTime": 0,
"logLevels": {
"additionalProp1": [
"string"
],
"additionalProp2": [
"string"
],
"additionalProp3": [
"string"
]
},
"reportProgress": true,
"liveLogInterval": 0,
"masterModel": "string"
}

The current status in CoSimulation Studio:

cosim config provided via /initialize/{sessionId}

The CoSimulation Studio uses this end point for providing multi-model config

The keys mentioned in the cosim config mentioned below are mandatory in CoSimulation Studio.

{
  "fmus": {
     ...
  },
  "connections": {
     ...
  },
  "parameters": {
     ...
  },
  "logVariables": {
    ...
  }
 }

Simulation / Experiment config via /simulate/${sessionId}

The CoSimulation Studio uses this end-point for providing experiment config.

⚠️ The experiment config mentioned below is hard coded in CoSimulation Studio.

{
"startTime": 0,
"endTime": 10,

1 reply

lausdahl Nov 28, 2024
Maintainer

Also see this https://into-cps-maestro.readthedocs.io/en/latest/user/file_formats.html#legacy-configuration-into-cps-multi-model-and-coe-config-file-format

prasadtalasila · 2024-11-27T14:40:19Z

prasadtalasila
Nov 27, 2024
Maintainer Author

I am thinking of adding cosimulation/cosimulation.yaml which contains a list of all the cosimulations to be run. For example, it can contain:

execute:
  category: # example, default / fault / no-fault
    experiments: [experiment-1, experiment-2,...]

This yaml is helpful to trigger cosimulation(s) in CoSimulation Studio. The INTO-CPS App user can manually select an experiment and execute the same.

Question:

The term cosimulation scenario has been used before in the context of Maestro. Does it mean: a pair of multi-model and experiment configuration?

1 reply

clagms Nov 27, 2024
Maintainer

The term cosimulation scenario has been used before in the context of Maestro. Does it mean: a pair of multi-model and experiment configuration?

Yes.

prasadtalasila · 2024-12-05T15:14:14Z

prasadtalasila
Dec 5, 2024
Maintainer Author

@lausdahl provided partial schemas.

Extracted from the code.
MaestroV1SimulationConfiguration.json
Extracted from API docs:
api-docs.json

0 replies

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Project Structure #197

{{title}}

Replies: 7 comments 8 replies

{{title}}

{{editor}}'s edit

{{editor}}'s edit

{{title}}

{{title}}

{{title}}

{{title}}

{{editor}}'s edit

{{editor}}'s edit

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

cosim config provided via /initialize/{sessionId}

Simulation / Experiment config via /simulate/${sessionId}

cosim config provided via /initialize/{sessionId}

Simulation / Experiment config via /simulate/${sessionId}

{{title}}

{{title}}

{{title}}

{{title}}

Select a reply

Project Structure #197

prasadtalasila Nov 23, 2024 Maintainer

Replies: 7 comments · 8 replies

CThuleHansen Nov 23, 2024 Maintainer

prasadtalasila Nov 25, 2024 Maintainer Author

clagms Nov 25, 2024 Maintainer

prasadtalasila Nov 25, 2024 Maintainer Author

prasadtalasila Nov 25, 2024 Maintainer Author

CThuleHansen Nov 25, 2024 Maintainer

clagms Nov 26, 2024 Maintainer

CThuleHansen Nov 26, 2024 Maintainer

prasadtalasila Nov 26, 2024 Maintainer Author

CThuleHansen Nov 27, 2024 Maintainer

prasadtalasila Nov 27, 2024 Maintainer Author

From Maestro API docs

cosim config provided via /initialize/{sessionId}

Simulation / Experiment config via /simulate/${sessionId}

The current status in CoSimulation Studio:

cosim config provided via /initialize/{sessionId}

Simulation / Experiment config via /simulate/${sessionId}

lausdahl Nov 28, 2024 Maintainer

prasadtalasila Nov 27, 2024 Maintainer Author

Question:

clagms Nov 27, 2024 Maintainer

prasadtalasila Dec 5, 2024 Maintainer Author

prasadtalasila
Nov 23, 2024
Maintainer

Replies: 7 comments 8 replies

CThuleHansen
Nov 23, 2024
Maintainer

prasadtalasila Nov 25, 2024
Maintainer Author

clagms Nov 25, 2024
Maintainer

prasadtalasila Nov 25, 2024
Maintainer Author

prasadtalasila
Nov 25, 2024
Maintainer Author

CThuleHansen
Nov 25, 2024
Maintainer

clagms Nov 26, 2024
Maintainer

CThuleHansen Nov 26, 2024
Maintainer

prasadtalasila
Nov 26, 2024
Maintainer Author

CThuleHansen Nov 27, 2024
Maintainer

prasadtalasila
Nov 27, 2024
Maintainer Author

lausdahl Nov 28, 2024
Maintainer

prasadtalasila
Nov 27, 2024
Maintainer Author

clagms Nov 27, 2024
Maintainer

prasadtalasila
Dec 5, 2024
Maintainer Author