Skip to content

perost/OpenModelicaLibraryTesting

 
 

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

OpenModelica Library Testing

This repository provides scripts and documentation to run the nightly Modelica library tests for OpenModelica.

OpenModelica nightly testsuite

Some of the open-source Modelica libraries managed by the Open Modelica Package Manager are tested on a daily basis on the OSMC servers.

Test results reports are publicly available.

The configuration file for the regular library nightly testsuite is conf.json. Additional old and non-standard libraries are listed in conf-old.json and conf-nonstandard.json, note that failures in those libraries may be due to the fact that they are not fully complying with the Modelica standard, rather than to OpenModelica issues. The setup of the configuration files is discussed in conf-howto.md.

Test results reports are collected in the https://libraries.openmodelica.org/branches/ directory. The overview.html report gives the results of the regular testsuite with the default C runtime and solvers. Other reports contain the results using the C++ runtime, FMI, daeMode, and the old frontend. Combined reports also include results from the old and nonstandard libraries. The https://libraries.openmodelica.org/branches/history/ directory contains regression reports and plots using different versions (including master) and simulation runtime configurations (C++, daeMode, FMI, old frontend) of OpenModelica.

If you want to include your open-source library in the testsuite, please open a pull request on conf.json, or open an issue on the OpenModelica issue tracker and ask us to do it for you.

Running the library testing infrastructure on your own server

License: OSMC-PL

The scripts from this repository can be used to run regression tests for public, private, and commercial Modelica libraries to keep track of coverage with different OpenModelica versions, according to the conditions of the OSMC-PL license.

Dependencies (Linux)

Set-Up

  • Install or build OpenModelica

  • Install Python requirements

    pip install -r requirements.txt
  • Install libraries you want to test

    • (Optional) Remove already installed libraries

      rm -rf ~/.openmodelica/libraries/
      rm -rf /path/to/OpenModelica/build/lib/omlibraries
    • Install all available libraries.

      /path/to/omc .CI/installLibraries.mos

      or

    • Install your libraries.

  • Create configs/myConf.json to specify what libraries to test.

    [
      {
        "library":"MyModelicaLibrary",
        "libraryVersion":"master",
        "referenceFileExtension":"mat",
        "referenceFileNameDelimiter":"/",
        "referenceFileNameExtraName":"$ClassName",
        "referenceFiles":{
          "giturl":"https://github.com/myName/MyModelicaLibrary-ref",
          "destination":"ReferenceFiles/MyModelicaLibrary"
        },
        "optlevel":"-Os -march=native"
      }
    ]

    You can add extra compiler settings

    "extraCustomCommands":["setCommandLineOptions(\"--std=3.2\");"]

    and extra simulation flags

    "extraSimFlags": "-s=ida -nls=kinsol"

    Check config/conf.json for more.

  • If you used .CI/installLibraries.mos to test all libraries you'll need to install reference results and set environment variables, see Reference Results.

    export MSLREFERENCE="/path/to/ReferenceFiles/"
    export REFERENCEFILES="/path/to/OpenModelica/testsuite/ReferenceFiles"
    export PNLIBREFS="/path/to/ReferenceFiles/PNlib/ReferenceFiles"
    export THERMOFLUIDSTREAMREFS="/path/to/ReferenceFiles/ThermofluidStream-main-regression/ReferenceData"
    export THERMOFLUIDSTREAMREFSOM="/path/to/ReferenceFiles/ThermofluidStream-OM-regression/ReferenceData"
  • Run the library test

    ./test.py --noclean configs/myConf.json

    Use configs/*.json to specify what to test. The test results are saved in sqlite3.db.

  • Generate HTML results

    ./report.py configs/myConf.json
  • Upload and backup

    • Upload HTML files somewhere
    • backup sqlite3.db

Reference Results

If you use the default configs config/conf.json and config/conf-c++.json to test all libraries you need to download the reference files and make them available by defining MSLREFERENCE and REFERENCEFILES.

Some result file locations:

To download the MSL reference files create a file installReferenceResults.sh with

#!/bin/sh

refdir="/some/path/to/ReferenceFiles"   # Change the path!

# Update git repo for MSL Reference files
mkdir -p $refdir/modelica.org/ReferenceResults
cd $refdir/modelica.org/ReferenceResults
rm -rf $refdir/MAP-LIB_ReferenceResults/

test -f MAP-LIB_ReferenceResults.git/config || git clone --bare https://github.com/modelica/MAP-LIB_ReferenceResults.git MAP-LIB_ReferenceResults.git
cd MAP-LIB_ReferenceResults.git
git fetch origin '*:*'
git fetch --tags

for tag in $(git for-each-ref --format="%(refname:lstrip=-1)" refs/heads/)
do
  echo "tag: $tag"
  base="$refdir/MAP-LIB_ReferenceResults/$tag"
  mkdir -p $base
  echo "mkdir -p $base"
  git archive --format=tar $tag | (cd $base && tar xvf -)
done

and run it

chmod a+rx installReferenceResults.sh
./installReferenceResults.sh
export MSLREFERENCE="/some/path/to/ReferenceFiles/"

For the other libraries just clone the repositories to /some/path/to/ReferenceFiles/.

About

Test script for OMCompiler+OpenModelicaLibraries

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages

  • Python 95.0%
  • Smarty 3.3%
  • Motoko 1.4%
  • Other 0.3%