General

Project status

Scope of this project is to

• provide a simple regression testing framework for Modelica models
• support continuous integration (CI) for Modelica library development, in particular
• test for reproducibility of results from Modelica models and to
• have it wrapped in the well-known Python unittest module

When developing Modelica models or libraries and releases have been made, there is definitely the need for testing. E.g.

• validation of successful model translation,
• reproducibility of simulation results, or
• making sure refactorings in Modelica libraries do not alter the behavior.

Other testing criteria could be memory consumption, CPU time and more.

The scope of MoPyRegtest is to have a lightweight and simple way to perform regression tests and to be able to integrate this with automated testing and continuous integration (CI) toolchains, like e.g. GitHub Actions. Without immediately requiring commercial simulation tools or creating strong dependencies from other Modelica libraries.

All that is needed to use MoPyRegtest is a Python 3 installation, an OpenModelica installation, and a text editor.

The current realization uses the Python unittest module and calls the OpenModelica compiler omc as an external executable for translation and simulation of Modelica models. Note that the implementation of MoPyRegtest is not specific to OpenModelica but can be replaced by any other Modelica simulation tool that comes with a suitable scripting API.

The test execution and orchestration is done by the Python unittest module. In particular, Python unittest supports test discovery. Also, unittest is a standard module that comes with virtually all Python distributions.

MoPyRegtest is work in progress!

With MoPyRegtest, users can define regression tests manually in Python for one Modelica model at a time, or automatically for many Modelica models simultaneously.

The user has to

• provide Modelica models that can be translated, built and executed and produce a result .csv file
• provide for every Modelica model a .csv file with a reference result against which an actual result is compared
• create a Python file and test methods that instantiate a mopyregtest.RegressionTest object, which needs to know which Modelica model to test and where to find the reference result.

Examples can be found in the examples folder. Also see the examples/README.md.

MoPyRegtest can generate regression test definitions for elements of a Modelica library automatically. Either by creating Generator class objects. Or using the mopyregtest command line tool.

The folder examples/generate_tests/README.md has a detailed explanation on how to automatically generate tests.

To use MoPyRegtest you need to have

• a Python 3 distribution including the modules unittest, numpy and pandas,
• a working OpenModelica installation (version 1.13.0 or later, works also for latest 1.22), and
• the OpenModelica compiler executable omc must be accessible via the PATH variable.

It is most convenient to install MoPyRegtest using Python's package manager pip. MoPyRegtest does not need any special privileges to run.

You can install MoPyRegtest from the Python Package Index by running

pip3 install --user mopyregtest

To install a specific version, e.g. v0.3.1, run pip3 install --user mopyregtest==0.3.1

To install from source, first clone this repository to your local <your-mopyregtest-dir> (adapt to your needs) with the command git clone https://github.com/pstelzig/MoPyRegtest.git <your-mopyregtest-dir>

Then run

cd <your-mopyregtest-dir> pip3 install --user .

to get the latest development head. To install a specific version from source, e.g. v0.3.1, run

cd <your-mopyregtest-dir> git checkout v0.3.1 pip3 install --user .

To uninstall MoPyRegtest, run

pip3 uninstall mopyregtest

To check if MoPyRegtest's command line tool is working, run mopyregtest --help from a terminal.

If an error message shows that the program mopyregtest cannot be found, then you have to check if your PATH variable knows where to find mopyregtest as installed by pip. To find out the location, run pip uninstall mopyregtest but do not confirm to proceed. The uninstallation will tell you which files pip would remove upon uninstalling MoPyRegtest, including the mopyregtest executable (or mopyregtest.exe under Windows) and its location.

Found existing installation: MoPyRegtest 0.4.0rc1 Uninstalling MoPyRegtest-0.4.0rc1: Would remove: /home/<your user name>/.local/bin/mopyregtest /home/<your user name>/.local/lib/python3.10/site-packages/MoPyRegtest-0.4.0rc1.dist-info/* /home/<your user name>/.local/lib/python3.10/site-packages/mopyregtest/* Proceed (Y/n)? 

Then, one could either call mopyregtest with its full path, or one could add its parent folder to the PATH variable (Be careful! First make sure that this does not cause any harm to your system).

• Allow for parallel execution of regression tests
• Improve readability of in particular failed test results
• Provide logs that help users in the analysis of failed tests
• Make definition of the tests even simpler, e.g. using a more human-readable BDD approach

MoPyRegtest is implemented in Python3 and uses the Python core modules (including pathlib and unittest) along with numpy and pandas.

MoPyRegtest is open source software. MoPyRegtest is provided "as is", without warranty of any kind. Usage is completely at your own risk. See the file LICENSE.

If you want to help me extend MoPyRegtest, please drop me a message! Contributions are welcome!

The MoPyRegtest is being developed by the following authors:

• Dr. Philipp Emanuel Stelzig