A SiL and HiL test system that automates the acceptance of every firmware version

The task

A power quality analyser measures voltage, current and power quality, and every firmware version has to show before release that its readings stay within the tolerances of the standard. At the client, that acceptance largely ran by hand: set the calibrator, read the device, keep the record. With every new version and every new device type the effort grew, and release became the bottleneck.

What was required was a framework that automates these acceptances and runs in the client's infrastructure, with its existing instruments, its test management and its development tools. The firmware itself was not to be modified for it.

The solution

The framework is built on Robot Framework. Each interface of the device has its own keyword library, for Modbus, REST, OPC UA and direct access to the firmware's memory. The client's high-precision calibrators are integrated through libraries of their own and are driven from within the test case: the test sets a waveform with harmonics or a modulation, reads the device under test and compares against the tolerance.

The normative test cases for power quality instruments are implemented as executable tests of this kind. Part of them is generated directly from the tables of the standard, so that a changed tolerance does not have to be updated in dozens of places.

The same tests run on two levels. In the SiL system the firmware runs without hardware against a simulated calibrator, the fast run after every change. In the HiL system it runs on the device against the real calibrator, the acceptance. The results of the unit tests on Google Test go, together with the coverage, automatically into the client's test management, where release is decided.

This grew over several years in very closely interlocked work with the client's development, in its infrastructure and with its tools.