MBP Testing Tool Testing IoT Applications
IoT applications usually consist of a large number of different IoT-Devices, which can violate the requirements of a stable and secure application by their respective misbehavior. Sensors are the main cause of incorrect behavior of an IoT application. Their data extracted from the external environment are analyzed and actuators are used to react to them. As a consequence of such a misbehavior, due to an anomaly within the sensor values, communication problems between the IoT devices can occur - in the worst case, the analysis triggers an action by actuators that does not fit the context. This can have fatal consequences, especially in safety critical applications (e.g. autonomous vehicles).
For this reason, the test platform was developed with different sensor simulators to effectively test the behavior of IoT applications for sensor anomalies and to provide a basis for taking action by means of a detailed test report.
The test platform can be found within the MBP under the menu item "Testing-Tool" (1). Here you will find a short introduction to the correct handling of the testing tool (2).
Below this you will find a table in which the tests are managed. You can find out if and which sensor and actuator simulators and the test device have already been registered by clicking on the tool icon in the upper right corner of the table (3). Once the respective operators have been created, one click is enough to automatically register the desired components.
With the plus sign (4) you can create a new test. Besides the name of the test you have to enter the type of sensor and other settings specific to the type of sensors. You can also select the rules you want to observe within the test and if they should or should not be violated.
Once the test has been created, it can be managed either in the table or on the details page which can be accessed by clicking on the name of the test (5).
The management within the table includes starting and stopping the test, the setting for reusing already generated data, downloading the last generated test report (if available) and deleting the test.
This page offers you a detailed view of the test and the possibility to edit your previous test settings. As in the administration table, the test can be started, stopped and already generated data can be reused.
In the upper left window (1) the most important information regarding the configuration and status of the test or Sensor Simulator is displayed. Here you have the possibility to edit the test (2) and stop it when the test is running.
In order to be able to compare already executed tests with each other, all already generated test reports are listed with date and time and are available for download (3).
In order to provide visual feedback on the controls belonging to the IoT application and the data generated by the sensor simulator, a table with detailed information of the rules (4) is provided as well as diagrams for live transmission and historical data of the sensor simulators (5).
Since sensors are difficult to manipulate from the outside in order to perform a test with certain anomalies in a planned way, different types of sensor simulators are available within the MBP. These include one-dimensional sensors:
- Temperature Sensor
- Humidity Sensor
As well as three-dimensional sensors like:
- GPS Sensor
- Accelerometer
These sensor simulators are able to simulate realistic changes of the external environment and to combine it with anomalies. As an anomaly within the sensor data
- outlier values, which are above or below the maximum rate of change of successive
- wrong value types, which for example return a string instead of a floating point,
- failures, where values are not delivered by the actual frequency, are considered.
In addition, for each of these sensors it is possible to plan the duration of the simulation/test in time, by the number of anomalies and events. The operators for creating these sensor simulators are provided here. There you also get information about which parameters have to be defined for each operator.
For the actuator simulator there are also operators available within the MBP which can be created in the platform. This is an actuator with no functionality. So if a rule is violated to which this actuator is assigned, the actuator is only informed about it, but no action is triggered.
How to use the Testing-Tool efficiently and correctly can be found in the linked How to.
