Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Enable the support of complex IoT data #591

Merged
merged 60 commits into from
Dec 8, 2021
Merged

Enable the support of complex IoT data #591

merged 60 commits into from
Dec 8, 2021

Conversation

schneiti
Copy link
Collaborator

@schneiti schneiti commented Jun 8, 2021

Adds functionality for the support of complex sensor data. This includes compound data types and a larger selection of primitive data types.

Main additions (short version):

  • Instead of units, each operator (excluding monitoring operators) gets a data model assigned which represents the json structure of the sensor values which will be send via MQTT.
  • Modular and configurable visualization charts for complex iot data
  • Access to sensor data fields via JsonPath for CEP conditions and visualization settings.
  • Utils DocumentReader class for accessing complex data with data model leaf nodes (read and write access to perform automatic data processing, if in future needed)

Steps to bind sensors/actuators to the MBP properly:

As this new version of the MBP adds new required fields to existing database entities it is necessary or at least recommended to drop the local MongoDB collections (at least for the collections: mongoValueLogs, operator, sensor, actuator).

  1. Create a new data model which represents the json sensor value MQTT payload and the desired data type mappings.
    grafik
  2. Have a look at the generated example mqtt sensor value payload to make sure the data model was defined correctly.
    grafik
  3. Connect the data model to the operator. (Make sure that the extraction scripts send sensor values which are in conformity with the data model specification). Examples of such operator scripts (generating random values for test purposes) can be found in
    MBP/resources/adapter-scripts/complex_iot_data_stub/
    grafik
  4. Create sensor/actuator instances.

Visualize sensor data

  1. In the sensor detail view choose visualization modules. If no module is available this means there is recently no suitable chart for this kind of iot data implemented.
    grafik
  2. Click on the settings buttons for a visualization module.
    grafik
  3. Choose the data fields you want to visualize
    grafik
  4. Click on save.
    grafik

Configure conditions for CEP.

  • For each data field which should be accessed for creating IoT conditions, a json path has to be provided via the json path input method next to the condition.
    grafik
  • Also multiple filter conditions are now available (connected via AND)
    grafik

Monitoring operators

  • Monitoring operators are not affected by the changes and do not need data models, as they are intended to provide simple device metrics.

Closes #17, #531, #530, #529, #528, #500, #501, #508, #509, #510, #511, #515, #514, #607, #621, #619, #620

…he database and adds pseudo operator scripts to simulate complex data
…ents to access and alter easily complex iot data
# Conflicts:
#	src/main/java/de/ipvs/as/mbp/repository/ValueLogRepository.java
#	src/main/resources/static/js/controllers/operators/OperatorListController.js
#	src/main/resources/static/js/directives/charts/HistoricalChart.js
#	src/main/resources/static/js/directives/charts/LiveChart.js
#	src/main/webapp/WEB-INF/views/index.html
#	src/main/webapp/WEB-INF/views/templates/operators.html
#	src/main/webapp/WEB-INF/views/templates/sensor-id.html
…tion settings also for the new modular geo map visualization
schneiti added 14 commits July 8, 2021 10:54
# Conflicts:
#	src/main/webapp/WEB-INF/views/templates/operators.html
# Conflicts:
#	src/main/java/de/ipvs/as/mbp/repository/OperatorRepository.java
#	src/main/webapp/WEB-INF/views/index.html
# Conflicts:
#	src/main/java/de/ipvs/as/mbp/service/receiver/ValueLogReceiverArrivalHandler.java
…fault components scripts as well as the rerun operator script (jars in simulation folder still missing)
…data model hint to home.html. Fix bug regarding the mbp_client.py file and corresponding user scripts.
… rerun strategy with a physical testing device
@schneiti schneiti changed the title Draft: Enable the support of complex IoT data Enable the support of complex IoT data Sep 5, 2021
@schneiti schneiti marked this pull request as ready for review September 5, 2021 16:40
…nging the MQTT payload in case of sensor value transmission
pom.xml Outdated Show resolved Hide resolved
@schneijan schneijan self-requested a review December 2, 2021 12:36
@schneijan schneijan self-assigned this Dec 2, 2021
Copy link
Member

@schneijan schneijan left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks great, only found some minor issues to consider before merge.

@schneijan schneijan merged commit 1f0b54b into master Dec 8, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Enable more complex sensor values
2 participants