Merge pull request #591 from IPVS-AS/complex-iot-data

Enable complex IoT data

pom.xml

@@ -190,6 +190,14 @@
             <artifactId>cglib</artifactId>
             <version>3.3.0</version>
         </dependency>
+
+        <!-- JsonPath -->
+        <dependency>
+            <groupId>com.jayway.jsonpath</groupId>
+            <artifactId>json-path</artifactId>
+            <version>2.5.0</version>
+        </dependency>
+
     </dependencies>
 
     <build>

resources/operators/control/README.md

@@ -28,3 +28,8 @@ The name convention for subfolders is `<actuator-type>_<IoT-device-type>`.
 - [loudspeaker_computer](loudspeaker_computer): operator scripts to control a loudspeaker actuator.
 - [relay_raspberry-pi](relay_raspberry-pi): operator scripts to control a relay.
 - [switch_raspberry-pi](switch_raspberry-pi): operator scripts to control a radio frequency (RF), Arduino-based switch.
+
+## Note on data model definition
+
+As many here listed actuators do not send any data to the MBP they can use an arbitrary data model. Therefore, for these actuators
+no data model json definition is provided in their folders.

resources/operators/control/camera_raspberry-pi/README.md

@@ -29,4 +29,9 @@ When a picture is taken, it is saved on the deployed folder on the Raspberry Pi
 
 - `running.sh`: This file checks if the python script is running.
 
-- `stop.sh`: This file stops the execution of the python script.
+- `stop.sh`: This file stops the execution of the python script.
+
+## Data model support
+
+Since the here provided scripts send a MQTT message for each camera image to a "image" MQTT topic (in a not normalized message format) it can't be handled like usual sensor data (must be considered separately by the MBP implementation). 
+Thefore no data model support is provided 

resources/operators/control/simulators/actuator_adapter_testing/README.md

@@ -17,4 +17,6 @@ This folder contains operator scripts to control an actuator that does not trigg
 
  - `stop.sh`: This file stops the execution of the python script.
 
- - `actuator_testingTool.py`: This python script contains a MQTT client, which subscribes to a MQTT topic in the form action/<actuator_id/#> to receive control commands. The actuator is only used for communication and does not execute any actions. 
+ - `actuator_testingTool.py`: This python script contains a MQTT client, which subscribes to a MQTT topic in the form action/<actuator_id/#> to receive control commands. The actuator is only used for communication and does not execute any actions. 
+
+ - `dataModel.json`: This file contains a data model definition which can be used for the creation of a respective operator entity in the MBP.

resources/operators/control/simulators/actuator_adapter_testing/dataModel.json

@@ -0,0 +1,21 @@
+{
+  "name": "DM_actuator_testingTool",
+  "treeNodes": [
+    {
+      "name": "value",
+      "type": "double",
+      "parent": "RootObj",
+      "children": []
+    },
+    {
+      "name": "RootObj",
+      "description": "",
+      "type": "object",
+      "unit": "",
+      "parent": "",
+      "children": [
+        "value"
+      ]
+    }
+  ]
+}

resources/operators/extraction/Adafruit_DHT-temperature_raspberry-pi/Adafruit_DHT-temperature_raspberry-pi.py

@@ -129,7 +129,7 @@ def main(argv):
 
          # measured_temp = aiReader.getTemperature(temp_adc_channel)
 
-         msg_pub = {"component": component.upper(), "id": component_id, "value": "%.3f" % (measured_temp)}
+         msg_pub = {"component": component.upper(), "id": component_id, "value": {"temperature": measured_temp}}
          publisher.sendMessage (topic_pub, json.dumps(msg_pub))
 
          time.sleep(measureInterval)

resources/operators/extraction/Adafruit_DHT-temperature_raspberry-pi/README.md

@@ -17,4 +17,6 @@ This folder contains operator scripts to extract sensor data from a LKAdafruit D
 
  - `running.sh`: This file checks if the python script is running.
 
- - `stop.sh`: This file stops the execution of the python script.
+ - `stop.sh`: This file stops the execution of the python script.
+
+ - `dataModel.json`: This file contains a data model definition which can be used for the creation of a respective operator entity in the MBP.

resources/operators/extraction/Adafruit_DHT-temperature_raspberry-pi/dataModel.json

@@ -0,0 +1,21 @@
+{
+  "name": "DM_Adafruit_DHT-temperature_raspberry-pi",
+  "treeNodes": [
+    {
+      "name": "temperature",
+      "type": "double",
+      "parent": "RootObj",
+      "children": []
+    },
+    {
+      "name": "RootObj",
+      "description": "",
+      "type": "object",
+      "unit": "",
+      "parent": "",
+      "children": [
+        "temperature"
+      ]
+    }
+  ]
+}

resources/operators/extraction/LK-light_raspberry-pi/LK-light_raspberry-pi.py

@@ -24,7 +24,7 @@ def main(argv):
       # retrieve sensor value
       received_value = reader.read_light()
       # send data to the MBP
-      mbp.send_data(float(received_value))
+      mbp.send_data(json.dumps({"light-val": float(received_value)}))
       # waits a time interval before sending new data
       time.sleep(int(interval))
   except:

resources/operators/extraction/LK-light_raspberry-pi/README.md

@@ -33,4 +33,5 @@ Optional parameter:
 - `install.sh`: This file installs the necessary libraries to run the python script.
 - `start.sh`: This file starts the execution of the python script.
 - `running.sh`: This file checks if the python script is running.
-- `stop.sh`: This file stops the execution of the python script.
+- `stop.sh`: This file stops the execution of the python script.
+- `dataModel.json`: This file contains a data model definition which can be used for the creation of a respective operator entity in the MBP.

resources/operators/extraction/LK-light_raspberry-pi/dataModel.json

@@ -0,0 +1,21 @@
+{
+  "name": "DM_LK-light_raspberry-pi",
+  "treeNodes": [
+    {
+      "name": "light-val",
+      "type": "double",
+      "parent": "RootObj",
+      "children": []
+    },
+    {
+      "name": "RootObj",
+      "description": "",
+      "type": "object",
+      "unit": "",
+      "parent": "",
+      "children": [
+        "light-val"
+      ]
+    }
+  ]
+}

resources/operators/extraction/LK-sound_raspberry-pi/LK-sound_raspberry-pi.py

@@ -40,7 +40,7 @@ def main(argv):
       now = time.perf_counter()
       if(now - last_send_time >= interval):
         # send data to the MBP
-        mbp.send_data(avg(last_20_values))
+        mbp.send_data(json.dumps({"sound-avg-val": avg(last_20_values)}))
         last_send_time = time.perf_counter()
         last_20_values = []
       time.sleep(0.05)

resources/operators/extraction/LK-sound_raspberry-pi/README.md

@@ -25,4 +25,4 @@ Optional parameter:
  - `start.sh`: This file starts the execution of the python script.
  - `running.sh`: This file checks if the python script is running.
  - `stop.sh`: This file stops the execution of the python script.
-
+ - `dataModel.json`: This file contains a data model definition which can be used for the creation of a respective operator entity in the MBP.

resources/operators/extraction/LK-sound_raspberry-pi/dataModel.json

@@ -0,0 +1,21 @@
+{
+  "name": "DM_LK-sound_raspberry-pi",
+  "treeNodes": [
+    {
+      "name": "sound-avg-val",
+      "type": "double",
+      "parent": "RootObj",
+      "children": []
+    },
+    {
+      "name": "RootObj",
+      "description": "",
+      "type": "object",
+      "unit": "",
+      "parent": "",
+      "children": [
+        "sound-avg-val"
+      ]
+    }
+  ]
+}

resources/operators/extraction/LK-temperature_raspberry-pi/LK-temperature_raspberry-pi.py

@@ -24,7 +24,7 @@ def main(argv):
       # retrieve sensor value
       received_value = reader.read_temperature()
       # send data to the MBP
-      mbp.send_data(float(received_value))
+      mbp.send_data(json.dumps({"temperature": float(received_value)}))
       # waits a time interval before sending new data
       time.sleep(int(interval))
   except:

resources/operators/extraction/LK-temperature_raspberry-pi/README.md

@@ -25,4 +25,4 @@ Optional parameter:
  - `start.sh`: This file starts the execution of the python script.
  - `running.sh`: This file checks if the python script is running.
  - `stop.sh`: This file stops the execution of the python script.
-
+ - `dataModel.json`: This file contains a data model definition which can be used for the creation of a respective operator entity in the MBP.

resources/operators/extraction/LK-temperature_raspberry-pi/dataModel.json

@@ -0,0 +1,21 @@
+{
+  "name": "DM_LK-temperature_raspberry-pi",
+  "treeNodes": [
+    {
+      "name": "temperature",
+      "type": "double",
+      "parent": "RootObj",
+      "children": []
+    },
+    {
+      "name": "RootObj",
+      "description": "",
+      "type": "object",
+      "unit": "",
+      "parent": "",
+      "children": [
+        "temperature"
+      ]
+    }
+  ]
+}

resources/operators/extraction/README.md

@@ -11,14 +11,15 @@ The expected message structure by the MBP is a json-formatted string with the fo
 {
 	"component": "SENSOR", 
 	"id": "596cafaa6c0ccd5d29da0e90", 
-	"value": 20
+	"value": {"temperature:" 20}
 }
 ```
+The `value` key can have an arbitrary JSON object as value. However the structure of this JSON object must be registered to the MBP by creating a data model entity.
 
 The following shows a command line example using the paho MQTT client to send sensor data to the MBP: 
 
 ```bash
-$ mosquitto_pub.exe -t 'sensor/596cafaa6c0ccd5d29da0e90' -m '{"component":"SENSOR","id":"596cafaa6c0ccd5d29da0e90","value":20}'
+$ mosquitto_pub.exe -t 'sensor/596cafaa6c0ccd5d29da0e90' -m '{"component":"SENSOR","id":"596cafaa6c0ccd5d29da0e90","value": {"temperature:" 20}}'
 ```
 
 
@@ -29,6 +30,7 @@ $ mosquitto_pub.exe -t 'sensor/596cafaa6c0ccd5d29da0e90' -m '{"component":"SENSO
 
 - [Adafruit_DHT-temperature_raspberry-pi](Adafruit_DHT-temperature_raspberry-pi) :thermometer: :droplet: : scripts to extract sensor data from a DHT sensor. 
 - [bosch-xdk_mqtt-gateway](bosch-xdk_mqtt-gateway) :thermometer: :partly_sunny: :droplet: : operator scripts to extract sensor data from Bosch XDK devices.
+- [complex_iot_data_stub](complex_iot_data_stub) :sensor:: operator scripts using a more complex data model demonstrating the complex IoT data capabilities of the MBP.
 - [LK-light_raspberry-pi](LK-light_raspberry-pi) :partly_sunny:: operator scripts to extract sensor data from a LK light sensor.
 - [LK-temperature_raspberry-pi](LK-temperature_raspberry-pi) :thermometer:: operator scripts to extract sensor data from a LK temperature sensor.
 - [LK-sound_raspberry-pi](LK-sound_raspberry-pi) :microphone:: operator scripts to extract sensor data from a LK sound sensor.
@@ -38,4 +40,4 @@ $ mosquitto_pub.exe -t 'sensor/596cafaa6c0ccd5d29da0e90' -m '{"component":"SENSO
 
 Further folders:
 
-* [simulators](simulators): simulators of different sensors, which can generate sensor data for different events combined with anomalies. These simulators can be used for testing rule-based IoT applications to detect failures and take actions against them.
+* [simulators](simulators): simulators of different sensors, which can generate sensor data for different events combined with anomalies. These simulators can be used for testing rule-based IoT applications to detect failures and take actions against them.

resources/operators/extraction/bosch-xdk_mqtt-gateway/README.md

@@ -68,3 +68,5 @@ Example of a received JSON string:
 - `running.sh`: This file checks if the python script is running.
 
 - `stop.sh`: This file stops the execution of the python script.
+
+- `dataModel.json`: This file contains a data model definition which can be used for the creation of a respective operator entity in the MBP.

resources/operators/extraction/bosch-xdk_mqtt-gateway/bosch-xdk_mqtt-gateway.py

@@ -159,7 +159,7 @@ def main(argv):
 
          value = subscriber.getLastValue()
 
-         msg_pub = {"component": component.upper(), "id": component_id, "value": "%f" % (float(value)) }
+         msg_pub = {"component": component.upper(), "id": component_id, "value": {"value": float(value)}}
          publisher.sendMessage (topic_pub, json.dumps(msg_pub))
 
          time.sleep(5)

resources/operators/extraction/bosch-xdk_mqtt-gateway/dataModel.json

@@ -0,0 +1,21 @@
+{
+  "name": "DM_bosch-xdk_mqtt-gateway",
+  "treeNodes": [
+    {
+      "name": "value",
+      "type": "double",
+      "parent": "RootObj",
+      "children": []
+    },
+    {
+      "name": "RootObj",
+      "description": "",
+      "type": "object",
+      "unit": "",
+      "parent": "",
+      "children": [
+        "value"
+      ]
+    }
+  ]
+}

resources/operators/extraction/complex_iot_data_stub/README.md

@@ -0,0 +1 @@
+# Operator Scripts for testing the complex IoT data functionality of the MBP

resources/operators/extraction/complex_iot_data_stub/all_data_types/README.md

@@ -0,0 +1,21 @@
+# Complex IoT data test operator: Scripts to simulate a sensor sending all available MBP data types
+
+This folder contains operator scripts and a data model to deploy a sensor simulator which sends sensor values using all data types of the MBP.
+
+## Hardware Setup 
+
+ - A computer running a Linux-based OS, such as a Raspberry Pi or a Laptop running the Ubuntu OS. (Optionally, you can also use a virtual machine)
+
+## Operator files 
+
+ - `install.sh`: This file installs the necessary libraries to run the .py file.
+
+ - `start.sh`: This file starts the execution of the .py file.
+
+ - `running.sh`: This file checks if the .py file is running.
+
+ - `stop.sh`: This file stops the execution of the .py file.
+
+ - `test_all_mbp_data_types.py`: This python script is responsible for connecting to the MBP via MQTT and sending the sensor values.
+
+ - `dataModel.json`: This file contains a data model definition which can be used for the creation of a respective operator entity in the MBP.