-
Notifications
You must be signed in to change notification settings - Fork 35
Low Bandwidth Devices
Documented / instrumented tests of mwp and the INAV Configurator against low bandwidth devices. Tested with a F405 flight controller with a hardware FC UART. Hosted on Arch Linux x86_64, June 2024.
Note that these are artificial bench tests at short range and may not fully represent real world performance.
For the INAV Configurator tests, "wireless" mode was used as data integrity was a key requirement.
The test mission comprises 120 points, and the test simply involves:
- Upload the mission to the flight controller
- Download the mission from the flight controller
- Verify that the mission had been correctly "round-tripped"
- Record the times taken (and any errors).
Probably the worst serial device I own. Ridiculously small h/w buffers, very slow.
Tests are performed using mwp-ble-bridge -t to generate a TCP server, as the Configurator appears not to recognise BLE devices on Linux. This means all applications are using the same infrastructure to access the device (mwp can access the BLE device directly; using the bridge just makes it the same everywhere, including WSL access to the device).
$ mwp --version --build-id
7.135.494 7997d2bc (development)
14:35:54.078829 Start mission upload for 120 points
14:36:30.068279 mission uploaded for 120 points
So c. 36 seconds to upload the mission.
14:37:02.154259 Received nav_wp_multi_mission_index: 1
14:37:21.466484 Download completed #120 (0)
So c. 18 seconds to download the mission.
$ git rev-parse --short HEAD
b717b2b3
Master (8.0.0) as of 2024-05-14
Visually timed from clicking icon until icon becomes sensitive again, took 101 seconds.
Visually timed from clicking icon until icon becomes sensitive again, took 44 seconds.
Both applications coped with the very weak CC2451 device.
| Application | Upload | Download |
|---|---|---|
| mwp linux | 36s | 18s |
| mwp WSL | 37s | 20s |
| Configurator (Linux) | 101s | 44s |
The Configurator is consistently slower and both applications take longer to upload than download. The WSL overhead is insignificant, particularly considering that there is double virtualisation (Win11 in Linux KVM host running the WSL VM for Ubuntu 22.04).
A c. 2016 vintage small form factor / low bandwidth data radio.
Tested with a CP2102 USB to UART Bridge at 9600 baud.
$ mwp --version --build-id
7.135.494 7997d2bc (development)
15:21:04.409928 Start mission upload for 120 points
15:21:19.521475 mission uploaded for 120 points
So c. 15 seconds to upload the mission.
15:21:40.003163 Received nav_wp_multi_mission_index: 1
15:21:55.023486 Download completed #120 (0)
So c. 15 seconds to download the mission.
$ git rev-parse --short HEAD
b717b2b3
Master (8.0.0) as of 2024-05-14
Visually timed from clicking icon until icon becomes sensitive again, took 38 seconds.
Visually timed from clicking icon until icon becomes sensitive again, took 39 seconds.
A c. 2017 vintage LoRa transparent serial radio. One of the earlier small factor commercial LoRa radios, so probably not as efficient as more modern examples.
Tested via a CP2102 USB to UART Bridge at 9600 baud (9600 airspeed).
mwp --version --build-id
7.140.407 752b6ad7
15:50:18.048333 Start mission upload for 120 points
15:50:53.106723 mission uploaded for 120 points
So c. 35 seconds to upload the mission.
16:16:32.215014 Start download for 120 WP
16:16:32.215035 Received nav_wp_multi_mission_index: 1
16:17:07.119474 Download completed #120 (0)
So c. 35 seconds to download the mission.
$ git rev-parse --short HEAD
71817afc
Master (8.0.0) as of 2024-05-19
Visually timed from clicking icon until icon becomes sensitive again, took 142 seconds.
Visually timed from clicking icon until icon becomes sensitive again, took 141 seconds.
Both applications coped with the old LoRa ET45-TTL-100.
| Application | Upload | Download |
|---|---|---|
| mwp Linux | 35s | 35s |
| Configurator (Linux) | 142s | 141s |
The Configurator is again consistently slower. WSL was not tested, but based on earlier results, I would expect mwp / WSL to be similar to the Linux performance.
A more recent LoRa transparent serial radio.
Tested via a CP2102 USB to UART Bridge at 9600 baud (9600 airspeed).
Only mwp was tested, requiring 30s for both upload and download, an improvement over the older LoRa device.
The old faithfully. Rock solid, well buffered. 64k air speed / 115k ground speed via HC05 Bluetooth (legacy RFCOMM)
mwp was tested, requiring 15s for both upload and download.
The INAV Configurator was also tested, requiring 160s seconds for both upload and download.
Still the short range wireless champion, at least for mwp.
To establish some ground truth; a wired device using 2400 baud.
mwp was tested, requiring 20s for both upload and download.
The INAV Configurator was also tested, requiring 75s seconds for both upload and download.
So wired is faster at 2400 baud that many wireless options at notionally faster speeds.
| Device | Application | Upload (s) | Download (s) |
|---|---|---|---|
| CC2541 | mwp | 36s | 18s |
| CC2541 | mwp (WSL) | 37s | 20s |
| CC2541 | Configurator | 101s | 44s |
| HC-12 | mwp | 15s | 15s |
| HC-12 | Configurator | 39s | 38s |
| E45-TTL-100 | mwp | 35s | 35s |
| E45-TTL-100 | Configurator | 142s | 141s |
| E220-900T22D | mwp | 30s | 30s |
| E220-900T22D | Configurator | - | - |
| SiK (3DR) | mwp | 15s | 15s |
| SiK (3DR) | Configurator | 161s | 160s |
| pl2303 (2400) | mwp | 20s | 20s |
| pl2303 (2400) | Configurator | 76s | 75s |
- Both applications work with low bandwidth / inadequately buffered devices.
- No errors / timeouts / retries were shown in any test or application
- WSL performs acceptably. Little impact from either double virtualisation or
ser2udpserial bridge.
In all cases, the mission was correctly "round-tripped" through the FC.
To my surprise, Configurator 7.1.1 was able to complete the HC-12 test as well (only devices tested on 7.1.1):
| Device | Applciation | Upload (s) | Download (s) |
|---|---|---|---|
| HC-12 | Configurator 7.1.1 | 35s | 35s |
Faster than Configurator 8.0.0, slower than mwp.
In order to test the performance with a single message, the tool msp_testmsg was used. This application sends a single MSP request, logging the payload (typically for analysis).
It has a 30 second response timeout.
The following messages are used:
-
MSP2_ADSB_VEHICLE_LIST: Id 0x2090, fixed size of 152 bytes -
MSP_BOXNAMES: Id 0x74, Variable sized response, typically around 400 bytes, depending on FC configuration.
115200 ground speed, 64k air speed.
$ ./msp_testmsg -d /dev/rfcomm2 0x74
2024/05/29 11:24:40 Using device /dev/rfcomm2
INAV v8.0.0 MATEKF405 (49008f47) API 2.5
"MISS PIGGY"
MSP 0x74/116 returns OK with payload 392
$ ./msp_testmsg -d /dev/rfcomm2 0x2090
2024/05/29 11:24:54 Using device /dev/rfcomm2
INAV v8.0.0 MATEKF405 (49008f47) API 2.5
"MISS PIGGY"
MSP 0x2090/8336 returns OK with payload 152
As expected, no problem.
9600 ground and air speeds.
$ ./msp_testmsg -d /dev/ttyUSB0@9600 0x2090
2024/05/29 11:49:12 Using device /dev/ttyUSB0
INAV v8.0.0 WINGFC (27194727) API 2.5
"BENCHYMCTESTY"
MSP 0x2090/8336 returns OK with payload 152
$ ./msp_testmsg -d /dev/ttyUSB0@9600 0x74
2024/05/29 11:53:13 Using device /dev/ttyUSB0
INAV v8.0.0 WINGFC (27194727) API 2.5
"BENCHYMCTESTY"
MSP 0x74/116 returns OK with payload 435
No problem.
9600 ground and air speeds.
./msp_testmsg -d /dev/ttyUSB0@9600 0x2090
2024/05/29 13:03:11 Using device /dev/ttyUSB0
INAV v8.0.0 WINGFC (27194727) API 2.5
"BENCHYMCTESTY"
MSP 0x2090/8336 returns OK with payload 152
./msp_testmsg -d /dev/ttyUSB0@9600 0x74
2024/05/29 13:03:32 Using device /dev/ttyUSB0
INAV v8.0.0 WINGFC (27194727) API 2.5
"BENCHYMCTESTY"
MSP 0x74/116 returns OK with payload 435
No problem.
9600 ground and air speeds.
./msp_testmsg -d /dev/ttyUSB0@9600 0x2090
2024/05/29 13:08:30 Using device /dev/ttyUSB0
INAV v8.0.0 WINGFC (27194727) API 2.5
"BENCHYMCTESTY"
MSP 0x2090/8336 returns OK with payload 152
./msp_testmsg -d /dev/ttyUSB0@9600 0x74
2024/05/29 13:08:50 Using device /dev/ttyUSB0
INAV v8.0.0 WINGFC (27194727) API 2.5
"BENCHYMCTESTY"
MSP 0x74/116 returns OK with payload 435
No problem.
9600 UART speed, via mwp-ble-bridge as msp_testmsg is not BLE aware.
$ mwp-ble-bridge -k -a BleTest00
2024-05-29T13:13:06+0100 BLE chipset CC2541, mtu 23 (may not end well)
BleTest00 <=> /dev/pts/2
$ ./msp_testmsg -d /dev/pts/2 0x2090
2024/05/29 13:13:10 Using device /dev/pts/2
INAV v8.0.0 WINGFC (27194727) API 2.5
"BENCHYMCTESTY"
MSP 0x2090/8336 returns OK with payload 152
$ ./msp_testmsg -d /dev/pts/2 0x74
2024/05/29 13:13:29 Using device /dev/pts/2
INAV v8.0.0 WINGFC (27194727) API 2.5
"BENCHYMCTESTY"
Timeout reached, read 280 (272 payload)
I was surprised that this device survived the 0x2090 test. However, if the UART speed is set to 115200 baud. it fails the 0x2090 test as well.
Surprisingly good results for all tested devices. I had not really expected many devices other than the 3DR and E220 to complete these tests successfully.
There is a suite for LTM Soak Testing that provides a LTM sink and source (soak) generator. The soak generator emulates INAV LTM in rate and content and thus may be used in a bench test environment to validate low bandwidth devices handling of LTM using a desktop computer.
All the viable telemetry devices listed above (3DR, HC-12, Ebyte E45 LoRa , Ebyte E220 LoRa) can sustain LTM fast rate without issues.
The CC2451 cannot sustain LTM at any rate when the generator is directed to the BT side, but does (all rates) when the generator is directed to the serial side (as would be the usual usage cases). In any case, it's not a practical telemetry device.