feat(scorch): new components, and updates to existing components

New Components

- collector: Collects and processes data from cyber-physical experiments
  and Scorch components
- iperf: Collects network performance measurements using
  [iperf3](https://github.com/esnet/iperf) or
  [rperf](https://github.com/opensource-3d-p/rperf)
- pcap: Collects PCAPs from VMs via minimega's `capture pcap` API
- qos: Apply Quality of Service (QoS) effects on network interfaces,
  including dropping packets (`loss`), delaying packets (`delay`), or
  limiting bandwidth (`rate`)
- rtds: Collects and verifies data from the Real-Time Dynamic Simulator
  (RTDS)
- disruption: Orchestrates and executes disruption scenarios for
  cyber-physical experiments

Updated Components

- tcpdump
  - Improve README
- cc
  - Improve README
- hoststats
  - Added README
  - Minor code tweaks and logging
- vmstats
  - Added README
  - Optimized miniccc commands by using minimega tags (`mm vm tag`).
    This enables a command to be farmed out to a bunch of VMs with one
    minimega command, instead of dozens or hundreds of commands.
  - Added check if VMs are compatiable. This means Linux VMs that aren't
    routers, since the `vmstat` in VyOS is older and has a different
    command syntax (and is missing the timestamp feature).
  - If `vms` isn't specified, the default is now to run on all
    compatiable VMs. Previously, it would run on *all* VMs, even
    Windows.
  - Improved logging

General Updates

- setup.py
  - Add dependencies: elasticsearch, requests, dateutil
  - Change minimum Python version to 3.8 (from 3.7). Given the current
    phenix image, this can be 3.9 or even 3.10 really, but leaving it at
    3.8 for now to avoid making anyone mad.
  - Fixup metadata
- SCEPTRE app (sceptre.py)
  - Add simulator type comparisons to `sceptre_start.mako`. This fixes
    an issue where it was matching based on the hostname and not the
    simulator type. This affected simulink provider, where it would only
    work if the provider VM had 'simulink-provider' in the name.
  - If groundtruth isn't configured (`metadata.gt`), then don't start
    `simulinkgt` in `sceptre-start.sh`
  - Generalize creation of `sceptre-start.sh/.ps1` in a new function,
    `render_sceptre_start()`. This greatly reduces code duplication, and
    fixes a few minor issues as well (like attempting to set files as
    executable while they were still open).
  - General code cleanup of sceptre.py
- phenix_apps and scorch
  - New setting: `PHENIX_TEMP_DIR` (defaults to `/tmp/phenix`)
  - New setting: `MM_FILEPATH` (defaults to `/phenix/images`)
  - Refactor minimega file receive util function to allow multiple files
    to be transferred using the same miniccc mount
  - Improve minimega util functions: improve performance, fixed bugs,
    and added new functions
  - Many new util functions in `phenix_apps/common/utils.py` for
    Elasticsearch, minimega, PCAP wrangling, timestamps, JSON
    read/writes, and many more.
  - Lots of code cleanup in phenix_apps
  - Add type annotations to many functions

.editorconfig

@@ -11,6 +11,16 @@ charset = utf-8
 insert_final_newline = true
 trim_trailing_whitespace = true
 
+[*.rst]
+indent_size = 3
+insert_final_newline = true
+trim_trailing_whitespace = true
+
+[*.md]
+indent_size = 2
+insert_final_newline = true
+trim_trailing_whitespace = true
+
 [*.{yml,yaml}]
 indent_size = 2
 insert_final_newline = true
@@ -27,6 +37,14 @@ end_of_line = crlf
 [*.{xml,xslt}]
 indent_size = 2
 
-[*.md]
+[tox.ini]
+insert_final_newline = true
+trim_trailing_whitespace = true
+
+[Dockerfile*]
 indent_size = 2
-trim_trailing_whitespace = true
+insert_final_newline = true
+
+[*.toml]
+insert_final_newline = true
+trim_trailing_whitespace = true

.gitignore

@@ -0,0 +1,17 @@
+.venv
+
+# Editors
+.vscode
+.idea
+
+# pyenv
+.python-version
+
+*.egg-info
+__pycache__
+
+build
+dist
+
+.DS_Store
+.vagrant

src/python/.gitignore

@@ -2,3 +2,4 @@
 __pycache__
 build
 dist
+.venv