Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

The next round of test work #3486

Open
wants to merge 11 commits into
base: dev
Choose a base branch
from
Open

The next round of test work #3486

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
11 commits
Select commit Hold shift + click to select a range
5fc1a36
tests: tests/README basic module requirements
nwf Dec 28, 2021
14f06e8
tests: introduce NTestEnv utility functions
nwf Dec 24, 2020
2b45361
tests: adc require running on DUT 0 using NTestEnv
nwf Dec 30, 2021
2f0cbfc
tests: Add script to make LFS for running tests
nwf Jun 17, 2020
9e5b352
tests: preflight script to grab host IP address
nwf Jun 17, 2020
42e67f4
tests: preflight-dut scripting
nwf Dec 31, 2020
9e4c0d5
tests: Slam docs together
nwf Dec 30, 2021
74554dd
expect test framework: net TCL module
nwf Dec 31, 2019
219b116
tests: add preflight TLS script
nwf Dec 30, 2021
742a041
expect tests: mqtt module
nwf Feb 2, 2020
2144d26
tests: top-level runner
nwf Dec 31, 2020
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
104 changes: 17 additions & 87 deletions tests/HardwareTestHarness.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -48,43 +48,13 @@ Peripherals

### I2C Bus

There is an I2C bus hanging off DUT 0. Attached hardware is used both as
tests of modules directly and also to facilitate testing other modules
(e.g., gpio).

Most of the positions on the board are connected to the DUT1 I2C bus.

#### MCP23017: I/O Expander

At address 0x20. An 16-bit tristate GPIO expander, this chip is used to
test I2C, GPIO, and ADC functionality. This chip's interconnections are
as follows:

MPC23017 | Purpose
---------|--------------------------------------------------------------
/RESET |DUT0 reset. This resets the chip whenever the host computer resets DUT 0 over its serial link (using DTR/RTS).
B 0 |4K7 resistor to DUT 0 ADC.
B 1 |2K2 resistor to DUT 0 ADC.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

again B2,3 and 4 are not described in the readme

B 2 |Direct to DUT1 RST
B 3 |Direct to DUT1 D3
B 4 |When low, connects the alternate UART pins on DUT0 to RX,TX on DUT1
B 5 |DUT1 GPIO16/WAKE via 4K7 resitor
B 6 |DUT0 GPIO13 via 4K4 resistor and DUT1 GPIO15 via 4K7 resistor (also feeds in the primary TX from DUT1 when enabled by B4)
B 7 |DUT0 GPIO15 via 4K7 resistor and DUT1 GPIO13 via 4K7 resistor (also feeds the primary RX on DUT1 when enabled by B4)

Notes:

- DUT 0's ADC pin is connected via a 2K2 reistor to this chip's port
B, pin 1 and via a 4K7 resistor to port B, pin 0. This gives us the
ability to produce approximately 0 (both pins low), 1.1 (pin 0 high,
pin 1 low), 2.2 (pin 1 high, pin 0 low), and 3.3V (both pins high)
on the ADC pin.
- Port B pins 6 and 7 sit on the UART cross-wiring between DUT 0 and
DUT 1. The 23017 will be tristated for inter-DUT UART tests, but
these
- All of port A, remain available for expansion and are routed to the breadboard area.

#### WS2812s
#### MCP23017: I/O Expander (DUT 0, 0x20)

As per the [Generic Framework](README.md).

### WS2812s

There are three WS2812s connected on DUT1/D4. The last Ws2812 is positioned so that a TCS34725 module
can be mounted upside down over it to read out the color of the WS2812. That device is connected to
Expand All @@ -94,22 +64,34 @@ LED is connected to the INT pin and so you can disable the LED under software co
#### Oled Displays

Each of the D1 Minis is connected to a position for a 128x64 OLED display, again on the primary I2C bus.
Note: Not yet used in the test harness.

#### Servo

On DUT1 pin D4/GPIO 2 there is a connection to a position for a small servo. The servo is powered by the
5V voltage rail.

Note: Not yet used in the test harness.

#### DHTxx

On DUT1 pin D6/GPIO 12 there is a connection to a position for a DHTxx device. The silk screen indicates the
orientation of the device.

Note: Not yet used in the test harness.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

you mean the software test harness? This is the documentation of the HWTH


#### DS18B20

There are two positions for DS18B20s -- one with the VCC pin connected and one without. The data pin is
connected to DUT1 pin D5/GPIO 14.

Note: Not yet used in the test harness.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

meaning no tests available yet?


Note: It would make sense to augment the 1-Wire testing facility to
include bus-drive power, perhaps via the MCP23017, especially if we ever
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As states above there are already 2 devices possible, one without VCC connected which then would be bus-driven

augment the driver as per
https://github.com/nodemcu/nodemcu-firmware/issues/1995

#### I2C devices with VCC/GND/SCL/SDA pin order

There are three positions for I2C devices which have the pins in the VCC/GND/SCL/SDA order. These
Expand All @@ -121,61 +103,9 @@ There are three positions for I2C devices with other pin orders. Each of these p
to a crossbar switch and so four blobs of solder can configure each of these headers into any
desired pin order. As far as I can tell, most of the cheap modules use the VCC/GND/SCL/SDA order.


Breadboard Area
===============

All the pins on each D1 Mini and the A port of the MCP23017 are brought out to a breadboard
area. This can be used to solder components and/or wires, or even a header could be soldered
on to transfer all the signals to a conventional breadboard.


ESP8266 Device 0 Connections
----------------------------
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The description here is more exact and exceeds the spec in the readme, so it should be kept.
e.g.; D4/5 of DUT 1 is not described at all.
Also in general the setup might evolve in the readme but this document describes a hardware version, which will remain unchanged (else it will be a v2)


ESP | Usage
----------|----------------------------------------------------------
D3/GPIO 0 |Used to enter programming mode; otherwise unused in test environment.
TX/GPIO 1 |Primary UART transmit; reserved for host communication
D4/GPIO 2 |[reserved for 1-Wire] [+ reserved for 23017 INT[AB] connections]
RX/GPIO 3 |Primary UART recieve; reserved for host communication
D2/GPIO 4 |I2C SDA. Connected to MCP23017, Oled display and the TCS34725 if present.
D1/GPIO 5 |I2C SCL
GPIO 6 |[Reserved for on-chip flash]
GPIO 7 |[Reserved for on-chip flash]
GPIO 8 |[Reserved for on-chip flash]
GPIO 9 |[Reserved for on-chip flash]
GPIO 10 |[Reserved for on-chip flash]
GPIO 11 |[Reserved for on-chip flash]
D6/GPIO 12 |
D7/GPIO 13 |Secondary UART RX; DUT 1 GPIO 15, I/O expander B 6
D5/GPIO 14 |
D8/GPIO 15 |Secondary UART TX; DUT 1 GPIO 13, I/O expander B 7
D0/GPIO 16 |
A0/ADC 0 |Resistor divider with I/O expander

ESP8266 Device 1 Connections
----------------------------

ESP | Usage
----------|----------------------------------------------------------
D3/GPIO 0 |Used to enter programming mode; otherwise unused in test environment.
TX/GPIO 1 |Primary UART transmit; reserved for host communication
D4/GPIO 2 |Connected to chain of 3 WS2812s. Also connected to the servo position.
RX/GPIO 3 |Primary UART recieve; reserved for host communication
D2/GPIO 4 |I2C SDA. Connected to all the other I2C positions on the board
D1/GPIO 5 |I2C SCL
GPIO 6 |[Reserved for on-chip flash]
GPIO 7 |[Reserved for on-chip flash]
GPIO 8 |[Reserved for on-chip flash]
GPIO 9 |[Reserved for on-chip flash]
GPIO 10 |[Reserved for on-chip flash]
GPIO 11 |[Reserved for on-chip flash]
D6/GPIO 12 |Connected to data pin for DHTxx
D7/GPIO 13 |Secondary UART RX; DUT 0 GPIO 15, I/O exp B 7 via 4K7 Also used as HSPI MOSI for SPI tests
D5/GPIO 14 |Connected to data pin for DS18B20s.
D8/GPIO 15 |Secondary UART TX; DUT 0 GPIO 13, I/O exp B 6 via 4K7 Also used as HSPI /CS for SPI tests
D0/GPIO 16 |I/O expander B 5 via 4K7 resistor, for deep-sleep tests
A0/ADC 0 |


10 changes: 6 additions & 4 deletions tests/NTest_adc_env.lua
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,21 +4,23 @@
local N = ...
N = (N or require "NTest")("adc-env")

-- TODO: Preflight test that we are in the correct environment with an I2C
-- expander in the right place with the right connections.
local NTE = require "NTestEnv"

-- TODO: Use the mcp23017 module in the main tree rather than hand-coding
-- the commands

N.test('setup', function()
-- Configure the ADC
-- Ensure that we're on DUT 0
assert(NTE.getFeat('DUT') == 0, "Not on DUT 0")

-- Configure the ADC (this implicitly checks for having the ADC module)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Early checks with distinct error message are a nice thing to have. So since we now have respective methods we should use them. Maybe even have such checks for basic modules as sjson before using them in NTestEnv

if adc.force_init_mode(adc.INIT_ADC)
then
node.restart()
error "Must reboot to get to ADC mode"
end

-- Configure the I2C bus
-- Configure the I2C bus (again, implicitly testing...)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

again ... ;-)

i2c.setup(0, 2, 1, i2c.FAST)

-- Set the IO expander port B to channels 0 and 1 as outputs
Expand Down
110 changes: 73 additions & 37 deletions tests/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,18 @@ NodeMCU DUTs, but there is reason to want to orchestrate DUTs and the
environment using the host. Files matching the glob `NTest_*.lua` are intended
for on-DUT execution.

## Modules Required

Basic test harness functionality depends on having at least the following C
modules available:

* `crypto`
* `encoder`
* `file`

Of course, most of the test functionality itself will require additional
modules.

## Manual Test Invocation

At the moment, the testing regime and host-based orchestration is still in
Expand Down Expand Up @@ -124,30 +136,34 @@ devices, as found on almost all ESP8266 boards with USB to UART
adapters, but the host does not necessarily need to use USB to connect,
so long as TXD, RXD, DTR, and RTS are wired across.

A particular implementation of this can be found at [Test Harness](HardwareTestHarness.md).
A particular implementation of this, with some additional goodies, can
be found at [Test Harness Board](HardwareTestHarness.md).

## Peripherals

### I2C Bus

There is an I2C bus hanging off DUT 0. Attached hardware is used both as
tests of modules directly and also to facilitate testing other modules
(e.g., gpio).
There is an I2C bus hanging off each DUT. Attached hardware is used both
as tests of modules directly and also to facilitate testing other
modules (e.g., gpio).

#### MCP23017: I/O Expander
#### MCP23017: I/O Expander (DUT 0, 0x20)

At address 0x20. An 16-bit tristate GPIO expander, this chip is used to
test I2C, GPIO, and ADC functionality. This chip's interconnections are
as follows:
At address 0x20 off DUT 0. An 16-bit tristate GPIO expander, this chip
is used to test I2C, GPIO, and ADC functionality. This chip's
interconnections are as follows:

MPC23017 | Purpose
---------|--------------------------------------------------------------
/RESET |DUT0 reset. This resets the chip whenever the host computer resets DUT 0 over its serial link (using DTR/RTS).
B 0 |4K7 resistor to DUT 0 ADC.
B 1 |2K2 resistor to DUT 0 ADC.
B 2 |Direct to DUT1 RST
B 3 |Direct to DUT1 D3
B 4 |When low, connects the alternate UART pins on DUT0 to RX,TX on DUT1 (NOTE: not yet used in the test harness)
B 5 |DUT1 GPIO16/WAKE via 4K7 resitor
B 6 |DUT0 GPIO13 via 4K7 resistor and DUT1 GPIO15 via 4K7 resistor
B 7 |DUT0 GPIO15 via 4K7 resistor and DUT1 GPIO13 via 4K7 resistor
B 6 |DUT0 GPIO13 via 4K4 resistor and DUT1 GPIO15 via 4K7 resistor (also feeds in the primary TX from DUT1 when enabled by B4)
B 7 |DUT0 GPIO15 via 4K7 resistor and DUT1 GPIO13 via 4K7 resistor (also feeds the primary RX on DUT1 when enabled by B4)

Notes:

Expand All @@ -156,39 +172,48 @@ Notes:
ability to produce approximately 0 (both pins low), 1.1 (pin 0 high,
pin 1 low), 2.2 (pin 1 high, pin 0 low), and 3.3V (both pins high)
on the ADC pin.

- Port B pins 6 and 7 sit on the UART cross-wiring between DUT 0 and
DUT 1. The 23017 will be tristated for inter-DUT UART tests, but
these
- Port B pins 2, 3, and 4, as well as all of port A, remain available
for expansion.
- The interrupt pins are not yet routed, but could be. We reserve DUT
0 GPIO 2 for this purpose with the understanding that the 23017's
interrupt functionality will be disabled (INTA, INTB set to
open-drain, GPINTEN set to 0) when not explicitly under test.
these are also useful for GPIO tests on DUT1.

- All of port A remains available for expansion and are routed to the
breadboard area.

- The interrupt pins are not yet routed to DUTs, but could be. We
reserve DUT 0 GPIO 2 for this purpose with the understanding that
the 23017's interrupt functionality will be disabled (INTA, INTB
set to open-drain, GPINTEN set to 0) when not explicitly under test.

### WS2812s

The test harness reserves DUT1/D4 for connection to WS2812.

# DUT Connections

ESP8266 Device 0 Connections
----------------------------

ESP | Usage
----------|----------------------------------------------------------
GPIO 0 |Used to enter programming mode; otherwise unused in test environment.
GPIO 1 |Primary UART transmit; reserved for host communication
GPIO 2 |[reserved for 1-Wire] [+ reserved for 23017 INT[AB] connections]
GPIO 3 |Primary UART recieve; reserved for host communication
GPIO 4 |I2C SDA
GPIO 5 |I2C SCL
GPIO 6 |[Reserved for on-chip flash]
GPIO 7 |[Reserved for on-chip flash]
GPIO 8 |[Reserved for on-chip flash]
GPIO 9 |[Reserved for on-chip flash]
GPIO 10 |[Reserved for on-chip flash]
GPIO 11 |[Reserved for on-chip flash]
GPIO 12 |
GPIO 13 |Secondary UART RX; DUT 1 GPIO 15, I/O expander B 6
GPIO 14 |
GPIO 15 |Secondary UART TX; DUT 1 GPIO 13, I/O expander B 7
GPIO 16 |
ADC 0 |Resistor divider with I/O expander
ESP | Usage
-----------|----------------------------------------------------------
D3/GPIO 0 |Used to enter programming mode; otherwise unused in test environment.
TX/GPIO 1 |Primary UART transmit; reserved for host communication
D4/GPIO 2 |(+ reserved for 23017 INT[AB] connections)
RX/GPIO 3 |Primary UART recieve; reserved for host communication
D2/GPIO 4 |I2C SDA bus 0. Connected to MCP23017, DUT0 Oled display and the TCS34725 if present.
D1/GPIO 5 |I2C SCL bus 0
GPIO 6 |(Reserved for on-chip flash)
GPIO 7 |(Reserved for on-chip flash)
GPIO 8 |(Reserved for on-chip flash)
GPIO 9 |(Reserved for on-chip flash)
GPIO 10 |(Reserved for on-chip flash)
GPIO 11 |(Reserved for on-chip flash)
D6/GPIO 12 |
D7/GPIO 13 |Secondary UART RX; DUT 1 GPIO 15, I/O expander B 6
D5/GPIO 14 |
D8/GPIO 15 |Secondary UART TX; DUT 1 GPIO 13, I/O expander B 7
D0/GPIO 16 |
A0/ADC 0 |Resistor divider with I/O expander

ESP8266 Device 1 Connections
----------------------------
Expand All @@ -214,4 +239,15 @@ GPIO 15 |Secondary UART TX; DUT 0 GPIO 13, I/O exp B 6 via 4K7 Also used as HS
GPIO 16 |I/O expander B 5 via 4K7 resistor, for deep-sleep tests
ADC 0 |

# Probing the Test Environment from Tests

The `NTestEnv` module provides convenient functions for preflight checks
and limited adaptation of test programs.

## Test Configuration File

Our tests expect a `testenv.conf` in SPIFFS to provide parameters to
some tests. This file is a JSON map with the following keys:

- "DUT". Its value is either 0 or 1 indicating which DUT is running the
given test.
Loading