README

NDI/VISCA PTZ Controller for Raspberry Pi and Rock Pi

This code is a complete, but basic console-based pan-tilt-zoom controller for the Raspberry Pi 4
and Rock Pi 4.  (In other words, this does not run in X11; it literally writes directly to the
framebuffer.)

---------
Features:
---------

* Requires the Pimoroni I/O Expander for input purposes.
* Supports three-axis analog joysticks.
* Supports five pushbuttons for loading stored positions, and a set button for storing new positions.
* Supports lights for showing which position was most recently loaded (reset when you move the camera)
  and for the set button (toggle) state.
* Supports remote tally light polling to show whether the camera is in preview mode, program mode,
  inactive, or unresponsive (e.g. a network failure).  (Requires a VISCA-compatible camera.)
* Supports zooming at variable speed.
* Supports both VISCA control and NDI control for maximum flexibility.


-------------------
A Note About VISCA:
-------------------

VISCA is a protocol for controlling PTZ cameras over IP.  Almost every NDI camera that I have tested so far
supports VISCA, though some devices may require you to use a different port.  This code has been tested
against cameras by AVKANS, NewTek, and Marshall.  Your mileage may vary.  Panasonic uses its own protocol,
so those are not supported unless they also supports NDI control.

The reason VISCA is required for full functionality is that the NDI protocol doesn't provide some
required features.  I've filed bugs with the NDI SDK team about some of these issues, and will continue
to try to find ways to eliminate the need for VISCA, because it's kind of a bloated, unreliable mess
(and this code is a particularly hackish implementation that I wrote very quickly to work around
those limitations).

Specifically, the following functionality does not work when working with a pure NDI camera:

* Tally light support.  The NDI SDK lets you set the tally light, but not obtain its current state.
  If you're using OBS, it would theoretically be possible to add OBS websocket support to this app,
  which would allow you to find out whether OBS has turned on the tally light or not.  However, all
  of my cameras support VISCA, so it wasn't worth the effort to do that when I already had a
  mostly working VISCA implementation.

* Zoom speed support.  This should work; the NDI SDK lets you pass a zoom speed value from -1.0 to 1.0.
  However, in real-world testing, exactly none of the cameras I tested were able to zoom at multiple
  speeds when driven through that function call (from either the Linux/Raspbery Pi SDK or the macOS SDK).
  Presumably, this will get fixed at some point on the camera side, at which point VISCA will no longer
  be required for this purpose.

* Exposure adjustment and manual exposure.  The NDI SDK lacks exposure compensation APIs, though it
  does provide manual exposure settings.  But because all my cameras support VISCA, it wasn't worth
  the extra effort to hook up the manual exposure calls via NDI.  (It's would take only about two
  lines of code, though, so if you need it, it should be an easy change.)

This PTZ controller software looks up the camera using Avahi, then connects to the VISCA UDP port on
that same device.  This may or may not be the best approach, but it works (with the caveat that you
have to provide the VISCA port).

This software can also be used with my VISCAPTZ software running on a second device, which provides
pan and tilt support and position recall on custom motor-based hardware, along with zoom control
for Panasonic cameras and tally light sourcing from Panasonic cameras (if connected by NDI to a
source that sets the tally light properly) or directly from OBS or Tricaster mixing systems.
(See https://github.com/dgatwood/VISCAPTZ for info.)


--------------
Prerequisites:
--------------

* Download my C translation of the Pimoroni I/O Expander library:

  https://github.com/dgatwood/ioe-c

  and symlink or copy the files constants.h and ioexpander.c into the
  source folder.

* Disable X11 in raspi-config (Raspberry Pi) or equivalent.

* Install the NDI SDK from NewTek (available separately) and move the resulting folder
  to /usr/local/NDISDK (or edit the Makefile for a different location).

* Install Avahi (client and common libraries), which is typically one of the following:

        sudo apt-get install avahi-utils avahi-daemon libavahi-client-dev libnss-mdns
        sudo apt-get install avahi-client3 avahi-daemon libavahi-client-dev libnss-mdns

* Install the correct GPIO library:
  - For Raspberry Pi, install pigpiod and enable the daemon.  (libmraa may also work.)
  - For Rock Pi, install libmraa with apt:

        sudo apt-get install libmraa-rockpi4 or
        sudo apt-get install libmraa

    or build and install it from source code (https://github.com/eclipse/mraa).

* Configure dhcpcd on your device to use a static IP fallback in the same IP range as your cameras
  (typically 192.168.100.x).  For example, on Raspberry Pi:

        profile static_eth0
        static ip_address=192.168.100.23/24
        static routers=192.168.100.1
        static domain_name_servers=192.168.100.1
        interface eth0
        fallback static_eth0

* Add NDI to the ld.so configuration file:

  - Create a file named /etc/ld.so.conf.d/ndi.conf and paste in one of the following:

        /usr/local/NDISDK/lib/arm-rpi4-linux-gnueabihf
        /usr/local/NDISDK/lib/aarch64-rpi4-linux-gnueabi

    depending on your CPU architecture, adjusting the path as needed based on where you
    installed the SDK.  Then run

        sudo ldconfig

    to update the cache.

* Follow the hardware-specific configuration instructions for your board below.

* Build the custom hardware.


--------------------------
Raspberry Pi Configuration
--------------------------

* Configure the console to use 32 bpp if necessary (Raspberry Pi).  In config.txt, add/change:

        disable_overscan=0
        framebuffer_depth=32
        display_rotate=2
        dtparam=i2c_arm=on
        dtparam=spi=on

* For HDMI panels (Raspberry Pi).  In config.txt, add/change:

        max_usb_current=1
        hdmi_force_hotplug=1
        hdmi_group=1
        hdmi_mode=16
        hdmi_drive=2

* For DSI panels (Raspberry Pi).  In config.txt, add/change:

        video=DSI-1:1920x1080@60
        framebuffer_width=1920
        framebuffer_height=1080

  If this doesn't work, try 1920x1080@30.

* Disable the videocore 3D driver (Raspberry Pi) by commenting out the following line
  in config.txt:

        # dtoverlay=vc4-fkms-v3d

  This may or may not be necessary.  It was in the version of Raspbian that I was using,
  without which I couldn't force the display into 24bpp or the correct resolution, and
  I was always doing software scaling, which isn't ideal.  But YMMV.

* Disable the blinking cursor on the virtual console that you plan to use by editing
  cmdline.txt.  Change the console to:

        console=tty3

  to ensure the output doesn't get clobbered by log messages, then add:

        vt.global_cursor_default=0

  to the end.  Then disable console blanking so that your screen doesn't go black by adding

        consoleblank=0

  to that line as well.

---------------------
Rock Pi Configuration
---------------------

I recommend starting with the official Debian distro, being sure to get the Bullseye version.
Unfortunately, I was unable to get DSI working on Armbian, so I gave up.  And the official
Ubuntu Server distro has nonfunctional Wi-Fi, or at least I only got it to connect twice,
with hundreds of failures in between.

On Rock Pi, you must also perform the following changes:

* Uncomment the following line in /etc/sysctl.conf to keep the console clean of
  kernel log messages:

        kernel.printk = 3 4 1 3

* Add the following to your /etc/rc.local:

        # Disable the flashing blue heartbeat LED that sinks so much power that your LEDs will
        # severely flash twice a second if you don't.
        echo none > /sys/class/leds/user-led2/trigger
        echo none > /sys/class/leds/status/trigger

  One of those two lines should work, depending on kernel version.

* Downgrade to the 5.10.x kernel for HDMI or 4.4.154-90 for DSI.  The 5.15.x kernel does not
  support GPIO correctly (10 pins are nonfunctional), and 5.10.x does not support DSI.

* Enable I2C
  - Edit the file /boot/hw_intfc.conf as root.
  - Change:

        intfc:i2c7=off

    to:

        intfc:i2c7=on

* Enable DSI (if not using HDMI)

  - Edit the file /boot/hw_intfc.conf as root and uncomment this line:

        intfc:dtoverlay=raspberrypi-7-inch-lcd

  - Edit the file extlinux/extlinux.conf as root.  In each of the "append" lines, add:

        append ...existing arguments here... vt.global_cursor_default=0 fake_vsync_isr=1 consoleblank=0

    all on one line.

  - If there's a console=tty1 in that line, change it to tty3 so that log messages won't clobber
    the video output.

  - There's no way to rotate the display in hardware on Rock Pi (that I could find), so you'll have
    to add the -F flag when running the tool if you need to flip the panel's output.

* If you want to run as a non-root user:
  - Create a GPIO group and add yourself to it:

        sudo addgroup gpiouser
        sudo usermod -a -G gpiouser yourusername

  - Add yourself to the i2c group (or whatever group owns /dev/i2c-7).

        sudo usermod -a -G i2c yourusername

  - Create a file called /etc/udev/rules.d/99-gpio.rules and paste the following:

        SUBSYSTEM=="gpio", PROGRAM="/bin/sh -c '\
          chown -R root:gpiouser /sys/devices/platform/pinctrl/gpio /sys/class/gpio &&\
          chmod -R 770 /sys/devices/platform/pinctrl/gpio /sys/class/gpio'"
        KERNEL=="gpio*", GROUP="gpiouser"
        KERNEL=="gpiochip*", GROUP="gpiouser"

  - Add the following to /etc/rc.local:

        # Change ownership of various stuff so that you can touch GPIO as a non-root user.
        chown -R root:gpiouser /sys/class/gpio
        chmod -R ug+rw /sys/class/gpio

  - Reboot.

-------------------
Command-line flags:
-------------------

To see a list of available cameras, type ./cameracontroller and press return.  This currently
loops forever, because cameras may not appear instantly.  Press Control-\ to terminate the app.
We should probably fix this at some point.

Usage:

    ./cameracontroller [flags] <camera_name>

where <camera_name> is typically something like "NDIHX-PTZUHD (chan 1, 192.168.100.168)".

Note that this tool searches for a camera with a matching name and IP address first, but then
falls back to a name match at a different IP.  Be certain that each camera has its own
distinct name, because there is no guarantee that this software will correctly detect the camera
in the initial scan before falling back and matching against identically named cameras at
different IP addresses.

General flags:

  -f / --fast                   -- Enables preview mode, in which a lower quality (typically 720p)
                                   stream is requested.  This can be helpful for cameras that
                                   produce higher bitrate streams.  (You can play NDI-HX streams
                                   from iOS devices without this flag.)

  -F / --flip                   -- Flips the screen while drawing.  Proper framebuffer flipping is
                                   better, but this is all we have on some platforms.

  -D / --duty_cycle             -- Sets the duty cycle that should be used for all LEDs
                                   (range 0 to 255).

VISCA command-line flags:

  -V / --enable_visca           -- Enables VISCA for camera control.  To enable visca for tally
                                   light purposes while using NDI camera control, set a port
                                   with -p, but don't pass -V.
  -u / --visca_use_udp          -- Enables UDP mode for VISCA and enables VISCA for tally light
                                   reporting.  At this point, TCP support is untested, so you
                                   should always pass this flag.
  -p / --visca-port             -- Sets the VISCA port and enables VISCA for tally light reporting.
                                   Most cameras use port 52381 UDP.  However, this code defaults to
                                   the PTZOptics port, 1259 UDP.  Note that PTZOptics cameras are
                                   entirely untested.
  -O / --onscreenlights         -- Configures the code to use on-screen boxes instead of physical
                                   status LEDs.  (Note that some status features are available
                                   only with VISCA.)

VISCA-only features:

  -e / --exposure_compensation  -- Sets exposure compensation amount (-5 to 5)
  -a / --auto_exposure          -- Enables automatic exposure (disables -i/-ig/-s).
  -i / --iris                   -- Sets manual exposure with the specified iris (range 0 to 20).
                                   The meaning of these values is camera-dependent.
  -g / --gain                   -- Sets manual exposure with the specified gain (range 0 to 15).
                                   The meaning of these values is camera-dependent.
  -s / --shutter                -- Sets manual exposure with the specified shutter (range 0 to 21).
                                   The meaning of these values is camera-dependent.

Debugging:

  -d / --debug                  -- Enables some basic debugging
  -B / --buttondebug            -- Enables button-specific debugging
  -P / --ptzdebug               -- Enables PTZ (joystick) debugging
  -v / --verbose                -- Enables more detailed debugging


--------------------
Other Configuration:
--------------------

The file LEDConfiguration.h lets you change the brightness of each
LED individually.

----------------
Common Mistakes:
----------------

If the tool can see your camera but cannot receive video:

1.  Reboot the camera.  Some cameras have known bugs.
2.  Make sure your IPv4 address is working.  Most cameras support IPv6 for
    discovery, but do NOT support streaming over IPv6.
3.  In particular, make sure that you aren't sharing an IP address with
    another camera or PTZ controller!
4.  Make sure you aren't running any weird firewalls that might interfere.


----------------------
Building the Hardware:
----------------------

The custom hardware construction (for actual PTZ control) is documented here:

https://docs.google.com/document/d/1Ss_mx8vgRw_ys8ocJK7LZhjXRocf9EI3qovyQXoxxJw/edit?usp=sharing

or in the hardware directory.

Enjoy!