Expunge PYOP2_NO_FORK_AVAILABLE

[connorjward]

See discussion.

[wence-]

I'll just note in passing that although this hasn't shown up as an issue anywhere, at large scale, infiniband network stacks are typically still fragile in the presence of fork after network bringup.

This workaround in pyop2 was never really a full solution anyway, so removing it would be fine.

To solve this generally, one needs to arrange that fork calls are made in a process that is "clean". pytools has a solution for this which is to spawn a "forkserver" before anything else happens. That is, you fork before network stack bringup and use that process to do all of your subsequent process forking.

As noted in that discussion, there are many places where the firedrake software stack calls subprocess.Popen. To fix this, all of those would have to be audited and fixed to use the prefork approach (possibly by importing and monkey-patching subprocess).

@inducer Do you still use prefork (and is it easy to do so with loopy)?

[inducer]

I still maintain it, just in case. As for use, the story is a bit complex. With OpenCL, the JIT is mostly in-process, however https://github.com/pocl/pocl/ does fork a linker AFAIK. None of this appears to have been a source of problems so far on Lassen (a DOE machine) that we run things on.

As for Loopy, if you use it to do the (C-based) JIT'ing, you'll be using codepy, which internally already calls into pytools.prefork.

[connorjward]

Closing as this has been moved to Firedrake firedrakeproject/firedrake#3820