Stdout/In in spawned terminal - Windows 11

[wilmer-suarez-simplisafe]

Related to this discussion: https://github.com/orgs/pyinstaller/discussions/8979

Using sys.executable to run a script that spawns a new terminal (via miniterm). I seem to be having issues with https://pyinstaller.org/en/stable/common-issues-and-pitfalls.html#sys-stdin-sys-stdout-and-sys-stderr-in-noconsole-windowed-applications-windows-only. I/O doesn't seem to be working in the spawned window.

I keep getting the OSError: [Errno 22] Invalid argument error for print().

[rokm]

Are you using windowed/noconsole mode? In that case, I don't expect the process to have console (and its standard I/O streams) attached, even if you launch it from console (in this case, apparently the miniterm-spawned terminal).

There's not enough context to go by, though. What part of the code raises this error (i.e., miniterm itself, the external script you are trying to run in the sub-process, etc.), and what is happening on the offending line? (I was under impression that print() itself properly handled the lack of standard streams...).

[wilmer-suarez-simplisafe]

Yes, I'm using the --windowed flag. The bundled app I'm working on is a Tkinter GUI that, if selected, uses sys.exectuable to re-execute the app entry point and uses exec() to spawn a terminal. The terminal is spawned in a thread using serial.tools.minterm.Miniterm().start(). An example of the entry point .py file is shared here.

The Invalid argument exception is raised by minterm since I'm using print() right before .start()'ing it. I assumed the spawned terminal would take over the I/O.

Traceback (most recent call last):
  File "run.py", line 52, in <module>
  File "<string>", line 130, in <module>
  File "<string>", line 122, in main
  File "l_tools\serial_terminal.py", line 281, in start_terminal
  File "<frozen codecs>", line 378, in write
  File "serial\tools\miniterm.py", line 103, in write
OSError: [Errno 22] Invalid argument

If I omit the --windowed flag, it works as expected. I would prefer to include that flag, so is there no way to redirect within the frozen app?

[rokm]

The miniterm console is, as far as I can tell, heavily relying on sys.stderr and sys.stdout. The error in question is raised by Out.write implementation, which does a os.write(self.fd, s). If that raises Invalid argument error, then that file descriptor is invalid. And considering it seems to be obtained via sys.stderr.fileno() and sys.stdoutfileno(), that shouldn't be much of a surprise.

I would expect it raise an error due to sys.stdout/sys.stderr being None, though. Perhaps you can try dumping the repr of sys.stdout and sys.stderr (and if they are not None, their .fileno()) to a log file (in the child process) to see what kind of stream objects you are actually dealing with.

Is there anything else you aren't telling us that is not obvious from the entry point, though? Are you overriding sys.stdout and sys.stderr so that they are not None? How are you spawning the sys.executable process - any special arguments to e.g. subprocess.call (for example, are you redirecting standard streams to pipes?)).

[wilmer-suarez-simplisafe]

❗ Yes, sorry, forgot to mention. I'm doing what is suggested here:

if platform.system() == "Windows" and getattr(sys, "frozen", False):
    if sys.stdout is None:
        sys.stdout = open(devnull, "w")
    if sys.stderr is None:
        sys.stderr = open(devnull, "w")

If I omit doing that, I get the same exact error. If I refrain from using printf() it won't error out but I/O still doesn't work (i.e, the terminal is just a blank screen).

Omitting the above (with --windowed), I get the following info (including the error I shared before):

The first time the app runs:

sys.stdout repr: None
sys.stderr repr: None

When re-opened using sys.executable():

sys.stdout repr: <_io.TextIOWrapper name='<stdout>' mode='w' encoding='cp932'>
sys.stderr repr: <_io.TextIOWrapper name='<stderr>' mode='w' encoding='cp932'>
sys.stdout.fileno(): 1
sys.stderr.fileno(): 2

With --windowed omitted:

The first time the app runs:

sys.stdout: <_io.TextIOWrapper name='<stdout>' mode='w' encoding='utf-8'>
sys.stderr: <_io.TextIOWrapper name='<stderr>' mode='w' encoding='utf-8'>
sys.stdout.fileno(): 1
sys.stderr.fileno(): 2

When re-opened using sys.executable():

sys.stdout: <_io.TextIOWrapper name='<stdout>' mode='w' encoding='utf-8'>
sys.stderr: <_io.TextIOWrapper name='<stderr>' mode='w' encoding='cp932'>
sys.stdout.fileno(): 1
sys.stderr.fileno(): 2

I'm setting the TX and RX encoding for miniterm to utf-8 in all cases.

How I'm spawning the sys.executable():

...
args = {
    "stderr": PIPE,
    "env": {**environ, "PYINSTALLER_RESET_ENVIRONMENT": "1"},
    "creationflags": CREATE_NEW_CONSOLE if system() == "Windows" else 0,
}
process = Popen(terminal_command, **args)
...

[rokm]

Hmmm... I think that explains the Invalid argument error.

Because you are redirecting stderr to PIPE, the streams become non-None in the child process. For some reason, both sys.stdout and sys.stderr become non-None, but only stderr has a valid backing fileno. So trying to os.write(sys.stderr.fileno(), b'...') succeeds while os.write(sys.stdout.fileno(), b'...') raises Invalid argument error.

If you had redirected both, that print would have likely succeeded.

I haven't dug deeply into miniterm code, but it seems it writes most if its own messages to stderr. So if that's redirected to a pipe, it might explain why you don't see anything in the console - you might be able to retrieve the captured output in the parent process, though. (And if anything was supposed to be written to sys.stdout, well, its backing file descriptor is invalid).

[wilmer-suarez-simplisafe]

Tried a few things but could not get the created console to be responsive. Decided to just live without using --windowed. Thanks for your help.