ModeulNotFoundError: No module named 'serial'

[wilmer-suarez-simplisafe]

Env:

• python 3.12
• pyinstaller 6.11.1
• macOS 14.7.1

I get the above error when my app reaches import serial during runtime.
I've tried adding the dependency to the build command using --hidden-import "pyserial" and still get the same error and PyInstaller outputs:

...
20578 INFO: Analyzing hidden import 'pyserial'
20578 ERROR: Hidden import 'pyserial' not found
...

I also tried using a hook named: hook-serial.py with the following content:

from PyInstaller.utils.hooks import collect_all

datas, binaries, hiddenimports = collect_all("pyserial", on_error="raise")

PyInstaller then complains that pyserial isn't a package.

...
4927 INFO: Processing standard module hook 'hook-serial.py' from '/Users/<user>/Desktop/Repositories/<repo>/hooks'
4928 DEBUG: Collecting data files for pyserial
4928 WARNING: collect_data_files - skipping data collection for module 'pyserial' as it is not a package.
4928 DEBUG: Collecting dynamic libraries for pyserial
4928 WARNING: collect_dynamic_libs - skipping library collection for module 'pyserial' as it is not a package.
4928 DEBUG: Collecting submodules for pyserial
4928 DEBUG: collect_submodules - pyserial is not a package.
...

I assume the issue stems from pyserial requiring the import to be import serial instead of import pyserial.
Anyway around that? Or am I doing something wrong?

[rokm]

Can you post the actual run-time error with full traceback?

Hiddenimports should be importable package/module names (i.e., the names used in import xyz, so serial in this case) and not dist names (pyserial in this case). That's why you are getting collect_submodules - pyserial is not a package.

But if your application does a straight-forward import serial, then you shouldn't need a hidden import at all; if you are getting an missing-module error, the most likely reason is that you are running a PyInstaller instance that is installed in a different python environment than your app's dependencies.

[rokm]

Is manager.py your frozen application's entry-point script? Or is it some script that you've collected as data file and are launching it via subprocess? If that's the case, how are you launching it?

[wilmer-suarez-simplisafe]

No, I have a run.py script that just executes the application as the scriptname arg. manager.py is added using "--add-data".
I'm launching my build script as a subprocess to make the PyInstaller output easier to review:

import os
import sys
import subprocess

pyinstaller_args = [
    "run.py",
    "--windowed",
    "--onefile",
    "--name",
    <name>,
    "--icon",
    "icon/logs.icns" if sys.platform == "darwin" else "icon/logs.ico",
    "--add-data",
    f"manager.py{os.pathsep}.",
    "--add-data",
    f"l_tools{os.pathsep}l_tools", # Local package (has associated hook)
    "--add-data",
    f"l_app{os.pathsep}l_app", # Local package (has associated hook)
    "--additional-hooks-dir",
    "hooks",
    "--distpath",
    "dist/macOS" if sys.platform == "darwin" else "dist/Windows",
    "--noconfirm",
    "--clean",
    "--log-level",
    "DEBUG",
]

log_file = "build_output.log"
with open(log_file, "w") as f:
    process = subprocess.Popen(
        [sys.executable, "-m", "PyInstaller"] + pyinstaller_args,
        stdout=f,
        stderr=subprocess.STDOUT
    )
    process.communicate()

[rokm]

How are you launching manager.py from your frozen application (run.py), then?

[rokm]

Are you using system python interpreter? I.e., something like subprocess.Popen(['python3', scriptname], ...)?

If so, that system interpreter has no access to python modules that are collected into executable-embedded PYZ archive and are accessible only to embedded interpreter that PyInstaller-frozen application is using. The only reason that system interpreter finds l_tools and l_app is that you are explicitly also collecting them as data files (and those directories end up being next to the manager.py script, whose parent directory ends up being added to search path when the script is launched). You would need to ensure that all dependencies are collected as source .py files.

[wilmer-suarez-simplisafe]

🤦

import xyx

if not getattr(sys, "frozen", False):
    sys.path.append(os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
    import manager

class App():
...

I had set this to be able to run things locally to test. import manager should be outside that if block.

I was hoping doing that would fix the issue; but it has not. Still getting the exact same error:

Traceback (most recent call last):
File "/var/folders/vh/g3n0s4115vz_6btxnnyt6wgh0000gq/T/_MEIjV9mWv/manager.py", line 11, in <module>
import l_tools
File "/private/var/folders/vh/g3n0s4115vz_6btxnnyt6wgh0000gq/T/_MEIjV9mWv/l_tools/__init__.py", line 3, in <module>
from . import serial_terminal
File "/private/var/folders/vh/g3n0s4115vz_6btxnnyt6wgh0000gq/T/_MEIjV9mWv/l_tools/serial_terminal.py", line 6, in <module>
import serial
ModuleNotFoundError: No module named 'serial'

[wilmer-suarez-simplisafe]

Are you using system python interpreter? I.e., something like subprocess.Popen(['python3', scriptname], ...)?

If so, that system interpreter has no access to python modules that are collected into executable-embedded PYZ archive and are accessible only to embedded interpreter that PyInstaller-frozen application is using. The only reason that system interpreter finds l_tools and l_app is that you are explicitly also collecting them as data files (and those directories end up being next to the manager.py script, whose parent directory ends up being added to search path when the script is launched). You would need to ensure that all dependencies are collected as source .py files.

No, i'm doing running the build_app.py like so: python build_app.py via the CLI. build_app.py is the code I shared above.

[rokm]

Traceback (most recent call last):
File "/var/folders/vh/g3n0s4115vz_6btxnnyt6wgh0000gq/T/_MEIjV9mWv/manager.py", line 11, in <module>
import l_tools
File "/private/var/folders/vh/g3n0s4115vz_6btxnnyt6wgh0000gq/T/_MEIjV9mWv/l_tools/__init__.py", line 3, in <module>
from . import serial_terminal
File "/private/var/folders/vh/g3n0s4115vz_6btxnnyt6wgh0000gq/T/_MEIjV9mWv/l_tools/serial_terminal.py", line 6, in <module>
import serial
ModuleNotFoundError: No module named 'serial'

Hmm, is this complete traceback?

Because I would expect it to start at run.py; what I see there implies that the manager.py is the entry-point (as if it was launched via external python interpreter).

[rokm]

Also:

• what happens if you add print(manager) after import manager - what is the module path that is printed when ran as frozen application? (if any - i.e., unless exception is really raised at that import)
• what happens if you do not collect manager.py as data at all - what kind of error do you get in that case?