PEP for enhanced Pyinstaller support?

[ElliotGarbus]

This is just a raw concept. I offer it as an idea for possible enhancement.

Pyinstaller is a critical element of the Python eco-system, but is not well supported by the "core".
Users often have issues with libraries that require using hidden-imports to bundle correctly.
What could be added to python or the standard library to better support building executables with pyinstaller?

The docs call out: "Some Python scripts import modules in ways that PyInstaller cannot detect: for example, by using the __import__() function with variable data, using importlib.import_module(), or manipulating the sys.path value at run time. If your script requires files that PyInstaller does not know about, you must help it:"

Could there be a simple mechanism that reported all dependencies to pyinstaller? Could the import module be extended to capture data and provide it to pyinstaller? Could a buildtime switch be added to python that reports dependencies - including dynamic dependencies? Could the python interpreter create the hooks file?

These are rhetorical questions designed only to inspire. Fundamentally can we enhance the user experience by adding features to Python?

Thanks for your attention.

[bwoodsend]

If we were willing to go down the path of people running their code with some tracing enabled in order to build a manifest of its dependencies then Python's sys.audithook() is probably as good as it could get. I've toyed with the idea but I can't ever really see it working.

It's more the attitudes of package authors rather than Python that I'd change. If packages stuck to one package == one purpose instead of stuffing anything and everything + its test suite + its build system into one big wheel then PyInstaller would no longer need to filter out the unused stuff and could replace its fallible dependency analysis with importlib.metadata.files() and importlib.metadata.requires().

What would be useful is if Python could take away the nasty features such as namespace packages or meta path finders but I already know that someone somewhere will claim that they're essential features and that their destructive wake is justified. I find that Python is shifting more and more towards the unnecessarily complicated...

[ElliotGarbus]

I don't know anything about sys.audithook, or importlib... but it seems to me there should be an automated mechanism that creates the required metadata for pyinstaller. Something built for purpose, not just leveraging what was built for another use.
I could imagine a python command line switch that creates a hook or a full spec file for pyinstaller.

I don't doubt there are technical challenges to achieve the goal. I do thing it would be a valuable addition. A PEP could be a mechanism to engage others in the community around this objective.