Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Infer python version from Makefile.envs in PyO3 #19

Merged
merged 16 commits into from
Aug 24, 2024
Merged

Infer python version from Makefile.envs in PyO3 #19

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
16 commits
Select commit Hold shift + click to select a range
dfac897
Don't hardcode python version in pyo3
ryanking13 Jul 29, 2024
bee4fb5
Merge remote-tracking branch 'upstream/main' into infer-python-versio…
ryanking13 Aug 14, 2024
b19d54a
[integration]
ryanking13 Aug 14, 2024
98ba829
set version
ryanking13 Aug 19, 2024
b47617a
[integration]
ryanking13 Aug 19, 2024
00846b6
Fix commnet
ryanking13 Aug 19, 2024
074a52a
[integration]
ryanking13 Aug 19, 2024
bebed71
comment [integration]
ryanking13 Aug 19, 2024
6d59255
Remove comment
ryanking13 Aug 19, 2024
04194aa
[integration]
ryanking13 Aug 19, 2024
2753b03
remove config file [integration]
ryanking13 Aug 19, 2024
45df8e7
Fix pyo3_cross_lib_dir
ryanking13 Aug 19, 2024
23292c4
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Aug 19, 2024
0e281d7
[integration]
ryanking13 Aug 19, 2024
6d867c4
changelog
ryanking13 Aug 20, 2024
72d2e2c
[integration]
ryanking13 Aug 20, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 7 additions & 0 deletions CHANGELOG.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,13 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0

## Unreleased

### Changed

- `pyo3_config_file` is no longer available in `pyodide config` command.
Pyodide now sets `PYO3_CROSS_PYTHON_VERSION`, `PYO3_CROSS_LIB_DIR` to specify the cross compilation environment
for PyO3.
[#19](https://github.com/pyodide/pyodide-build/pull/19)

## [0.28.0] - 2024/08/14

- `pyodide xbuildenv` subcommand is now publicly available.
Expand Down
1 change: 0 additions & 1 deletion pyodide_build/cli/config.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,6 @@
"python_version": "PYVERSION",
"rustflags": "RUSTFLAGS",
"cmake_toolchain_file": "CMAKE_TOOLCHAIN_FILE",
"pyo3_config_file": "PYO3_CONFIG_FILE",
"rust_toolchain": "RUST_TOOLCHAIN",
"cflags": "SIDE_MODULE_CFLAGS",
"cxxflags": "SIDE_MODULE_CXXFLAGS",
Expand Down
6 changes: 3 additions & 3 deletions pyodide_build/config.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -143,6 +143,7 @@ def to_env(self) -> dict[str, str]:
"pyminor": "PYMINOR",
"pyo3_cross_include_dir": "PYO3_CROSS_INCLUDE_DIR",
"pyo3_cross_lib_dir": "PYO3_CROSS_LIB_DIR",
"pyo3_cross_python_version": "PYO3_CROSS_PYTHON_VERSION",
"pyodide_emscripten_version": "PYODIDE_EMSCRIPTEN_VERSION",
"pyodide_jobs": "PYODIDE_JOBS",
"pyodide_root": "PYODIDE_ROOT",
Expand All @@ -161,7 +162,6 @@ def to_env(self) -> dict[str, str]:
"sysconfig_name": "SYSCONFIG_NAME",
"targetinstalldir": "TARGETINSTALLDIR",
"cmake_toolchain_file": "CMAKE_TOOLCHAIN_FILE",
"pyo3_config_file": "PYO3_CONFIG_FILE",
"meson_cross_file": "MESON_CROSS_FILE",
"cflags_base": "CFLAGS_BASE",
"cxxflags_base": "CXXFLAGS_BASE",
Expand Down Expand Up @@ -192,7 +192,6 @@ def to_env(self) -> dict[str, str]:
DEFAULT_CONFIG: dict[str, str] = {
# Paths to toolchain configuration files
"cmake_toolchain_file": str(TOOLS_DIR / "cmake/Modules/Platform/Emscripten.cmake"),
"pyo3_config_file": str(TOOLS_DIR / "pyo3_config.ini"),
"meson_cross_file": str(TOOLS_DIR / "emscripten.meson.cross"),
# Rust-specific configuration
"rustflags": "-C link-arg=-sSIDE_MODULE=2 -C link-arg=-sWASM_BIGINT -Z link-native-libraries=no",
Expand All @@ -213,8 +212,9 @@ def to_env(self) -> dict[str, str]:
"cxxflags": "$(CXXFLAGS_BASE)",
"ldflags": "$(LDFLAGS_BASE) -s SIDE_MODULE=1",
# Rust-specific configuration
"pyo3_cross_lib_dir": "$(CPYTHONINSTALL)/lib",
"pyo3_cross_lib_dir": "$(CPYTHONINSTALL)/sysconfigdata", # FIXME: pyodide xbuildenv stores sysconfigdata here
Copy link
Member

@hoodmane hoodmane Aug 20, 2024 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What about $(CPYTHONINSTALL)/lib/python$(PYMAJOR).$(PYMINOR)/?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We don't have that directory in our xbuildenv now. Normally sysconfigdata would be placed in that directory but we don't vendor lib/python3.X.Y/... part in the xbuildenv and only store sysconfigdata in the top-level directory.

xbuildenv/pyodide-root$ ls -R | grep ":$" | sed -e 's/:$//' -e 's/[^-][^\/]*\//--/g' -e 's/^/ /' -e 's/-/|/'
 .
 |-cpython
 |---installs
 |-----python-3.12.1 (this is CPYTHONINSTALL)
 |-------include
 |---------python3.12
 |-----------cpython
 |-----------internal
 |-------sysconfigdata
 |---------__pycache__

Maybe we can change the path in the xbuildenv, but for now the sysconfigdata directory works like a directory where first-party libraries are stored.

"pyo3_cross_include_dir": "$(PYTHONINCLUDE)",
"pyo3_cross_python_version": "$(PYMAJOR).$(PYMINOR)",
# Misc
"stdlib_module_cflags": "$(CFLAGS_BASE) -I$(PYTHONINCLUDE) -I Include/ -I. -IInclude/internal/", # TODO: remove this
# Paths to build dependencies
Expand Down
7 changes: 0 additions & 7 deletions pyodide_build/tools/pyo3_config.ini
View file
Open in desktop

This file was deleted.

Loading