GH-45278: [Python][Packaging] Updated delvewheel install command and updated flags used with delvewheel repair

[protokoul]

Rationale for this change

It is already explained in the issue.

What changes are included in this PR?

This PR installs delvewheel by using its latest version instead of using a github branch. The new flag --with-mangle introduced in the latest version of delvewheel is used with delvewheel repair command. Removed comments that referred to the use of the github branch for delvewheel installation.

Are these changes tested?

No, these changes are not tested because to run Windows containers I will need Windows 11 Pro or Enterprise. I do not have a machine that satisfies this requirement.

• GitHub Issue: [Python][Packaging] Update python_wheel_windows_build.bat to use released delvewheel #45278

[github-actions]

⚠️ GitHub issue #45278 has been automatically assigned in GitHub to PR creator.

[raulcd]

@github-actions crossbow submit wheel-windows-cp39-cp39-amd64

[github-actions]

Revision: 9308397

Submitted crossbow builds: ursacomputing/crossbow @ actions-3fbb6f8b60

Task	Status
wheel-windows-cp39-cp39-amd64

[raulcd]

@github-actions crossbow submit wheel-windows-cp39-cp39-amd64

[github-actions]

Revision: d7c2e48

Submitted crossbow builds: ursacomputing/crossbow @ actions-c8bb220277

Task	Status
wheel-windows-cp39-cp39-amd64

[raulcd]

From what I understand this is currently not mangling msvcp40.dll because we are copying it on the wheel. I think we have to remove this copy:

arrow/ci/scripts/python_wheel_windows_build.bat

Lines 136 to 137 in 4ede48c

	@REM Bundle the C++ runtime
	cp C:\Windows\System32\msvcp140.dll pyarrow\

[protokoul]

Sure, let me push this change. Also, would it be better if I also add delvewheel show as discussed here if it helps to see any changes?

[raulcd]

@github-actions crossbow submit wheel-windows-cp39-cp39-amd64

[github-actions]

Revision: 4135fb9

Submitted crossbow builds: ursacomputing/crossbow @ actions-13055f6c30

Task	Status
wheel-windows-cp39-cp39-amd64

[raulcd]

Great! This seems to work but it's copying the dll on pyarrow.libs/ an this was not expected to be previously:

 C:\>C:\Python39\python C:\arrow\ci\scripts\python_wheel_validate_contents.py --path C:\arrow\python\repaired_wheels   || exit /B 1 
Traceback (most recent call last):
  File "C:\arrow\ci\scripts\python_wheel_validate_contents.py", line 48, in <module>
    main()
  File "C:\arrow\ci\scripts\python_wheel_validate_contents.py", line 44, in main
    validate_wheel(args.path)
  File "C:\arrow\ci\scripts\python_wheel_validate_contents.py", line 35, in validate_wheel
    assert not outliers, f"Unexpected contents in wheel: {sorted(outliers)}"
AssertionError: Unexpected contents in wheel: ['pyarrow.libs/', 'pyarrow.libs/.load-order-pyarrow-20.0.0.dev14', 'pyarrow.libs/msvcp140-a4b2ab3dc6cde85d4f2c0e24432d66e1.dll']

We might want to update our python_wheel_validate_contents.py script to allow this folder to be available on the Windows wheel.

[raulcd]

@github-actions crossbow submit wheel-windows-cp39-cp39-amd64

[github-actions]

Revision: 657d971

Submitted crossbow builds: ursacomputing/crossbow @ actions-3a4d87d41c

Task	Status
wheel-windows-cp39-cp39-amd64

[raulcd]

Thanks for taking the effort @protokoul
This seems to work now with the newer delvewheel version:

2025-01-22T13:31:33.2489094Z External dependencies to copy into the wheel are
2025-01-22T13:31:33.2489352Z {'msvcp140.dll'}
...
2025-01-22T13:31:33.2497910Z copying DLLs into pyarrow.libs
2025-01-22T13:31:33.2498831Z copying C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.28.29333\bin\HostX64\x64\msvcp140.dll -> C:\Users\ContainerAdministrator\AppData\Local\Temp\tmpgwwi5dn3\pyarrow.libs\msvcp140.dll
...
2025-01-22T13:31:33.2509034Z repairing msvcp140.dll -> msvcp140-a4b2ab3dc6cde85d4f2c0e24432d66e1.dll
...

My only concern is that it seems to add C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.28.29333\bin\HostX64\x64\msvcp140.dll to the wheel instead of the downloaded C:\Windows\System32\msvcp140.dll.
@pitrou from what I understand we were manually upgrading it because the system provided one was an old version, is this an issue that prevents us from using the released delvewheel?
I suppose the solution would be to build our own Image and use a newer Microsoft Runtime Library but that's a bigger effort.

[pitrou]

@pitrou from what I understand we were manually upgrading it because the system provided one was an old version, is this an issue that prevents us from using the released delvewheel?

As long as we're using the same one that we've built against, it should be ok.

I suppose the solution would be to build our own Image and use a newer Microsoft Runtime Library but that's a bigger effort.

Definitely, but it's desirable anyway: #45156

[protokoul]

Thank you @raulcd for reviewing the changes and helping with the PR.