Restore previous builds of JLLs which (currently) depend on JLLWrappers 1.7 #122129
Conversation
giordano
force-pushed
the
mg/yank-jllwrappers
branch
from
f02e0d2 to
0d1d178
Compare
|
New approach: duplicate previous build, which didn't depend on JLLWrappers. New version of the script: #!/bin/bash
set -euo pipefail
GENERAL_REPO="${HOME}/repo/General"
JLLS=(
OpenSSL_jll
Hwloc_jll
JpegTurbo_jll
libpng_jll
libsixel_jll
Bzip2_jll
FFTW_jll
Giflib_jll
LERC_jll
Expat_jll
Libuuid_jll
Libmount_jll
OpenSpecFun_jll
Lz4_jll
LZO_jll
Pixman_jll
Libffi_jll
Qhull_jll
Zstd_jll
SQLite_jll
XZ_jll
libaec_jll
boost_jll
brotli_jll
pugixml_jll
yaml_cpp_jll
snappy_jll
MPItrampoline_jll
Blosc_jll
Blosc2_jll
libzip_jll
libsodium_jll
ZeroMQ_jll
Libgpg_error_jll
libusb_jll
Nettle_jll
Xorg_libXau_jll
Xorg_xcb_proto_jll
Xorg_libpthread_stubs_jll
Xorg_xtrans_jll
Xorg_libXdmcp_jll
Xorg_libXext_jll
Xorg_libX11_jll
Xorg_libxcb_jll
)
for jll in "${JLLS[@]}"; do
dir="jll/$(echo ${jll:0:1} | tr [:lower:] [:upper:])/${jll}"
compat="${dir}/Compat.toml"
versions="${dir}/Versions.toml"
last_hash=$(git -C "${GENERAL_REPO}" log -1 --oneline --format=%H -- "${compat}")
# Restore Compat.toml to previous state
git -C "${GENERAL_REPO}" reset "${last_hash}^" -- "${compat}"
git -C "${GENERAL_REPO}" restore -- "${compat}"
# Clean up any local changes to Versions.toml
git -C "${GENERAL_REPO}" restore --staged -- "${versions}"
git -C "${GENERAL_REPO}" checkout -- "${versions}"
# Find second to last version and bump build number by 2
old_version_entry=$(tail -n 5 "${GENERAL_REPO}/${versions}"|head -n 2)
new_build_version=$(($(echo "${old_version_entry}"|perl -lne 'print $1 if /[0-9.]+\+([0-9]+)/;') + 2))
# Append new entry, duplicate of the second to last one but change the build number.
echo >> "${GENERAL_REPO}/${versions}"
echo "$(echo "${old_version_entry}" | sed -E "s/([0-9.]+\+)[0-9]+/\1${new_build_version}/")" >> "${GENERAL_REPO}/${versions}"
git -C "${GENERAL_REPO}" add "${versions}"
done |
giordano
changed the title
* New version: FFTW_jll v3.3.10+2 UUID: f5851436-0d7a-5f13-b9de-f02708fd171a Repo: https://github.com/JuliaBinaryWrappers/FFTW_jll.jl.git Tree: 5cf2433259aa3985046792e2afc01fcec076b549 Registrator tree SHA: 191228b6dd8b9d0e2965ae3e705fe54c51dcfee8 * Update Compat.toml --------- Co-authored-by: Mosè Giordano <[email protected]>
|
@giordano Should the PR title say "Restore previous builds of JLLs which depend on JLLWrappers 1.2" instead? |
|
Ah, yeah that was my confusion. Maybe let's update the PR title to clarify that. |
DilumAluthge
changed the title
Yeah, I should've written "1.2 through 1.6". |
|
Any comment on the content of the PR besides the title? |
|
Yeah. I'm a little confused by the PR changes. Give me a minute, let me find an example. |
|
Okay, so using OpenSpecFun_jll as an example. So, originally, we had OpenSpecFun_jll And then OpenSpecFun_jll So now this PR makes two changes to OpenSpecFun_jll:
So this PR technically breaks OpenSpecFun_jll |
That's what @IanButterworth suggested, yes. |
|
The premise being far fewer manifests will have +1 than +0 |
|
Also, this is unlikely to effectively break the build versions we're overriding: a re-resolution of the package manager would also need to downgrade |
|
Ah, okay, yeah this PR makes sense to me. Do we want to get any other eyes on it before merging? |
|
Also, I haven't personally had the chance to test this PR to make sure it fixes the problem in existing manifests (although by glancing at the PR diff, and based on the discussion above, it does sound like this PR should fix the problem). Has anyone tested this PR branch, to make sure it fixes the problem in existing manifests? |
|
Passing along feedback from a colleague: We can verify that this PR fixed the issue for one of our use cases. |
|
Unfortunately, it looks like the fix causes issues when these JLLs end up in a system image. Specifically, if one of the fixed versions, e.g. Script to reproduce the error#!/usr/bin/env bash
export JLL_NAME="Xorg_libXau_jll"
export JLL_VERSION="v1.0.11"
JULIA_DEPOT_PATH="$(mktemp -d)"
export JULIA_DEPOT_PATH
export SYSIMAGE_PATH="${JULIA_DEPOT_PATH}/${JLL_NAME}.so"
julia --eval "using Pkg" \
--eval "Pkg.add(\"PackageCompiler\")" \
--eval "using PackageCompiler" \
--eval "Pkg.add(PackageSpec(; name=ENV[\"JLL_NAME\"], version=ENV[\"JLL_VERSION\"]))" \
--eval "create_sysimage([ENV[\"JLL_NAME\"]]; sysimage_path=ENV[\"SYSIMAGE_PATH\"])"
julia --sysimage "${SYSIMAGE_PATH}" \
--eval "using Pkg" \
--eval "Pkg.resolve()"Example output using
|
|
I just hit this. Investigating a fix here JuliaLang/Pkg.jl#4130 |
|
That error should be fixed on julia master later today, and is queued up for 1.10.8 and 1.11.3 |
Bash script used to produce this PR:
With
you can then see that all packages which still refer to
JLLWrappers1.7.0 are either stdlibs or packages which which had a brand new version (Reactant, Enzyme, SCIP_PaPILO, Fmt, OpenLDAPClient_jll, HelloWorldC, HELICS, Wayland_protocols, PDAL, Wasmtime).