Opened 10 months ago
Last modified 3 weeks ago
#31396 needs_review enhancement
relocatable wheel version of package sage_conf — at Version 25
Reported by:  mkoeppe  Owned by:  

Priority:  major  Milestone:  sage9.5 
Component:  relocation  Keywords:  
Cc:  culler, ghkliem, dimpase, jhpalmieri, slelievre  Merged in:  
Authors:  Matthias Koeppe  Reviewers:  
Report Upstream:  N/A  Work issues:  
Branch:  u/mkoeppe/relocatable_wheel_version_of_package_sage_conf (Commits, GitHub, GitLab)  Commit:  b871e09db5d847470fcf33eab2960554e0787bc1 
Dependencies:  #29039, #31409, #31521  Stopgaps: 
Description (last modified by )
This version of sage_conf
is for making a wheel that packages the precompiled nonPython bits of the Sage distribution (and the wheelhouse local/var/lib/sage/wheels
), making SAGE_ROOT
(and thus SAGE_LOCAL=$SAGE_ROOT/local
) relocatable using the method proposed in #31076, using SAGE_ROOT=/var/tmp/sage...
and a symlink.
The sage
script invokes sageconfig
to determine SAGE_ROOT
and SAGE_LOCAL
. In the version of sageconfig
supplied by this version of sage_conf
, we ensure that the symlink from /var/tmp/sage....
to the actual install location is set.
To test:
./bootstrap && (cd src/pkgs/sage_confrelocatable/ && tox v v v)
Specifically for building a wheel on macOS that uses XCode python 3.8:
./bootstrap && (cd src/pkgs/sage_confrelocatable/ && tox v v v e pythonmacos10.15python3.8)
This builds a 900MB wheel, an order of magnitude above the standard file size limit on PyPI. Opened a PR to request a file size limit increase: https://github.com/pypa/pypisupport/issues/985
Followup step: Do not package up the wheelbuilding venv.
Change History (25)
comment:1 Changed 10 months ago by
 Branch set to u/mkoeppe/relocatable_wheel_version_of_package_sage_conf
comment:2 Changed 10 months ago by
 Commit set to 9573b7c12afc5de1de743e53ff182d20490b340c
comment:3 Changed 10 months ago by
 Cc ghkliem added
 Description modified (diff)
comment:4 followup: ↓ 7 Changed 10 months ago by
 Description modified (diff)
On macOS, some shared libraries from homebrew still leak into the build:
find src/pkgs/sage_confrelocatable/.tox/python/lib/python3.9/sitepackages/sage_root name "*.so" o name "*.dylib"  xargs otool L libgiac.dylib: /usr/local/opt/gettext/lib/libintl.8.dylib R/library/cluster/libs/*.dylib: /usr/local/opt/gettext/lib/libintl.8.dylib (compatibility version 11.0.0, current version 11.0.0) /usr/local/opt/pcre/lib/libpcre.1.dylib (compatibility version 4.0.0, current version 4.12.0) /usr/local/opt/xz/lib/liblzma.5.dylib (compatibility version 8.0.0, current version 8.5.0)
Definitely should disable the R build (#30383).
Also check whether setting CC=gcc sysroot....
helps getting rid of /usr/local
leaking in.
Also need to add enablefatbinary
.
setup.py build_py
should probably use a symlink from /var/tmp...
into ~/.sage
as well so that it is easy to test relocation after removing the symlink.
Also copytree
from sage_root_source
should be changed so it overwrites (updates) the existing contents of SAGE_ROOT.
the tox test should include the test with otool
for selfcontainedness.
comment:5 Changed 10 months ago by
 Commit changed from 9573b7c12afc5de1de743e53ff182d20490b340c to f785d3cde259485597664306e191a21ad0ab9d58
Branch pushed to git repo; I updated commit sha1. New commits:
f785d3c  src/pkgs/sage_confrelocatable/tox.ini: Add environment macos10.15

comment:6 Changed 10 months ago by
Using isysroot helps but giac configuration needs more work  #31403
comment:7 in reply to: ↑ 4 Changed 10 months ago by
Replying to mkoeppe:
setup.py build_py
should probably use a symlink from/var/tmp...
into~/.sage
as well so that it is easy to test relocation after removing the symlink.
Does not work because sage resolves symlinks when it determines SAGE_ROOT.
comment:8 Changed 10 months ago by
 Dependencies changed from #29039 to #29039, #31409
comment:9 Changed 10 months ago by
 Commit changed from f785d3cde259485597664306e191a21ad0ab9d58 to e56a2a956a96bd70dbb443869697e89b7fb2b58a
Branch pushed to git repo; I updated commit sha1. New commits:
6da7ba2  Add variable TARGETS

b2d4ab8  build/pkgs/{r,rpy2}: Downgrade to optional

8732076  src/sage/repl/ipython_tests.py: Mark R interface tests # optional  rpy2

dbdf084  src/sage/structure/sage_object.pyx: Mark R interface test # optional  rpy2

737b21c  src/sage/stats/r.py: Mark all 2 doctests in this file as # optional  rpy2

4403924  src/sage/interfaces/r.py: Mark all tests # optional  rpy2

e56a2a9  Merge branch 't/31409/cygwin_standard__r_build_fails_____downgrade_r__rpy2_to_optional' into t/31396/relocatable_wheel_version_of_package_sage_conf

comment:10 Changed 10 months ago by
 Commit changed from e56a2a956a96bd70dbb443869697e89b7fb2b58a to a2289744baf90e3dcb288f96ef52d471ab8251bb
Branch pushed to git repo; I updated commit sha1. New commits:
a228974  src/pkgs/sage_confrelocatable/setup.py: Move the SAGE_ROOT directory away after building

comment:11 Changed 10 months ago by
 Commit changed from a2289744baf90e3dcb288f96ef52d471ab8251bb to c260942d3085fa369117746c635b48da5a572489
Branch pushed to git repo; I updated commit sha1. New commits:
0cac965  build/pkgs/ecl: Update to 21.2.1, remove patches included in new version

6a62484  delete all merged upstream patches: MR 210, MR 231

62fac00  Merge branch 't/31336/upgrade_ecl_to_21_2_1' into t/31396/relocatable_wheel_version_of_package_sage_conf

c260942  src/pkgs/sage_confrelocatable/setup.py: Use enabledownloadfromupstreamurl

comment:12 Changed 10 months ago by
 Commit changed from c260942d3085fa369117746c635b48da5a572489 to 4f17a047ec84130462cb7f91f28ed284816cfa74
Branch pushed to git repo; I updated commit sha1. Last 10 new commits:
dc5e225  build/pkgs/python3/spkgconfigure.m4: On macOS, if the distutils test fails, try using empty ARCHFLAGS

df17ad4  tox.ini: Add configuration factors for specific homebrew python3.x

914ea58  build/pkgs/python3/spkgconfigure.m4: Get rid of stray AC_SUBST

986739b  m4/sage_check_python_for_venv.m4: Fix reason

0b3e70d  build/pkgs/python3/spkgconfigure.m4: If PYTHON_FOR_VENV is configured to build multiarch extensions, set SAGE_ARCHFLAGS to disable it

372adcb  .github/workflows/tox.yml: Update xcode versions

fc8b676  SAGE_CHECK_PYTHON_FOR_VENV: Rework with less nesting

5ae8044  Merge branch 'u/mkoeppe/accept__usr_bin_python3_from_xcode_12_3_on_macos_10_15__catalina_' of git://trac.sagemath.org/sage into t/31396/relocatable_wheel_version_of_package_sage_conf

c5c73a9  src/pkgs/sage_confrelocatable/tox.ini: Define environments macospython3.8, macospython3.7, macospython3_xcode

4f17a04  src/pkgs/sage_confrelocatable/setup.py: Use enablefatbinary

comment:13 Changed 10 months ago by
 Description modified (diff)
comment:14 Changed 10 months ago by
 Commit changed from 4f17a047ec84130462cb7f91f28ed284816cfa74 to 851435289634de431fc8708e0c4c59a37b578b71
Branch pushed to git repo; I updated commit sha1. New commits:
8514352  src/pkgs/sage_confrelocatable/tox.ini: Run delocatelistdeps

comment:15 Changed 10 months ago by
 Description modified (diff)
comment:16 Changed 9 months ago by
 Commit changed from 851435289634de431fc8708e0c4c59a37b578b71 to 1baf390bc2bc5c29a7bfa6d7c73d31f39a1b5e65
Branch pushed to git repo; I updated commit sha1. Last 10 new commits:
c3e4093  Rename SAGE_OPTIONAL_CLEANED_PACKAGES to SAGE_OPTIONAL_UNINSTALLED_PACKAGES

4916415  Merge tag '9.3.beta9' into t/30383/new_package_type__optional_enabled_by_default

a35428e  Merge #30383

e098133  src/pkgs/sage_conf/sage_conf.py.in: Add SAGE_SPKG_WHEELS

5dbd712  src/pkgs/sage_confpypi/setup.py: Use 'configure disablenotebook', run 'make build' so that wheels are built

839c534  src/pkgs/sage_confpypi/setup.py: Fix typo

100ff51  src/pkgs/sage_confpypi/MANIFEST.in: Graft sage_root/build/pkgs/sage_docbuild/src

afa2ffd  Add configure option disablesagelib

a4f75b8  src/pkgs/sage_confpypi: Use configure disablesagelib, update README

1baf390  Merge branch 't/29039/pip_installable_package_sage_bootstrap' into t/31396/relocatable_wheel_version_of_package_sage_conf

comment:17 Changed 9 months ago by
 Commit changed from 1baf390bc2bc5c29a7bfa6d7c73d31f39a1b5e65 to 31bba97d13f86c531eeef0a26383b98fa3671c45
Branch pushed to git repo; I updated commit sha1. Last 10 new commits:
f04c134  Merge branch 't/30383/new_package_type__optional_enabled_by_default' into t/31409/cygwin_standard__r_build_fails_____downgrade_r__rpy2_to_optional

27c4bb9  Merge #31409

66c2987  build/pkgs/{sage_sws2rst,sage_docbuild}/installrequires.txt: New

7f8ec6c  build/pkgs/sagelib/src/tox.ini: Add factor sitepackages

0283da5  build/make/Makefile.in: Add wheel, setuptools_wheel to PYTHON_TOOLCHAIN to make sure that PEP 517 packages have a complete build system

f720722  build/pkgs/sagelib/src/tox.ini: Add factor nobuildisolation

c451b31  src/setup.cfg.m4 (install_requires): Add sage_conf

6700223  Merge tag '9.3.rc0' into t/30913/sagelib__add_setup_cfg__install_requires_

ac0f563  Merge branch 't/30913/sagelib__add_setup_cfg__install_requires_' into t/29039/pip_installable_package_sage_bootstrap

31bba97  Merge #29039

comment:18 Changed 9 months ago by
 Cc dimpase added
comment:19 Changed 9 months ago by
 Commit changed from 31bba97d13f86c531eeef0a26383b98fa3671c45 to f5f4a157e3a1630fc789f8f61655e123b6573eb6
Branch pushed to git repo; I updated commit sha1. New commits:
f5f4a15  src/pkgs/sage_confrelocatable/setup.py: Build the wheelhouse, but disablenotebook disabler disablesagelib

comment:20 Changed 9 months ago by
 Description modified (diff)
comment:21 Changed 9 months ago by
 Description modified (diff)
comment:22 Changed 9 months ago by
 Commit changed from f5f4a157e3a1630fc789f8f61655e123b6573eb6 to 87db1d0a371b70040b4d7448f0fadc9b50f18b54
Branch pushed to git repo; I updated commit sha1. New commits:
87db1d0  src/pkgs/sage_confrelocatable/setup.py: Use withmp=gmp instead of broken withoutsystemgmp option

comment:23 Changed 9 months ago by
 Dependencies changed from #29039, #31409 to #29039, #31409, #31521
comment:24 Changed 9 months ago by
 Commit changed from 87db1d0a371b70040b4d7448f0fadc9b50f18b54 to b871e09db5d847470fcf33eab2960554e0787bc1
comment:25 Changed 9 months ago by
 Description modified (diff)
Branch pushed to git repo; I updated commit sha1. New commits:
Fix up package_data
Closer to working