Mercurial

annotate third_party/emsdk/scripts/update_python.py @ 188:32ce881452fa hg-web

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
Fixing few stuff.
author MrJuneJune <me@mrjunejune.com>
date Fri, 23 Jan 2026 22:50:28 -0800
parents 8d17f6e6e290
children
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
179
8d17f6e6e290 [ThirdParty] Added emsdk bazel rules that can be supported by bazel 9.0.0
MrJuneJune <me@mrjunejune.com>
parents:
diff changeset
1 #!/usr/bin/env python3
8d17f6e6e290 [ThirdParty] Added emsdk bazel rules that can be supported by bazel 9.0.0
MrJuneJune <me@mrjunejune.com>
parents:
diff changeset
2 # Copyright 2020 The Emscripten Authors. All rights reserved.
8d17f6e6e290 [ThirdParty] Added emsdk bazel rules that can be supported by bazel 9.0.0
MrJuneJune <me@mrjunejune.com>
parents:
diff changeset
3 # Emscripten is available under two separate licenses, the MIT license and the
8d17f6e6e290 [ThirdParty] Added emsdk bazel rules that can be supported by bazel 9.0.0
MrJuneJune <me@mrjunejune.com>
parents:
diff changeset
4 # University of Illinois/NCSA Open Source License. Both these licenses can be
8d17f6e6e290 [ThirdParty] Added emsdk bazel rules that can be supported by bazel 9.0.0
MrJuneJune <me@mrjunejune.com>
parents:
diff changeset
5 # found in the LICENSE file.
8d17f6e6e290 [ThirdParty] Added emsdk bazel rules that can be supported by bazel 9.0.0
MrJuneJune <me@mrjunejune.com>
parents:
diff changeset
6
8d17f6e6e290 [ThirdParty] Added emsdk bazel rules that can be supported by bazel 9.0.0
MrJuneJune <me@mrjunejune.com>
parents:
diff changeset
7 """Updates the python binaries that we cache store at
8d17f6e6e290 [ThirdParty] Added emsdk bazel rules that can be supported by bazel 9.0.0
MrJuneJune <me@mrjunejune.com>
parents:
diff changeset
8 http://storage.google.com/webassembly.
8d17f6e6e290 [ThirdParty] Added emsdk bazel rules that can be supported by bazel 9.0.0
MrJuneJune <me@mrjunejune.com>
parents:
diff changeset
9
8d17f6e6e290 [ThirdParty] Added emsdk bazel rules that can be supported by bazel 9.0.0
MrJuneJune <me@mrjunejune.com>
parents:
diff changeset
10 We only supply binaries for windows and macOS, but we do it very different ways for those two OSes.
8d17f6e6e290 [ThirdParty] Added emsdk bazel rules that can be supported by bazel 9.0.0
MrJuneJune <me@mrjunejune.com>
parents:
diff changeset
11 On Linux, we depend on the system version of python.
8d17f6e6e290 [ThirdParty] Added emsdk bazel rules that can be supported by bazel 9.0.0
MrJuneJune <me@mrjunejune.com>
parents:
diff changeset
12
8d17f6e6e290 [ThirdParty] Added emsdk bazel rules that can be supported by bazel 9.0.0
MrJuneJune <me@mrjunejune.com>
parents:
diff changeset
13 Windows recipe:
8d17f6e6e290 [ThirdParty] Added emsdk bazel rules that can be supported by bazel 9.0.0
MrJuneJune <me@mrjunejune.com>
parents:
diff changeset
14 1. Download precompiled version of python from NuGet package manager,
8d17f6e6e290 [ThirdParty] Added emsdk bazel rules that can be supported by bazel 9.0.0
MrJuneJune <me@mrjunejune.com>
parents:
diff changeset
15 either the package "python" for AMD64, or "pythonarm64" for ARM64.
8d17f6e6e290 [ThirdParty] Added emsdk bazel rules that can be supported by bazel 9.0.0
MrJuneJune <me@mrjunejune.com>
parents:
diff changeset
16 2. Set up pip and install pywin32 and psutil via pip for emrun to work.
8d17f6e6e290 [ThirdParty] Added emsdk bazel rules that can be supported by bazel 9.0.0
MrJuneJune <me@mrjunejune.com>
parents:
diff changeset
17 3. Re-zip and upload to storage.google.com
8d17f6e6e290 [ThirdParty] Added emsdk bazel rules that can be supported by bazel 9.0.0
MrJuneJune <me@mrjunejune.com>
parents:
diff changeset
18
8d17f6e6e290 [ThirdParty] Added emsdk bazel rules that can be supported by bazel 9.0.0
MrJuneJune <me@mrjunejune.com>
parents:
diff changeset
19 macOS recipe:
8d17f6e6e290 [ThirdParty] Added emsdk bazel rules that can be supported by bazel 9.0.0
MrJuneJune <me@mrjunejune.com>
parents:
diff changeset
20 1. Clone cpython
8d17f6e6e290 [ThirdParty] Added emsdk bazel rules that can be supported by bazel 9.0.0
MrJuneJune <me@mrjunejune.com>
parents:
diff changeset
21 2. Use homebrew to install and configure openssl (for static linking!)
8d17f6e6e290 [ThirdParty] Added emsdk bazel rules that can be supported by bazel 9.0.0
MrJuneJune <me@mrjunejune.com>
parents:
diff changeset
22 3. Build cpython from source and use `make install` to create archive.
8d17f6e6e290 [ThirdParty] Added emsdk bazel rules that can be supported by bazel 9.0.0
MrJuneJune <me@mrjunejune.com>
parents:
diff changeset
23 """
8d17f6e6e290 [ThirdParty] Added emsdk bazel rules that can be supported by bazel 9.0.0
MrJuneJune <me@mrjunejune.com>
parents:
diff changeset
24
8d17f6e6e290 [ThirdParty] Added emsdk bazel rules that can be supported by bazel 9.0.0
MrJuneJune <me@mrjunejune.com>
parents:
diff changeset
25 import glob
8d17f6e6e290 [ThirdParty] Added emsdk bazel rules that can be supported by bazel 9.0.0
MrJuneJune <me@mrjunejune.com>
parents:
diff changeset
26 import multiprocessing
8d17f6e6e290 [ThirdParty] Added emsdk bazel rules that can be supported by bazel 9.0.0
MrJuneJune <me@mrjunejune.com>
parents:
diff changeset
27 import os
8d17f6e6e290 [ThirdParty] Added emsdk bazel rules that can be supported by bazel 9.0.0
MrJuneJune <me@mrjunejune.com>
parents:
diff changeset
28 import platform
8d17f6e6e290 [ThirdParty] Added emsdk bazel rules that can be supported by bazel 9.0.0
MrJuneJune <me@mrjunejune.com>
parents:
diff changeset
29 import shutil
8d17f6e6e290 [ThirdParty] Added emsdk bazel rules that can be supported by bazel 9.0.0
MrJuneJune <me@mrjunejune.com>
parents:
diff changeset
30 import sys
8d17f6e6e290 [ThirdParty] Added emsdk bazel rules that can be supported by bazel 9.0.0
MrJuneJune <me@mrjunejune.com>
parents:
diff changeset
31 import urllib.request
8d17f6e6e290 [ThirdParty] Added emsdk bazel rules that can be supported by bazel 9.0.0
MrJuneJune <me@mrjunejune.com>
parents:
diff changeset
32 from subprocess import check_call
8d17f6e6e290 [ThirdParty] Added emsdk bazel rules that can be supported by bazel 9.0.0
MrJuneJune <me@mrjunejune.com>
parents:
diff changeset
33
8d17f6e6e290 [ThirdParty] Added emsdk bazel rules that can be supported by bazel 9.0.0
MrJuneJune <me@mrjunejune.com>
parents:
diff changeset
34 from zip import unzip_cmd, zip_cmd
8d17f6e6e290 [ThirdParty] Added emsdk bazel rules that can be supported by bazel 9.0.0
MrJuneJune <me@mrjunejune.com>
parents:
diff changeset
35
8d17f6e6e290 [ThirdParty] Added emsdk bazel rules that can be supported by bazel 9.0.0
MrJuneJune <me@mrjunejune.com>
parents:
diff changeset
36 version = '3.13.3'
8d17f6e6e290 [ThirdParty] Added emsdk bazel rules that can be supported by bazel 9.0.0
MrJuneJune <me@mrjunejune.com>
parents:
diff changeset
37 major_minor_version = '.'.join(version.split('.')[:2]) # e.g. '3.9.2' -> '3.9'
8d17f6e6e290 [ThirdParty] Added emsdk bazel rules that can be supported by bazel 9.0.0
MrJuneJune <me@mrjunejune.com>
parents:
diff changeset
38 # This is not part of official Python version, but a repackaging number appended by emsdk
8d17f6e6e290 [ThirdParty] Added emsdk bazel rules that can be supported by bazel 9.0.0
MrJuneJune <me@mrjunejune.com>
parents:
diff changeset
39 # when a version of Python needs to be redownloaded.
8d17f6e6e290 [ThirdParty] Added emsdk bazel rules that can be supported by bazel 9.0.0
MrJuneJune <me@mrjunejune.com>
parents:
diff changeset
40 revision = '0'
8d17f6e6e290 [ThirdParty] Added emsdk bazel rules that can be supported by bazel 9.0.0
MrJuneJune <me@mrjunejune.com>
parents:
diff changeset
41
8d17f6e6e290 [ThirdParty] Added emsdk bazel rules that can be supported by bazel 9.0.0
MrJuneJune <me@mrjunejune.com>
parents:
diff changeset
42 PSUTIL = 'psutil==7.0.0'
8d17f6e6e290 [ThirdParty] Added emsdk bazel rules that can be supported by bazel 9.0.0
MrJuneJune <me@mrjunejune.com>
parents:
diff changeset
43
8d17f6e6e290 [ThirdParty] Added emsdk bazel rules that can be supported by bazel 9.0.0
MrJuneJune <me@mrjunejune.com>
parents:
diff changeset
44 upload_base = 'gs://webassembly/emscripten-releases-builds/deps/'
8d17f6e6e290 [ThirdParty] Added emsdk bazel rules that can be supported by bazel 9.0.0
MrJuneJune <me@mrjunejune.com>
parents:
diff changeset
45
8d17f6e6e290 [ThirdParty] Added emsdk bazel rules that can be supported by bazel 9.0.0
MrJuneJune <me@mrjunejune.com>
parents:
diff changeset
46
8d17f6e6e290 [ThirdParty] Added emsdk bazel rules that can be supported by bazel 9.0.0
MrJuneJune <me@mrjunejune.com>
parents:
diff changeset
47 # Detects whether current python interpreter architecture is ARM64 or AMD64
8d17f6e6e290 [ThirdParty] Added emsdk bazel rules that can be supported by bazel 9.0.0
MrJuneJune <me@mrjunejune.com>
parents:
diff changeset
48 # If running AMD64 python on an ARM64 Windows, this still intentionally returns AMD64
8d17f6e6e290 [ThirdParty] Added emsdk bazel rules that can be supported by bazel 9.0.0
MrJuneJune <me@mrjunejune.com>
parents:
diff changeset
49 def find_python_arch():
8d17f6e6e290 [ThirdParty] Added emsdk bazel rules that can be supported by bazel 9.0.0
MrJuneJune <me@mrjunejune.com>
parents:
diff changeset
50 import sysconfig
8d17f6e6e290 [ThirdParty] Added emsdk bazel rules that can be supported by bazel 9.0.0
MrJuneJune <me@mrjunejune.com>
parents:
diff changeset
51 arch = sysconfig.get_platform().lower()
8d17f6e6e290 [ThirdParty] Added emsdk bazel rules that can be supported by bazel 9.0.0
MrJuneJune <me@mrjunejune.com>
parents:
diff changeset
52 if 'amd64' in arch:
8d17f6e6e290 [ThirdParty] Added emsdk bazel rules that can be supported by bazel 9.0.0
MrJuneJune <me@mrjunejune.com>
parents:
diff changeset
53 return 'amd64'
8d17f6e6e290 [ThirdParty] Added emsdk bazel rules that can be supported by bazel 9.0.0
MrJuneJune <me@mrjunejune.com>
parents:
diff changeset
54 if 'arm64' in arch:
8d17f6e6e290 [ThirdParty] Added emsdk bazel rules that can be supported by bazel 9.0.0
MrJuneJune <me@mrjunejune.com>
parents:
diff changeset
55 return 'arm64'
8d17f6e6e290 [ThirdParty] Added emsdk bazel rules that can be supported by bazel 9.0.0
MrJuneJune <me@mrjunejune.com>
parents:
diff changeset
56 raise f'Unknown Python sysconfig platform "{arch}" (neither AMD64 or ARM64)'
8d17f6e6e290 [ThirdParty] Added emsdk bazel rules that can be supported by bazel 9.0.0
MrJuneJune <me@mrjunejune.com>
parents:
diff changeset
57
8d17f6e6e290 [ThirdParty] Added emsdk bazel rules that can be supported by bazel 9.0.0
MrJuneJune <me@mrjunejune.com>
parents:
diff changeset
58
8d17f6e6e290 [ThirdParty] Added emsdk bazel rules that can be supported by bazel 9.0.0
MrJuneJune <me@mrjunejune.com>
parents:
diff changeset
59 def make_python_patch():
8d17f6e6e290 [ThirdParty] Added emsdk bazel rules that can be supported by bazel 9.0.0
MrJuneJune <me@mrjunejune.com>
parents:
diff changeset
60 python_arch = find_python_arch()
8d17f6e6e290 [ThirdParty] Added emsdk bazel rules that can be supported by bazel 9.0.0
MrJuneJune <me@mrjunejune.com>
parents:
diff changeset
61 package_name = 'pythonarm64' if python_arch == 'arm64' else 'python'
8d17f6e6e290 [ThirdParty] Added emsdk bazel rules that can be supported by bazel 9.0.0
MrJuneJune <me@mrjunejune.com>
parents:
diff changeset
62 download_url = f'https://www.nuget.org/api/v2/package/{package_name}/{version}'
8d17f6e6e290 [ThirdParty] Added emsdk bazel rules that can be supported by bazel 9.0.0
MrJuneJune <me@mrjunejune.com>
parents:
diff changeset
63 filename = f'python-{version}-win-{python_arch}.zip'
8d17f6e6e290 [ThirdParty] Added emsdk bazel rules that can be supported by bazel 9.0.0
MrJuneJune <me@mrjunejune.com>
parents:
diff changeset
64 out_filename = f'python-{version}-{revision}-win-{python_arch}.zip'
8d17f6e6e290 [ThirdParty] Added emsdk bazel rules that can be supported by bazel 9.0.0
MrJuneJune <me@mrjunejune.com>
parents:
diff changeset
65
8d17f6e6e290 [ThirdParty] Added emsdk bazel rules that can be supported by bazel 9.0.0
MrJuneJune <me@mrjunejune.com>
parents:
diff changeset
66 if not os.path.exists(filename):
8d17f6e6e290 [ThirdParty] Added emsdk bazel rules that can be supported by bazel 9.0.0
MrJuneJune <me@mrjunejune.com>
parents:
diff changeset
67 print(f'Downloading python: {download_url} to {filename}')
8d17f6e6e290 [ThirdParty] Added emsdk bazel rules that can be supported by bazel 9.0.0
MrJuneJune <me@mrjunejune.com>
parents:
diff changeset
68 urllib.request.urlretrieve(download_url, filename)
8d17f6e6e290 [ThirdParty] Added emsdk bazel rules that can be supported by bazel 9.0.0
MrJuneJune <me@mrjunejune.com>
parents:
diff changeset
69
8d17f6e6e290 [ThirdParty] Added emsdk bazel rules that can be supported by bazel 9.0.0
MrJuneJune <me@mrjunejune.com>
parents:
diff changeset
70 os.mkdir('python-nuget')
8d17f6e6e290 [ThirdParty] Added emsdk bazel rules that can be supported by bazel 9.0.0
MrJuneJune <me@mrjunejune.com>
parents:
diff changeset
71 check_call(unzip_cmd() + [os.path.abspath(filename)], cwd='python-nuget')
8d17f6e6e290 [ThirdParty] Added emsdk bazel rules that can be supported by bazel 9.0.0
MrJuneJune <me@mrjunejune.com>
parents:
diff changeset
72 os.remove(filename)
8d17f6e6e290 [ThirdParty] Added emsdk bazel rules that can be supported by bazel 9.0.0
MrJuneJune <me@mrjunejune.com>
parents:
diff changeset
73
8d17f6e6e290 [ThirdParty] Added emsdk bazel rules that can be supported by bazel 9.0.0
MrJuneJune <me@mrjunejune.com>
parents:
diff changeset
74 src_dir = os.path.join('python-nuget', 'tools')
8d17f6e6e290 [ThirdParty] Added emsdk bazel rules that can be supported by bazel 9.0.0
MrJuneJune <me@mrjunejune.com>
parents:
diff changeset
75 python_exe = os.path.join(src_dir, 'python.exe')
8d17f6e6e290 [ThirdParty] Added emsdk bazel rules that can be supported by bazel 9.0.0
MrJuneJune <me@mrjunejune.com>
parents:
diff changeset
76 check_call([python_exe, '-m', 'ensurepip', '--upgrade'])
8d17f6e6e290 [ThirdParty] Added emsdk bazel rules that can be supported by bazel 9.0.0
MrJuneJune <me@mrjunejune.com>
parents:
diff changeset
77 check_call([python_exe, '-m', 'pip', 'install', 'pywin32==310', '--no-warn-script-location'])
8d17f6e6e290 [ThirdParty] Added emsdk bazel rules that can be supported by bazel 9.0.0
MrJuneJune <me@mrjunejune.com>
parents:
diff changeset
78 check_call([python_exe, '-m', 'pip', 'install', PSUTIL])
8d17f6e6e290 [ThirdParty] Added emsdk bazel rules that can be supported by bazel 9.0.0
MrJuneJune <me@mrjunejune.com>
parents:
diff changeset
79
8d17f6e6e290 [ThirdParty] Added emsdk bazel rules that can be supported by bazel 9.0.0
MrJuneJune <me@mrjunejune.com>
parents:
diff changeset
80 check_call(zip_cmd() + [os.path.join('..', '..', out_filename), '.'], cwd=src_dir)
8d17f6e6e290 [ThirdParty] Added emsdk bazel rules that can be supported by bazel 9.0.0
MrJuneJune <me@mrjunejune.com>
parents:
diff changeset
81 print('Created: %s' % out_filename)
8d17f6e6e290 [ThirdParty] Added emsdk bazel rules that can be supported by bazel 9.0.0
MrJuneJune <me@mrjunejune.com>
parents:
diff changeset
82
8d17f6e6e290 [ThirdParty] Added emsdk bazel rules that can be supported by bazel 9.0.0
MrJuneJune <me@mrjunejune.com>
parents:
diff changeset
83 # cleanup if everything went fine
8d17f6e6e290 [ThirdParty] Added emsdk bazel rules that can be supported by bazel 9.0.0
MrJuneJune <me@mrjunejune.com>
parents:
diff changeset
84 shutil.rmtree('python-nuget')
8d17f6e6e290 [ThirdParty] Added emsdk bazel rules that can be supported by bazel 9.0.0
MrJuneJune <me@mrjunejune.com>
parents:
diff changeset
85
8d17f6e6e290 [ThirdParty] Added emsdk bazel rules that can be supported by bazel 9.0.0
MrJuneJune <me@mrjunejune.com>
parents:
diff changeset
86 if '--upload' in sys.argv:
8d17f6e6e290 [ThirdParty] Added emsdk bazel rules that can be supported by bazel 9.0.0
MrJuneJune <me@mrjunejune.com>
parents:
diff changeset
87 upload_url = upload_base + out_filename
8d17f6e6e290 [ThirdParty] Added emsdk bazel rules that can be supported by bazel 9.0.0
MrJuneJune <me@mrjunejune.com>
parents:
diff changeset
88 print('Uploading: ' + upload_url)
8d17f6e6e290 [ThirdParty] Added emsdk bazel rules that can be supported by bazel 9.0.0
MrJuneJune <me@mrjunejune.com>
parents:
diff changeset
89 cmd = ['gsutil', 'cp', '-n', out_filename, upload_url]
8d17f6e6e290 [ThirdParty] Added emsdk bazel rules that can be supported by bazel 9.0.0
MrJuneJune <me@mrjunejune.com>
parents:
diff changeset
90 print(' '.join(cmd))
8d17f6e6e290 [ThirdParty] Added emsdk bazel rules that can be supported by bazel 9.0.0
MrJuneJune <me@mrjunejune.com>
parents:
diff changeset
91 check_call(cmd)
8d17f6e6e290 [ThirdParty] Added emsdk bazel rules that can be supported by bazel 9.0.0
MrJuneJune <me@mrjunejune.com>
parents:
diff changeset
92
8d17f6e6e290 [ThirdParty] Added emsdk bazel rules that can be supported by bazel 9.0.0
MrJuneJune <me@mrjunejune.com>
parents:
diff changeset
93
8d17f6e6e290 [ThirdParty] Added emsdk bazel rules that can be supported by bazel 9.0.0
MrJuneJune <me@mrjunejune.com>
parents:
diff changeset
94 def build_python():
8d17f6e6e290 [ThirdParty] Added emsdk bazel rules that can be supported by bazel 9.0.0
MrJuneJune <me@mrjunejune.com>
parents:
diff changeset
95 if sys.platform.startswith('darwin'):
8d17f6e6e290 [ThirdParty] Added emsdk bazel rules that can be supported by bazel 9.0.0
MrJuneJune <me@mrjunejune.com>
parents:
diff changeset
96 # Take some rather drastic steps to link openssl and liblzma statically
8d17f6e6e290 [ThirdParty] Added emsdk bazel rules that can be supported by bazel 9.0.0
MrJuneJune <me@mrjunejune.com>
parents:
diff changeset
97 # and avoid linking libintl completely.
8d17f6e6e290 [ThirdParty] Added emsdk bazel rules that can be supported by bazel 9.0.0
MrJuneJune <me@mrjunejune.com>
parents:
diff changeset
98 osname = 'macos'
8d17f6e6e290 [ThirdParty] Added emsdk bazel rules that can be supported by bazel 9.0.0
MrJuneJune <me@mrjunejune.com>
parents:
diff changeset
99 check_call(['brew', 'install', 'openssl', 'xz', 'pkg-config'])
8d17f6e6e290 [ThirdParty] Added emsdk bazel rules that can be supported by bazel 9.0.0
MrJuneJune <me@mrjunejune.com>
parents:
diff changeset
100 if platform.machine() == 'x86_64':
8d17f6e6e290 [ThirdParty] Added emsdk bazel rules that can be supported by bazel 9.0.0
MrJuneJune <me@mrjunejune.com>
parents:
diff changeset
101 prefix = '/usr/local'
8d17f6e6e290 [ThirdParty] Added emsdk bazel rules that can be supported by bazel 9.0.0
MrJuneJune <me@mrjunejune.com>
parents:
diff changeset
102 min_macos_version = '11.0'
8d17f6e6e290 [ThirdParty] Added emsdk bazel rules that can be supported by bazel 9.0.0
MrJuneJune <me@mrjunejune.com>
parents:
diff changeset
103 elif platform.machine() == 'arm64':
8d17f6e6e290 [ThirdParty] Added emsdk bazel rules that can be supported by bazel 9.0.0
MrJuneJune <me@mrjunejune.com>
parents:
diff changeset
104 prefix = '/opt/homebrew'
8d17f6e6e290 [ThirdParty] Added emsdk bazel rules that can be supported by bazel 9.0.0
MrJuneJune <me@mrjunejune.com>
parents:
diff changeset
105 min_macos_version = '11.0'
8d17f6e6e290 [ThirdParty] Added emsdk bazel rules that can be supported by bazel 9.0.0
MrJuneJune <me@mrjunejune.com>
parents:
diff changeset
106
8d17f6e6e290 [ThirdParty] Added emsdk bazel rules that can be supported by bazel 9.0.0
MrJuneJune <me@mrjunejune.com>
parents:
diff changeset
107 # Append '-x86_64' or '-arm64' depending on current arch. (TODO: Do
8d17f6e6e290 [ThirdParty] Added emsdk bazel rules that can be supported by bazel 9.0.0
MrJuneJune <me@mrjunejune.com>
parents:
diff changeset
108 # this for Linux too, move this below?)
8d17f6e6e290 [ThirdParty] Added emsdk bazel rules that can be supported by bazel 9.0.0
MrJuneJune <me@mrjunejune.com>
parents:
diff changeset
109 osname += '-' + platform.machine()
8d17f6e6e290 [ThirdParty] Added emsdk bazel rules that can be supported by bazel 9.0.0
MrJuneJune <me@mrjunejune.com>
parents:
diff changeset
110
8d17f6e6e290 [ThirdParty] Added emsdk bazel rules that can be supported by bazel 9.0.0
MrJuneJune <me@mrjunejune.com>
parents:
diff changeset
111 for f in [os.path.join(prefix, 'lib', 'libintl.dylib'),
8d17f6e6e290 [ThirdParty] Added emsdk bazel rules that can be supported by bazel 9.0.0
MrJuneJune <me@mrjunejune.com>
parents:
diff changeset
112 os.path.join(prefix, 'include', 'libintl.h'),
8d17f6e6e290 [ThirdParty] Added emsdk bazel rules that can be supported by bazel 9.0.0
MrJuneJune <me@mrjunejune.com>
parents:
diff changeset
113 os.path.join(prefix, 'opt', 'xz', 'lib', 'liblzma.dylib'),
8d17f6e6e290 [ThirdParty] Added emsdk bazel rules that can be supported by bazel 9.0.0
MrJuneJune <me@mrjunejune.com>
parents:
diff changeset
114 os.path.join(prefix, 'opt', 'openssl', 'lib', 'libssl.dylib'),
8d17f6e6e290 [ThirdParty] Added emsdk bazel rules that can be supported by bazel 9.0.0
MrJuneJune <me@mrjunejune.com>
parents:
diff changeset
115 os.path.join(prefix, 'opt', 'openssl', 'lib', 'libcrypto.dylib')]:
8d17f6e6e290 [ThirdParty] Added emsdk bazel rules that can be supported by bazel 9.0.0
MrJuneJune <me@mrjunejune.com>
parents:
diff changeset
116 if os.path.exists(f):
8d17f6e6e290 [ThirdParty] Added emsdk bazel rules that can be supported by bazel 9.0.0
MrJuneJune <me@mrjunejune.com>
parents:
diff changeset
117 os.remove(f)
8d17f6e6e290 [ThirdParty] Added emsdk bazel rules that can be supported by bazel 9.0.0
MrJuneJune <me@mrjunejune.com>
parents:
diff changeset
118 os.environ['PKG_CONFIG_PATH'] = os.path.join(prefix, 'opt', 'openssl', 'lib', 'pkgconfig')
8d17f6e6e290 [ThirdParty] Added emsdk bazel rules that can be supported by bazel 9.0.0
MrJuneJune <me@mrjunejune.com>
parents:
diff changeset
119 else:
8d17f6e6e290 [ThirdParty] Added emsdk bazel rules that can be supported by bazel 9.0.0
MrJuneJune <me@mrjunejune.com>
parents:
diff changeset
120 osname = 'linux'
8d17f6e6e290 [ThirdParty] Added emsdk bazel rules that can be supported by bazel 9.0.0
MrJuneJune <me@mrjunejune.com>
parents:
diff changeset
121
8d17f6e6e290 [ThirdParty] Added emsdk bazel rules that can be supported by bazel 9.0.0
MrJuneJune <me@mrjunejune.com>
parents:
diff changeset
122 src_dir = 'cpython'
8d17f6e6e290 [ThirdParty] Added emsdk bazel rules that can be supported by bazel 9.0.0
MrJuneJune <me@mrjunejune.com>
parents:
diff changeset
123 if os.path.exists(src_dir):
8d17f6e6e290 [ThirdParty] Added emsdk bazel rules that can be supported by bazel 9.0.0
MrJuneJune <me@mrjunejune.com>
parents:
diff changeset
124 check_call(['git', 'fetch'], cwd=src_dir)
8d17f6e6e290 [ThirdParty] Added emsdk bazel rules that can be supported by bazel 9.0.0
MrJuneJune <me@mrjunejune.com>
parents:
diff changeset
125 else:
8d17f6e6e290 [ThirdParty] Added emsdk bazel rules that can be supported by bazel 9.0.0
MrJuneJune <me@mrjunejune.com>
parents:
diff changeset
126 check_call(['git', 'clone', 'https://github.com/python/cpython'])
8d17f6e6e290 [ThirdParty] Added emsdk bazel rules that can be supported by bazel 9.0.0
MrJuneJune <me@mrjunejune.com>
parents:
diff changeset
127 check_call(['git', 'checkout', 'v' + version], cwd=src_dir)
8d17f6e6e290 [ThirdParty] Added emsdk bazel rules that can be supported by bazel 9.0.0
MrJuneJune <me@mrjunejune.com>
parents:
diff changeset
128
8d17f6e6e290 [ThirdParty] Added emsdk bazel rules that can be supported by bazel 9.0.0
MrJuneJune <me@mrjunejune.com>
parents:
diff changeset
129 env = os.environ
8d17f6e6e290 [ThirdParty] Added emsdk bazel rules that can be supported by bazel 9.0.0
MrJuneJune <me@mrjunejune.com>
parents:
diff changeset
130 if sys.platform.startswith('darwin'):
8d17f6e6e290 [ThirdParty] Added emsdk bazel rules that can be supported by bazel 9.0.0
MrJuneJune <me@mrjunejune.com>
parents:
diff changeset
131 # Specify the min OS version we want the build to work on
8d17f6e6e290 [ThirdParty] Added emsdk bazel rules that can be supported by bazel 9.0.0
MrJuneJune <me@mrjunejune.com>
parents:
diff changeset
132 min_macos_version_line = '-mmacosx-version-min=' + min_macos_version
8d17f6e6e290 [ThirdParty] Added emsdk bazel rules that can be supported by bazel 9.0.0
MrJuneJune <me@mrjunejune.com>
parents:
diff changeset
133 build_flags = min_macos_version_line + ' -Werror=partial-availability'
8d17f6e6e290 [ThirdParty] Added emsdk bazel rules that can be supported by bazel 9.0.0
MrJuneJune <me@mrjunejune.com>
parents:
diff changeset
134 # Build against latest SDK, but issue an error if using any API that would not work on the min OS version
8d17f6e6e290 [ThirdParty] Added emsdk bazel rules that can be supported by bazel 9.0.0
MrJuneJune <me@mrjunejune.com>
parents:
diff changeset
135 env = env.copy()
8d17f6e6e290 [ThirdParty] Added emsdk bazel rules that can be supported by bazel 9.0.0
MrJuneJune <me@mrjunejune.com>
parents:
diff changeset
136 env['MACOSX_DEPLOYMENT_TARGET'] = min_macos_version
8d17f6e6e290 [ThirdParty] Added emsdk bazel rules that can be supported by bazel 9.0.0
MrJuneJune <me@mrjunejune.com>
parents:
diff changeset
137 configure_args = ['CFLAGS=' + build_flags, 'CXXFLAGS=' + build_flags, 'LDFLAGS=' + min_macos_version_line]
8d17f6e6e290 [ThirdParty] Added emsdk bazel rules that can be supported by bazel 9.0.0
MrJuneJune <me@mrjunejune.com>
parents:
diff changeset
138 else:
8d17f6e6e290 [ThirdParty] Added emsdk bazel rules that can be supported by bazel 9.0.0
MrJuneJune <me@mrjunejune.com>
parents:
diff changeset
139 configure_args = []
8d17f6e6e290 [ThirdParty] Added emsdk bazel rules that can be supported by bazel 9.0.0
MrJuneJune <me@mrjunejune.com>
parents:
diff changeset
140 check_call(['./configure'] + configure_args, cwd=src_dir, env=env)
8d17f6e6e290 [ThirdParty] Added emsdk bazel rules that can be supported by bazel 9.0.0
MrJuneJune <me@mrjunejune.com>
parents:
diff changeset
141 check_call(['make', '-j', str(multiprocessing.cpu_count())], cwd=src_dir, env=env)
8d17f6e6e290 [ThirdParty] Added emsdk bazel rules that can be supported by bazel 9.0.0
MrJuneJune <me@mrjunejune.com>
parents:
diff changeset
142 check_call(['make', 'install', 'DESTDIR=install'], cwd=src_dir, env=env)
8d17f6e6e290 [ThirdParty] Added emsdk bazel rules that can be supported by bazel 9.0.0
MrJuneJune <me@mrjunejune.com>
parents:
diff changeset
143
8d17f6e6e290 [ThirdParty] Added emsdk bazel rules that can be supported by bazel 9.0.0
MrJuneJune <me@mrjunejune.com>
parents:
diff changeset
144 install_dir = os.path.join(src_dir, 'install')
8d17f6e6e290 [ThirdParty] Added emsdk bazel rules that can be supported by bazel 9.0.0
MrJuneJune <me@mrjunejune.com>
parents:
diff changeset
145
8d17f6e6e290 [ThirdParty] Added emsdk bazel rules that can be supported by bazel 9.0.0
MrJuneJune <me@mrjunejune.com>
parents:
diff changeset
146 # Install requests module. This is needed in particular on macOS to ensure
8d17f6e6e290 [ThirdParty] Added emsdk bazel rules that can be supported by bazel 9.0.0
MrJuneJune <me@mrjunejune.com>
parents:
diff changeset
147 # SSL certificates are available (certifi in installed and used by requests).
8d17f6e6e290 [ThirdParty] Added emsdk bazel rules that can be supported by bazel 9.0.0
MrJuneJune <me@mrjunejune.com>
parents:
diff changeset
148 pybin = os.path.join(src_dir, 'install', 'usr', 'local', 'bin', 'python3')
8d17f6e6e290 [ThirdParty] Added emsdk bazel rules that can be supported by bazel 9.0.0
MrJuneJune <me@mrjunejune.com>
parents:
diff changeset
149 pip = os.path.join(src_dir, 'install', 'usr', 'local', 'bin', 'pip3')
8d17f6e6e290 [ThirdParty] Added emsdk bazel rules that can be supported by bazel 9.0.0
MrJuneJune <me@mrjunejune.com>
parents:
diff changeset
150 check_call([pybin, '-m', 'ensurepip', '--upgrade'])
8d17f6e6e290 [ThirdParty] Added emsdk bazel rules that can be supported by bazel 9.0.0
MrJuneJune <me@mrjunejune.com>
parents:
diff changeset
151 check_call([pybin, pip, 'install', 'requests==2.32.3'])
8d17f6e6e290 [ThirdParty] Added emsdk bazel rules that can be supported by bazel 9.0.0
MrJuneJune <me@mrjunejune.com>
parents:
diff changeset
152
8d17f6e6e290 [ThirdParty] Added emsdk bazel rules that can be supported by bazel 9.0.0
MrJuneJune <me@mrjunejune.com>
parents:
diff changeset
153 # Install psutil module. This is needed by emrun to track when browser
8d17f6e6e290 [ThirdParty] Added emsdk bazel rules that can be supported by bazel 9.0.0
MrJuneJune <me@mrjunejune.com>
parents:
diff changeset
154 # process quits.
8d17f6e6e290 [ThirdParty] Added emsdk bazel rules that can be supported by bazel 9.0.0
MrJuneJune <me@mrjunejune.com>
parents:
diff changeset
155 check_call([pybin, pip, 'install', PSUTIL])
8d17f6e6e290 [ThirdParty] Added emsdk bazel rules that can be supported by bazel 9.0.0
MrJuneJune <me@mrjunejune.com>
parents:
diff changeset
156
8d17f6e6e290 [ThirdParty] Added emsdk bazel rules that can be supported by bazel 9.0.0
MrJuneJune <me@mrjunejune.com>
parents:
diff changeset
157 dirname = 'python-%s-%s' % (version, revision)
8d17f6e6e290 [ThirdParty] Added emsdk bazel rules that can be supported by bazel 9.0.0
MrJuneJune <me@mrjunejune.com>
parents:
diff changeset
158 if os.path.isdir(dirname):
8d17f6e6e290 [ThirdParty] Added emsdk bazel rules that can be supported by bazel 9.0.0
MrJuneJune <me@mrjunejune.com>
parents:
diff changeset
159 print('Erasing old build directory ' + dirname)
8d17f6e6e290 [ThirdParty] Added emsdk bazel rules that can be supported by bazel 9.0.0
MrJuneJune <me@mrjunejune.com>
parents:
diff changeset
160 shutil.rmtree(dirname)
8d17f6e6e290 [ThirdParty] Added emsdk bazel rules that can be supported by bazel 9.0.0
MrJuneJune <me@mrjunejune.com>
parents:
diff changeset
161 os.rename(os.path.join(install_dir, 'usr', 'local'), dirname)
8d17f6e6e290 [ThirdParty] Added emsdk bazel rules that can be supported by bazel 9.0.0
MrJuneJune <me@mrjunejune.com>
parents:
diff changeset
162 tarball = 'python-%s-%s-%s.tar.gz' % (version, revision, osname)
8d17f6e6e290 [ThirdParty] Added emsdk bazel rules that can be supported by bazel 9.0.0
MrJuneJune <me@mrjunejune.com>
parents:
diff changeset
163 shutil.rmtree(os.path.join(dirname, 'lib', 'python' + major_minor_version, 'test'))
8d17f6e6e290 [ThirdParty] Added emsdk bazel rules that can be supported by bazel 9.0.0
MrJuneJune <me@mrjunejune.com>
parents:
diff changeset
164 shutil.rmtree(os.path.join(dirname, 'include'))
8d17f6e6e290 [ThirdParty] Added emsdk bazel rules that can be supported by bazel 9.0.0
MrJuneJune <me@mrjunejune.com>
parents:
diff changeset
165 for lib in glob.glob(os.path.join(dirname, 'lib', 'lib*.a')):
8d17f6e6e290 [ThirdParty] Added emsdk bazel rules that can be supported by bazel 9.0.0
MrJuneJune <me@mrjunejune.com>
parents:
diff changeset
166 os.remove(lib)
8d17f6e6e290 [ThirdParty] Added emsdk bazel rules that can be supported by bazel 9.0.0
MrJuneJune <me@mrjunejune.com>
parents:
diff changeset
167 check_call(['tar', 'zcvf', tarball, dirname])
8d17f6e6e290 [ThirdParty] Added emsdk bazel rules that can be supported by bazel 9.0.0
MrJuneJune <me@mrjunejune.com>
parents:
diff changeset
168
8d17f6e6e290 [ThirdParty] Added emsdk bazel rules that can be supported by bazel 9.0.0
MrJuneJune <me@mrjunejune.com>
parents:
diff changeset
169 print('Created: %s' % tarball)
8d17f6e6e290 [ThirdParty] Added emsdk bazel rules that can be supported by bazel 9.0.0
MrJuneJune <me@mrjunejune.com>
parents:
diff changeset
170 if '--upload' in sys.argv:
8d17f6e6e290 [ThirdParty] Added emsdk bazel rules that can be supported by bazel 9.0.0
MrJuneJune <me@mrjunejune.com>
parents:
diff changeset
171 print('Uploading: ' + upload_base + tarball)
8d17f6e6e290 [ThirdParty] Added emsdk bazel rules that can be supported by bazel 9.0.0
MrJuneJune <me@mrjunejune.com>
parents:
diff changeset
172 check_call(['gsutil', 'cp', '-n', tarball, upload_base + tarball])
8d17f6e6e290 [ThirdParty] Added emsdk bazel rules that can be supported by bazel 9.0.0
MrJuneJune <me@mrjunejune.com>
parents:
diff changeset
173
8d17f6e6e290 [ThirdParty] Added emsdk bazel rules that can be supported by bazel 9.0.0
MrJuneJune <me@mrjunejune.com>
parents:
diff changeset
174
8d17f6e6e290 [ThirdParty] Added emsdk bazel rules that can be supported by bazel 9.0.0
MrJuneJune <me@mrjunejune.com>
parents:
diff changeset
175 def main():
8d17f6e6e290 [ThirdParty] Added emsdk bazel rules that can be supported by bazel 9.0.0
MrJuneJune <me@mrjunejune.com>
parents:
diff changeset
176 if sys.platform.startswith('win') or '--win32' in sys.argv:
8d17f6e6e290 [ThirdParty] Added emsdk bazel rules that can be supported by bazel 9.0.0
MrJuneJune <me@mrjunejune.com>
parents:
diff changeset
177 make_python_patch()
8d17f6e6e290 [ThirdParty] Added emsdk bazel rules that can be supported by bazel 9.0.0
MrJuneJune <me@mrjunejune.com>
parents:
diff changeset
178 else:
8d17f6e6e290 [ThirdParty] Added emsdk bazel rules that can be supported by bazel 9.0.0
MrJuneJune <me@mrjunejune.com>
parents:
diff changeset
179 build_python()
8d17f6e6e290 [ThirdParty] Added emsdk bazel rules that can be supported by bazel 9.0.0
MrJuneJune <me@mrjunejune.com>
parents:
diff changeset
180 return 0
8d17f6e6e290 [ThirdParty] Added emsdk bazel rules that can be supported by bazel 9.0.0
MrJuneJune <me@mrjunejune.com>
parents:
diff changeset
181
8d17f6e6e290 [ThirdParty] Added emsdk bazel rules that can be supported by bazel 9.0.0
MrJuneJune <me@mrjunejune.com>
parents:
diff changeset
182
8d17f6e6e290 [ThirdParty] Added emsdk bazel rules that can be supported by bazel 9.0.0
MrJuneJune <me@mrjunejune.com>
parents:
diff changeset
183 if __name__ == '__main__':
8d17f6e6e290 [ThirdParty] Added emsdk bazel rules that can be supported by bazel 9.0.0
MrJuneJune <me@mrjunejune.com>
parents:
diff changeset
184 sys.exit(main())