[Build] Linux x86 runner#3
Conversation
|
Note: we have to upgrade vulkan version, because old version no longer avialble. |
hughperkins
force-pushed
the
hp/linux-x86-runner
branch
from
13820c5 to
ee692a5
Compare
| - uses: actions/checkout@v4 | ||
| - name: Linux x86 Build | ||
| run: | | ||
| bash .github/workflows/scripts_new/linux_x86.sh |
There was a problem hiding this comment.
I do get the point of putting the CI script in a dedicated file instead of just pasting itself contain here. If the script is only used for CI, avoid unnecessary layers for indirection that are just obfuscating the logic.
There was a problem hiding this comment.
It would also allow splitting it into multiple steps or jobs for better modularity and readability of failures.
There was a problem hiding this comment.
Ok, so my understanding of the desires here are:
- nice to see multiple steps
- nice to be able to track progress across multiple steps in UI, eg something like https://github.com/duburcqa/jiminy/actions/runs/14775364707/job/41482561535
I in my turn personally like to factorize code out of the runner configuration because:
- decouples code from runner configuration
- means I can run/test the code easily, on arbitrary platforms, without needing to somehow trigger full CI flow etc
- eg spin up a container, try running the script
- run on my mac
- etc ...
- personally, I also find it easier to format code in standalone bash scripts, without needing to indent, pay attention to various yaml quirks etc
- lastly, I feel that we shouldn't require potential developers to be able to read github runner config files, when setting up their development environment
- yet I feel that the scripts used by CI are the best 'golden source' for what works today, when someone wants to install
To comply with all the above 😓 , what I have done is:
- split the runner configuration into 4 steps:
- prerequisites
- build
- install
- test
- split the script corresondingly into 4 parts, one part for each step
- added documentation to the dev install page, linking to each of these steps, as a reference
Now I admit that we should ideally turn on the markup link checker, to run across all .md files, systematically, even if they didn't change, since someone could rename one of these scripts, and break the build page 😓 Well, baby steps. We do at least have the link checker running, and we'll turn it on, to run across all .md files, systematically, in a bit.
| outdir.mkdir(parents=True, exist_ok=True) | ||
| unzip(local_cached, outdir, strip=strip) | ||
| elif name.endswith(".tar.gz") or name.endswith(".tgz"): | ||
| elif any(name.endswith(ext) for ext in [".tar.gz", ".tgz", ".xz", ".tar.bz2"]): |
There was a problem hiding this comment.
No need to change here, but I recommend immutable tuple instead of mutable list if mutability is irrelevant. better use the minimal required container, this makes the code more self-explanatory and may avoid silly mistakes (i.e. default values for method in Python).
There was a problem hiding this comment.
Personally, I feel that list is more intuitive to read. tuple on the whole I feel represents something more analogous to a struct, even though it's technically iterable. I don't have a strong preference to keep it as a list, but I do have a gentle preference.
Default values for a method, totally agree with using tuple, instead of list, for the reason you allude to: that parameters to methods should not be mutaable, otherwise if you modify them inside the method, then you just modified thd efault parametrsr 😓
There was a problem hiding this comment.
I guess stachexchange tends to align with your own pov on this: when not mutable, use a tuple https://stackoverflow.com/questions/1708510/list-vs-tuple-when-to-use-each. i'll change to tuple
There was a problem hiding this comment.
actually, I might ask in #code
| - uses: actions/checkout@v4 | ||
| - name: Linux x86 Build | ||
| run: | | ||
| bash .github/workflows/scripts_new/linux_x86.sh |
There was a problem hiding this comment.
It would also allow splitting it into multiple steps or jobs for better modularity and readability of failures.
hughperkins
force-pushed
the
hp/linux-x86-runner
branch
from
0a047ed to
451f379
Compare
|
Thank you! |
Introduce the helper machinery that the per-class to_torch / to_numpy methods will migrate to in subsequent commits. Existing public symbols (can_zerocopy, dlpack_to_torch, invalidate_zerocopy_cache, current_arch_is_cpu) are preserved as deprecated shims so the in-tree pre-rework callers continue to work; they will be removed once every call site is migrated. New surface: - _ZerocopyCache: per-instance container with two independent slots (torch tensor + numpy ndarray), each filled lazily on first access via torch.utils.dlpack.from_dlpack and numpy.from_dlpack respectively. Numpy zero-copy now bypasses torch entirely (closes review #6). - make_zerocopy_cache_if_supported(owner, ...): constructs a cache when zero-copy is supported and registers `owner` with `pyquadrants.cache_holders` so invalidation is wired automatically (closes review #18). - get_zerocopy_torch / get_zerocopy_numpy: thin entry points that implement the always-zerocopy-then-clone semantic (closes review #15, #16, #21) and the Apple Metal double-sync (qd.sync() on read AND torch.mps.synchronize() after .clone()/.to(); closes review #1, #22, #23). Also applies the small lints from the review: - Module-level constant for the torch>2.9.1 MPS bytes_offset probe; drops the pointless lru_cache wrapper around a zero-arg helper (closes review #2). - ASCII '...' instead of Unicode horizontal ellipsis '\u2026' in the docstring (closes review #3). - Top-level imports for numpy and torch (try/except for the no-torch CI case); no per-call lazy imports in the new code path (closes review #7, #9). The deprecated shim still does what the existing per-class methods expect; the new helpers are torch-clean. cache_holders is still empty until the next commits register Ndarray / ScalarField / MatrixField; this commit alone is no-op behaviourally.
3 participants
Issue: #
Brief Summary
Add linux x86 runner
Notes:
copilot:summary
Walkthrough
copilot:walkthrough