Image upgrades! Impls for CUDA + numpy, along with an abstraction and full backwards compatibility

[mdaiter]

@paul-nechifor is leading this integration, according to @spomichter .

Full on abstraction for Image away from a locked-in platform.

Added all functions @alexlin2 requested in native CUDA + OpenCV numpy fallbacks.

Also, CUDA IPC and CPU IPC are now native.

[mdaiter]

(yes, I know, there are a few object points accepted in np.ndarray and put on-device. gonna go after this in a bit

[mdaiter]

the reason being that you might not need these parameters on the GPU by the time you're calling this function. It's basically user-convenience. Can shift to GPU or keep them on CPU. @alexlin2, feedback please.

[mdaiter]

nit: repeat

[mdaiter]

(it's a YUV conversion, taking the Y channel. It's luminosity. lemme know if I should drop a comment here)

[paul-nechifor]

Yeah, you could mention that's the BT.601 standard YUV conversion to make the numbers less magical.

[mdaiter]

Stats:
run.py
CUDA run:
Wall clock (real): 1m09s → a full 30 seconds faster than CPU.
User CPU time: 3m56s → slightly lower than CPU (≈ −25s).
System CPU time: 0m42s → dramatically lower (≈ −1m25s vs CPU).
CPU run:
Wall clock (real): 1m39s
User CPU time: 4m22s
System CPU time: 2m07s
What it shows:
Throughput win: CUDA pipeline finishes ~30% faster wall-time.
CPU relief:
User time drops modestly (Python + preprocessing still dominate).
System time drops by ~⅔, which is big — kernel overhead from memcopies, etc., is being offloaded to the GPU stack.
Total CPU time:
CPU: ~6m28s
CUDA: ~4m38s
→ That’s ~30% less CPU consumed overall.
unitree_go2.py
CUDA: Real: 1m12s User: 2m26s Sys: 0m34s
CPU: Real: 1m38s User: 3m56s Sys: 2m25s
^to break these down:
CUDA run
Real (wall time): 1m12s: finished faster overall
User CPU time: 2m26s: much less CPU spent than CPU version
System CPU time: 0m34s: also lower (fewer memory copies / syscalls bottlenecking)
CPU run
Real (wall time): 1m38s -> slower
User CPU time: 3m56s -> almost 2x the CPU load of CUDA
System CPU time: 2m25s -> kernel overhead way higher than CUDA

[spomichter]

@paul-nechifor I am reviewing as well but can you take a look at this specifically the changes to the Image type

[paul-nechifor]

Just a question. What's your opinion on using actually images for testing instead of zeros or noise?

That is:

1. add a small sample image
2. perform the modification on the image and save it as expected.png
3. in the test assert the result image is similar to expected.png

This way we can see what it's supposed to look like. And if we change the implementation and a test fails, we can compare what it should have been and what the new code is generating.

[mdaiter]

We could do that! Hit me with an image, happy to include.

[leshy]

should throw an error here, not return a magic number, is there a situation in which cp is None here?

[leshy]

is this a diff algo then cpu image sharpness? does this return the same value? is it fair to compare in tests?

[mdaiter]

1. Agreed, that makes sense. Technically, a zero sharp image is just blurry. But I understand where this comment is derivative from.
2. Check the parity tests. It's explicitly compared.

[leshy]

we need to measure aloc time as well, not init once then measure calculation, and this needs to be done for all tests, since every module will need to initialize image before doing operations on it, and init will happen on every frame on ingest