bridge: function parameter IOSurfaces — 30% faster than spatial packing (76.9ms vs 110ms/step)

[fspecii]

What

An alternative dynamic weight approach where weights are declared as native MIL function parameters backed by persistent IOSurfaces, rather than packed into the spatial dimension of a single large input tensor.

Two approaches compared

Current training_dynamic/ — spatial dimension packing

// Input: [1, DIM, 1, SEQ + 4*DIM] — activations at sp[0:SEQ], weights at sp[SEQ:]
func main<ios18>(tensor<fp32, [1, DIM, 1, SEQ+4*DIM]> x) {
    Wq = slice_by_size(x=x, begin=[0,0,0,SEQ], size=[1,DIM,1,DIM]);  // overhead
    Wq2 = reshape(shape=[1,1,DIM,DIM], x=Wq);                         // overhead
    xnt = transpose(perm=[0,1,3,2], x=xn2);                           // overhead
    qm = matmul(x=xnt, y=Wq2);
    ...
}

This PR — native function parameters

// Each weight is its own IOSurface argument, no unpacking needed
func main<ios18>(tensor<fp16,[1,K,1,M]> x, tensor<fp16,[1,N,K]> W) {
    out = matmul(x=W, y=x);
} -> (out);

The spatial packing approach adds slice_by_size + reshape + transpose overhead for every weight matrix per kernel call. With 4 weights per attention kernel that's 12 extra ops before the actual matmul.

Performance

Same hardware (M-series), same model (Stories110M, 12 layers, DIM=768, SEQ=256):

Approach	ms/step	Kernels
training_dynamic/ (spatial packing)	110ms	9
Function parameters (this PR)	76.9ms	74

30% faster per step. Both compile once at startup, both update weights via IOSurface writes (~0.001ms).

New bridge API

// Compile once — weights declared as MIL function parameters
ANEKernelHandle *ane_bridge_compile_dyn(
    const char *mil_text, size_t mil_len,
    int n_inputs, const size_t *input_sizes,
    int n_weights, const size_t *weight_sizes,
    size_t output_size);

// Update weights — direct IOSurface write, ~0.001ms
void ane_bridge_write_weight(ANEKernelHandle *k, int idx,
                              const void *fp16_data, size_t bytes);
void ane_bridge_write_weight_f32(ANEKernelHandle *k, int idx,
                                  const float *fp32_data, size_t count);

// Chain kernels without CPU round-trip
void ane_bridge_copy_io(ANEKernelHandle *src, int src_out_idx,
                         ANEKernelHandle *dst, int dst_in_idx);

// 90.6% p99 jitter reduction (plain p99=35ms → with RT task p99=3.3ms)
void ane_bridge_begin_realtime(void);
void ane_bridge_end_realtime(void);

Cache fix

try_cache_restore previously required a data file that ANE never creates for parameter-based models (only net.plist is written to tmpDir). Fixed to require only net.plist; data is saved/restored conditionally for BLOBFILE models.

Tested

test_bridge.m — 15/15 assertions:

• compile_dyn, write_weight (ratio=2.0 confirmed), write_weight_f32, copy_io (1→2→4 chain), begin/end_realtime, compile cache hit (compile count does not increment on second call with same MIL), free