Fix neuron pruning

[ppolewicz]

As users of sn17 observed this week, pruning neurons can sometimes be unfair - it might cause two miners to be deregistered in wrong order.

When we looked into it closely, Fish summarized:

pruning score and emissions are stored differently as storage values so you can have cases where they might have different emissions but the same pruning score meaning it would default to registration date. This is likely happening here. Pruning score is not exactly the same as emission even though it's based on it.

I'd like to propose a slight simplification of the registration process:

• align the registration cycle with tempo (not removing the concept of the registration cycle - we might need it if dynamic tempo is ever implemented)
• remove trust
• remove pruning score
• remove rank
• replace the logic contained in

  subtensor/pallets/subtensor/src/subnets/registration.rs

  Line 419 in 6309d35

  pub fn get_neuron_to_prune(netuid: NetUid) -> u16 {

  with a rust equivalent of list(sorted(neurons, lambda neuron: not neuron.is_immune, neuron.emission, neuron.registration_block, neuron.uid))[0] - so every time we need to select a neuron for pruning, we'd sort the neurons by non_immunity, emission, registration block and the actual uid number and pick the lowest one for pruning.

The current system seems to be half-designed for thousands of registrations per block. It's confusing, unnecessarily stateful, unfair due to some implementation detail and hard to explain, when it doesn't have to be (even for me lol). If we ever needed something like this, then we'd calculate it once during the first deregistration in the given tempo and we'd store a queue of THREE uids which can potentially be deregistered in the current tempo after the one we are deregistering right now... but even that is premature optimization because chatgpt says sorting a thousand structs by 4 integer properties using a modern CPU should take about 5-50 microseconds. We don't need an elaborate caching scheme here when reading from the cache would take more time than recalculating the result.

[ppolewicz]

This needs some more design work because 1/dividends is not a good indicator of... deregistrability of a validator

[ppolewicz]

Ok while that statement is true, the impacted validator would have to be really small and the subnet would have to be really slow for it to matter, so we can proceed with the fix as planned.

Meanwhile

subtensor/pallets/subtensor/src/epoch/run_epoch.rs

Line 886 in 6309d35

let cloned_pruning_scores: Vec<u16> = vec_max_upscale_to_u16(&pruning_scores);

was identified as a culprit of the precision error which sometimes scores uids the same while their emission was slightly different.

[ppolewicz]

This is going to be split into two PRs:

• one fixes neuron pruning
• the other one cleans up unused storage maps

[ppolewicz]

Update to neuron pruning logic:

• add a new global parameter for the governor which sets the minumum stakeweight of immortal validators (default 150k)
• immortal validators cannot be deregistered by standard neuron pruning. Even if all uids are immune, the validator has zero emission and it is the oldest uid in the subnet - it still should not deregister. One could say it's double immune.
• the governor should be able to deregister a (netuid, uid) pair manually (if there is no such functionality yet). This should only be used as last resort and probably in a series of transactions to 1. disable registration, 2. (later) in a batch deregister the uid, register a hotkey (probably using a proxy) and re-enable registration, unless the code of the subnet is known to not panic when the number of uids decreases

Immune uids is a bad idea because it can be used to create a market of miner immunity.

We can do something else instead, I'll describe it in another comment.

[ppolewicz]

Update to neuron pruning logic:

• add a new global parameter for the governor which sets the minimum non-immune uids (one value for every subnet), by default it's going to be 10. If all but 10 uids are immune or immortal, then the next uid to register, instead of choosing the one with the lowest emission from a set of the 10 remaining uids, will select the one from the set which consists of the 10 remianing uids and the immune (until today) uid which registered into the subnet first. Actually this can be multiple uids if they all registered in the same block (something not currently possible due to configuration, but that configuration might change in the future and this feature here needs to be prepared for it).

This is also described in #2058