sidechain block broadcasting logic

[haerdib]

broadcast: point-to-point.
After receiving, update state accordingly

[clangenb]

Some design thoughts:

1. I believe the number of workers per shard are going to be within a reasonable range n << 100. Hence, each worker will end up being the producing validator quite regularly (multiple times per minute?) if we target subsecond block production time. I therefore choose to keep connections to other workers open once established. Any objections?
2. In the end the workers need to be aware if they are producing blocks or waiting for broadcasted blocks. I saw in the polkassembly that there will be a pallet enforcing those rules. However, sidechain block production time will be faster than layer one production time, which implies that workers need to compute themselves if it is their turn or not.

These two points call for a sidechain worker state that tracks at least:

struct WorkerState = {
  peers: HashMap<PeerId, SomePeerState>,
  role: ProducingOrWaitingForImport
}

Abstracting this into a state struct, makes it easily extendable in the future. It could also contain the last synced by the enclave or such (simple example, not sure if this makes sense).

In the initial broadcasting PoC, I would statically assign these roles, i.e., have strictly one block producer and one/several importers. Later, this will be computed based on the worker_scheduler pallet's storage state.

@brenzi, @haerdib is this somewhat coherent with what you had in mind?

[brenzi]

I agree with 1 and 2
however: be aware that we will want to randomize the block production order to mitigate DoS

[brenzi]

In that sense, "producingorwaiting" may not be necessary. I think the scheduling of blocks happens on the untrusted worker side anyway, but the enclave should define the random order, or the next time to produce a block

[clangenb]

I have implemented the individual components. Now I see some problems when thinking about integration:

From what I see in the code, a newly onboarded worker does not sync itself with another worker's state. Instead, it computes its state on it own, while syncing blocks in the chain relay. Currently, a worker does not update its state, when it observes that a sidechain confirmation is included in a layer 1 block.

Hence, when a worker broadcasts its sidechain block, it will fail upon import by other workers, as the apriori state hashes do not match.

So, before wrapping up this issue, I believe we should think about how to properly onboard new workers, #273.

[brenzi]

@haerdib has already implemented state snapshotting AFAIR. This way new workes have to request the latest state from an active validator (I suggest not to use the term worker for sidechain validators. Maybe validaTEErs? 8-P )

[haerdib]

State snapshotting is not really implemented yet. But since the sgx externalities are state snapshots, we just need to implement a fast access & share it with other workers.