Add StorageList support in decl_storage

[sorpaas]

Add back storage list support in decl_storage macro. Can fix issues like #2228.

A QoL function StorageList::push is also added.

[xlc]

please put this to the end to improve backward compatibility

[xlc]

no pop method? how do I delete the last item?

[bkchr]

I thought we won't bring back storage list because of the performance issues. See here #772

[tomusdrw]

We do have linked_map now that should be more efficient

[tomusdrw]

To clarify on my "more efficient" comment, after discussing that with @bkchr .
linked_map and StorageList have many similarities and in most cases the performance would be similar, so iterating over unbounded collection inside runtime is still not recommended.

However:

1. linked_map offers arbitrary (sparse) keys (not only a consistent range of 0..len)
2. That makes removals more efficient (no need to re-order) and also makes the key persistent, hence reliable

[xlc]

I see some use cases of storage list. Every data structure have its trade offs.
Storage list can be used as a stack, an append only storage (like events). It use less storage than linked map (which needs to store next and prev for every item).

I think Substrate should offer (secure and safe) options and guidances to help developers to make right decisions.

[bkchr]

You will need to iterate events at the end of each block and that will create a lot of database accesses.

[xlc]

Why do I need to iterate events? To conver to Vec for compatibility reason? I believe polkadotjs is able to somehow detects the format and handle both cases just like metadata versions

[bkchr]

Yeah, good point^^

[sorpaas]

@bkchr

You will need to iterate events at the end of each block and that will create a lot of database accesses.

It does not necessarily need any database access. For something like Events, things are directly stored in the overlay. We only need to generate trie structure when storage_root is called, which usually happens after Events are killed.

[bkchr]

Yeah I know that we don't always access the real database. Nevertheless, we still need to have 1 lookups + 2 writes for storing one event.

(events are also part of the storage root, as they stay for the client to inspect them)

[gavofyork]

Indeed. Events are not free - the stick around until the next block and therefore bloat the trie and increase hashing required and then you'll need to iterate over all of the last block's events to delete them, increasing I/O.

[bkchr]

Events bottleneck is also already fixed in master.

So how do we continue with this one? Close?