api to store and retrieve user's encrypted data

[wighawag]

Hi,
I am building a game that can potentially run without server and use only web3 api. The only gotcha is that it require to store data between transactions.

Currently I store data in localStorage but this means that a player has to go back to the device where the game started in order to continue playing.

I was thinking to encrypt that data and store it on our server. Unfortunately web3 does not have such api yet. I could of course encrypt using another key that the player memories/record somewhere but a web3 api would be much better.

But I would like to go further. I think it will be common for dapps to require secret data to be kept between several transaction (ens registrar dapp for example) and web3 should provide an api to not only encrypt data but to store and retrieve it. If it become a standard, dapps can then rely on that feature to give a better experience to their user.

Node themeselves could provide a simple api to store and retreive encrypted data.

There are of courses security issues arround that but we could follow what browser do with domains and only page retrived for a specific domain could retrive data saved at that same domain.

What do you think?

Would metamask be interested to provide an encrypted storage space for its user ?

It would be of course synced across all metamask instance using the same vault

[danfinlay]

For context, pasting a conversation we had on Gitter:

Ronan @wighawag 03:00
Hey, is there anybody who would like to give feedback on this issue : #2000 I plan to create a EIP for that as it is not specific to metamask but I'd like to hear some feedback first.

Dan Finlay @FlySwatter 08:10
We'll be discussing it at our team meeting today @ronan, I think it's interesting, but there are definitely a number of things to make sure we do right.

Ronan @wighawag 09:51
@FlySwatter great, looking forward to hear what you think. I agree, the way browser handle cookies and local storage is far from perfect and it is worth thinking properly before doing something similar with web3

Dan Finlay @FlySwatter 09:52
For one thing, if a Ðapp asked metamask to store too much, it could exceed our own storage allocation!
I do very much want a second set of encryption keys

Ronan @wighawag 09:53
how much do you have and how much do you use currently?

Dan Finlay @FlySwatter 09:53
I don’t recall offhand, we don’t use nearly the allocation, but that doesn’t mean we couldn’t hit it.
There is an API for requesting more from the user
But anyways, that all turns into a larger feature
It might be nice if we could encrypt, and then allow the Ðapp to store the encrypted data in its own local storage?

Ronan @wighawag 09:54
ok, yes localstroage and cookie have limit too, forgot what they are. One thing with encryption is that you can displace them on a backend and retroiev on demand for each dapp to keep low local use
yes, dapp woudl just need a api to retrieve latest data

Dan Finlay @FlySwatter 09:56
Part of me thinks this kind of encrypted storage could also just be handled by something like Swarm or FileCoin
Once we start storing data, you could say are we just storing it on that local computer, or wouldn’t you want it on every computer with that account?
And so then you’re talking about distributed storage,

Ronan @wighawag 09:56
that's right

Dan Finlay @FlySwatter 09:56
and we’d need incentive for that storage, rather than just throwing more free backend at it.
We still don’t even do that for MetaMask accounts itself
Hence why you have to re-enter nicknames after restoring on a new computer

Ronan @wighawag 09:58
I think the main problem is authentication for distributed storage. we need a way to ensure that another dapp cannot ask for decrypting data put by a another dapp

Dan Finlay @FlySwatter 10:00
That might just be a client-enforced constraint.
Ultimately if someone has the keys, and the app is in the open, that user can decrypt.
But yeah there’s also a UX question about teaching people about publicly encrypted data. “This data is private, for now, but as computers advance, it may become publicly viewable."

Ronan @wighawag 10:02
yes the user decrypting is ok, the problem is malicious dapp that ask for decryption and steal the data for malicious purpose
what a ui could do is check a prefix or something and that prefix need to match the domain name on which the page is serve. or something like that

[step21]

If I may make a suggestion, this seems like it would turn metamask into a general purpose sync engine, independent of how it would be implemented exactly or how data would be stored. If I would do something like that, I think it would be sufficient for now to use chrome/firefox sync, which every browser user has access to. For firefox it is described here https://developer.mozilla.org/en-US/Add-ons/WebExtensions/API/storage/sync

[wighawag]

Hi @step21 , this looks like a potential implementation mechanism for metamask to support syncing

though while I created the issue in the first place I understand @danfinlay concern. This might be the case that it should not be signer responsibility to store data. Metamask currently do not require any storage backend to operate (except that it needs to be link to a blockchain node) and adding that responsibility would increase the scope of a signer significantly.

Nevertheless I think we need a way for dapps to be able to sync local data across devices without having to make the user remember an extra passphrase and ideally without requiring confirmation from the user.

one way to do it currently is to store data encrypted on ipfs/swarm. This unfortunately require the user/dapp to recall the hash on the other devices. This is not convenient.

While you could use ENS to associate that hash with a fixed name, it would require a tx and thus confirmation for every update (not practical, nor cheap)

Another solution is for dapps to provide the storage backend and only allow signed data to be stored at deterministic location. Since confirmation is undesirable, the signing key could be generated from a passphrase that the user could retrieve from one device and use it to sync on another device.

This is not ideal still as the user would need to remember to keep that passphrase in mind or would lost the information forever if it lost the original device and did not sync it yet to another.

The ideal solution would be to rely on the user only needing to remember the signer seed (metamask seed words for example)

This is where the api mentioned on this issue would be helpful

What if the signer responsibility would be to store the hash of data per dapps? This would reduce the amount of data required but not remove the burden of a new responsibility.

The solution could be that swarm nodes could reserve deterministic storage space (addressable not by content hash as usual but by fixed address based on user public key) that only the specific public key would be allowed to write to and that signers would allow dapps to write to a subspace of it only if their origin match the subspace index/key.

In other word, swarm nodes would provide deterministic location that can be segmented / indexed. Signers like metamask would ensure that dapps can only write to the segment that belongs to their origin (the ens name, hash or dns name from where the javasript code is coming from).

When I mention origin this is in the same context as web browser "same origin policy" (https://en.wikipedia.org/wiki/Same-origin_policy) and is described in the context of encryption here : ethereum/EIPs#130

This way we could sync data securely across devices without signer needed to implement the storage solution nor needing user intervention but I am not sure how it fit with swarm goal and capabilities.

Hope it makes sense too :)

[bdresser]

@oed or @christianlundkvist do you have any thoughts on this?

@danfinlay is this blocked (or at least helped) by MetaMask/eth-sig-util#18 ?

[oed]

Thanks for the ping @bdresser. I'll provide some insights that we have gathered from working with uPort on these kinds of issues below.

To me there are three major types of data that we would like to store.

Dapp specific data (cookies?)

Basically data that should only be accessed by the dapp that created it. An open question here is why should we be siloing data? What a dapp goes offline and a new dapp that speaks ineracts with the same protocol is developed, shouldn't the new dapp be able to access the user data?

Shared data

This is the most exciting use case imo. Dapps can store data in a shared space, this data can then be accessed by other dapps. Here we might want to store personal data that shouldn't be public, but that is useful to multiple dapps.

Here it would also useful to have namespaces. Dapps can request read/write access to different namespaces so that the user is in control of what dapps can access which namespaces.

Attestation data

Attestations is basically a claim signed another entity about some attrubute concerning you. Here the user probably want more granular controll over what data is shared.

Implementation

We are using a concept called Identity Hubs developed by the identity foundation. This specification provides a standard API for storing personal data. An identity hub can be implemented on ipfs/swarm or using regular servers. Users could even run hubs for themself if they like. Data in the hub is of course encrypted by default.

It would indeed be cool if there was a stanard way of interacting with these kind of hubs from web3 directly. However I still think there are a bunch of open questions that needs to be considered before standardizing such a critical component.

Hope this was helpful 😃

[bdresser]

thanks @oed! lots of possibilities here – i think this could enable a lot of sophisticated dapps.

do you have any plans to work on an MVP?

[oed]

@bdresser we are already working on an mvp. Should have something to share soon :)

[danfinlay]

That's interesting stuff, @oed, looking forward to seeing what you're cooking up!

[wighawag]

By the way, swarm is coming with a solution for what I briefly mentioned above : They call it Mutable Resource Updates

release blog post : https://blog.ethereum.org/2018/06/21/announcing-swarm-proof-of-concept-release-3/