Atomic update operation?

[phreeza]

Not sure how hard this would be to implement, but it would be neat to have a safe way of updating an entry. Right now I am deleting and then inserting the new entry, which seems the only way to do this, but it is quite unsafe, for example if the program is terminated right between the write and delete steps the database may become corrupted.

If there is interest in this, I would be willing to put in some work, if someone can give me guidance. Does it fundamentally go against the philosophy, or would it make sense?

[msiemens]

Hey phreeza!

First about the update operation: It would defenitely be useful to have it. I thought about something like this:

db.update({'int': 2}, where('char') == 'a')

That would update all elements, where the char field equals 'a' to have a field int equal to 2.
Running this command would result in only one write operation, so it's definitely better that a remove + insert.

I've implemented this update function and would update TinyDB soon.

Concerning the atomicity: At the moment TinyDB does not have real, fault-tolerant atomicity. I've found a interesting article about that topic: Reliable file updates with Python. If you need something closer to real atomacity, feel free to take a method from the article and implement a new storage class (see TinyDB docs) (and submit a PR).

[phreeza]

that interface is exactly what I had in mind, great. I assume this would update all matches to the query correspondingly?

I hadn't even gone as far as thinking about 'true' atomicity on the filesystem level. I think this will do just fine for my purposes.

[msiemens]

That's exactly what it does :) I'll publish the new version of TinyDB right now.

[alexreg]

I hope you don't mind me asking here: is the present update method atomic for a single-document update, or not even that?

[msiemens]

I hope you don't mind me asking here: is the present update method atomic for a single-document update, or not even that?

That depends on how exactly you define atomicity. If you're looking for atomicity as used in the ACID acronym, TinyDB doesn't come with atomicity by default. As mentioned above, one could implement some sort of atomicity by writing data to a temporary file first and then renaming it to the actual file.

But in general, TinyDB simply writes the whole database to disk on every write operation. If the Python interpreter dies or crashes mid-way, TinyDB doesn't provide any guarantees about data integrity out of the box (aside from a custom storage that provides a basic level of atomicity as mentioned above).

[alexreg]

Okay, thanks for clarifying. I think I'll go with tinyrecord, which as I understand, provides that.