[tx] Interpret transact input with set semantics, like Datomic

[rnewman]

Consider simple input like this:

[:db/add "foo" :thing/unique "x"]
[:db/add "foo" :thing/v 92]
[:db/add "bar" :thing/unique "x"]
[:db/add "bar" :thing/s true]

"foo" and "bar" are intended to be temporary IDs for the same entity. However, if that entity does not already exist in the store — that is, if a search with :thing/unique "x" returns no results — a new entid will be allocated for each, and insertion will fail.

.t [{:db/ident :thing/unique :db/valueType :db.type/string :db/cardinality :db.cardinality/one :db/unique :db.unique/identity :db/index true}]
.t [{:db/ident :thing/v :db/valueType :db.type/long :db/cardinality :db.cardinality/many}]
.t [{:db/ident :thing/s :db/valueType :db.type/boolean :db/cardinality :db.cardinality/one}]
.t [[:db/add "foo" :thing/unique "x"]
[:db/add "foo" :thing/v 92]
[:db/add "bar" :thing/unique "x"]
[:db/add "bar" :thing/s true]]

Error(MentatError(DbError(Msg("Could not update datoms: failed to add datoms not already present"))), State { next_error: Some(SqliteFailure(Error { code: ConstraintViolation, extended_code: 2067 }, Some("UNIQUE constraint failed: datoms.a, datoms.value_type_tag, datoms.v"))), backtrace: None }).

Fixing this means deducing equivalence between a graph of related tempids, so that only a single entid is allocated for equivalent entity placeholders. This is what the transactor does when it handles upserts, so it might be best to fix it there, perhaps by inserting collections of datoms grouped by tempid-entity in order that other tempid searches can succeed.

[ncalexan]

I thought deeply about this transaction this morning, and I conclude that the behaviour is correct. (The error message, of course, could use improvement.) Suppose that you knew that there was an existing entity with :thing/unique "x", so that you could replace the tempids with (lookup-ref :thing/unique "x") without altering the meaning of the transaction. Then you'd have:

[:db/add (lookup-ref :thing/unique "x") :thing/unique "x"]
[:db/add (lookup-ref :thing/unique "x") :thing/v 92]
[:db/add (lookup-ref :thing/unique "x") :thing/unique "x"]
[:db/add (lookup-ref :thing/unique "x") :thing/s true]

and here we clearly see that

[:db/add (lookup-ref :thing/unique "x") :thing/unique "x"]

is repeated. That's not allowed, to the best of my knowledge, by Datomic; and Mentat has tried to be as strict as possible and not accept overlapping transaction inputs either.

I'll admit that it took me a long time to realize that this input should be rejected, which I interpret to mean that the expectation is that it is valid and should "do what I mean". However, I'm very reluctant to weaken a fundamental assumption of the transactor, which is that inputs are non-overlapping.

If Datomic accepts the input in this ticket, I think I would try to be consistent -- but I doubt Datomic does accept the input.

[ncalexan]

If Datomic accepts the input in this ticket, I think I would try to be consistent -- but I doubt Datomic does accept the input.

A little testing shows that Datomic does accept this input, and allocates the two tempids only a single entid. Interesting...

[rnewman]

I would imagine that Datomic accumulates unique identity mappings in a persistent data structure during the transact, based on the unique identity mappings present in the store at the beginning, resulting in this behavior. That's just the Clojure way.

We interpret each datom independently against the current state. However, clearly the datoms mentioned in a transact are not entirely independent — they share tempid tokens. It's not entirely unreasonable to think that if "foo" means the same entity in:

[:db/add "foo" :x/y 123]
[:db/add "foo" :x/z 456]

that "foo" and "bar" will mean the same entity in

[:db/add "bar" :x/y 123]
[:db/add "foo" :x/y 123]

As with so many of these questions, this comes down to a bit of open/closed world perspective.

Another way of looking at this is:

• Mentat attempts to resolve the input datom set individually against the current store, then insert each.
• Datomic effectively accumulates the input datom set as a new store, then smushes it into the current store. Transact ~= persistent (with).