Foreign syntax doesn't recognize tables in other modules

[parsonsmatt]

Suppose you have the following modules:

-- Schema/Foo.hs
module Schema.Foo where

mkPersist $(persistFileWith "foo.persistentmodels")

-- foo.persistentmodels
Foo
    name Text
    Primary name

-- Schema/Bar.hs

module Schema.Bar where

import Schema.Foo

mkPersist $(persistFileWith "bar.persistentmodels")

-- bar.persistentmodels
Bar
    foo Text
    Foreign Foo fk_bar_to_foo foo

This code will currently fail, because the HaskellName "Foo" is not present in the EmbedEntityMap, because it was not parsed from the same block of text.

I think there needs to be some way to lookup the entity without requiring it be present in the map. Attempting to call entityDef (Nothing :: Maybe $(Foo)) instead of relying on what's in the Map might work.

[korayal]

It seems this issue still exists on persistent-2.11 where setting foreign key constrains on a non-primary key is introduced. It would be unfortunate to not be able to use it on a split-file setup.

Are there any solutions other than combining these separated model files back? looks like we can do parent FooId using the primary key of the Foo table, but not this:

-- Schema/Foo.hs
module Schema.Foo where

mkPersist $(persistFileWith "foo.persistentmodels")

-- foo.persistentmodels
DBFoo
    name Text
    UniqueName name

-- Schema/Bar.hs

module Schema.Bar where

import Schema.Foo

mkPersist $(persistFileWith "bar.persistentmodels")

-- bar.persistentmodels
DBBar
    foo Text
    parent Text
    Foreign DBFoo fk_foo_name parent References name

    ? Exception when trying to run compile-time code:
        could not find table HaskellName {unHaskellName = "DBFoo"} fdef=ForeignDef
{foreignRefTableHaskell = HaskellName {unHaskellName = "DBFoo"}, foreignRefTableDBName =
DBName {unDBName = "d_b_foo"}, foreignConstraintNameHaskell = HaskellName {unHaskellName =
"fk_foo_name"}, foreignConstraintNameDBName = DBName {unDBName = "d_b_barfk_foo_name"},
foreignFieldCascade = FieldCascade {fcOnUpdate = Just Cascade, fcOnDelete = Just Cascade},
foreignFields = [], foreignAttrs = [], foreignNullable = False, foreignToPrimary = False}
allnames=["DBBar"]

[parsonsmatt]

Good catch, thanks.

That error is getting thrown here: Database.Persist.Quasi.fixForeignKeysAll. We use the pent :: EntityDef to get the DBName for the entity, as well as the length of fields on the primary key (unnecessary and also a bug that probably slipped in).

fixForeignKeysAll is called here, and parse is called in parseReferences, which is used in the quasiquoter, not in the mkPersist code, which would be able to resolve this by performing some other lookup.

parseReferences is in Q, which means that we could translate these functions to Q. But then we're not returning EntityDef, we're returning Q Exp, though we could use the typed stuff and return Q (TExp EntityDef).

The UnboundForeignDef is created in takeForeign, which just puts a dummy default. So in fixForeignKeysAll, it's trying to overwrite the bad DBName with a good DBName from the EntityDef. We could simply defer that check until the liftAndFixKeys step that occurs in the TemplateHaskell code.

tl;dr:

1. We need an EntityDef for the referenced Entity in order to know the right DBName for the referenced table.
2. We don't necessarily have an EntityDef at parse-time, but we will at compile-time.
3. We need to defer the check of the validity of the foreign key until we are doing liftAndFixKeys, which takes place in the mkPersist stage of things. That will allow us to splice in the EntityDef that has been "fixed" already.

I really want to refactor the QQ and TH machinery to use more special-cased datatypes. The QQ shouldn't be outputting EntityDef since it can't possibly know. UnboundEntityDef is the right approach, but that messes up also by containing an EntityDef that needs to be "fixed." A better approach would be a separate type, and then the necessary function would be something like translate :: UnboundEntityDef -> Q (TExp EntityDef).

[jkachmar]

Just as a small addition to this, but it looks like parent FooId (in @korayal's example) has a more subtle problem.

It compiles just fine, but the migration that persistent generates will fail to include foreign key constraints.

e.g. Following the original example

-- bar.persistentmodels
Bar
    foo Text
    parent FooId

...will produce a table that contains parent with NOT NULL constraints and a BIGINT type, but no foreign key constraints at all.

[parsonsmatt]

I'm going to close this out, since we have a work-around.

See #1256 for more details.

tl;dr:

import Foo
import Bar

mkPersistWith sqlSettings $(discoverEntities) [persistLowerCase| ... |]