Foreign key constraint not included when model definitions separated

[brandon-leapyear]

We had our persistent model definitions in one file, but now we're trying to split them up into multiple files. But when we run migrations after compiling the multiple files, we notice that the database doesn't include foreign key constraints between models in different files. Specifically, the model1_model2_fkey constraint in PostgreSQL is created when the persistent model definitions are in one file, but not when they are in different files.

In debugging this, I built the following modules with -ddump-splices:

{- User.hs -}
share [mkPersist sqlSettings, mkMigrate "migrateAll"] [persistLowerCase|
  User
    name String
|]

{- Photo.hs -}
import User

share [mkPersist sqlSettings, mkMigrate "migrateAll"] [persistLowerCase|
  Photo
    owner UserId
|]

{- Both.hs -}
share [mkPersist sqlSettings, mkMigrate "migrateAll"] [persistLowerCase|
  User
    name String

  Photo
    owner UserId
|]

Looking at the splices, the FieldDef for the owner column in Photo.hs is incorrectly marked as NoReference, whereas the FieldDef for the owner column in Both.hs is correctly marked as ForeignRef "User". I believe this discrepancy causes the migration code to not set the fkey constraint in the database.

# separate files
CREATe TABLE "photo"("id" SERIAL8  PRIMARY KEY UNIQUE,"owner" INT8 NOT NULL);
CREATe TABLE "user"("id" SERIAL8  PRIMARY KEY UNIQUE,"name" VARCHAR NOT NULL);

# same file
CREATe TABLE "user"("id" SERIAL8  PRIMARY KEY UNIQUE,"name" VARCHAR NOT NULL);
CREATe TABLE "photo"("id" SERIAL8  PRIMARY KEY UNIQUE,"owner" INT8 NOT NULL);
ALTER TABLE "photo" ADD CONSTRAINT "photo_owner_fkey" FOREIGN KEY("owner") REFERENCES "user"("id");

repro.zip

[parsonsmatt]

Unfortunately this is part of why I recommend the approach in here. persistent actually doesn't know that UserId is a foreign key, it's just "some type," and since it isn't parsed in the same QQ block, it doesn't know to make a reference.

Related to #1073 #1042

[brandon-leapyear]

✨ This is an old work account. Please reference @brandonchinn178 for all future communication ✨

---

Thanks! I did see that article, but it didn't click until right now. I think I know what we can do to alleviate this problem. Thanks!

[brandon-leapyear]

✨ This is an old work account. Please reference @brandonchinn178 for all future communication ✨

---

But in reference to the underlying issue, can't you reify the type and check to see if it's a type alias for ''Database.Persist.Sql.Key record? Wouldn't that be a pretty decent way of checking to see if "some type" is a persistent foreign key?

[parsonsmatt]

That's a great idea - but we need the EntityDef in order to make the key. I think the presence of Key Foo ensures that you have an instance PersistEntity Foo, which in turn lets you do that!

[brandon-leapyear]

✨ This is an old work account. Please reference @brandonchinn178 for all future communication ✨

---

Why would you need the EntityDef? To create a ForeignRef, don't you only need HaskellName and FieldType? When would HaskellName not be equivalent to record in Key record? And when would FieldType not be FTTypeCon (GHC.Maybe.Just (TH.packPTH "Data.Int")) (TH.packPTH "Int64")))?

[parsonsmatt]

The logic does a lookup on a Map HaskellName EntityDef. We need the EntityDef to know exactly what the Key record is - it's not necessarily a newtype over Int64. For custom Id syntax, it could be any type. For composite keys, it could be several fields (this is unfortunately poorly supported right now). And for non-composite natural keys, the name of the Id is not necessarily id. Even for Id custom fields, the sql= lets you specify a different column name.

Fortunately, we can call entityDef (Proxy @record) to get the EntityDef. If we do the reify typeSyn and it gives us an alias for the data family, then we know that the instance exists.

[brandon-leapyear]

✨ This is an old work account. Please reference @brandonchinn178 for all future communication ✨

---

I see. But wouldn't entityDef (Proxy @record) need to be run at compile-time, in order to generate the ForeignRef? Or would we leave it as entityDef (Proxy @record) to be run at runtime?

[parsonsmatt]

I think that it would be fine to run it at compile-time.

GHC will compile module Foo, which will a) run Template Haskell, b) splice that Template Haskell into code, and then compile that code. In part b it will create the instance of PersistEntity Foo.

Then, in the importing module, we have the instance PersistEntity Foo available to use at compile-time.

[brandon-leapyear]

✨ This is an old work account. Please reference @brandonchinn178 for all future communication ✨

---

Right; my question is, after we reify UserId we would get back the Template Haskell Type, right? Something like AppT (ConT ''Database.Persist.Key) (ConT ''MyModule.User). Is there a function that will convert Language.Haskell.TH.Type into Proxy record, where record is the realized type represented by Type?

[parsonsmatt]

Oh, huh. Hmm. That's a great question and I don't know the answer!

[danbroooks]

Could this be re-opened if this is an issue? I have come across this when trying to split my DB models down into separate files. Unfortunately the system I work on exposes a JS based scripting system, and we generate some automatic DB fetching functions for references from one entity to another in JS... When models are split up, the reference goes from ForeignRef to NoReference, so breaks scripts we have in our scripting system.

I think I know what we can do to alleviate this problem.

What did you do instead @brandon-leapyear ? @parsonsmatt AFAIKT I am following what you're doing in that blogpost, but I still get this issue. What is it in particular that needs to be changed to stop it changing to NoReference, but allows us to define definitions in separate places (and thus win some compile time).

Thanks!

[brandon-leapyear]

✨ This is an old work account. Please reference @brandonchinn178 for all future communication ✨

---

For us, I moved the persistent model definitions into external files, using persistFileWith. Then I have the models defined in separate modules (with mkPersist), but then I use persistManyFileWith for mkMigrate in one file

[parsonsmatt]

@danbroooks How are you getting the EntityDef?

[danbroooks]

@brandon-leapyear thanks for that! Sounds like what I am doing currently though..

@parsonsmatt :

    register_con
      :: ToBackendKey SqlBackend e
      => PersistEntityBackend e ~ SqlBackend
      => FromScriptEntity e
      => ToScriptEntity e
      => Proxy e
      -> [ScriptMethod]
      -> [ScriptMethod]
      -> IO ()
    register_con proxy =
      let edef = entityDef (Nothing `asProxyTypeOf` (Just <$> proxy))
       in registerPersistCon pool cxt idx proxy edef

register_con is called like:

register_con (Proxy :: Proxy User) [] []