Add proof validation to tactic mode #63
Merged
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR introduces proof validation in tactic mode (see #44).
Highlights:
getProofStatusmethod inspired by LeanDojovalidateProofmethod. It is called bycreateProofStepReponseafter each tactic application.ProofSnapshotarootGoalsattribute (type:List MVarId) containing the initial goals of the declaration before applying any tactic.proofStatusattribute toProofStepResponseshowing after each tactic the status of the proof up to this point.rootGoalsattribute toCommandOptionto extract initial goals and return (synthetic) sorries from each declaration in a command. It differs fromallTacticsin that it only extracts initial goals. Useful to test automated provers on existing Lean projects. It is similar to what LeanDojo can achieve after the tracing/parsing step.Issues:
applytheorem itself issue reported in [BUG] REPL accepts incorrect proofs #44 and returns an error (see test/self_proof_apply_check.in). However, it doesn't catch this issue when using theexact?tactic. From what I understand, it seems theexact?tactic should first fail at evaluation (runStringmethod) but doesn't.Example:
`exact?\` could not close the goal. Try `apply?` to see partial suggestions.However, in Lean REPL in tactic mode, when we run:
{"cmd": "theorem ex : False := sorry"} {"proofState": 0, "tactic": "exact?"}Try this: exact _root_.exand the tactic successfully applies.And I am surprised that
getProofStatusdoesn't catch this either. My best guess is that the proof state is decoupled from the declaration at some point. Not sure if this is indeed the case though.