Error message that code must be compilable before gen will work

[jackc]

It seems that gen will not if the current package is not compilable. This caused me some confusion as I was refactoring some hand-duplicated code to use gen. I first tried it with one type and it worked. So then I deleted all my hand implementations and added the annotations to generate them.

But now gen would fail with what looks like a compile error that my types I wanted to generate were not found (because the rest of my original package depended on what I now wanted gen to generate for me). Confusing to be told that it is failing because Foo is not defined when gen is supposed to generate Foo.

Finally I figured out that gen must be parsing the Go code to read out the annotations and that was what was failing.

It'd be nice to get an error message that gen needs a compilable project before it can generate code instead of just getting what looks like a Go build error.

On the bright side, overall gen is easy to use and extend. Even with this I was able to refactor my hand-duplicated code to use gen (including a custom typewriter) in about an hour and a half.

[clipperhouse]

Yes, thanks for the feedback. It’s true that gen needs a compilable project – first it needs valid syntax to get an AST, and in turn it needs semantic validity to evaluate types.

What was the text of the error in your case? I wonder if we might communicate it better.

There have been suggestions in the past that there may be scenarios where gen can proceed even in the absence of correct code, but I haven’t been convinced. Basically, too heuristic (non-deterministic) for my taste, but deserves more thinking.

[jackc]

You can duplicate the problem by deleting a gen generated file that is used by the rest of the package.

jack@edi:~/dev/go/src/github.com/jackc/jchat/backend$ rm user_signal.go 
jack@edi:~/dev/go/src/github.com/jackc/jchat/backend$ gen
repository.go:33:23: undeclared name: UserSignal
exit status 1

[clipperhouse]

Yeah. Correct but a usability problem. Not sure what can be done, ideas?

[jackc]

Maybe just an explanation with the error message. It's reasonable to require a compilable project, it's just not immediately obvious that that is the problem.

Maybe something like "Project must be compilable to generate code" before the line number of the compile error.

[clipperhouse]

@jackc I may have a way forward here. https://groups.google.com/d/msg/golang-nuts/0uA0E1kf0Y4/chNYS6gYk3oJ

[jackc]

That would be pretty cool.

[dgnorton]

clipperhouse/typewriter#4

[rickb777]

I hit this problem too. I had some nicely working generated code. Then something broke and I very rapidly got cornered. Things snowballed. I had to throw away work to recover it.

The cryptic error messages made it worse. We could improve the messages, but the deeper problem remains.

Code generation tools should never get you into a situation where you are forced to go backwards. This is a serious problem for 'gen' because people will typically give up immediately and stop using it, instead of persevering. That would be a shame.

[clipperhouse]

@rickb777 Entirely right. It’s a usability problem, but also kind of fundamental to the nature of codegen. Good news is the types package (appears to) give us some tolerance that will cover a decent percentage of the frustrating situations. Let’s see if I can push my attempt this weekend.

[rickb777]

One thought: it's nice that gen doesn't need lots of command line options. But perhaps when things go wrong, some verbose mode would be valuable.

[rickb777]

One more thought: an easy trap to fall into is generating code containing a syntax error. This is hard to correct if you have lots of generated files. Version control comes to the rescue if you're lucky or if you planned for this, but maybe a way is needed to revert a particular change somehow, so as to get unstuck in such cases.

(Just thinking out loud.)

[clipperhouse]

Yes, syntactic problems are hard. If we can’t parse it, we can know very little, so correctness or safety go out the window.

However, in type-checking case, we can retain certain guarantees while other semantic errors are present. That’s my current approach. I’ll offer a command-line flag to override certain classes of errors.

In the case of syntactic failure, we might consider falling back to really crude string matching to get out of a jam. Would also be a command-line flag. But yuck.

[dgnorton]

How about parsing into AST, generating the new types in their files, parsing those, and THEN pass it all off to the type checker.

[clipperhouse]

@dgnorton The typewriters need type information to do their thing. Type checking happens before code generation.

[clipperhouse]

I’ve just pushed a bunch of experimental changes to allow gen to tolerate a class of type-check errors. Here are some docs: http://clipperhouse.github.io/gen/force/

Would appreciate your testing it out and letting me know what you think. @rickb777 @jackc @dgnorton