[Discussion] Migrate the template system from Mustache to other alternatives

[wing328]

Currently, Swagger Codegen mainly uses mustache as the template system for code generation.

Mustache, a logic-less template system, may not be the best template system for this project and we would like to gather feedback from the community to see if anyone has a better suggestion.

This is not to say we hit some showstoppers with mustache and it must be replaced. If there are better alternatives and tools to ease the migration, we would definitely consider as our goal has always been making developers' (specially contributors) life easier.

[jimschubert]

I would suggest we consider using Liquid. (see Liqp). We may be able to reuse quite a bit of the current templates.

Another thing I'd like to consider is how we define the templating structure. I think it would be interesting to maintain the target directory structure within the language's resources directory. This would work for C# and ASP.NET generators, but I don't know how well it would work for generators supporting libraries.

[fehguy]

Thanks for starting this conversation, @wing328. If you look in the way-back machine, we chose STL for the first version, but it's lack of traction and unfamiliar syntax was a big barrier to using this project and contributing. I'm going to guess that the web world has some really good templates now that we should consider. Mustache was a great choice when we selected it, because it was easy to use and fast to iterate.

If the web experts could chime in with what they're using now, that'd be great input. We can always keep with Mustache but it's definitely worth considering alternatives, even it's sibling handlebars because we're really having to bake a lot of logic into them.

Lastly, it's very important that whatever template syntax we use has good tool support. I know that Golang has a nice template framework but we'd be seriously constrained by the runtime (for now at least).

[cbornet]

If we want to stay on a JVM runtime, Thymeleaf is pretty popular in the Spring MVC community for server rendering and it has a non-html mode

[pelepelin]

IMHO, in web world the most popular syntax style is Django/Jinja/Twig, but Mustache/Handlebars is popular enough, so unless you have a pressing matter, the style change does not worth it.

[cbornet]

To continue the discussion (that can lead us very far...), I think most modern web sites do client-side rendering with logic-less templating and an intermediate presentation layer (MVVM way). I'm not sure this fits well with our (current) architecture where we have language libraries that share java code and have distinct templates.
But maybe we could use a subclass or interface impl for each library where they would manage their own presentation. This also means that the libraries should be able to manipulate the template engine (add a handlebars helper function for instance).
If we want to go that way (which I think is the cleanest if it is possible), handlebars is probably the best choice as we can reuse our existing mustache and enhance them, it is very well known by the community and very easy to learn.

[ClayAtWork]

The only requirement I would like to add is creating custom functions to be used in the template files, without having to modify the java source. I think Mustache refers to these as Lambdas, and Handlebars or other templating systems might call them filters. For instance, we have a swagger spec that generates codebases for multiple platforms, and we might want to do a simple string transform for a different syntax style. SOME_ENUM -> SomeEnum. It's be great if we didn't have to modify the java source, because this would cause us to have to fork the repo and publish our own artifact, where we'd rather continue to depend on the public one.

I'm actually not sure how this would work in practice, does the Java Mustache implementation support custom Javascript functions for the templates to use?

[nzjoel1234]

As outlined in this talk https://www.youtube.com/watch?v=Vur2dAFZ4GE#t=4m25s a big problem with templates is that a whole heap of concepts need to be re-invented (i.e. loops, conditionals etc.)

Would it make sense to instead just use something like javascript to build the code?

[fehguy]

@nzjoel1234 I believe if you're just using javascript, you don't need codegen to generate the code. Templates are a must from my POV, we're trying to avoid writing more code...

[wing328]

FYI. @tgmuender has filed #5365 to support handlebars (as an option)

[cbornet]

Handlebars has the concept of helper methods which could help us escape our current problem of "statically typed language + logic-less template". So shouldn't we try to make it our default template engine ? Seems that option would be the less painful (compared to rewriting the code or our templates in another language/template). We could have a transition phase where each language impl declares what template engine it shall use.

[heapoutofspace]

Since it might be relevant for others, here's an excerpt of a discussion between me and @wing328:

Do you guys need to convert the mustache templates to handlebars somehow?

No, the reason we chose handlebars and not something a little more full-featured (like Jinja) is the backwards compatibility it offers for mustache templates.

I would like to know more on why you made the switch and the estimated effort to do so for the whole project.

The thing that bugged me most about mustache is the lack of whitespace control. It really made for some ugly, barely readable templates and was a pain to work with. We carried on for a while until we hit another bump when we tried to add a simple conditional to our template which wasn't backed by a boolean property on the model object. Sure, we could have added another such property to one of the classes in the mustache context, or even the additionalProperties, but we thought it ought to be possible to do something like that in the template itself. Proper templating engines often offer expression evaluation in if statements, which mustache clearly doesn't, and handlebars doesn't excel at it either. We debated over a few possible engines and finally settled on handlebars since it allowed us to reuse all existing templates and even merge in upstream changes with little hassle. Like I mentioned before, handlebars doesn't offer anything like Jinja's {% if dataType == "String" %} out of the box, but it was easy to add our own {{#equals "String" dataType}} tag which served our purpose well enough.

After using it for a couple of weeks now I really believe that handlebars offers enough flexibility in this environment and results in both more readable templates and more predictable output. Through custom tags it is even possible to extend the functionality available to templates in some ways, should the need ever arise. Because of the backwards compatibility with mustache I would even suggest to switch the whole project to the handlebars engine without keeping mustache around as the default. It certainly has to be verified, but I believe there should be no difference in output for existing templates.

I suggest we let the discussion in #3950 play out a bit, but I think the 2.3.0 release would be a good opportunity to introduce such a feature. Since I have the impression that you're generally interested in the idea, I'll have a discussion with Tobias (@tgmuender) about how we can improve on his changes and most likely file another pull request with handlebars as the only engine later this week