Including a test project

[Zylann]

Currently, with this repo alone, there is no way to test wether or not changes work, create leaks or even compile. The latter is partly because some features are header-only and only get compiled when a project uses them, so when we do PRs we have to test them on our projects.

I'm considering the possibility of adding a folder to the repo, in which we'd put a Godot project with a GDNative library inside, which would run a series of simple tests. I'm not necessarily thinking of a full unit test suite right away. At least having a simple project for the CI to compile and run would be a good start, so new PRs can automatically get better test coverage on multiple platforms.

The goals of this:

• Check if the header-only parts of the project compile
• Check if it runs without crashing
• Check if it doesn't leak memory

My only concern is how much of a deal is it to have these extra files. Perhaps it's not too bad considering the size it would take (about a hundred kilobytes?), compared to build symbols which are measured in hundreds of megabytes.

[follower]

Would including the project as a submodule be an option?

[follower]

Also, it might be one level of abstraction too much but as part of Foreigner the libffi wrapper for Godot there's a test of basic functionality in: https://github.com/and3rson/foreigner/blob/cd1c9aee8c920990c989f103e2ebe6c525f5a414/test/test.gd#L1

Admittedly it adds an extra layer because the data goes GDScript -> GDNative -> Foreigner -> libffi -> testlib and back again but it also already exists. :D

I've got a WIP branch here that adds Mac/Linux/+cross-compiling and some other stuff:

• https://github.com/follower/foreigner/blob/6e4b7d5fd2462a826ba4b7c36b630aace57d9f98/test/test.gd

(I still need to commit/push my changes that use the recent fixes in the godot-cpp master that fix memory leaks.)

Anyway, just thought I'd mention it in case it was useful as a starting point even with the extra layer...

(Another option might be to turn https://github.com/and3rson/foreigner/blob/cd1c9aee8c920990c989f103e2ebe6c525f5a414/testlib/testlib.cpp#L1 into a GDNative plugin, exercise it with a modified test.gd & not actually use the libffi part. :) )

[Zylann]

What would be the benefit of a submodule? If the test project is one, how would that project compile on its own? If it depends back on godot-cpp wouldn't it ask for a cyclic submodule?

[piiertho]

Maybe a simple thing that can be done is using gut (godot unit testing) and run tests in ci ?

[Zylann]

Isn't GUT for GDScript? Also I'm not sure if that would address the kind of tests such project would aim for

[follower]

What would be the benefit of a submodule?

@Zylann Primarily I suggested it in regard to your comment about "My only concern is how much of a deal is it to have these extra files."

A separate submodule for the test project would mean any additional files weren't cluttering/bloating the godot-cpp repository directly.

If the test project is one, how would that project compile on its own?

FWIW I don't think the test project necessarily has to compile on its own when it's not included as a submodule.

If it depends back on godot-cpp wouldn't it ask for a cyclic submodule?

I hadn't considered that point but I seem to recall one can be selective in which submodules are updated & how deeply/recursive sub-submodules are updated, so I imagine it would be possible to handle a potentially cyclic submodule.

Alternatively, the test project could be kept separate & include godot-cpp as a submodule--which in some ways might be considered a more realistic "test" scenario which would exactly match how godot-cpp is intended to be used by other projects.

[follower]

If it's decided to go down the road of adding unit tests from the C++ side, I've got a repo that demonstrates how to use Catch2 with GDNative: https://gitlab.com/RancidBacon/godoterrortest

Re: GUT, presumably at least some of the testing is in relation to crossing the GDScript<-->godot-cpp boundary, so some testing from GDScript is presumably required but I don't know where it's best to "drive" it from.