Are there any server stubs that can respond with examples from the schema?

[ndbroadbent]

Hello,

My goal is to automatically generate an integration test for every API client, to make sure that all my API clients are working properly and don't crash. I think the easiest way to do this would be to start a mock server, and make my API clients connect to this server by setting the host. My API changes fairly regularly, so I also want this whole process to be automated and based on the openapi specification.

So I would like to generate a server stub that I can run for some automated tests, but I need it to send valid responses. Are there are any auto-generated server stubs that can respond by sending examples from the schema?

Or maybe there is a better way to do this? I'm also looking at Prism to set up a mock server.

[ybelenko]

@ndbroadbent I'm working on PHP Slim enhancements and I'm really want to create proper mock server generation. But... it will take a lot of time before stable release. For instance, right now I try to create DataMocker class which can produce any Openapi Data Type example. The biggest issue is to take in account complex models with multidimensional arrays. The other issue is that when there is more than one response per endpoint, it's hard to say which one we should use on mock server. My prototype does work with dummy simple models, but it's not enough.

[ndbroadbent]

Hi @ybelenko, that sounds great!

I've actually realized that it would be better if I just start my application server and run API client tests against my real backend code. I ended up integrating the API client tests with my server test suite, so it starts a real server with mock data, and then runs the API client tests against that server. One advantage of that approach is that my API clients are now tightly integrated with my server code, so I'll never accidentally break one of them.

But having said that, I think mock server generation would still be really helpful and would provide some more possibilities.

[ybelenko]

@ndbroadbent
Frankly, client side unit tests doesn't bother me so much. What you can test from the client side? Response to valid request and response to invalid request, that's all. It would be waste of time to write client side unittests to test every request parameter. But it's rational to spend more time to write server side unit tests, let me explain why.

Let's say we have basic api endpoint called /authentication/login which accepts username and password in form data or JSON body. To cover this endpoint with UnitTests I usually write 3 tests: testLoginWithValidRequest, testLoginWithInvalidUsername, testLoginWithInvalidPassword. Each test sends builded request and checks response body and status code. Right after that I write DBUnitTests, I use different database fixtures to check database content before and after successfull/failed call. In other words, from server side perspective we can tests not only responses, but the whole process behind the curtains and that's much more important to me.

Mock server is essential for client and server side unit testing. But what's even more important is to test actual database, that's why I'm so interested in MysqlSchemaCodegen #955. That codegen is also can help us to generate database fixtures automatically. For instance, right now there is no way to generate SQL statements to init database in PHP server stubs.

[ybelenko]

I've spend a lot of time on php-slim4 server. With a small configuration it can respond with fake data. I would say, that mock feature is in beta state right now.

[ndbroadbent]

Hi @ybelenko, I would love to know more about the status of this work! This is still my number one priority for openapi-generator, because I've been doing this myself and writing all of the end-to-end integration tests by hand. It's extremely important that I run actual API calls to my real test server, because this has helped me discover and fix lots of bugs in the past.

But if openapi generator can now run tests with a mock server, then this should achieve the same goal. (I can start up my real application as a mock server.)

How does the php-slim4 mock server integrate with automated tests for each API client? Is it now possible to generate a full set of integration test cases where each API client will communicate with the php-slim4 mock server?

EDIT: To respond to your earlier comment, I should also mention that I use rswag for Ruby on Rails, so I am actually describing my API endpoints as a set of test cases, and this also tests the whole process on the server side and records requests/responses, etc.

So I'd really love to find a way to integrate this with openapi-generator, so that each API client library is running through the same list of automated tests, and check that the behavior is consistent across each language. That's my high-level goal that I would love to achieve at some point, but it will take a lot of work.

[ybelenko]

How does the php-slim4 mock server integrate with automated tests for each API client? Is it now possible to generate a full set of integration test cases where each API client will communicate with the php-slim4 mock server?

Not integrated at all. 😄 We're far away from this point I think.

I've ended up with my own php package(even three) to generate fake data from OpenAPI spec:

• Openapi Data Mocker
• Openapi Data Mocker Interfaces
• Openapi Data Mocker Server Middleware

There are no documentation for the last two repos yet.