Edge to Edge Testing & Dependency Injection

[jonathanmeier5]

Great seeing how things have evolved.

Since re-reading the chapter on Abstractions, I've been interested in your thoughts on testing interfaces that allow for DI and fakes vs those that don't. For example, without patching I'm not sure how one would inject a fake into a test calling a REST API.

The list of 6 items at the end of this section in Chapter 12 is helpful in this respect.

I do have one question though. Working through an example using that list, imagine that the example directory sync code from Chapter 3 is called from a CLI.

Would you have a full blown integration test calling from the CLI that does real file IO and then use the fake in the file IO code system's unittests?

A slightly related question about your list of 6 items:

At my current work place we use docker-compose to test our code and run our local environments.

However, our dev/production deployments derive from different configuration files than our local/test environments. In this case dev/prod runs off Kubernetes manifests vs the local and test docker-compose files.

From what I've heard, the old advice is to try to keep your local and test environments as close as possible to production.
Do you have advice for maintaining parity between your test/local configurations and your deployments? This question might be fairly platform specific, so no worries if it's out of band.

[bobthemighty]

Since re-reading the chapter on Abstractions, I've been interested in your thoughts on testing interfaces that allow for DI and fakes vs those that don't. For example, without patching I'm not sure how one would inject a fake into a test calling a REST API.

Let's say I have a ReST API that returns info on books. I can create a client for it

class BookClient:
    
    def get(self, book_id):
       ...


    def borrow(self, book_id):
       ... 

I would separate two kinds of test: a fast test suite that uses a fake client and checks:

• When my client returns a book, I format the response correctly for the client
• When we try to borrow, if there is a conflict, we raise a BookAlreadyBorrowedException
• When we get a book, if the book doesn't exist, we raise a BookNotFoundException

etc etc

Separately, I would write some acceptance tests that can run against either a mocked API or the real thing and check that my client works with the external contract. I write a lot of JS these days and there I tend to run a real HTTP server that returns pre-canned responses. I did a similar thing on a Python project, and it works pretty well.

Would you have a full blown integration test calling from the CLI that does real file IO and then use the fake in the file IO code system's unittests?

This.

Do you have advice for maintaining parity between your test/local configurations and your deployments?

We have a similar setup at Made. Developers write code using docker compose locally, and then we deploy to a Nomad cluster in AWS. I don't honestly have much advice for you, except that the 12-Factor Manifesto is under-rated in this regard. If your apps are 12-factor compliant, then it's straightforward to move them between environments and it's basically fine.

When configuration is that straightforward, and everything is addressable via URIs that are injected with environment vars, the differences between local/prod tend to come down to things like DB versions and load balancer configurations, some of which are unavoidable.

[hjwp]

Would you have a full blown integration test calling from the CLI that does real file IO and then use the fake in the file IO code system's unittests?

re: integration (or e2e) tests doing real file IO, absolutely yes. re calling from the CLI, i'm more of a maybe.

in the appendix csvs chapter we have a CLI tool. i initially wrote the tests using subrocess.run, but i've iterated on that a little, and i prefer a solution where the e2e/integration tests call a main() function from the CLI tool directly. i like it, for example, because it lets exceptions bubble up, which is helpful for debugging, and i feel it's not too much of a sacrifice in terms of making the test less realistic.

example here, including some weird python import-path hackery magic: https://github.com/python-leap/code/blob/appendix_csvs/tests/e2e/test_csv.py#L10

[hjwp]

folks, is there anything in here we should capture in the book? maybe a pointer to the CLI tests in the abstractions chapter?

re: api fakes, i wonder if we should talk more about this:

I tend to run a real HTTP server that returns pre-canned responses.

i've done this too, built a single-file http server that returns some hardcoded/simple responses. for a fake external API, for example, you can have a server that stores anything it gets sent via POST into a list, and then just returns it if you ever do a GET. zero logic, but it's a real server that you run in the same docker-compose environment as the rest of your dev env, and you configure connecting to it using the same env vars as for the prod servers.

in the book we sort of do this with that fake-email-server docker image thing, mailhog, altho we didn't write our own.

so we have a hierarchy of fakes + tests:

• unit tests use an in-process, in-memory fake version of an abstraction, like FakeRepository
• e2e/acceptance tests can use a separate-container fake, either handrolled like a dummy fake api, or off-the-shelf if something like mailhog exists. (there's a fake ec2/aws thing we use too isn't there bob?)
• e2e/integration might also use a "real" external API, but connect up to a sandbox/test environment, if the 3rd party provides one (and one that's not total crap, i'm looking at you paypal).

[daniel-butler]

When and what to test is always a challenge. I find it comes down to time, effort and assessed risk. None of which is easy to get right especially when first learning to program.

The level of explanation around why choosing a given test has been adequate. It does feel like we don’t get the full picture of the trade offs and the whys on some tests when reading the book. I can try to highlight where it felt like that. The focus of the book is the design patterns not the trade offs for testing so I think it’s inevitable.

That’s my two cents with that and $5 you can buy yourself a coffee :)

[hjwp]

hey daniel, the book is almost finished now, if you want to give it another look and point out where you think we could do with more testing chat, we'd love that? we want to talk more about how these architectural patterns help you get more out of tdd. so anywhere you think we haven't justified a particular test strategy well, there's an opportunity to talk more about it...

re-open (a) new issue(s) if you get a chance?

[xtaje]

By far my favorite comment ever about this topic is this.

Some addtional tools that may help are librarlies like HTTMock at the unit level. At the acceptance/e2e level for HTTP we are using Mountebank and WireMock. If I were to do it over again I might try Hoverfly instead.