Add integration tests.

[sharkinsspatial]

Initial deployment of this construct revealed several small CDK and configuration related issues that made the resulting infrastructure fail when used in a stack. ref be40af7 and ref #8

We should build an integration test suite which deploys an ephemeral stack using the construct, it should

• Create a collection via the Ingestor API.
• Ingest an item via the Ingestor API.
• Verify the creation of the item via the STAC API endpoint.

Given the expense (in deployment time and resources) these integration tests should likely only run against new release tags.

### Tasks
- [ ] https://github.com/developmentseed/cdk-pgstac/issues/47
- [x] Create a test suite : https://github.com/developmentseed/eoapi-tests
- [x] Create a github action that runs the demo deployment, runs the tests on it, and tears down the deployment

[emileten]

We would like to see this implemented since we use cdk-pgstac to deploy the MAAP data system.

I am wondering if, now that we also have a titiler-pgstac endpoint, we also want to add an item such as this one in your list @sharkinsspatial :

• create a mosaic with the deployed titiler endpoint
• verify the creation of that mosaic through the titiler APIs

[emileten]

Other suggestion : because it takes a significant time to deploy a complete app with all these constructs, and even longer to destroy all the resources after the fact, I've been exploring LocalStack to test the deployment 'locally'. To test whether localstack is able to emulate all the services we use, I've been trying to deploy maap-cdk-pgstac out of the box with it. Although I still have a couple bugs to resolve to complete my attempt, it works better than what I expected and it's an order of magnitude faster than a real deployment.

So, assuming :

1. a localstack deployment can be used to actually try to publish and query collections (I don't know about that yet).
2. we manage to use their pro version (which we need), either by using the credits they provide to open source repos or actually getting a membership.

What do people think of that, rather than doing a full real deployment and then tear it down ? @alukach, thoughts ?

[sharkinsspatial]

@emileten I'd be curious to hear others' thoughts but I have mixed feelings about LocalStack testing. For me integration testing mainly serves the purpose of verifying that the sometimes opaque Cloudformation deployment process is successful and that our various resources work well together. I think LocalStack testing only captures the second point and even then, it may not capture some AWS edge cases well. As for the time involved in full integration testing, this is why I suggest

Given the expense (in deployment time and resources) these integration tests should likely only run against new release tags.

I'll also include some internal notes on a wider discussion about integration testing from Slack

I think there a few questions to ask here most of which revolve around feedback and thorough testing. First, the things we want immediate feedback for when developing

1. If I make a business logic code change I should be able to verify its behavior via unit testing. If the external interactions for this business logic code can’t be easily mocked, that is a code smell that likely needs refactoring.
2. Having code which works against an idealized mock is not the same as working in a real environment but it can provide us the feedback we need to sketch out multiple code changes necessary for a feature or fix.

We need to test the following things

1. That our business code changes work in an idealized model world. This is why we unit test.
2. That our business code or infrastructure code changes can be deployed to pre-existing stack successfully.
3. That our business code or infrastructure code changes in real world stack can run against real world services. This is why we integration test.
4. That our infrastructure code can be deployed completely from scratch to a new stack successfully (we often miss this one as we were continually on a project with an existing dev stack).

To achieve all this I think we need

1. Robust unit test coverage with great mocking for all business logic. This is a big ask for some projects but pays dividends later in velocity and self documentation.
2. An always running development stack where spot changes can be deployed and integration tests can can be run for fast feedback. The big issue with this shared environment is the fact that it is shared and conflicting changes running simultaneously can cause failing behavior.
3. Full integration tests for releases, where a new stack gets spun up from scratch and all integration tests are run. This is our slowest, most expensive testing and should likely only be run for releasing code to a new stage.

Because this is a construct rather than an active, production system we can forgo the "always running development stack" and focus only on 1, 3 and 4.