Skip to content

add ec2 instance with pgbouncer#39

Merged
hrodmn merged 28 commits intomainfrom
feat/pgbouncer
Dec 12, 2024
Merged

add ec2 instance with pgbouncer#39
hrodmn merged 28 commits intomainfrom
feat/pgbouncer

Conversation

@hrodmn
Copy link
Collaborator

@hrodmn hrodmn commented Dec 5, 2024

Overview

titiler-pgstac and any other serverless functions that can send many simultaneous requests to the pgstac database can quickly overwhelm the database resources. This PR adds an EC2 instance running pgbouncer to the MAAP eoAPI stack, providing connection pooling for titiler-pgstac requests.

Features

  • Connection pooling sets the maximum connections to the actual database instance to 25
  • pool_mode = transaction provides high performance for many inbound requests
  • CloudWatch logging is set up for the pgbouncer process
  • With these changes, the test deployment which is running a t3.micro RDS instance is performing significantly better* than the t3.small instance on the dev deployment
    • based on subjective assessment of slippy map tile rendering.
    • Without these changes the test deployment is basically unusable for rendering large-scale mosaics.

Questions

  • Are there any other security measures we need to consider with this approach? @jjfrench
  • Should the pgbouncer configuration settings be dynamic according to the pgstac database size? @bitner

This is one approach for solving https://github.com/NASA-IMPACT/active-maap-sprint/issues/1052

@hrodmn hrodmn marked this pull request as ready for review December 10, 2024 13:13
@hrodmn hrodmn requested a review from jjfrench December 10, 2024 13:14
Copy link
Member

@jjfrench jjfrench left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry for the late review! I left some comments/questions but everything looks good overall! 🚀

@hrodmn hrodmn merged commit 7d675df into main Dec 12, 2024
@hrodmn hrodmn deleted the feat/pgbouncer branch December 12, 2024 01:58
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants