Skip to content

feat(rds): add aws_s3 postgres extension (table_import / query_export)#806

Merged
vieiralucas merged 2 commits intomainfrom
worktree-rds-aws-s3-extension
Apr 28, 2026
Merged

feat(rds): add aws_s3 postgres extension (table_import / query_export)#806
vieiralucas merged 2 commits intomainfrom
worktree-rds-aws-s3-extension

Conversation

@vieiralucas
Copy link
Copy Markdown
Member

@vieiralucas vieiralucas commented Apr 28, 2026

Summary

  • Add Aurora-postgres-compatible aws_s3 extension: aws_s3.table_import_from_s3 and aws_s3.query_export_to_s3 (positional + aws_commons._s3_uri_1 composite overloads).
  • Bump aws_commons to 1.1 with the _s3_uri_1 type and create_s3_uri() constructor, plus a 1.0->1.1 upgrade script.
  • Two new bridge endpoints (/_fakecloud/rds/s3-import, /_fakecloud/rds/s3-export) wired straight into the in-memory S3 state.
  • Bake the new extension files into the prebuilt fakecloud-postgres image.

Test plan

  • cargo clippy --workspace --all-targets -- -D warnings
  • cargo fmt --all
  • cargo test -p fakecloud-rds
  • cargo test -p fakecloud-e2e --test rds_aws_s3 (Docker required — runs in CI)

Summary by cubic

Adds the RDS PostgreSQL aws_s3 extension so SQL can import from and export to S3. Also adds server bridges and upgrades aws_commons to 1.1; the extensions are baked into the prebuilt fakecloud-postgres image.

  • New Features

    • aws_s3.table_import_from_s3 and aws_s3.query_export_to_s3 (positional and aws_commons._s3_uri_1 overloads via aws_commons.create_s3_uri()).
    • New bridges: /_fakecloud/rds/s3-import and /_fakecloud/rds/s3-export (read/write in-memory S3).
    • Prebuilt image now includes aws_commons 1.1, aws_lambda, and aws_s3.
    • E2E test covers S3 → table import → query export → S3, uses a scratch table for the composite import to keep the export deterministic; docs and README updated.
  • Migration

    • If you have aws_commons 1.0 installed, run: ALTER EXTENSION aws_commons UPDATE TO '1.1' before using aws_s3.

Written for commit 3880c77. Summary will update on new commits. Review in cubic

Aurora-postgres parity for the third extension in the aws_lambda /
aws_commons family. SQL inside RDS PostgreSQL can now import S3 objects
into tables (`aws_s3.table_import_from_s3`) and export query results
back to S3 (`aws_s3.query_export_to_s3`), with the same composite-typed
overload pattern as `aws_lambda.invoke`.

- New `aws_s3` extension (control + 1.0 SQL) baked into the prebuilt
  fakecloud-postgres image. Functions talk to fakecloud over two new
  bridge endpoints, `/_fakecloud/rds/s3-import` and `/_fakecloud/rds/s3-export`.
- `aws_commons` bumped to 1.1 with the `_s3_uri_1` composite type and
  `create_s3_uri()` constructor, plus a 1.0->1.1 upgrade script for
  users of the older extension.
- Bridge handlers read/write the in-memory S3 state directly, so any
  bucket reachable via standard `GetObject`/`PutObject` is reachable
  from `aws_s3` SQL.
- E2E test (`rds_aws_s3.rs`) covers the round trip: PutObject -> table
  import (positional + composite overloads) -> query export -> GetObject.
- Docs and README sync the new extension surface.
Copy link
Copy Markdown

@cubic-dev-ai cubic-dev-ai Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

1 issue found across 13 files

Prompt for AI agents (unresolved issues)

Check if these issues are valid — if so, understand the root cause of each and fix them. If appropriate, use sub-agents to investigate and fix each issue separately.


<file name="crates/fakecloud-e2e/tests/rds_aws_s3.rs">

<violation number="1" location="crates/fakecloud-e2e/tests/rds_aws_s3.rs:118">
P2: Importing the CSV into the same table a second time changes the data used by the export step. Use a scratch table for the composite overload (or export before the second import) so the exported rows stay deterministic.</violation>
</file>

Reply with feedback, questions, or to request a fix. Tag @cubic-dev-ai to re-run a review.

Comment thread crates/fakecloud-e2e/tests/rds_aws_s3.rs Outdated
Cubic flagged the second `table_import_from_s3` call appending into
`people` after the first import, which polluted the table before the
export step could read it deterministically. Reorder so the export
runs against the 3-row state, then exercise the composite overload
into a separate `people_scratch` table.
@codecov
Copy link
Copy Markdown

codecov Bot commented Apr 28, 2026

Codecov Report

❌ Patch coverage is 0% with 126 lines in your changes missing coverage. Please review.

Files with missing lines Patch % Lines
crates/fakecloud-server/src/main.rs 0.00% 121 Missing ⚠️
crates/fakecloud-rds/src/runtime.rs 0.00% 5 Missing ⚠️

📢 Thoughts on this report? Let us know!

@vieiralucas vieiralucas merged commit 842a491 into main Apr 28, 2026
56 of 57 checks passed
@vieiralucas vieiralucas deleted the worktree-rds-aws-s3-extension branch April 28, 2026 01:09
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant