Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .claude-plugin/marketplace.json
Original file line number Diff line number Diff line change
Expand Up @@ -123,7 +123,7 @@
"name": "databases-on-aws",
"source": "./plugins/databases-on-aws",
"tags": ["aws", "database", "aurora", "dsql", "serverless", "postgresql"],
"version": "1.0.0"
"version": "1.1.0"
},
{
"category": "deployment",
Expand Down
2 changes: 1 addition & 1 deletion plugins/databases-on-aws/.claude-plugin/plugin.json
Original file line number Diff line number Diff line change
Expand Up @@ -22,5 +22,5 @@
"license": "Apache-2.0",
"name": "databases-on-aws",
"repository": "https://github.com/awslabs/agent-plugins",
"version": "1.0.0"
"version": "1.1.0"
}
2 changes: 1 addition & 1 deletion plugins/databases-on-aws/.codex-plugin/plugin.json
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
{
"name": "databases-on-aws",
"version": "1.0.0",
"version": "1.1.0",
"description": "Expert database guidance for the AWS database portfolio. Design schemas, execute queries, handle migrations, and choose the right database for your workload.",
"author": {
"name": "Amazon Web Services",
Expand Down
10 changes: 4 additions & 6 deletions plugins/databases-on-aws/skills/dsql/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -153,13 +153,11 @@ defaults that may change — when a user's decision depends on an exact limit, v
| Max indexes per table | 24 | `aurora dsql index limits` |
| Max columns per index | 8 | `aurora dsql index limits` |
| IDENTITY/SEQUENCE CACHE values | 1 or >= 65536 | `aurora dsql sequence cache` |
| Supported column data types | See docs | `aurora dsql supported data types` |

**When to verify:** Before recommending batch sizes, connection pool settings, or schema designs
where hitting a limit would cause failures. No need to verify for general guidance or when
the exact number doesn't affect the user's decision.
**When to verify:** Before recommending batch sizes, connection pool settings, or schema designs where hitting a limit would cause failures; any time the exact number can affect user decision.

**Fallback:** If `awsknowledge` is unavailable, use the defaults above and note to the user
that limits should be verified against [DSQL documentation](https://docs.aws.amazon.com/aurora-dsql/latest/userguide/).
**Fallback:** If `awsknowledge` is unavailable, use the defaults above and flag that limits should be verified against [DSQL documentation](https://docs.aws.amazon.com/aurora-dsql/latest/userguide/).

## CLI Scripts Available

Expand Down Expand Up @@ -208,7 +206,7 @@ ALTER COLUMN TYPE, DROP COLUMN, DROP CONSTRAINT → Table Recreation Pattern (Wo
- MUST include tenant_id in all tables
- MUST use `CREATE INDEX ASYNC` exclusively
- MUST issue each DDL in its own transact call: `transact(["CREATE TABLE ..."])`
- MUST store arrays/JSON as TEXT
- MUST serialize arrays as TEXT or JSON; cast back at query time (`string_to_array(text, ',')` or `jsonb_array_elements_text(json::jsonb)`)

### Workflow 2: Safe Data Migration

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,8 @@ effortless scaling, multi-region viability, among other advantages.
- **REQUIRED: Follow DDL Guidelines** - Refer to [DDL Rules](#schema-ddl-rules)
- **SHALL repeatedly generate fresh tokens** - Refer to [Connection Limits](auth/authentication-guide.md#connection-rules)
- **ALWAYS use ASYNC indexes** - `CREATE INDEX ASYNC` is mandatory
- **MUST Serialize arrays/JSON as TEXT** - Store arrays/JSON as TEXT (comma separated, JSON.stringify)
- **MUST serialize arrays as TEXT or JSON** - see [Schema Design Rules](#schema-design-rules)
- **MUST cast to `JSONB` at query time** for JSONB operators — see [Supported Data Types](#supported-data-types)
- **ALWAYS Batch within row limit** - maintain transaction limits (verify via `awsknowledge`: `aurora dsql transaction limits`)
- **REQUIRED: Build and sanitize all SQL with `safe_query.build()`** - See [Input Validation](../mcp/tools/input-validation.md#required-pattern)
- **MUST follow correct Application Layer Patterns** - when multi-tenant isolation or application referential integrity are required; refer to [Application Layer Patterns](#application-layer-patterns)
Expand Down Expand Up @@ -53,9 +54,8 @@ effortless scaling, multi-region viability, among other advantages.

### Schema Design Rules

- MUST use **simple PostgreSQL types:** VARCHAR, TEXT, INTEGER, BOOLEAN, TIMESTAMP
- MUST store arrays as TEXT (comma-separated is recommended)
- MUST store JSON objects as TEXT (JSON.stringify)
- MUST verify column types via `awsknowledge`: `aurora dsql supported data types` or the [DSQL supported data types list](https://docs.aws.amazon.com/aurora-dsql/latest/userguide/working-with-postgresql-compatibility-supported-data-types.html)
- MUST serialize arrays as TEXT or JSON; cast back at query time via `string_to_array(text, ',')` or `jsonb_array_elements_text(json::jsonb)`
- ALWAYS include tenant_id in tables for multi-tenant isolation
- SHOULD create async indexes for tenant_id and common query patterns

Expand Down Expand Up @@ -124,9 +124,9 @@ UPDATE table SET c = 'default' WHERE c IS NULL; ← AFTER ADD COLUMN

### Supported Data Types

```
VARCHAR, TEXT, INTEGER, DECIMAL, BOOLEAN, TIMESTAMP, UUID
```
**MUST verify** column types against the [DSQL supported data types docs](https://docs.aws.amazon.com/aurora-dsql/latest/userguide/working-with-postgresql-compatibility-supported-data-types.html) or via `awsknowledge`: `aurora dsql supported data types` — the supported set evolves, so do not treat any static list as exhaustive.

`JSONB`, arrays, and `INET` are **[runtime-only](https://docs.aws.amazon.com/aurora-dsql/latest/userguide/working-with-postgresql-compatibility-supported-data-types.html#working-with-postgresql-compatibility-query-runtime)** — cast at query time

### Supported Key

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -54,7 +54,7 @@ async function batchInsert(pool, tenantId, items) {
await client.query(
`INSERT INTO entities (tenant_id, name, metadata)
VALUES ($1, $2, $3)`,
[tenantId, item.name, JSON.stringify(item.metadata)]
[tenantId, item.name, item.metadata]
);
}

Expand Down Expand Up @@ -105,7 +105,7 @@ async function processBatches(pool, tenantId, batches, startIdx, step) {
for (const item of batch) {
await client.query(
'INSERT INTO entities (tenant_id, name, metadata) VALUES ($1, $2, $3)',
[tenantId, item.name, JSON.stringify(item.metadata)]
[tenantId, item.name, item.metadata]
);
}

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -129,9 +129,12 @@ INSERT INTO distributors VALUES (nextval('order_seq'), 'nothing');

---

## Data Serialization
## Runtime-Only Types

**Pattern:** MUST store arrays and JSON as TEXT (runtime-only types). Per [DSQL docs](https://docs.aws.amazon.com/aurora-dsql/latest/userguide/working-with-postgresql-compatibility-supported-data-types.html), cast to JSON at query time.
`JSONB`, arrays, and `INET` are [runtime-only](https://docs.aws.amazon.com/aurora-dsql/latest/userguide/working-with-postgresql-compatibility-supported-data-types.html#working-with-postgresql-compatibility-query-runtime) — not valid as column types.

- **MUST** serialize arrays as `TEXT` or `JSON` — use `TEXT` (comma-separated) for homogeneous short strings; use `JSON` when elements may contain commas or aren't homogeneous
- **MUST** cast back at query time — `string_to_array(text, ',')` for TEXT, `jsonb_array_elements_text(json::jsonb)` for JSON

```javascript
function toTextArray(values) {
Expand All @@ -142,32 +145,21 @@ function fromTextArray(textValue) {
return textValue ? textValue.split(',').map(v => v.trim()) : [];
}

function toTextJSON(object) {
return JSON.stringify(object);
}

function fromTextJSON(textValue) {
if (!textValue) return null;
try {
return JSON.parse(textValue);
} catch (err) {
console.warn('Invalid JSON in column:', err.message);
return null;
}
}

const categoriesText = toTextArray(['backend', 'api', 'database']);
await pool.query('INSERT INTO projects (project_id, categories) VALUES ($1, $2)', [projectId, categoriesText]);

const configText = toTextJSON({ theme: 'dark', notifications: true });
await pool.query('INSERT INTO user_settings (user_id, preferences) VALUES ($1, $2)', [userId, configText]);
await pool.query(
'INSERT INTO user_settings (user_id, preferences) VALUES ($1, $2)',
[userId, { theme: 'dark', notifications: true }],
);
```

Query-time operations:

```sql
SELECT user_id, preferences::jsonb->>'theme' as theme
FROM user_settings WHERE preferences::jsonb->>'notifications' = 'true';
SELECT user_id, preferences::jsonb->>'theme' AS theme
FROM user_settings
WHERE preferences::jsonb->>'notifications' = 'true';

SELECT project_id, string_to_array(categories, ',') as category_array FROM projects;
SELECT project_id, string_to_array(categories, ',') AS category_array FROM projects;
```
Original file line number Diff line number Diff line change
Expand Up @@ -45,7 +45,7 @@ transact([
price DECIMAL(10,2) NOT NULL,
category VARCHAR(255) DEFAULT 'other' CHECK (category IN ('electronics', 'clothing', 'food', 'other')),
tags TEXT,
metadata TEXT,
metadata JSON,
stock INTEGER DEFAULT 0 CHECK (stock >= 0),
is_active BOOLEAN DEFAULT true,
created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP,
Expand All @@ -70,7 +70,7 @@ transact(["CREATE INDEX ASYNC idx_products_category ON products(tenant_id, categ
| `MEDIUMTEXT` | `TEXT` |
| `ENUM(...)` | `VARCHAR(255)` with `CHECK` constraint |
| `SET(...)` | `TEXT` (comma-separated) |
| `JSON` | `TEXT` (JSON.stringify) |
| `JSON` | `JSON` |
| `UNSIGNED` | `CHECK (col >= 0)` |
| `TINYINT(1)` | `BOOLEAN` |
| `DATETIME` | `TIMESTAMP` |
Expand Down Expand Up @@ -99,7 +99,6 @@ transact(["CREATE INDEX ASYNC idx_products_category ON products(tenant_id, categ
- **MUST convert** AUTO_INCREMENT to UUID with gen_random_uuid(), IDENTITY column with `GENERATED AS IDENTITY (CACHE ...)`, or explicit SEQUENCE -- ALWAYS use `GENERATED AS IDENTITY` for auto-incrementing columns (see [AUTO_INCREMENT Migration](ddl-auto-increment.md#auto_increment-migration))
- **MUST replace** ENUM with VARCHAR and CHECK constraint
- **MUST replace** SET with TEXT (comma-separated)
- **MUST replace** JSON columns with TEXT
- **MUST replace** FOREIGN KEY constraints with application-layer referential integrity
- **MUST replace** ON UPDATE CURRENT_TIMESTAMP with application-layer updates
- **MUST convert** all index creation to use CREATE INDEX ASYNC
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -97,7 +97,7 @@ Map MySQL data types to their DSQL equivalents.

| MySQL Type | DSQL Equivalent | Notes |
| -------------- | --------------------------------------------------------- | ---------------------------------------------------------------------------------------------------- |
| JSON | TEXT | MUST store as TEXT |
| JSON | `JSON` | Direct equivalent |
| AUTO_INCREMENT | UUID with gen_random_uuid(), IDENTITY column, or SEQUENCE | See [AUTO_INCREMENT Migration](ddl-auto-increment.md#auto_increment-migration) for all three options |

---
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -35,7 +35,7 @@ These guidelines apply when users say "Get started with DSQL" or similar phrases
- Example:
- "What column names would you like in this table?"
- "What is the column name of the primary key?"
- "JSON must be serialized. Would you like to stringify the JSON to serialize it as TEXT?"
- "Would you like to store this in a `JSON` column, or serialize as TEXT?"

**Examples:**

Expand Down Expand Up @@ -252,7 +252,9 @@ cargo add aws-sdk-dsql tokio --features full
- If yes, MUST verify DSQL compatibility:
- No SERIAL types (use `GENERATED AS IDENTITY` with sequences, or UUID)
- No foreign keys (implement in application)
- No array/JSON column types (serialize as TEXT)
- Serialize arrays as TEXT or JSON; cast back at query time (`string_to_array(text, ',')` / `jsonb_array_elements_text(json::jsonb)`)
- Cast to `JSONB` at query time for JSONB operators (`JSONB` is not a valid column type)
- Verify column types against the [supported data types list](https://docs.aws.amazon.com/aurora-dsql/latest/userguide/working-with-postgresql-compatibility-supported-data-types.html)
- Reference [`./development-guide.md`](./development-guide.md) for full constraints

**If no schema found:**
Expand Down Expand Up @@ -349,7 +351,7 @@ Let them know you're ready to help with more:
**ALWAYS follow these rules:**

1. **Indexes:** Use `CREATE INDEX ASYNC` - synchronous index creation not supported
2. **Serialization:** Store arrays/JSON as TEXT (comma-separated or JSON.stringify)
2. **Runtime-only types:** Serialize arrays as TEXT or JSON; cast to `JSONB` at query time for JSONB operators
3. **Referential Integrity:** Implement foreign key validation in application code
4. **DDL Operations:** Execute one DDL per transaction, no mixing with DML
5. **Transaction Limits:** Maximum 3,000 row modifications, 10 MiB data size per transaction
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -52,6 +52,18 @@ Before referring to any listed errors, refer to the complete [DSQL troubleshooti
- Use native TLS libraries (not OpenSSL 1.0.x)
- Set `server_name_indication` to cluster endpoint in SSL config

## Cluster Lifecycle

See [cluster lifecycle](https://docs.aws.amazon.com/aurora-dsql/latest/userguide/cluster-lifecycle.html) for state definitions and behavior.

### Error: "FATAL: unable to accept connection, waking up cluster, please retry later"

The cluster is `INACTIVE` and waking up. Poll `aws dsql get-cluster --identifier <id> --region <region> --query status --output text` until `ACTIVE`, then retry.

### Error: `FailedPrecondition` when backing up an `IDLE` / `INACTIVE` cluster

Connect to the cluster to wake it, then retry the backup.

## Incompatibility

When migrating from PostgreSQL, remember DSQL doesn't support:
Expand Down Expand Up @@ -83,10 +95,8 @@ See [full list of unsupported features](https://docs.aws.amazon.com/aurora-dsql/
**Cause:** Using TEXT[] or other array types
**Solution:**

1. Change column to TEXT
2. Store as comma-separated: `"tag1,tag2,tag3"`
3. Or use JSON.stringify: `"["tag1","tag2","tag3"]"`
4. Deserialize in application layer
1. Change column to TEXT and store as comma-separated (`"tag1,tag2,tag3"`), or use a `JSON` column (`tags JSON`)
2. Deserialize in application layer; cast to `JSONB` at query time for JSONB operators

### Error: "Please use CREATE INDEX ASYNC"

Expand Down
Loading
Loading