diff --git a/content/influxdb3/enterprise/admin/pachatree/_index.md b/content/influxdb3/enterprise/admin/pachatree/_index.md new file mode 100644 index 0000000000..25387ae717 --- /dev/null +++ b/content/influxdb3/enterprise/admin/pachatree/_index.md @@ -0,0 +1,88 @@ +--- +title: Performance upgrade preview +seotitle: Performance upgrade preview for InfluxDB 3 Enterprise +description: > + Preview performance upgrades in InfluxDB 3 Enterprise with improved + single-series query performance, consistent resource usage, wide-and-sparse + table support, and automatic distinct value caching. +menu: + influxdb3_enterprise: + name: Performance upgrade preview + parent: Administer InfluxDB +weight: 115 +influxdb3/enterprise/tags: [storage, performance, beta, preview] +related: + - /influxdb3/enterprise/get-started/setup/ + - /influxdb3/enterprise/admin/pachatree/configure/ + - /influxdb3/enterprise/admin/pachatree/monitor/ + - /influxdb3/enterprise/admin/performance-tuning/ +--- + +> [!Warning] +> #### Private preview beta +> The performance upgrade preview is available to {{% product-name %}} Trial +> and Commercial users as a private beta. These features are subject to breaking changes +> and **should not be used for production workloads**. +> +> To share feedback on this preview, see [Support and feedback options](#bug-reports-and-feedback). +> Your feedback on stability +> and performance at scale helps shape the future of InfluxDB 3. + +{{% product-name %}} includes a private preview of major performance and +feature updates. These updates represent the foundation for the upcoming 3.10 +and 3.11 releases. Available in beta now for InfluxDB 3 Enterprise, many of +these improvements will be coming to InfluxDB 3 Core in subsequent release cycles. + +## Key improvements + +- **Faster single-series queries**: Optimized data path for fetching specific + trends over long time windows, with single-digit millisecond response times. +- **Consistent resource usage**: Reduced CPU and memory spikes during heavy + compaction or ingestion bursts through key-range partitioning. +- **Wide-and-sparse table support**: Handles schemas with up to hundreds of + thousands of columns and dynamic schema evolution without expensive rewrites. +- **Automatic distinct value caches**: Dramatically reduced latency for metadata + queries like `SHOW TAG VALUES` in InfluxQL. + +## Enable the preview + +Add the `--use-pacha-tree` flag to your normal +[`influxdb3 serve` startup command](/influxdb3/enterprise/get-started/setup/): + +```bash +influxdb3 serve \ + --node-id host01 \ + --cluster-id cluster01 \ + --object-store file \ + --data-dir ~/.influxdb3 \ + --use-pacha-tree +``` + +The flag exposes additional configuration options prefixed with `--pt-`. +See [Configure the preview](/influxdb3/enterprise/admin/pachatree/configure/) +for tuning options, or [Monitor the preview](/influxdb3/enterprise/admin/pachatree/monitor/) +for system tables and telemetry. + +## Who should try the preview + +Consider enabling the preview in your staging or development environment if +you have workloads with high cardinality, wide tables, frequent backfill, or +query-heavy access patterns. + +Your feedback on stability and speed at scale helps inform the development of +features planned for general availability in the 3.10 and 3.11 releases. + +> [!Important] +> #### Important: Upgraded file format +> +> These upgrades use a new file format (`.pt` files). +> When you enable the preview, +> your existing data is automatically upgraded to the new format in place. +> While downgrading is possible, any data written after the upgrade will not +> be included in the downgrade. +> +> Systems with large amounts of data may take longer to complete the +> conversion. For the beta, we recommend starting with a fresh setup for +> testing and evaluation rather than converting existing data. + +{{< children hlevel="h2" readmore=true hr=true >}} diff --git a/content/influxdb3/enterprise/admin/pachatree/configure.md b/content/influxdb3/enterprise/admin/pachatree/configure.md new file mode 100644 index 0000000000..410da02582 --- /dev/null +++ b/content/influxdb3/enterprise/admin/pachatree/configure.md @@ -0,0 +1,452 @@ +--- +title: Configure the performance upgrade preview +seotitle: Performance upgrade preview configuration reference for InfluxDB 3 Enterprise +description: > + Complete reference for all configuration options available with the InfluxDB 3 Enterprise + performance upgrades, including memory, compaction, WAL, caching, and replication settings. +menu: + influxdb3_enterprise: + name: Configuration reference + parent: Performance upgrade preview +weight: 202 +influxdb3/enterprise/tags: [storage, configuration, beta, preview, reference] +related: + - /influxdb3/enterprise/admin/pachatree/ + - /influxdb3/enterprise/admin/pachatree/monitor/ + - /influxdb3/enterprise/admin/performance-tuning/ + - /influxdb3/enterprise/reference/config-options/ +--- + +> [!Warning] +> #### Private preview beta +> The performance upgrade preview is available to {{% product-name %}} Trial +> and Commercial users as a private beta. These features are subject to breaking changes +> and **should not be used for production workloads**. +> +> To share feedback on this preview, see [Support and feedback options](#bug-reports-and-feedback). +> Your feedback on stability +> and performance at scale helps shape the future of InfluxDB 3. + + +This page provides a complete reference for all configuration options available +with these performance upgrades. All options require the `--use-pacha-tree` flag. + +## Core settings + +| Option | Description | Default | +|:-------|:------------|:--------| +| `--use-pacha-tree` | Enable the performance upgrade preview | `false` | +| `--pt-max-columns` | Maximum total columns across the instance | `6500000` (~6.5M) | + +### Set maximum columns + +Limit the total number of columns across all databases and tables: + +```bash +influxdb3 serve \ + # ... + --use-pacha-tree \ + --pt-max-columns 1000000 +``` + +## Memory management + +Control how the upgraded storage uses memory for buffers, caches, and processing. + +| Option | Description | Default | +|:-------|:------------|:--------| +| `--pt-wal-max-buffer-size` | Maximum WAL buffer size before flush | `15MB` | +| `--pt-snapshot-max-unmerged-size` | Unmerged data threshold for snapshot | `500MB` | +| `--pt-merge-threshold-size` | Size threshold to trigger merge | `125MB` | +| `--pt-replica-max-buffer-size` | Query node buffer maximum | 50% system memory (max 16 GB) | + +### WAL buffer size + +The WAL buffer accumulates incoming writes before flushing to object storage. +Larger buffers reduce flush frequency but increase memory usage: + +```bash +influxdb3 serve \ + # ... + --use-pacha-tree \ + --pt-wal-max-buffer-size 30MB +``` + +### Snapshot threshold + +Control when the snapshot buffer triggers a snapshot creation: + +```bash +influxdb3 serve \ + # ... + --use-pacha-tree \ + --pt-snapshot-max-unmerged-size 1GB +``` + +### Merge threshold + +Set the size threshold that triggers background merge operations: + +```bash +influxdb3 serve \ + # ... + --use-pacha-tree \ + --pt-merge-threshold-size 250MB +``` + +### Query node buffer (distributed) + +For query nodes in distributed deployments, control the maximum buffer size: + +```bash +influxdb3 serve \ + # ... + --use-pacha-tree \ + --mode query \ + --pt-replica-max-buffer-size 8GB +``` + +## WAL configuration + +Configure Write-Ahead Log (WAL) behavior for durability and performance. + +| Option | Description | Default | +|:-------|:------------|:--------| +| `--pt-wal-flush-interval` | Automatic flush interval | `1s` | +| `--pt-wal-merge-on-interval` | Enable interval-based merging | `true` (all-in-one), `false` (multi-node) | +| `--pt-wal-flush-concurrency` | WAL flush worker concurrency | `max(io_threads - 2, 2)` | +| `--pt-wal-replication-interval` | Query node WAL polling interval | `250ms` | + +### Flush interval + +Control how frequently the WAL buffer flushes to object storage: + +```bash +# More frequent flushes (lower latency, more I/O) +influxdb3 serve \ + # ... + --use-pacha-tree \ + --pt-wal-flush-interval 500ms +``` + +```bash +# Less frequent flushes (higher latency, less I/O) +influxdb3 serve \ + # ... + --use-pacha-tree \ + --pt-wal-flush-interval 5s +``` + +### Flush concurrency + +Increase flush workers for high-throughput workloads: + +```bash +influxdb3 serve \ + # ... + --use-pacha-tree \ + --pt-wal-flush-concurrency 8 +``` + +### Interval-based merging + +In all-in-one mode, WAL files are merged on a time interval. +Disable for more control over merge timing: + +```bash +influxdb3 serve \ + # ... + --use-pacha-tree \ + --pt-wal-merge-on-interval false +``` + +## Compaction settings + +Configure background compaction behavior. + +| Option | Description | Default | +|:-------|:------------|:--------| +| `--pt-compactor-concurrency` | Compaction worker concurrency | `max(io_threads - 2, 2)` | +| `--pt-snapshot-merge-concurrency` | Snapshot merge concurrency | `max(io_threads - 2, 2)` | +| `--pt-compactor-cache` | Compaction cache size | 20% of file cache | + +### Compaction concurrency + +Increase compaction workers for faster background processing: + +```bash +influxdb3 serve \ + # ... + --use-pacha-tree \ + --pt-compactor-concurrency 8 +``` + +### Snapshot merge concurrency + +Control parallelism for snapshot merge operations: + +```bash +influxdb3 serve \ + # ... + --use-pacha-tree \ + --pt-snapshot-merge-concurrency 4 +``` + +### Compaction cache + +Dedicated cache for compaction operations (separate from query cache): + +```bash +influxdb3 serve \ + # ... + --use-pacha-tree \ + --pt-compactor-cache 2GB +``` + +## Cache configuration + +Configure data caching for query performance. + +| Option | Description | Default | +|:-------|:------------|:--------| +| `--pt-file-cache-size` | Data file cache size | ~20% system memory | +| `--pt-file-cache-recency` | Only cache files newer than this | `5h` | +| `--pt-file-cache-evict-after` | Evict entries not read within this duration | `24h` | +| `--pt-compactor-cache-ttl` | Compaction cache TTL | `5m` | +| `--pt-disable-data-file-cache` | Disable data file caching | `false` | + +### File cache size + +Set the maximum size for the data file cache: + +```bash +influxdb3 serve \ + # ... + --use-pacha-tree \ + --pt-file-cache-size 8GB +``` + +### Cache recency filter + +Only cache files containing data from the specified time window: + +```bash +influxdb3 serve \ + # ... + --use-pacha-tree \ + --pt-file-cache-recency 24h +``` + +### Cache eviction + +Control how long cached entries persist without access: + +```bash +influxdb3 serve \ + # ... + --use-pacha-tree \ + --pt-file-cache-evict-after 12h +``` + +### Disable caching + +For memory-constrained environments or when using external caching: + +```bash +influxdb3 serve \ + # ... + --use-pacha-tree \ + --pt-disable-data-file-cache +``` + +## File size limits + +Control the size of generated files. + +| Option | Description | Default | +|:-------|:------------|:--------| +| `--pt-gen0-max-rows-per-file` | Maximum rows per Gen0 file | `10000000` (10M) | +| `--pt-gen0-max-bytes-per-file` | Maximum bytes per Gen0 file | `100MB` | + +### Gen0 file size limits + +Control the size of Gen0 files for query and compaction performance: + +```bash +influxdb3 serve \ + # ... + --use-pacha-tree \ + --pt-gen0-max-rows-per-file 5000000 \ + --pt-gen0-max-bytes-per-file 50MB +``` + +## Enterprise replication + +Configure replication behavior for distributed deployments. + +| Option | Description | Default | +|:-------|:------------|:--------| +| `--pt-wal-replica-recovery-concurrency` | Concurrent WAL fetches during recovery | `8` | +| `--pt-wal-replica-steady-concurrency` | Concurrent fetches during steady-state | `8` | +| `--pt-wal-replica-queue-size` | Queue between fetching and merging | `100` | +| `--pt-wal-replica-recovery-tail-skip-limit` | Missing files before stopping recovery | `128` | +| `--pt-replica-gen0-load-concurrency` | Gen0 loads at startup | `16` | +| `--pt-replica-merge-interval` | Query node merge cycle interval | `100ms` | + +### Recovery concurrency + +Control parallelism during query node recovery: + +```bash +influxdb3 serve \ + # ... + --use-pacha-tree \ + --mode query \ + --pt-wal-replica-recovery-concurrency 16 +``` + +### Steady-state replication + +Configure ongoing replication performance: + +```bash +influxdb3 serve \ + # ... + --use-pacha-tree \ + --mode query \ + --pt-wal-replica-steady-concurrency 4 \ + --pt-wal-replica-queue-size 200 +``` + +### Merge interval + +Control how frequently query nodes merge replicated data: + +```bash +influxdb3 serve \ + # ... + --use-pacha-tree \ + --mode query \ + --pt-replica-merge-interval 50ms +``` + +## Other settings + +| Option | Description | Default | +|:-------|:------------|:--------| +| `--pt-enable-retention` | Enforce retention policies | `true` | +| `--pt-disable-hybrid-query` | Disable hybrid Parquet queries | `false` | + +### Retention policies + +Retention policies are enforced by default. Disable for testing or debugging: + +```bash +influxdb3 serve \ + # ... + --use-pacha-tree \ + --pt-enable-retention false +``` + +### Hybrid query mode + +Disable access to legacy Parquet data: + +```bash +influxdb3 serve \ + # ... + --use-pacha-tree \ + --pt-disable-hybrid-query +``` + +## Example configurations + +### Development (minimal resources) + +```bash { placeholders="dev01|dev-cluster"} +influxdb3 serve \ + --node-id dev01 \ + --cluster-id dev-cluster \ + --object-store file \ + --data-dir ~/.influxdb3 \ + --use-pacha-tree \ + --pt-file-cache-size 512MB \ + --pt-wal-max-buffer-size 5MB \ + --pt-snapshot-max-unmerged-size 100MB \ + --pt-compactor-concurrency 2 +``` + +### Production all-in-one (8 cores, 32 GB RAM) + +```bash { placeholders="prod01|prod-cluster|S3_BUCKET|AWS_ACCESS_KEY_ID|AWS_SECRET_ACCESS_KEY"} +influxdb3 serve \ + --node-id prod01 \ + --cluster-id prod-cluster \ + --object-store s3 \ + --bucket S3_BUCKET \ + --aws-access-key-id AWS_ACCESS_KEY_ID \ + --aws-secret-access-key AWS_SECRET_ACCESS_KEY \ + --use-pacha-tree \ + --num-io-threads 8 \ + --pt-file-cache-size 8GB \ + --pt-wal-max-buffer-size 30MB \ + --pt-snapshot-max-unmerged-size 1GB \ + --pt-compactor-concurrency 4 \ + --pt-wal-flush-concurrency 4 +``` + +### High-throughput ingest node + +```bash { placeholders="ingest01|prod-cluster|S3_BUCKET|AWS_ACCESS_KEY_ID|AWS_SECRET_ACCESS_KEY"} +influxdb3 serve \ + --node-id ingest01 \ + --cluster-id prod-cluster \ + --object-store s3 \ + --bucket S3_BUCKET \ + --aws-access-key-id AWS_ACCESS_KEY_ID \ + --aws-secret-access-key AWS_SECRET_ACCESS_KEY \ + --use-pacha-tree \ + --mode ingest \ + --num-io-threads 16 \ + --pt-wal-max-buffer-size 50MB \ + --pt-wal-flush-interval 2s \ + --pt-wal-flush-concurrency 8 \ + --pt-snapshot-max-unmerged-size 2GB +``` + +### Query-optimized node + +```bash { placeholders="query01|prod-cluster|S3_BUCKET|AWS_ACCESS_KEY_ID|AWS_SECRET_ACCESS_KEY"} +influxdb3 serve \ + --node-id query01 \ + --cluster-id prod-cluster \ + --object-store s3 \ + --bucket S3_BUCKET \ + --aws-access-key-id AWS_ACCESS_KEY_ID \ + --aws-secret-access-key AWS_SECRET_ACCESS_KEY \ + --use-pacha-tree \ + --mode query \ + --num-io-threads 16 \ + --pt-file-cache-size 16GB \ + --pt-file-cache-recency 24h \ + --pt-replica-max-buffer-size 8GB \ + --pt-replica-merge-interval 50ms +``` + +### Dedicated compactor + +```bash { placeholders="compact01|prod-cluster|S3_BUCKET|AWS_ACCESS_KEY_ID|AWS_SECRET_ACCESS_KEY"} +influxdb3 serve \ + --node-id compact01 \ + --cluster-id prod-cluster \ + --object-store s3 \ + --bucket S3_BUCKET \ + --aws-access-key-id AWS_ACCESS_KEY_ID \ + --aws-secret-access-key AWS_SECRET_ACCESS_KEY \ + --use-pacha-tree \ + --mode compact \ + --num-io-threads 8 \ + --pt-compactor-concurrency 8 \ + --pt-compactor-cache 4GB \ + --pt-compactor-cache-ttl 10m +``` diff --git a/content/influxdb3/enterprise/admin/pachatree/monitor.md b/content/influxdb3/enterprise/admin/pachatree/monitor.md new file mode 100644 index 0000000000..57cf556b6b --- /dev/null +++ b/content/influxdb3/enterprise/admin/pachatree/monitor.md @@ -0,0 +1,322 @@ +--- +title: Monitor the performance upgrade preview +seotitle: Monitor the performance upgrade preview in InfluxDB 3 Enterprise +description: > + Use system tables and query telemetry to monitor file status, query execution, + and overall performance when using InfluxDB 3 Enterprise performance upgrades. +menu: + influxdb3_enterprise: + name: Monitor + parent: Performance upgrade preview +weight: 203 +influxdb3/enterprise/tags: [storage, monitoring, beta, preview, system tables] +related: + - /influxdb3/enterprise/admin/pachatree/ + - /influxdb3/enterprise/admin/pachatree/configure/ + - /influxdb3/enterprise/admin/query-system-data/ +--- + +> [!Warning] +> #### Private preview beta +> The performance upgrade preview is available to {{% product-name %}} Trial +> and Commercial users as a private beta. These features are subject to breaking changes +> and **should not be used for production workloads**. +> +> To share feedback on this preview, see [Support and feedback options](#bug-reports-and-feedback). +> Your feedback on stability +> and performance at scale helps shape the future of InfluxDB 3. + +{{% product-name %}} provides system tables and a query telemetry endpoint to +monitor file status, query execution, and overall performance of these upgrades. + +## System tables + +The upgraded storage exposes internal state through system tables that you can +query with SQL. + +### system.pt_ingest_wal + +View WAL files and their partitions: + +```sql +SELECT * FROM system.pt_ingest_wal; +``` + +Example output: + +| wal_file_id | partition_id | database_id | table_id | min_time | max_time | row_count | size_bytes | +|:------------|:-------------|:------------|:---------|:---------|:---------|:----------|:-----------| +| wal_001 | p_1 | db_1 | t_1 | 2024-01-01T00:00:00Z | 2024-01-01T00:10:00Z | 50000 | 2456789 | +| wal_002 | p_1 | db_1 | t_1 | 2024-01-01T00:10:00Z | 2024-01-01T00:20:00Z | 48000 | 2345678 | + +Use this table to monitor: + +- **WAL accumulation**: Track the number and size of unmerged WAL files +- **Partition distribution**: See how data is distributed across partitions +- **Time coverage**: Verify data time ranges + +#### Monitor WAL backlog + +Check for WAL accumulation that may indicate merging is falling behind: + +```sql +SELECT + COUNT(*) as wal_file_count, + SUM(size_bytes) / 1024 / 1024 as total_size_mb, + MIN(min_time) as oldest_data, + MAX(max_time) as newest_data +FROM system.pt_ingest_wal; +``` + +### system.pt_ingest_files + +View Gen0 files with metadata: + +```sql +SELECT * FROM system.pt_ingest_files; +``` + +Example output: + +| file_id | generation | database_id | table_id | min_time | max_time | row_count | size_bytes | +|:--------|:-----------|:------------|:---------|:---------|:---------|:----------|:-----------| +| gen0_001 | 0 | db_1 | t_1 | 2024-01-01T00:00:00Z | 2024-01-01T01:00:00Z | 500000 | 45678901 | +| gen0_002 | 0 | db_1 | t_1 | 2024-01-01T01:00:00Z | 2024-01-01T02:00:00Z | 480000 | 43567890 | + +Use this table to monitor: + +- **File counts per generation**: Track compaction progress +- **File sizes**: Verify files are within configured limits +- **Time ranges**: Identify time-unbounded Gen0 files + +#### Monitor file distribution + +Check file distribution and compaction status: + +```sql +SELECT + generation, + COUNT(*) as file_count, + SUM(row_count) as total_rows, + SUM(size_bytes) / 1024 / 1024 as total_size_mb, + AVG(size_bytes) / 1024 / 1024 as avg_file_size_mb +FROM system.pt_ingest_files +GROUP BY generation +ORDER BY generation; +``` + +## Query telemetry + +The query telemetry endpoint provides detailed execution statistics for +analyzing query performance. + +### Enable query telemetry + +Query the telemetry endpoint after executing a query: + +```bash { placeholders="AUTH_TOKEN" } +curl -X GET "http://localhost:8181/api/v3/query_sql_telemetry" \ + -H "Authorization: Bearer AUTH_TOKEN" +``` + +Replace {{% code-placeholder-key %}}`AUTH_TOKEN`{{% /code-placeholder-key %}} +with your {{% token-link %}}. + +### Telemetry response + +The response includes: + +| Field | Description | +|:------|:------------| +| `query_id` | Unique identifier for the query | +| `execution_time_us` | Total execution time in microseconds | +| `chunks` | Per-chunk statistics | +| `cache_stats` | Cache hit rates by type | +| `file_stats` | File-level read statistics | + +### Example telemetry output + +```json +{ + "query_id": "q_12345", + "execution_time_us": 4523, + "chunks": [ + { + "chunk_id": "c_1", + "files_scanned": 3, + "blocks_processed": 12, + "rows_read": 24000, + "rows_returned": 150, + "bytes_read": 1234567 + } + ], + "cache_stats": { + "gen0_hits": 5, + "gen0_misses": 1, + "compacted_hits": 8, + "compacted_misses": 2 + } +} +``` + +## Performance analysis + +### Query performance metrics + +Track these key metrics for query performance: + +| Metric | Good | Warning | Action | +|:-------|:-----|:--------|:-------| +| Cache hit rate | >80% | <60% | Increase cache size or recency window | +| Rows read vs returned ratio | <100:1 | >1000:1 | Add more selective predicates | + +### Ingest performance metrics + +Monitor these metrics for write performance: + +| Metric | Healthy | Warning | Action | +|:-------|:--------|:--------|:-------| +| WAL file count | <50 | >100 | Increase merge concurrency | +| Unmerged size | <500 MB | >1 GB | Check compaction status | +| Gen0 file count | <100 | >200 | Increase compaction concurrency | + +### Monitor with SQL + +Create a performance summary query: + +```sql +-- File generation summary +SELECT + 'Gen0 files' as metric, + COUNT(*) as count, + SUM(size_bytes) / 1024 / 1024 as size_mb +FROM system.pt_ingest_files +WHERE generation = 0 + +UNION ALL + +SELECT + 'Compacted files' as metric, + COUNT(*) as count, + SUM(size_bytes) / 1024 / 1024 as size_mb +FROM system.pt_ingest_files +WHERE generation > 0 + +UNION ALL + +SELECT + 'WAL files' as metric, + COUNT(*) as count, + SUM(size_bytes) / 1024 / 1024 as size_mb +FROM system.pt_ingest_wal; +``` + +## Troubleshooting + +### High WAL file count + +**Symptom**: `system.pt_ingest_wal` shows many accumulated files. + +**Possible causes**: + +- Merge operations falling behind write rate +- Insufficient merge concurrency +- Object storage latency + +**Solutions**: + +1. Increase merge concurrency: + ```bash + --pt-snapshot-merge-concurrency 8 + ``` + +2. Increase WAL flush interval to create larger, fewer files: + ```bash + --pt-wal-flush-interval 5s + ``` + +3. Check object storage performance and connectivity + +### High cache miss rate + +**Symptom**: `cache_stats` shows >40% miss rate. + +**Possible causes**: + +- Cache size too small for working set +- Cache recency window too narrow +- Random access patterns across time ranges + +**Solutions**: + +1. Increase cache size: + ```bash + --pt-file-cache-size 16GB + ``` + +2. Extend cache recency window: + ```bash + --pt-file-cache-recency 24h + ``` + +3. Extend eviction timeout: + ```bash + --pt-file-cache-evict-after 48h + ``` + +### Slow compaction + +**Symptom**: Gen0 file count continues to grow. + +**Possible causes**: + +- Compaction concurrency too low +- Compaction cache undersized +- High write rate overwhelming compaction + +**Solutions**: + +1. Increase compaction concurrency: + ```bash + --pt-compactor-concurrency 8 + ``` + +2. Increase compaction cache: + ```bash + --pt-compactor-cache 4GB + ``` + +3. For distributed deployments, add dedicated compactor nodes: + ```bash + influxdb3 serve \ + # ... + --use-pacha-tree \ + --mode compact + ``` + +### Query node lag + +**Symptom**: Query nodes return stale data. + +**Possible causes**: + +- Replication falling behind +- Network latency to object storage +- Insufficient replica concurrency + +**Solutions**: + +1. Increase replication concurrency: + ```bash + --pt-wal-replica-steady-concurrency 8 + ``` + +2. Reduce merge interval: + ```bash + --pt-replica-merge-interval 50ms + ``` + +3. Increase replica queue size: + ```bash + --pt-wal-replica-queue-size 200 + ``` diff --git a/data/notifications.yaml b/data/notifications.yaml index 5c513b0ff5..8668e7fa9c 100644 --- a/data/notifications.yaml +++ b/data/notifications.yaml @@ -40,10 +40,42 @@ # - [The plan for InfluxDB 3.0 Open Source](https://influxdata.com/blog/the-plan-for-influxdb-3-0-open-source) # - [InfluxDB 3.0 benchmarks](https://influxdata.com/blog/influxdb-3-0-is-2.5x-45x-faster-compared-to-influxdb-open-source/) +- id: influxdb3.9-performance-preview + level: note + scope: + - /influxdb3/enterprise/ + exclude: + - /influxdb3/enterprise/admin/pachatree/ + title: 'InfluxDB 3.9: Performance upgrade preview' + slug: | + InfluxDB 3 Enterprise 3.9 includes a private preview of major performance upgrades + with faster single-series queries, wide-and-sparse table support, and more. + + message: | + InfluxDB 3 Enterprise 3.9 includes a private preview of major performance and + feature updates. + + **Key improvements:** + + - Faster single-series queries + - Consistent resource usage + - Wide-and-sparse table support + - Automatic distinct value caches for reduced latency with metadata queries + + _Preview features are subject to breaking changes._ + + For more information, see: + + - [Performance upgrade preview](/influxdb3/enterprise/admin/pachatree/) + - [InfluxDB 3 Enterprise release notes](/influxdb3/enterprise/release-notes/) + - [InfluxData Blog post](https://www.influxdata.com/blog/influxdb-3-9/") + - id: influxdb3.8-explorer-1.6 level: note scope: - / + exclude: + - /influxdb3/enterprise/ title: New in InfluxDB 3.8 slug: | Key enhancements in InfluxDB 3.8 and the InfluxDB 3 Explorer 1.6.