ScheduledBackup with plugin method causes barman-cloud sidecar memory growth (0.11.0)

[rf-ramos]

Summary

Despite upgrading to barman-cloud 0.11.0, the barman-cloud sidecar memory keeps growing during a ScheduledBackup (plugin method) and never shrinks.
Database size: 21.1 GiB.

I saw a similar report here (issue closed):
cloudnative-pg/plugin-barman-cloud#711 (comment)

Environment

• barman-cloud plugin version: 0.11.0 (sidecar included)
• CloudNativePG version: 1.27.0
• PostgreSQL version: 18
• Kubernetes: AKS v1.34.2

Expected behavior

ScheduledBackup completes without unbounded memory growth in the barman-cloud sidecar.

Actual behavior

During ScheduledBackup, the barman-cloud sidecar memory usage keeps increasing over time.

Reproduction

1. Deploy a CNPG Cluster with the barman-cloud plugin (see config below).
2. Create a ScheduledBackup using method: plugin.
3. Observe sidecar memory growth during backup.

ScheduledBackup manifest

apiVersion: postgresql.cnpg.io/v1
kind: ScheduledBackup
metadata:
  name: <database_name>
spec:
  schedule: 0 2 0 * * *
  backupOwnerReference: self
  cluster:
    name: <database_name>
  target: primary
  method: plugin
  pluginConfiguration:
    name: barman-cloud.cloudnative-pg.io

Cluster manifest

apiVersion: postgresql.cnpg.io/v1
kind: Cluster
metadata:
  name: <database_name>
spec:
  inheritedMetadata:
    labels:
      azure.workload.identity/use: "true"
  bootstrap:
    recovery:
      database: <database_name>
      owner: <database_name>
      secret:
        name: <redacted>
      source: production
  imageCatalogRef:
    apiGroup: postgresql.cnpg.io
    kind: ClusterImageCatalog
    name: postgresql
    major: 18
  imagePullSecrets:
    - name: <redacted>
  instances: 3
  primaryUpdateMethod: switchover
  managed:
    roles:
      - name: <database_name>
        ensure: present
        login: true
        replication: true
        passwordSecret:
          name: <redacted>
  resources:
    requests:
      cpu: "1"
      memory: 4Gi
    limits:
      memory: 4Gi
  storage:
    size: 64Gi
    storageClass: postgres-<database_name>
  walStorage:
    size: 64Gi
  monitoring:
    enablePodMonitor: false
  serviceAccountTemplate:
    metadata:
      annotations:
        azure.workload.identity/client-id: <redacted>
        azure.workload.identity/service-account-token-expiration: "86400"
        azure.workload.identity/tenant-id: <redacted>
  plugins:
    - name: barman-cloud.cloudnative-pg.io
      isWALArchiver: true
      parameters:
        barmanObjectName: <database_name>
        serverName: <database_name>-<timestamp>
  externalClusters:
    - name: production
      plugin:
        name: barman-cloud.cloudnative-pg.io
        parameters:
          barmanObjectName: <database_name>
          serverName: <database_name>-<previous_timestamp>

StorageClass manifest

apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
  name: postgres-<database_name>
provisioner: disk.csi.azure.com
reclaimPolicy: Delete
parameters:
  skuName: PremiumV2_LRS
  diskIOPSReadWrite: "3000" # default value
  diskMBpsReadWrite: "125" # default value
  cachingMode: None
  kind: Managed
allowVolumeExpansion: true

Logs from primary instance

{"level":"info","ts":"2026-02-09T00:02:00.654902071Z","msg":"Starting backup","logging_pod":"<redacted>"}
{"level":"info","ts":"2026-02-09T00:02:00.67376384Z","msg":"Starting barman-cloud-backup","logging_pod":"<redacted>","options":["--user","postgres","--name","backup-20260209000200","--cloud-provider","azure-blob-storage","--credential","managed-identity","https://<redacted>/backup","<redacted>"]}
{"level":"info","ts":"2026-02-09T00:02:01.025849551Z","logger":"barman-cloud-backup","msg":"/venv/lib/python3.13/site-packages/barman/cloud.py:231: FutureWarning: functools.partial will be a method descriptor in future Python versions; wrap it in staticmethod() if you want to preserve the old behavior","pipe":"stderr","logging_pod":"<redacted>"}
{"level":"info","ts":"2026-02-09T00:02:01.025866821Z","logger":"barman-cloud-backup","msg":"  self.buffer = self._buffer()","pipe":"stderr","logging_pod":"<redacted>"}
{"level":"info","ts":"2026-02-09T00:05:12.4727951Z","msg":"Executing barman-cloud-wal-archive","logging_pod":"<redacted>","walName":"/var/lib/postgresql/data/pgdata/pg_wal/<redacted>","options":["--cloud-provider","azure-blob-storage","--credential","managed-identity","https://<redacted>/backup","<redacted>","/var/lib/postgresql/data/pgdata/pg_wal/<redacted>"]}
{"level":"info","ts":"2026-02-09T00:05:13.31387298Z","msg":"Archived WAL file","walName":"/var/lib/postgresql/data/pgdata/pg_wal/<redacted>","startTime":"2026-02-09T00:05:12.472786965Z","endTime":"2026-02-09T00:05:13.313845555Z","elapsedWalTime":0.841058596,"logging_pod":"<redacted>"}
{"level":"info","ts":"2026-02-09T00:06:08.389816639Z","msg":"Skipping retention policy enforcement, no retention policy specified","logging_pod":"<redacted>"}
{"level":"info","ts":"2026-02-09T00:08:37.208953145Z","msg":"Executing barman-cloud-wal-archive","logging_pod":"<redacted>","walName":"/var/lib/postgresql/data/pgdata/pg_wal/<redacted>","options":["--cloud-provider","azure-blob-storage","--credential","managed-identity","https://<redacted>/backup","<redacted>","/var/lib/postgresql/data/pgdata/pg_wal/<redacted>"]}
{"level":"info","ts":"2026-02-09T00:08:37.87194076Z","msg":"Archived WAL file","walName":"/var/lib/postgresql/data/pgdata/pg_wal/<redacted>","startTime":"2026-02-09T00:08:37.208941104Z","endTime":"2026-02-09T00:08:37.871878767Z","elapsedWalTime":0.66293767,"logging_pod":"<redacted>"}
{"level":"info","ts":"2026-02-09T00:08:37.903039304Z","msg":"Executing barman-cloud-wal-archive","logging_pod":"<redacted>","walName":"/var/lib/postgresql/data/pgdata/pg_wal/<redacted>.backup","options":["--cloud-provider","azure-blob-storage","--credential","managed-identity","https://<redacted>/backup","<redacted>","/var/lib/postgresql/data/pgdata/pg_wal/<redacted>.backup"]}
{"level":"info","ts":"2026-02-09T00:08:38.499234327Z","msg":"Archived WAL file","walName":"/var/lib/postgresql/data/pgdata/pg_wal/<redacted>.backup","startTime":"2026-02-09T00:08:37.903030477Z","endTime":"2026-02-09T00:08:38.499195848Z","elapsedWalTime":0.596165375,"logging_pod":"<redacted>"}
{"level":"info","ts":"2026-02-09T00:08:41.759347667Z","msg":"Completed barman-cloud-backup","logging_pod":"<redacted>","options":["--user","postgres","--name","backup-20260209000200","--cloud-provider","azure-blob-storage","--credential","managed-identity","https://<redacted>/backup","<redacted>"]}
{"level":"info","ts":"2026-02-09T00:08:42.46143316Z","msg":"Backup completed","logging_pod":"<redacted>","backup":"20260209T000200"}
{"level":"info","ts":"2026-02-09T00:08:42.461450254Z","msg":"Refreshing the recovery window","logging_pod":"<redacted>","backupName":"backup-20260209000200"}

[dosubot]

Hi @rf-ramos! I'm Dosu and I’m helping the plugin-barman-cloud team.

The memory growth you’re seeing in the barman-cloud sidecar during ScheduledBackup is a known and tracked issue. The main root cause—Python bytecode recompilation on a read-only filesystem—was fixed in v0.11.0, which resolved most of the unbounded memory growth during WAL archiving and retention policy enforcement (PR #746, issue #711). However, even with this fix, it’s expected that memory usage can still grow significantly during large backups and may not drop back to baseline immediately after the backup completes. This is due to a combination of factors: OS-level caching (kernel page/slab cache), the plugin’s sidecar architecture, and the way Kubernetes reports container memory usage (which includes cache, not just process memory) (issue #711).

There’s no evidence of an ongoing memory leak in the plugin logic after v0.11.0, but memory may remain elevated after large or frequent backups. The only way to force memory reclamation is to restart the sidecar pod. Setting memory limits in the sidecar config can help mitigate OOM kills, but won’t prevent temporary spikes or high usage due to OS caching. Monitoring both the container’s memory and the actual process memory inside the sidecar can help distinguish between real leaks and cache effects.

If your usage pattern seems much worse than what’s described here (e.g., memory never drops at all, or grows without bound even when idle), sharing details like process vs. container memory, backup size, and frequency could help pinpoint if something else is going on. Otherwise, this is a known limitation and is still being discussed in the community.

To reply, just mention @dosu.

---

^{How did I do? Good | Irrelevant | Incorrect | Verbose | Hallucination | Report 🐛 | Other}

   

[rf-ramos]

I enabled the pprof server on barman and noticed a discrepancy between the memory used by the heap and the memory reported by the container_memory_working_set_bytes metric.
The container_memory_rss metric was consistent with what I observed in the pprof profile.
By adding a RAM limit on the plugin-barman-cloud container, the problem disappeared, without triggering an OOMKilled or any other issues.
The problem is now resolved. I'm sharing this in case it helps others encountering the same issue.