Skip to content

[Feature] Add set_at_, set_, update_ methods to ReplayBuffer#3590

Merged
vmoens merged 1 commit intopytorch:mainfrom
jashshah999:feat/replay-buffer-inplace-update
Apr 4, 2026
Merged

[Feature] Add set_at_, set_, update_ methods to ReplayBuffer#3590
vmoens merged 1 commit intopytorch:mainfrom
jashshah999:feat/replay-buffer-inplace-update

Conversation

@jashshah999
Copy link
Copy Markdown
Contributor

Summary

buffer[index]["key"] = value silently fails because tensor indexing returns a copy, not a view. This adds three in-place write methods to ReplayBuffer following the TensorDict nomenclature as suggested by @vmoens:

  • set_at_(key, value, index) -- set a key at specific indices
  • set_(key, value) -- set a key across the entire buffer
  • update_(input_dict_or_td) -- batch update multiple keys in-place

All methods acquire _replay_lock and write through directly to the underlying storage TensorDict.

Changes

  • Add set_at_, set_, update_ to ReplayBuffer in replay_buffers.py
  • Add tests for all three methods

Fixes #2810

@pytorch-bot
Copy link
Copy Markdown

pytorch-bot Bot commented Apr 3, 2026

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/rl/3590

Note: Links to docs will display an error until the docs builds have been completed.

❌ 2 New Failures, 1 Cancelled Job

As of commit babe1c9 with merge base e2c8a8d (image):

NEW FAILURES - The following jobs have failed:

CANCELLED JOB - The following job was cancelled. Please retry:

This comment was automatically generated by Dr. CI and updates every 15 minutes.

@meta-cla meta-cla Bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Apr 3, 2026
Copy link
Copy Markdown
Collaborator

@vmoens vmoens left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM thanks for this!

@vmoens vmoens merged commit bb1c168 into pytorch:main Apr 4, 2026
139 of 145 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. Feature New feature ReplayBuffers

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[BUG] An unexpected behavior in replacing/updaing a part of data stored in ReplayBuffer

2 participants