This repository was archived by the owner on Feb 7, 2025. It is now read-only.

This repository was archived by the owner on Feb 7, 2025. It is now read-only.

Missing scale in CrossAttention class #146

Closed

Assignees

Labels

opened

on Dec 16, 2022

The attention mechanism in the cross-attention class is missing the multiplication by the scale factor.

GenerativeModels/generative/networks/nets/diffusion_model_unet.py

Line 141 in dd39c70

attention_scores = torch.matmul(query, key.transpose(-1, -2))

Metadata

Assignees

Warvito

Labels

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests