Refactor bert-based models to use global attention function

[Marcel256]

Feature request

Refactoring of the attention modules in bert-based models to use global attention function

Motivation

Enabling easier support of SDPA and flash attention while minimizing code duplication in Bert copies

Your contribution

I already created a draft PR #37494 to outline the changes required. Would love to get feedback and would continue working on this PR if needed

[ccdv-ai]

is flex_attention planned?

transformers/src/transformers/modeling_utils.py

Lines 5928 to 5932 in 40cba20

	_global_mapping = {
	"flash_attention_2": flash_attention_forward,
	"flex_attention": flex_attention_forward,
	"sdpa": sdpa_attention_forward,
	}

[Marcel256]

I am currently considering it, but I would probably do it in a followup PR

[Marcel256]

@ArthurZucker should I continue working on this?

[ArthurZucker]

I think it is coming in #38108

[ccdv-ai]

I dont think #38108 covers encoder only models yet

[vasqu]

Hey there 👋 I'll probably cover encoder models in a follow-up PR to #38108 @ccdv-ai

The necessary attention types are already covered by the refactor (bart encoder for example) - will hopefully be smoother to go through :)

[vasqu]

For those that are interested: #38301

[ArthurZucker]

Yep sorry it's the follow up! 🤗

[Marcel256]

Done in #38301