Add ReadMe about MFU

[gobbleturk]

Description

Add a ReadMe section discussing Model flops utilization MFU (definition and how we report it).

We may want to add sections to this in the future (e.g. hardware utilizations or memory usage)

This is meant to help clarify the recent change about our attention flop calculation change (accounting for causality) in #1988

Note for reviewers: Click on display rich diff to see resultant markdown: https://screenshot.googleplex.com/9WxhjW8EV6PWJ9B

Tests

N/A readme

Checklist

Before submitting this PR, please make sure (put X in square brackets):

• I have performed a self-review of my code.
• I have necessary comments in my code, particularly in hard-to-understand areas.
• I have run end-to-end tests tests and provided workload links above if applicable.
• I have made or will make corresponding changes to the doc if needed.

[shralex]

Not only causal, also chunked and local

[gobbleturk]

Done.

[shralex]

Its an upper bound - doesn't mean its actually achievable, right ?

[gagika]

Yes, should we add something like this: "While achieving 100% is not practical due to many factors, the MFU score effectively shows how much room is left for optimization."

[gobbleturk]

Done!

Note we've gotten 70% MFU before on v5p, I've heard 80%+ MFU (even bf16, probably also v5p), its theoretically possible to get pretty close

[shralex]

Thanks Matt!

[gagika]

Yes, should we add something like this: "While achieving 100% is not practical due to many factors, the MFU score effectively shows how much room is left for optimization."

[shralex]

Do we want to say anything more about local or chunked attention ?

[gobbleturk]

Done.

[gagika]

dividing the attention the flops -> dividing the attention flops (no the)

[gobbleturk]

Done!