Fix typo in Language Modeling example scripts and update TPU type

[framoncg]

What does this PR do?

It fixes a couple of typos under the Language Modeling Pytorch examples directory that prevented the example to be run successfully. It also changes the DsitributedType in the accelerator module from TPU to TP according to the accelerate repository. This changes allow the examples to be run as they are suggested for testing in the README file

Before submitting

• This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
• Did you read the contributor guideline,
  Pull Request section?
• Was this discussed/approved via a Github issue or the forum? Please add a link
  to it if that's the case.
• Did you make sure to update the documentation with your changes? Here are the
  documentation guidelines, and
  here are tips on formatting docstrings.
• Did you write any new necessary tests?

Who can review?

@ArthurZucker

[Rocketknight1]

Hi @framoncg, "TPU" is correct! See the docs

The fixes for available are valid, though - if you remove the TPU changes and ping me I'm happy to merge those

[framoncg]

Hi @framoncg, "TPU" is correct! See the docs

The fixes for available are valid, though - if you remove the TPU changes and ping me I'm happy to merge those

Hi @Rocketknight1 thanks for taking a look! The thing is, the docs you shared are for version 0.14.0 and the latest stable 1.7.0 does not show TPU as a DataClass in the docs. The tag on the repo does show it here and when trying to run the examples as the README says it fails suggesting to change from TPU to TP. Not sure which version I should tackle for this case. I tested with the latest.

[framoncg]

TPU to TP

I did some more digging and as far as I could tell, I am wrong in this approach. Not sure what happened yet, but what I read on the stable version documentation is that TPU support for pytorch now goes through XLA and the TP I added is not the correct change. I am backtracking that part to merge the correction on the typo and I might raise an issue on the TPU problem in latest stable.

[framoncg]

Hi @Rocketknight1 if you can take a look, I rolled back the changes for TP in a new commit. Thank for the quick check

[Rocketknight1]

Hi @framoncg, yes, my bad! I didn't realize I was still on the old docs version. However, we definitely don't want to confuse TPU (Google's Tensor Processing Units) with TP (Tensor Parallel) - they're totally different concepts! I think the updated PR is good, though - we definitely want to fix those typos. Thank you!

[HuggingFaceDocBuilderDev]

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.