README: Update Bert Japanese model card

[KeshavSingh29]

What does this PR do?

As mentioned in #36979 , contributing to HF model cards, specifically to Bert Japanese.

Fixes # (issue)

Before submitting

• This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
• Did you read the contributor guideline,
  Pull Request section?
• Was this discussed/approved via a Github issue or the forum? Please add a link
  to it if that's the case.
• Did you make sure to update the documentation with your changes? Here are the
  documentation guidelines, and
  here are tips on formatting docstrings.
• Did you write any new necessary tests?

Who can review?

@stevhliu

[stevhliu]

Thanks!

[stevhliu]

Suggested change

	BertJapanese is a bidirectional transformer model that keeps the same architecture as original BERT and keeps the same learning objective, i.e., to predict masked tokens in a sentence and to predict whether one sentence follows another. While the architecture is same, BertJapanese relies on specific tokenization methods (wordpiece / character) that are more suitable for Japanese text.
	BertJapanese is a [BERT](./bert) model pretrained on Japanese text. It uses the MeCab and WordPiece tokenizers or character tokenization.
	You can find all the original BERTJapanese checkpoints under the [tohoku-nlp](https://huggingface.co/tohoku-nlp) organization.
	> [!TIP]
	> This model was contributed by [tohoku-nlp](https://huggingface.co/tohoku-nlp).
	>
	> Refer to the [BERT](./bert] docs for usage examples.

[KeshavSingh29]

Thanks for the comments / edits @stevhliu
div tag on line 24 is still needed.
Apart from that I have made the changes.

[stevhliu]

Suggested change

Check Notes for additional details.

[stevhliu]

Suggested change

	The example below demonstrates how to predict the [MASK] token with [`Pipeline`] or the [`AutoModel`] class using model with MeCab and WordPiece tokenization.
	The example below demonstrates how to predict the [MASK] token with MeCab and WordPiece tokenization with [`Pipeline`], [`AutoModel`], and from the command line.

[stevhliu]

Suggested change

	> [!TIP]
	> Note that this is the base model, you need to add a task specific head and further fine-tune it to make sure you get accurate results.

[stevhliu]

import torch
from transformers import AutoModelForMaskedLM, AutoTokenizer

model = AutoModelForMaskedLM.from_pretrained("tohoku-nlp/bert-base-japanese")
tokenizer = AutoTokenizer.from_pretrained("tohoku-nlp/bert-base-japanese", torch_dtype=torch.float16, device_map="auto")

text = "今日は[MASK]天気ですね。"

inputs = tokenizer(text, return_tensors="pt")
with torch.no_grad():
    outputs = model(**inputs)
    predictions = outputs.logits

masked_index = torch.where(inputs['input_ids'] == tokenizer.mask_token_id)[1]
predicted_token_id = predictions[0, masked_index].argmax(dim=-1)
predicted_token = tokenizer.decode(predicted_token_id)

print(f"The predicted token is: {predicted_token}")

[stevhliu]

Add command line example

[stevhliu]

Don't need to change any of the API references here

[KeshavSingh29]

@stevhliu In the original doc for bert_japanese
only the tokenizer ref. is mentioned. Would you like to keep the status quo here?
For ref:

## BertJapaneseTokenizer

[[autodoc]] BertJapaneseTokenizer

[stevhliu]

Yeah lets keep it how it is originally

[stevhliu]

Remove all these notes and replace with an example of character tokenization.

Suggested change

	- The model architecture(same as original BERT by Google) comes in two variants:
	- The example below demonstrates character tokenization.
	```py
	add code example here

[stevhliu]

pip install transformers["ja"]

import torch
from transformers import pipeline

pipeline = pipeline("fill-mask", model="tohoku-nlp/bert-base-japanese", torch_dtype=torch.float16, device=0)
pipeline("今日は[MASK]天気ですね。")