From dca0d160fde574697d04b737c7771561e0d53c01 Mon Sep 17 00:00:00 2001 From: rafakatri Date: Sun, 17 Aug 2025 18:54:48 -0300 Subject: [PATCH 1/6] Updated OLMo model card --- docs/source/en/model_doc/olmo.md | 108 +++++++++++++++++++++++++++---- 1 file changed, 95 insertions(+), 13 deletions(-) diff --git a/docs/source/en/model_doc/olmo.md b/docs/source/en/model_doc/olmo.md index 0cfd1fb51daf..bed12cc13387 100644 --- a/docs/source/en/model_doc/olmo.md +++ b/docs/source/en/model_doc/olmo.md @@ -15,28 +15,110 @@ rendered properly in your Markdown viewer. --> *This model was released on 2024-02-01 and added to Hugging Face Transformers on 2024-04-17.* +
+
+ PyTorch + FlashAttention + SDPA + Tensor parallelism +
+
+ # OLMo +[OLMo](https://huggingface.co/papers/2402.00838) is a 7B-parameter dense language model. It uses SwiGLU activations, non-parametric layer normalization, rotary positional embeddings, and a BPE tokenizer that masks personally identifiable information. It is pretrained on [Dolma](https://huggingface.co/datasets/allenai/dolma), a 3T-token dataset. -
-PyTorch -FlashAttention -SDPA -Tensor parallelism -
+You can find all the original OLMo checkpoints under the [OLMo](https://huggingface.co/collections/allenai/olmo-suite-65aeaae8fe5b6b2122b46778) collection. + +> [!TIP] +> This model was contributed by [shanearora](https://huggingface.co/shanearora). +> +> Click on the OLMo models in the right sidebar for more examples of how to apply OLMo to different language tasks. + +The example below demonstrates how to generate text with [`Pipeline`] or the [`AutoModel`] class. + + + + + +```py +import torch +from transformers import AutoModelForCausalLM, AutoTokenizer + +tokenizer = AutoTokenizer.from_pretrained( + "allenai/OLMo-7B-hf" +) + +model = AutoModelForCausalLM.from_pretrained( + "allenai/OLMo-7B-hf", + torch_dtype=torch.float16, + device_map="auto", + attn_implementation="sdpa" +) +input_ids = tokenizer("Plants create energy through a process known as", return_tensors="pt").to(model.device) + +output = model.generate(**input_ids, max_length=50, cache_implementation="static") +print(tokenizer.decode(output[0], skip_special_tokens=True)) +``` + + + + +```bash +echo -e "Plants create energy through a process known as" | transformers-cli run --task text-generation --model allenai/OLMo-7B-hf --device 0 +``` + + + + +Quantization reduces the memory burden of large models by representing the weights in a lower precision. Refer to the [Quantization](../quantization/overview) overview for more available quantization backends. + +The example below uses [insert quantization method here](link to quantization method) to only quantize the weights to __. -## Overview +```py +import torch +from transformers import AutoModelForCausalLM, AutoTokenizer, BitsAndBytesConfig -The OLMo model was proposed in [OLMo: Accelerating the Science of Language Models](https://huggingface.co/papers/2402.00838) by Dirk Groeneveld, Iz Beltagy, Pete Walsh, Akshita Bhagia, Rodney Kinney, Oyvind Tafjord, Ananya Harsh Jha, Hamish Ivison, Ian Magnusson, Yizhong Wang, Shane Arora, David Atkinson, Russell Authur, Khyathi Raghavi Chandu, Arman Cohan, Jennifer Dumas, Yanai Elazar, Yuling Gu, Jack Hessel, Tushar Khot, William Merrill, Jacob Morrison, Niklas Muennighoff, Aakanksha Naik, Crystal Nam, Matthew E. Peters, Valentina Pyatkin, Abhilasha Ravichander, Dustin Schwenk, Saurabh Shah, Will Smith, Emma Strubell, Nishant Subramani, Mitchell Wortsman, Pradeep Dasigi, Nathan Lambert, Kyle Richardson, Luke Zettlemoyer, Jesse Dodge, Kyle Lo, Luca Soldaini, Noah A. Smith, Hannaneh Hajishirzi. +quantization_config = BitsAndBytesConfig( + load_in_4bit=True, + bnb_4bit_compute_dtype=torch.float16, + bnb_4bit_use_double_quant=True, + bnb_4bit_quant_type="nf4" +) -OLMo is a series of **O**pen **L**anguage **Mo**dels designed to enable the science of language models. The OLMo models are trained on the Dolma dataset. We release all code, checkpoints, logs (coming soon), and details involved in training these models. +model = AutoModelForCausalLM.from_pretrained( + "allenai/OLMo-7B-hf", + attn_implementation="sdpa", + torch_dtype=torch.float16, + device_map="auto", + quantization_config=quantization_config +) -The abstract from the paper is the following: +tokenizer = AutoTokenizer.from_pretrained("allenai/OLMo-7B-hf") -*Language models (LMs) have become ubiquitous in both NLP research and in commercial product offerings. As their commercial importance has surged, the most powerful models have become closed off, gated behind proprietary interfaces, with important details of their training data, architectures, and development undisclosed. Given the importance of these details in scientifically studying these models, including their biases and potential risks, we believe it is essential for the research community to have access to powerful, truly open LMs. To this end, this technical report details the first release of OLMo, a state-of-the-art, truly Open Language Model and its framework to build and study the science of language modeling. Unlike most prior efforts that have only released model weights and inference code, we release OLMo and the whole framework, including training data and training and evaluation code. We hope this release will empower and strengthen the open research community and inspire a new wave of innovation.* +inputs = tokenizer("Bitcoin is", return_tensors="pt") +inputs = {k: v.to(model.device) for k, v in inputs.items()} -This model was contributed by [shanearora](https://huggingface.co/shanearora). -The original code can be found [here](https://github.com/allenai/OLMo/tree/main/olmo). +output = model.generate(**inputs, max_length=64) +print(tokenizer.decode(output[0])) +``` ## OlmoConfig From b596fbfb72b872c9bce02f4b5a47260b82438f75 Mon Sep 17 00:00:00 2001 From: rafakatri <81170796+rafakatri@users.noreply.github.com> Date: Mon, 18 Aug 2025 15:41:44 -0300 Subject: [PATCH 2/6] Update OLMo description Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com> --- docs/source/en/model_doc/olmo.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/source/en/model_doc/olmo.md b/docs/source/en/model_doc/olmo.md index bed12cc13387..30f52a09f5a0 100644 --- a/docs/source/en/model_doc/olmo.md +++ b/docs/source/en/model_doc/olmo.md @@ -25,7 +25,7 @@ rendered properly in your Markdown viewer. # OLMo -[OLMo](https://huggingface.co/papers/2402.00838) is a 7B-parameter dense language model. It uses SwiGLU activations, non-parametric layer normalization, rotary positional embeddings, and a BPE tokenizer that masks personally identifiable information. It is pretrained on [Dolma](https://huggingface.co/datasets/allenai/dolma), a 3T-token dataset. +[OLMo](https://huggingface.co/papers/2402.00838) is a 7B-parameter dense language model. It uses SwiGLU activations, non-parametric layer normalization, rotary positional embeddings, and a BPE tokenizer that masks personally identifiable information. It is pretrained on [Dolma](https://huggingface.co/datasets/allenai/dolma), a 3T-token dataset. OLMo was released to provide complete transparency of not just the model weights but the training data, training code, and evaluation code to enable more research on language models. You can find all the original OLMo checkpoints under the [OLMo](https://huggingface.co/collections/allenai/olmo-suite-65aeaae8fe5b6b2122b46778) collection. From b6122188ce17a85c67fcecc21a3ef00f7d9907c6 Mon Sep 17 00:00:00 2001 From: rafakatri <81170796+rafakatri@users.noreply.github.com> Date: Mon, 18 Aug 2025 15:42:18 -0300 Subject: [PATCH 3/6] Fix typo Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com> --- docs/source/en/model_doc/olmo.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/source/en/model_doc/olmo.md b/docs/source/en/model_doc/olmo.md index 30f52a09f5a0..3c50197828b8 100644 --- a/docs/source/en/model_doc/olmo.md +++ b/docs/source/en/model_doc/olmo.md @@ -37,7 +37,7 @@ You can find all the original OLMo checkpoints under the [OLMo](https://huggingf The example below demonstrates how to generate text with [`Pipeline`] or the [`AutoModel`] class. - ```py import torch From 2f382601feb9ffd1dc478b9ea00822e6bbaad215 Mon Sep 17 00:00:00 2001 From: rafakatri <81170796+rafakatri@users.noreply.github.com> Date: Mon, 18 Aug 2025 15:43:07 -0300 Subject: [PATCH 4/6] Fix cli typo Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com> --- docs/source/en/model_doc/olmo.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/source/en/model_doc/olmo.md b/docs/source/en/model_doc/olmo.md index 3c50197828b8..73ecfeb5cded 100644 --- a/docs/source/en/model_doc/olmo.md +++ b/docs/source/en/model_doc/olmo.md @@ -78,7 +78,7 @@ print(tokenizer.decode(output[0], skip_special_tokens=True)) ``` - + ```bash echo -e "Plants create energy through a process known as" | transformers-cli run --task text-generation --model allenai/OLMo-7B-hf --device 0 From 31d0be86cb03ac441c7f6eb2ebb993ebc22cbe6d Mon Sep 17 00:00:00 2001 From: rafakatri <81170796+rafakatri@users.noreply.github.com> Date: Mon, 18 Aug 2025 15:43:42 -0300 Subject: [PATCH 5/6] Fix cli example Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com> --- docs/source/en/model_doc/olmo.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/source/en/model_doc/olmo.md b/docs/source/en/model_doc/olmo.md index 73ecfeb5cded..6b59c2fc0500 100644 --- a/docs/source/en/model_doc/olmo.md +++ b/docs/source/en/model_doc/olmo.md @@ -81,7 +81,7 @@ print(tokenizer.decode(output[0], skip_special_tokens=True)) ```bash -echo -e "Plants create energy through a process known as" | transformers-cli run --task text-generation --model allenai/OLMo-7B-hf --device 0 +echo -e "Plants create energy through a process known as" | transformers run --task text-generation --model allenai/OLMo-7B-hf --device 0 ``` From a4cd78e8242b161065531156bb5d51a5008a3224 Mon Sep 17 00:00:00 2001 From: rafakatri <81170796+rafakatri@users.noreply.github.com> Date: Mon, 18 Aug 2025 15:44:07 -0300 Subject: [PATCH 6/6] Add bitsandbytes info Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com> --- docs/source/en/model_doc/olmo.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/source/en/model_doc/olmo.md b/docs/source/en/model_doc/olmo.md index 6b59c2fc0500..56165dcf4501 100644 --- a/docs/source/en/model_doc/olmo.md +++ b/docs/source/en/model_doc/olmo.md @@ -89,7 +89,7 @@ echo -e "Plants create energy through a process known as" | transformers run --t Quantization reduces the memory burden of large models by representing the weights in a lower precision. Refer to the [Quantization](../quantization/overview) overview for more available quantization backends. -The example below uses [insert quantization method here](link to quantization method) to only quantize the weights to __. +The example below uses [bitsandbytes](../quantization/bitsandbytes) to only quantize the weights to 4-bits. ```py import torch