Skip to content

Reproducing results for “From Reasoning to Answer…” — Hugging Face changes? #12

@Ykid

Description

@Ykid

Hello!

Thank you for releasing the code for From Reasoning to Answer: Empirical, Attention-Based and Mechanistic Insights into Distilled DeepSeek R1 Models. I’m attempting to reproduce the experiments and ran into an issue that seem related to recent changes on the Hugging Face repo.

Could you advise on how best to address this? Thank you!

(r2a) a@a:~/x/ACV/misc/R2A$ python trace_collection.py \
    --model_path "deepseek-ai/DeepSeek-R1-Distill-Qwen-1.5B" \
    --dataset "WildBench" \
    --with_reasoning \
    --limit 1
Output path set to: output/reasoning_traces/WildBench_DeepSeek-R1-Distill-Qwen-1.5B_withR_sampling_0.jsonl
Loading model: deepseek-ai/DeepSeek-R1-Distill-Qwen-1.5B
Loading dataset: allenai/WildBench
README.md: 8.22kB [00:00, 26.2MB/s]
^CTraceback (most recent call last):
  File "/opt/conda/envs/r2a/lib/python3.11/site-packages/huggingface_hub/utils/_http.py", line 409, in hf_raise_for_status
    response.raise_for_status()
  File "/opt/conda/envs/r2a/lib/python3.11/site-packages/requests/models.py", line 1026, in raise_for_status
    raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 404 Client Error: Not Found for url: https://huggingface.co/datasets/allenai/WildBench/resolve/26c49eb39d7d5ce2099b0bbafed5a88dcce954ec/WildBench.py

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions