Skip to content

cannot load ckpt #1

@cosine123

Description

@cosine123

╭─────────────────────────────── Traceback (most recent call last) ────────────────────────────────╮
│ /data/wangks/Language-Integrated-VI-main/blocksworld/run.py:287 in │
│ │
│ 284 │ prompt_path = args.prompt_path │
│ 285 │ ckpt_path = args.ckpt_path │
│ 286 │ │
│ ❱ 287 │ tasks_obj = ReasoningTasks(model_name=model_name, data_path=data_path, ckpt_path=ckp │
│ 288 │ config_file = 'data/blocksworld/bw_config.yaml' │
│ 289 │ │
│ 290 │ plan_method=forward_plan │
│ │
│ /data/wangks/Language-Integrated-VI-main/blocksworld/run.py:108 in init
│ │
│ 105 │ │ │ llm = ckpt_path │
│ 106 │ │ │ # the parent directory of the checkpoint directory │
│ 107 │ │ │ tokenizer_path = os.path.join(os.path.dirname(llm), "tokenizer.model") │
│ ❱ 108 │ │ │ llama = load(llm, tokenizer_path, local_rank, world_size, 3) │
│ 109 │ │ │ self.model = QueryLlama(llama, max_response_length=100, log_file=log_file) │
│ 110 │ │ elif self.model_name == "Vicuna": │
│ 111 │ │ │ self.model = QueryVicuna(model_path, num_gpus) │
│ │
│ /data/wangks/Language-Integrated-VI-main/blocksworld/run.py:54 in load │
│ │
│ 51 def load(ckpt_dir: str, tokenizer_path: str, local_rank: int, world_size: int, max_batch │
│ 52 │ start_time = time.time() │
│ 53 │ checkpoints = sorted(Path(ckpt_dir).glob("*.pth")) │
│ ❱ 54 │ assert ( │
│ 55 │ │ │ world_size == len(checkpoints) │
│ 56 │ ), f"Loading a checkpoint for MP={len(checkpoints)} but world size is {world_size}" │
│ 57 │ ckpt_path = checkpoints[local_rank] │
╰──────────────────────────────────────────────────────────────────────────────────────────────────╯
AssertionError: Loading a checkpoint for MP=0 but world size is 1

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions