Training Stuck in KorPuzzleMinesweeperbootcamp

[Linjunjie99]

It seems that RL training can be stuck in the extract_output func in KorPuzzleMinesweeperbootcamp.

My own training process has been stuchk in the func for days.

It is advised to apply time out mechanism in the compute score function.

[FlyingDutchman26]

Thank you for reporting this issue and for your patience, we sincerely apologize for the delay in training .

We appreciate your suggestion regarding the timeout mechanism. To help us better reproduce and fix the issue: could you please share more details on how you observed the stall in extract_output? For example, any logs, stack traces, or behavior patterns (e.g., hanging workers, CPU usage) would be very helpful.

The extract_output func in KorPuzzleMinesweeperbootcamp is here:

InternBootcamp/internbootcamp/bootcamp/kor_puzzle_minesweeper/kor_puzzle_minesweeper.py

Line 350 in f231ec6

def extract_output(output):

We also suspect there might be underlying issues related to veRL, as similar freezing problems have been reported by our colleagues when using veRL for training on mathematical reasoning tasks.

Thanks again for your support!

[Linjunjie99]

During my training using the verl framework, the process would get stuck after a few dozen steps. Monitoring showed that GPU utilization remained at 0%, indicating a complete halt in training progress.

I used the Cursor large language model to help diagnose the problem. By examining the Python process call stack, I observed that only the process responsible for calculating the reward function was active, and it was specifically executing the extract_output function from KorPuzzleMinesweeperbootcamp.

Unfortunately, I didn't save the specific logs or stack traces at the time. However, after implementing a timeout mechanism in the reward calculation function as a workaround, the training process was able to proceed normally without further stalls.

This suggests that the extract_output function may occasionally enter an infinite loop or take an excessively long time to return under certain conditions, causing the entire training pipeline to hang.

[LinyangLee]

During my training using the verl framework, the process would get stuck after a few dozen steps. Monitoring showed that GPU utilization remained at 0%, indicating a complete halt in training progress.

I used the Cursor large language model to help diagnose the problem. By examining the Python process call stack, I observed that only the process responsible for calculating the reward function was active, and it was specifically executing the extract_output function from KorPuzzleMinesweeperbootcamp.

Unfortunately, I didn't save the specific logs or stack traces at the time. However, after implementing a timeout mechanism in the reward calculation function as a workaround, the training process was able to proceed normally without further stalls.

This suggests that the extract_output function may occasionally enter an infinite loop or take an excessively long time to return under certain conditions, causing the entire training pipeline to hang.

Thanks for your tests, is it ok if you can submit a pr for your implementation?