Enhancement: Use multiprocessing for timeout to enable hard kill of stuck tasks

[FernandoCelmer]

Description

The current timeout implementation uses ThreadPoolExecutor, which cannot kill a running thread — a Python limitation. When a task exceeds its timeout, the TimeoutError is raised to the caller, but the thread continues running in the background until it finishes naturally.

This enhancement proposes using multiprocessing.Process instead of threads for timeout enforcement, enabling process.terminate() to hard-kill stuck tasks.

Current behavior

# engine.py — _execute_with_timeout
executor = ThreadPoolExecutor(max_workers=1)
future = executor.submit(self._execute_single)
result = future.result(timeout=seconds)
# If timeout: TimeoutError raised, but thread keeps running

The thread consuming CPU/memory/connections continues until the function returns or the process exits.

Proposed behavior

import multiprocessing

def _execute_with_timeout(self, seconds: int):
    result_queue = multiprocessing.Queue()
    process = multiprocessing.Process(
        target=self._run_in_process,
        args=(result_queue,)
    )
    process.start()
    process.join(timeout=seconds)

    if process.is_alive():
        process.terminate()  # hard kill
        process.join(timeout=5)
        if process.is_alive():
            process.kill()  # SIGKILL as last resort
        raise TimeoutError(f"Task timed out after {seconds}s")

    return result_queue.get()

Trade-offs

	Threads (current)	Processes (proposed)
Kill on timeout	No	Yes (terminate())
Shared memory	Yes (same process)	No (serialization needed)
Overhead	Low	Higher (fork/spawn)
Context passing	Direct reference	Must pickle/serialize
Platform	All	fork issues on macOS (already handled)

Considerations

• Context objects must be picklable to pass between processes
• The spawn context (used on macOS/Windows) requires all arguments to be picklable
• This could be opt-in via @action(timeout=30, timeout_mode="process") to avoid breaking existing behavior

Related

• BUG: Timeout via ThreadPoolExecutor does not kill the running thread #131 (closed — documents the Python thread limitation)
• macOS fork handling already in workflow.py