[HPU][Critical Issue Fix] ThreadPool instead of Pool for parallel pre-processing

[dsmertin]

There had been a problem with Gaudi (HPU) handling multiprocessed pre-processing which was patched with #38790
Indeed there're limitations when different processes try to use one HPU device.

So I changed Pool which is based on creating new processes with ThreadPool which uses threads in the same process.

[dsmertin]

Regarding the issue you created.
@IlyasMoutawwakil please review.

[IlyasMoutawwakil]

Hi @dsmertin I don't think this solves the issue, but rather propagates it to non-HPU devices.
Correct me if I'm wrong but threadpool is subject to the GIL and the task we're parallelizing here is CPU-bound (not I/O bound), so using threads almost adds no parallelism ?

[IlyasMoutawwakil]

We can accept the use for threadpool but only with HPU, since on other devices we still wanna make full use of the multiple processes, it could be something like:

pool_cls = ThreadPool if is_torch_hpu_available() else Pool
with pool_cls(...

[dsmertin]

We can accept the use for threadpool but only with HPU, since on other devices we still wanna make full use of the multiple processes, it could be something like:

pool_cls = ThreadPool if is_torch_hpu_available() else Pool
with pool_cls(...

Let me prepare the change.

UPD:
It's ready.

[HuggingFaceDocBuilderDev]

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

[ydshieh]

LGTM, but would leave @IlyasMoutawwakil for a final ✅ and merge it

thanks

[IlyasMoutawwakil]

@dsmertin no need to update the branch if the tests are passing (or waiting for approval) 😁