Skip to content

[Question] Does qmap work for multithreads of cross workers (on cluster)? #37

@draftman9

Description

@draftman9

Nice work! qmap efficiently solved the nonuniform-load task problems. Now I want to ask a deeper question. I have 2 nodes (or any HPC platforms) connecting to each other by ssh. I have verified this by 

addprocs([("user@node02", 1)], exeflags = ["-t 16"])
addprocs([("user@node03", 1)], exeflags = ["-t 32"])

After executing the codes above, I obtain 2 workers [2, 3]. worker 2 is opened on node02 and has 16 threads while worker 3 is opened on node03 and has 32 threads. So totally I have 48 threads to execute nonuniform-load tasks. I found this ThreadPools.jl seems to only support qmap in a single node, which means only one node is busy calculating and other nodes are idling. Is there anything I missed, or is there any Pkg I did not notice?

Any discussions are welcome since the author of ThreadPools.jl has been inactive for more than two years...

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions