Run on any background thread

[SimonDanisch]

I created some code, to run work on any background thread:

const AVAILABLE_THREADS = Base.RefValue{Channel{Int}}()

function spawn_background(f)
    # -1, because we don't spawn on foreground thread 1
    nbackground = Threads.nthreads() - 1
    if nbackground == 0
        # we don't run in threaded mode, so we just run things async
        log("Warning: no threads available, running in foreground thread")
        return @async f()
    end
    # Initialize dynamically, could also do this in __init__ but it's nice to keep things in one place
    if !isassigned(AVAILABLE_THREADS)
        # Allocate a Channel with n background threads
        c = Channel{Int}(nbackground)
        AVAILABLE_THREADS[] = c
        # fill queue with available threads
        foreach(i -> put!(c, i + 1), 1:nbackground)
    end
    # take the next free thread... Will block/wait until a thread becomes free
    thread_id = take!(AVAILABLE_THREADS[])

    return ThreadPools.@tspawnat thread_id begin
        try
            return f()
        catch e
            # If we don't do this, we get pretty bad stack traces... not sure why!?
            return CapturedException(e, Base.catch_backtrace())
        finally
            # Make thread available again after work is done!
            put!(AVAILABLE_THREADS[], thread_id)
        end
    end
end

Would this fit into ThreadPools?

[tro3]

Seems like a lot of overhead. If you have a single job, you can just spawnat 2. If you have a bunch, you can use @bthreads or @qbthreads to spread them out. Even if one needed a "spawn on any background thread" function, one could reduce to

function spawn_background2(f)
    t = @task f()
    @qbthreads for _ in [0]
        schedule(t)
    end
    return t
end

since the schedule will not leave the assigned thread. If you want to PR the simplified version as spawnbg (to keep consistent naming convention, compared to spawnat) and add some test cases, I'll be happy to merge.

[SimonDanisch]

If you have a single job, you can just spawnat 2. If you have a bunch, you can use @BTHREADS or @qbthreads to spread them out

Yeah, that's not the use case here. It's rather a work queue like thing, that runs on the foreground thread, which then spawns the work on a background thread:

while eof(io_socket)
 request = read!(io_socket)
 response = spawn_background(()-> process(request ))
 write(io_socket, fetch(response))
end

And there are multiple loops like that for each open socket, so each loop runs in serial on the foreground thread, but the heavy number crunching happens on background threads ;)

Great, your function looks much simpler and seems to work - didn't realize one could write it that way :)
It does look a bit like a hack though... I guess that happens, when the most core API is based on for loops.
Looks like it should be the other way around, and the @qbthreads should use spawnbackground in its implementation.
Ah well, I think this is about the scheduler having more information via the for loop & historical reasons, right?

[SimonDanisch]

Hm, that implementation doesn't pass my first test:

# Use busy wait, to make 100% sure, we don't have idle threads doing weird stuff
# and approximate the real world use case better
function busy_wait(time_s)
    t = time()
    while time() - t < time_s
    end
    return
end

function uses_all_threads(spawn_func)
    bg_nthreads = Threads.nthreads() - 1
    futures = map(1:bg_nthreads) do i
        return spawn_func() do
            # this should give plenty of time to have all scheduling run through, before any thread can finish
            busy_wait(0.1) 
            return Threads.threadid()
        end
    end
    return Set(fetch.(futures))
end
@test uses_all_threads(spawn_background2) == Set(2:Threads.nthreads())

It uses significantly fewer threads, which must mean it schedules on busy threads and needs to wait for them, even though there are free threads...
And it is indeed much slower:

@btime uses_all_threads(spawn_background2) #   1.520 s (4596 allocations: 313.85 KiB)
@btime uses_all_threads(spawn_background) #     100.128 ms (111 allocations: 8.45 KiB)

So my implementation nicely gives the expected 100ms, resulting from the tasks being run perfectly in parallel.

[SimonDanisch]

I made a PR: #29
Guess this may not be what you would want to include in ThreadPools, but like this you have the tests and therefore the expected behavior more well defined.

[SimonDanisch]

It would be amazing if we can find an implementation that is fast & correct, and integrates well with the current task system, so that I can get rid of this warning:

    ** Warning ** this doesn't compose with other ways of scheduling threads
    So, one should use `spawn_background` exclusively in each Julia process.

[tro3]

I'll take a look at the above. Thx

[tro3]

Pull merged, some cleanup done, and submitted for registration