solve concurrency

[andrewrk]

On top of the ability to create a kernel thread, Zig should have an abstraction that allows code to express concurrency.

• coroutines (async / await)
• kernel threads for linux
• atomic primitives
• see the TODO for atomicRmw in the docs. support bool and integers with bit counts not powers of 2. for atomicrmw as well as atomicload. @atomicRmw - support bool and non-power-of-2 int types #1220
• cmpxchg function is missing the loaded value #461
• implement kernel threads for macos and windows
• Proof of concept M:N threading. (multiplex coroutines onto a thread pool)
• Builtin function to tell you the maximum stack size of a given function #157

[winksaville]

Are you able to use Atomic code generation natively in Zig? If not that would be a great place to start.

[andrewrk]

We have some built-in functions for this currently with plans to for many more atomic primitives.

In addition this issue is calling to research M:N threading and perhaps some kind of promise abstraction and/or cooperative multithreading.

[winksaville]

Nice to hear atomics are supported. I'd like to caution on keeping things simple, one thing I really like about C is that I have very good control on how big programs are. You can create programs which don't use any "std" libraries and create code that are just a few bytes in size. Another words the programmer is in complete control.

So I suggest, if you want to have features like M:N threading or whatever, there should be no cost in speed or size in programs that don't use them.

[andrewrk]

Your comments are in complete alignment with Zig's philosophy. Whatever concurrency features we have, they will be built on top of the kernel thread abstraction that will be provided in the standard library, and they will not depend on a black box runtime or hide errors such as memory allocation failure.

[pavildon]

Just my opinion: i think LibUV has an excellent and industry proven abstraction for tasks and sockets/filesystem IO, and sits just in the middle of ASIO/STDC++ and threads in C11, I'm familiarising with zig trying to port parts of the async loop of that library using BSD's kqueue, but first I have to finish an allocator using mmap.

[andrewrk]

but first I have to finish an allocator using mmap.

ahh, very exciting. How do you plan to handle the problem of different threads allocating and deallocating memory?

[pavildon]

Im currently reading the malloc related source code in http://opensource.apple.com//source/Libc/Libc-583/gen/malloc.c it seems that it take pages with mmap, the expensive syscall ( it seems mmap is thread safe) is done less frequently and in the actual malloc call that subdivides the page for allocations just lock a mutex releasing it when it finish (macros MALLOC_LOCK and MALLOC_UNLOCK) , note : osx deprecated the use of sbrk and brk.

edit: good reading on the subject : Mac OS X Internals: A Systems Approach (memory chapter)

[kyle-github]

Very belated comment.

I strongly suggest not supporting M:N threading.

TL;DR: nearly every system that did use it in the past has abandoned it including Java, Solaris, Linux...

Some light reading. Dan Kegel has some good links in his C10k web pages:

Dan Kegel's C10k notes

There was an interesting discussion on the Rust lists a few years ago about this (a bit of a slog to read):

The future of M:N threading

[kyle-github]

In view of systems selling now with 32+ cores on the desktop, it seems like the benefits of M:N threading are less and less obvious. At the same time, there seems to have been little progress made on the problems with it.