Make the global logger thread local

[raphaelcohn]

It'd be very useful to have the global logger be init'd so it is thread local. This could be a significant breaking change to library users, so I'd suggest it was done using a compile-time feature.

Why?

1. Occasionally, logging can be quite expensive for some binaries, particularly if they are using one of the newer cloud services. In this world, one runs a second logging thread and has log requests forwarded to it (eg using thread::mpsc).
2. For larger server projects, logging needs may also vary for different parts of the code base:-
   • The main thread is used only for startup, option parsing and shutdown, and has very little in the way of logging needs, but if it does log, should log to standard error
   • The main application threads (eg socket listeners) need to log to a monitored cloud service, so pass their logging requests to a third logging thread
   • The logging thread receives log requests and forwards them on. It itself may need a logger that talks, say, to local syslog.

What do you think?

[sfackler]

You can pretty easily make a logger that sends its log messages wherever they need to go asynchronously via a background thread, and that fact seems a bit unrelated to whether you then install that logger once globally or explicitly in every thread you spawn.

More generally, the log crate used to have a thread local logger, but we explicitly moved away from that in #12. The background is in #3, but the important bit is this:

There is now a single global logger instead of thread-local loggers. The current liblog implementation does provide the ability to set the logger, but since it's task local, it's impossible to make sure the right logger is installed in each thread - think of threads started by TaskPool for example. Having thread-local loggers also results in a log of duplicated data and computation. The RUST_LOG environment variable has to be reparsed every time a program logs on a new thread, for example. In addition, it's not totally clear to me that anyone actually wants to have different loggers on different threads. Even if one does, it's pretty easy to make a thread local logger implementation for the logging facade.

Systems that want to partition the log data they produce into different places traditionally use something like logback, log4net, etc, that filter based on the "location" the log message is logged to rather than the thread it happens to be running on. An equivalent in the rust world would be something like log4rs.

[raphaelcohn]

Thank you for the reasoned reply - it's much appreciated. I've experimented with a logger that forwards messages. It's a bit clunky for now, but I can see a way through - and it was a useful introduction to Rust's take on thread locals and static initialization. (It'd be nice to have more powerful const functions, but I can see that's being heavily discussed).

I'd agree that in the general case, when there is library code that can instantiate threads indirectly, then it's really hard to make a thread-local logger work simply and transparently. The final part of the
reasoning (

In addition, it's not totally clear to me that anyone actually wants to have different loggers on different threads. Even if one does, it's pretty easy to make a thread local logger implementation for the logging facade.
) makes sense but possibly has performance implications for high-volume logging (eg on a HTTP server under DoS attack).

Many thanks once again.