[Zeek-Dev] Proposal: Make Zeek's debug logging thread-safe

[Bob Murphy]

> On Jul 17, 2020, at 3:01 AM, Robin Sommer <robin at corelight.com> wrote:
> 
> On Wed, Jul 15, 2020 at 14:57 -0700, Bob Murphy wrote:
> 
>> use a single, common logging API, but let it send its output to
>> different output mechanisms that support different use cases.
> 
> I get that in general. It's just that afaik this is the first time
> this need comes up. Adding a full-featured, thread-safe logging
> framework is a trade-off against complexity and maintainance costs.
> Not saying it's impossible, but I'd like to hear more people thinking
> this is a good idea before committing to such a route. 
> 
> Robin

I completely agree about that trade-off, which is why the work I’ve done so far is pretty simple. It doesn’t change the existing DebugLogger system other than adding thread safety. Then on the side, there are a few optional features like a scoping utility class and some preprocessor macros.

That said, different developers have different debugging styles, and I'm a big fan of using feature-rich debug logging frameworks with multiple operating modes and destinations, because they let me fix bugs and write new code much faster than I could otherwise.

Writing a powerful debug logging system does take time and effort, but my experience has been that once it’s finished, it usually doesn't require much ongoing maintenance. Working on open-source and commercial projects with lifetimes of more than a few years, I’ve always seen that time and effort pay for itself many, many times over by making it quicker and easier to diagnose bugs, write new features, and do performance enhancements.

That’s especially been true when I’ve worked on code that handled large volumes of data, like Zeek does. If I need to track down a bug in a stream of data that doesn’t manifest until megabytes have gone by, I usually find it the quickest approach is to run the software and search for a diagnostic pattern in a gigantic log file, compared to other approaches like spending hours hitting the same debugger breakpoint over and over again.

- Bob