[Bro-Dev] [JIRA] (BIT-1133) topic/jsiwek/dns-perf

Jon Siwek (JIRA) jira at bro-tracker.atlassian.net
Fri Feb 14 10:13:37 PST 2014

    [ https://bro-tracker.atlassian.net/browse/BIT-1133?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15554#comment-15554 ] 

Jon Siwek commented on BIT-1133:

-O0. I'm using instructions as a proxy for "work"; so far the
instruction number has turned out to be nicely stable across run on
the same machine. So I'm not saying that there's a precise 2% impact
but only that the DNS changes lead to now more work being done,

Makes sense.

which I think warrants understanding given that DNS is just a small of piece
of everything

Small piece relative to code or traffic?

Not saying it's actually the case here (I agree, would be nice to fully understand), just that generally small changes might inevitably have observable perf. difference due traffic patterns/volume.  In that case, it might be hard to fully optimize away the perf. hit without doing unrelated/low-level changes.

So with optimizations enabled, I compared user time between 741ae7a368f and master. Averaged over 3 runs each, I do see a rather consistent difference of 3-4%.

Looks like either system-specific difference or our methods differ.  On brotestbed:

$ cc --version
cc (GCC) 4.4.7 20120313 (Red Hat 4.4.7-4)
$ cat /etc/issue
Red Hat Enterprise Linux Server release 6.5 (Santiago)

$ git checkout 741ae7a368f && git fix-submodules && ./configure --builddir=build-opt --disable-perftools && cd build-opt && make -j8 && . bro-path-dev.sh

$ for i in {1..3}; do time bro -r ~/2009-M57-day11-18.trace local "Site::local_nets+={}"; done

real    0m38.847s
user    0m41.886s
sys 0m3.086s

real    0m39.148s
user    0m41.990s
sys 0m3.579s

real    0m38.963s
user    0m42.336s
sys 0m3.016s

$ git checkout 7d0fbcd7b7551e9 && git fix-submodules && ./configure --builddir=build-opt --disable-perftools && cd build-opt && make -j8 && . bro-path-dev.sh

$ for i in {1..3}; do time bro -r ~/2009-M57-day11-18.trace local "Site::local_nets+={}"; done

real    0m39.239s
user    0m42.414s
sys 0m3.033s

real    0m39.829s
user    0m42.580s
sys 0m3.424s

real    0m39.376s
user    0m42.353s
sys 0m3.274s

So I get avg. 42.07s user time before and avg. 42.449 user time after, about 1% diff.

So dunno. If it's just necessary to get the dns.log right, there's not much we can do. And given that you aren't seeing a difference, it may also be system-specific effect. So should we close the ticket and move on?

At least the results I got seem a reasonable tradeoff for pushing things towards correctness.

There's probably bigger performance wins if someone were to do a more generalized profiling/optimization pass over Bro in its entirety than trying to eke out more cycles from the new DNS script logic in particular?

> topic/jsiwek/dns-perf
> ---------------------
>                 Key: BIT-1133
>                 URL: https://bro-tracker.atlassian.net/browse/BIT-1133
>             Project: Bro Issue Tracker
>          Issue Type: Improvement
>          Components: Bro
>    Affects Versions: git/master
>            Reporter: Jon Siwek
>              Labels: dns, performance
>             Fix For: 2.3
> Refactored DNS script's state management to improve performance (due to reduced timer volume).
> Branch is in bro, bro-testing, bro-testing-private.

This message was sent by Atlassian JIRA

More information about the bro-dev mailing list