[Bro] Script profiling?

[Gilbert Clark]

Hi:

>> Are there any good script profilers out there that would tell me that I'm spending a lot of time in some script?  I've considered using vTune or some other profiler, but that only gives me second-order profiling information.
> Gilbert Clark is doing some profiling work but I'm not sure if that's going to address your needs.  I'll let him comment on his work if he's reading this thread.

Unfortunately, I don't think anything I have would be in a good place to 
release at the moment.  If this isn't a deterrent, feel free to get in 
touch off-list and we can chat: this isn't because the work is any big 
secret, but instead just because I'm a little hesitant to discuss 
anything tentative in a public setting.  In my experience, discussions 
of this nature have a funny way of turning into commitments, and I'd 
hate to be making commitments that I wouldn't be able to later follow 
through on :)

A few other things that might be more practical in the interim, though:

* There is a small script that ships with bro: 
aux/bro-aux/devel-tools/cpu-bench-with-trace.  This script will find all 
the scripts loaded in local.bro and init-default and run bro with each 
of these scripts loaded in sequence, producing time output for each 
one.  Assuming there's a trace available, this can be a simple way to 
help identify (relatively speaking) which scripts are introducing the 
most overhead.

* There are a few configuration settings that can be used to enable 
per-packet(ish) profiling: check out pkt_profile in init-bare.bro 
(http://www.bro.org/sphinx-git/scripts/base/init-bare.bro.html#id-pkt_profile_freq).

* Bro provides an option (-t) that will dump pretty much everything it's 
doing in script land into a log file.  This is *incredibly* verbose and 
*very* expensive but, with a little scripting, can yield some useful 
information about what bro is doing.  The overhead is high enough that I 
really doubt this could be used as an accurate timing mechanism, though.

* There is an option to enable segment profiling 
(http://www.bro.org/sphinx-git/scripts/base/init-bare.bro.html#id-segment_profiling) 
in prof.log.  Segment profiling will show time spent in particular areas 
of bro.  With that said, I have run into a few traces where bro crashes 
when the segment profiler is run.  This could be because I'm doing 
something wrong, though :)

* If systemtap / dtrace is available and you don't mind getting your 
hands dirty (read: feel comfortable building bro and possibly tweaking a 
few things here and there), we can work through finding a few sane probe 
points in the code that might help with this task in a practical way.

Hope something in there is useful.

-Gilbert