[Bro-Dev] Performance Issues after the fe7e1ee commit?

[Robin Sommer]

On Thu, Jun 07, 2018 at 17:05 -0500, you wrote:

> Thanks, if you pull master changes [1] again there's likely some improvement.

Yeah, a little bit, not much though.

> # zcat 2009-M57-day11-18.trace.gz | time bro -r - tests/m57-long.bro
> Known::use_host_store=F Known::use_service_store=F
> Known::use_cert_store=F

That indeed gets it way down, though still not back to the same level
on my box:

170.49user 58.14system 1:55.94elapsed 197%CPU

(pre-master: 73.72user 7.90system 1:06.53elapsed 122%CPU)

Are there more places where Bro's waiting for Broker in pcap mode?

> With that, I get the same timings as from before pre-Broker.  At least
> a good chunk of the difference when using data stores is that, for
> every query, Bro will immediately block until getting a response back
> from the data store thread/actor.

Yeah, I remember that discussion. It's the trade-off between
performance and consistency. Where's the code that's doing this
blocking? Would it be possible to not block right away, but sync up
with Broker when events are flushed the next time? (Like we had
discussed before as a general mechanism for making async operations
consistent)

Robin

-- 
Robin Sommer * ICSI/LBNL * robin at icir.org * www.icir.org/robin