[Bro] huge weird.log/conn.log

[erik clark]

Hm. It looks like this may be related to af_packet and bro2.5 in general. I
did a subset of production weird and took a subset of development weird,
sorted it out and compared the two. From the looks of things, the ratio of
items in the files take in identical number of events is pretty close to
identical.

This leads me to believe that I am just not dropping traffic either at Bro
or the interface on the dev box. Right now I have dropped only 70k packets
out of 49TiB of traffic according to ifconfig, and bro is reporting packet
loss of ~1%.

The 2.4.1 production box on the other hand is seeing 2-5% packet loss and
some packet loss at the interface. The services (http, dns, so on so forth)
on the dev box all have equal or more than the number of events on the
production box. All I can think of right now is that tuned af_packet on rh7
w/ 2.5 is so much better than tuned pf_ring on rh61 w/ 2.4.1 that it has
been noticeable.

Also, memory consumption on 2.5 is a significant fraction less than on the
production box with the same link. Wish I could say why this is, but it
really impresses me. Load is still high though at ~16, but MEH.


On Thu, Dec 1, 2016 at 11:14 AM, Vlad Grigorescu <vladg at illinois.edu> wrote:

> Can you take a look at what weirds, specifically, you're getting?
> Something like:
>
> > cat weird.log | bro-cut name| sort | uniq -c | sort -n
>
>   --Vlad
>
> erik clark <philosnef at gmail.com> writes:
>
> > I have two bro sensors. One is running 2.5, one is running 2.4.1. Both
> are
> > running on the same link off the tap.
> >
> > The weird.log on the 2.5 box is 6 times bigger than the weird.log on the
> > 2.4.1 log. Any idea why this might be? How can I troubleshoot this.
> >
> > My conn.log is 3 times bigger. For reference:
> >
> > conn.log -> 2.5 (45 minutes) 17 gig
> > conn.log -> 2.4.1 (45 min) 5.5 gig
> >
> > weird.log -> 2.5 (45 minutes) 11 gig
> > weird.log -> 2.4.1 (45 minutes) 1.2 gig
> >
> > These numbers seem to be WAY off. I have no idea how to even try and
> parse
> > this to see what is going on.
> >
> > Packet loss on 2.4.1 is 6%
> > Packet loss on 2.5 is 1%.
> > _______________________________________________
> > Bro mailing list
> > bro at bro-ids.org
> > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20161201/54b4dff7/attachment-0001.html