[Bro] how to merge rx and tx from different pcaps / slightly off-topic

[Jeff Barber]

I ran into some problems trying to process pcaps. One is the checksums
issue but I see you've already handled that. The other seems like it might
possibly be related:

If you don't specify --pseudo-realtime, BRO will apparently run connection
timers based on the current wall clock time, comparing the wall clock with
the start time recorded in conjunction with the packets in the pcap. This
means it may see a connection start, then immediately expire it as having
passed the session time limit. [What? That session is six months old!]

(This seems fundamentally broken to me, but it's also quite likely that I
didn't fully understand the code and/or that there's some good reason for
it to work this way; in any case, the --pseudo-realtime switch seems to
make it behave more sanely -- for this particular case anyway.)

Cheers.



On Wed, Sep 9, 2015 at 10:04 AM, Frank Meier <franky.meier.1 at gmx.de> wrote:

> Hi!
>
> Sorry if this is off-topic, but I hope to find the right audience here.
>
> I want to create bro-logs of around 900 Gb of data in 20.000 pcaps.
> Capturing was done on different interfaces for upstream and downstream
> (rx/tx).
>
> Because of the large number of files I cannot merge them in one step ("to
> many open files"),
> so I merged them to one pcap per day with mergecap.  After that Bro is
> called like this:
> # mergecap -F pcap -w - *.pcap | bro -r - foo.bro
>
> foo.bro reads:
>
> redef bits_per_uids = 128;
> redef ignore_checksums = T;
> redef Log::default_rotation_interval = 1day;
>
> No real service logs are written, except for a weird.log full of:
>
> connection_originator_SYN_ack
> data_after_reset
> data_before_established
> inappropriate_FIN
> possible_split_routing
> simultaneous_open
> SYN_after_close
> SYN_after_reset
> SYN_inside_connection
> SYN_seq_jump
> TCP_ack_underflow_or_misorder
> TCP_seq_underflow_or_misorder
> window_recision
>
> It looks like Bro not seeing the data in the correct order. But from what
> I read in mergecap
> source in merge_read_packet() this should work as intended: "Read the next
> packet,
> in chronological order, from the set of files to be merged."
>
> I am thankful for any ideas.
>
> Bye,
>
> Franky
>
> _______________________________________________
> Bro mailing list
> bro at bro-ids.org
> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20150909/ad20362b/attachment.html