[Bro] Differences in processing multiple traces with BRO and ipsumdump

[sridhar basam]

It looks like ipsumdump might be changing the snaplen to 2000 bytes when it
writes out the pcap file. I don't see an runtime option to change the
snaplen.

Another tool you can try to merge those files is tcpslice from
ftp://ftp.ee.lbl.gov/tcpslice.tar.gz. I have been able to preserve the
snaplen using tcpslice.

tcpslice trace*.pcap -w - | bro -r - ...

 Sri

On Fri, Jun 18, 2010 at 2:09 AM, Veronica Estrada <
estrada.veronica at gmail.com> wrote:

>
> Hi everyone,
>
> I am puzzled about the outcomes of using ipsumdump or BRO for processing
> multiple pcap files.
>
> I am using BRO to analyze anomalities in my 12 hours captured network
> traffic which was saved in 4 Gb pcap files. I want that BRO consider the
> cases when a connection may have been split in two or more files. I was
> using ipsumdump to solve this, but I found that some files have errors and
> cause ipsumdump to crush with this message:
> ToDump(bigPcap1.pcap): Inappropriate ioctl for device
>
> Using the capinfo tool I detected that some of my files have packet size
> larger than normal (65535), so using tshark I cut the part of the file with
> problems. For example:
> capinfos: An error occurred after reading 3830659 packets from
> "trace2.pcap": File contains a record that's not valid.
> (pcap: File has 4065648712-byte packet, bigger than maximum of 65535)
> So I create a reduced version of trace2.pcap with tshark:
> /usr/sbin/tshark -c 3830659 -r trace2.pcap -w trace2-new.pcap
>
> This solution seemed to work fine, all the ***-new.pcap have no errors
> while reading with capinfo or wireshark, but even so  there are some that
> still cause problems for processing. For example:
>
> I processed the following files in 3 different ways:
> trace1.pcap, trace2-new.pcap, trace3.pcap (trace2.pcap was replaced because
> of the packet size error)
>
> FIRST TRY - using ipsumdump with collate option:
> ipsumdump --collate -w - trace* |bro -r - brolite myenvironment -f "tcp or
> udp or icmp" dpd_conn_logs=T dpd detect_protocols dyn_disable irc-bot proxy
> ftp
>
> Output> 9.7 MB conn.log with 114861 lines (number of connections)
>
> SECOND TRY - using ipsumpdump without collate option
> ipsumdump --collate -w - trace* |bro -r - brolite myenvironment -f "tcp or
> udp or icmp" dpd_conn_logs=T dpd detect_protocols dyn_disable irc-bot proxy
> ftp
>
> Output:
> 19 Mbytes conn.log with 228922 lines  with 950 repeated connections
>
> THIRD TRY - without ipsumdump:
> /usr/local/bro/bin/bro -r trace1.pcap -r trace2-new.pcap -r trace3.pcap
> brolite todai -f "tcp or udp or icmp" dpd_conn_logs=T dpd detect-protocols
> dyn-disable irc-bot proxy ftp 2>bro-error3.log
>
> Output:
> 15 Mbytes conn.log with 169168 lines, connections are not repeated
>
> COMMENTS:
> pcap files has not overlap traffic (it was checked with trace-summary using
> first packet seen and last packet seen).
> I tried the ipsumdump with both collate and no collate option because when
> I used ipsumdump only (without bro), with collate option the resulted larger
> pcap file was a 7.9 GB file but without collate option the resulted file was
> 12.GB (trace1.pcap: 4 MB, trace2-new.pcap: 3.9GB, trace3.pcap: 4GB).
> Besides, while using ipsumpdump --collate alone, the progress bar showed
> something like this:
> 66%******************              |8017MB ETAToDump(LargerTrace.pcap):
> Success
> 100%****************************|12113MB
> But the progress bar for ipsumdump without the collate option didn't split
> and reach the 100% 12113MB.
>
> If anyone can illuminate this matter, it will be a great help.
>
> Veronica
>
>
>
>
> _______________________________________________
> Bro mailing list
> bro at bro-ids.org
> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro
>



-- 
Sridhar
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20100618/3fdcdc6e/attachment.html