From romhiest at ee.ethz.ch Tue Mar 1 02:21:58 2005 From: romhiest at ee.ethz.ch (Roman Hiestand) Date: Tue, 1 Mar 2005 10:21:58 +0000 Subject: [Bro] Memory consumption of a single table entry Message-ID: <1109672518.42244246e2c79@email.ee.ethz.ch> hello! I'm using Bro since quite a while and so far I'm quite happy with the tool. Now, I have a questions concerning the memory consumption. I'm running my own Bro script which is given below. I have installed the latest version 0.9a8 of Bro on a Suse Linux. I started Bro, sent 9'844 packets to random IPs. Afterwards, prof.log gives me the following information: 1109672099.923449 Global_sizes > 100k: 0K 1109672099.923449 test_table = 1340K (9844 entries) The tables is, as given below, indexed by destination IP and the entries are of type bool. One entry uses 139 Bytes (= 1340 KB / 9844 entries). I do not understand why one entry uses that much of memory. An IP can be stored in 4 Byte and the table content of type bool in 1 bit. I can imagine that Bro uses memory for the Hash or other information. But this does not require 139 Bytes. Can you please explain me, why Bro uses this much of memory for one table entry and what I can do to avoid this high memory consumption. Cheers, Roman #script test.bro ---------------- @load profiling @load reduce-memory redef ignore_checksums = T; redef profiling_interval = 10secs &redef; global test_table: table[addr] of bool; event icmp_unreachable(icmp: icmp_conn, code: count, context: icmp_context) { local destIP = context$id$resp_h; test_table[destIP] = T; } From jp-www at dcs.gla.ac.uk Wed Mar 2 09:22:06 2005 From: jp-www at dcs.gla.ac.uk (Jonathan Paisley) Date: Wed, 2 Mar 2005 17:22:06 +0000 Subject: [Bro] Attempting to use SSL analyser in 0.9a8 Message-ID: Hi all, I'm attempting to have a go with the SSL analyser, using bro 0.9a8. I'm encountering a problem that I've tracked down be caused by DataBlocks being deallocated but their data still being used following reassignment of a contents processor. I realise the above description is a bit vague, but I wanted to check if anybody was interested in hearing more about this problem or not. I understand that the SSL analyser is a work-in-progress, so may not be expected to work at all at the moment. Thanks for any pointers. Jonathan From sommer at in.tum.de Wed Mar 2 09:44:01 2005 From: sommer at in.tum.de (Robin Sommer) Date: Wed, 2 Mar 2005 18:44:01 +0100 Subject: [Bro] Attempting to use SSL analyser in 0.9a8 In-Reply-To: References: Message-ID: <20050302174401.GA32370@net.informatik.tu-muenchen.de> On Wed, Mar 02, 2005 at 17:22 +0000, Jonathan Paisley wrote: > I'm attempting to have a go with the SSL analyser, using bro 0.9a8. I'm > encountering a problem that I've tracked down be caused by DataBlocks > being deallocated but their data still being used following > reassignment of a contents processor. > > I realise the above description is a bit vague, but I wanted to check > if anybody was interested in hearing more about this problem or not. I > understand that the SSL analyser is a work-in-progress, so may not be > expected to work at all at the moment. Yes, unfortunately, the SSL analyzer is broken at the moment. However, there are two guys working on fixing it (cc'ed). I don't really know what the current state is but I am sure they'll appreciate any help in tracking the problem down. Thanks! Robin -- Robin Sommer * Room 01.08.055 * www.net.in.tum.de TU Muenchen * Phone (089) 289-18006 * sommer at in.tum.de From jp-www at dcs.gla.ac.uk Wed Mar 2 13:58:39 2005 From: jp-www at dcs.gla.ac.uk (Jonathan Paisley) Date: Wed, 2 Mar 2005 21:58:39 +0000 Subject: [Bro] Attempting to use SSL analyser in 0.9a8 In-Reply-To: <20050302174401.GA32370@net.informatik.tu-muenchen.de> References: <20050302174401.GA32370@net.informatik.tu-muenchen.de> Message-ID: On 2 Mar 2005, at 17:44, Robin Sommer wrote: > On Wed, Mar 02, 2005 at 17:22 +0000, Jonathan Paisley wrote: > >> I'm attempting to have a go with the SSL analyser, using bro 0.9a8. >> I'm >> encountering a problem that I've tracked down be caused by DataBlocks >> being deallocated but their data still being used following >> reassignment of a contents processor. >> >> I realise the above description is a bit vague, but I wanted to check >> if anybody was interested in hearing more about this problem or not. I >> understand that the SSL analyser is a work-in-progress, so may not be >> expected to work at all at the moment. > > Yes, unfortunately, the SSL analyzer is broken at the moment. > However, there are two guys working on fixing it (cc'ed). I don't > really know what the current state is but I am sure they'll > appreciate any help in tracking the problem down. Thanks for the info. I'll go into a bit more detail here in case it's any use. I have a trace file with a single HTTPS connection. I'm running bro like this: bro -r bro-test-ssl.pcap ssl weird.log gives: 1109703962.146861 ** 130.209.243.215/32874 > 130.209.240.41/https: SSLv2: FATAL: recordLength doesn't match data block length! A bit of debugging shows that the first two words of the stream content being parsed have been stomped. More investigation reveals that the malloc-ed block has been freed. Here is some more info (including a bit of a stack trace) from using valgrind: ==22627== Invalid read of size 1 ==22627== at 0x812DC6B: SSLv2_Interpreter::NewSSLRecord(SSL_InterpreterEndpoint*, int, unsigned char*) (SSLv2.cc:150) ==22627== by 0x812D102: SSL_ConnectionProxy::DoDeliver(TCP_Endpoint*, int, int, unsigned char*) (SSLProxy.cc:728) ==22627== by 0x812D039: SSL_ConnectionProxy::NewSSLRecord(SSL_ProxyEndpoint*, int, unsigned char*) (TCP_Contents.h:116) ==22627== by 0x812D404: SSL_ProxyEndpoint::DoDeliver(int, unsigned char*) (TCP_Contents.h:116) ==22627== by 0x812C540: SSL_RecordBuilder::addSegment(unsigned char*, int) (SSLProxy.cc:206) ==22627== by 0x812D38C: SSL_ProxyEndpoint::Deliver(int, int, unsigned char*) (SSLProxy.cc:798) ==22627== by 0x81045A6: TCP_Contents::DeliverBlock(int, int, unsigned char*) (TCP_Contents.cc:397) ==22627== by 0x810395E: TCP_Reassembler::BlockInserted(DataBlock*) (TCP_Contents.cc:215) ==22627== Address 0x41B8F680 is 0 bytes inside a block of size 126 free'd ==22627== at 0x4002B6AE: operator delete[](void*) (vg_replace_malloc.c:190) ==22627== by 0x80DA498: Reassembler::ClearBlocks() (Reassem.cc:160) ==22627== by 0x80DA186: Reassembler::~Reassembler() (Reassem.cc:68) ==22627== by 0x810353C: TCP_Reassembler::~TCP_Reassembler() (Obj.h:200) ==22627== by 0x8104124: TCP_Contents::~TCP_Contents() (Obj.h:204) ==22627== by 0x812D305: SSL_ProxyEndpoint::~SSL_ProxyEndpoint() (SSLProxy.cc:767) ==22627== by 0x8105744: TCP_Endpoint::AddContentsProcessor(TCP_Contents*) (TCP_Endpoint.cc:89) ==22627== by 0x8103D22: TCP_Contents::TCP_Contents(TCP_Endpoint*, int) (TCP_Contents.cc:284) I don't understand the object hierarchies fully yet, but my impression so far is that a new contents processor is assigned (TCP_Endpoint.c:89) which causes the old one to be deleted. This deletes the Reassembler instance, which frees the DataBlocks. However, there are still references (uchar*) to the data in the DataBlock which is soon to be used in NewSSLRecord. I tried patching around this by transferring the DataBlocks ownership between Reassembler instances (before the old one gets deleted), but came up against another crashing issue that I haven't had time to track down. I hope this is of some help. Please let me know if there's anything more I can do. Thanks. PS The above error isn't the first reported by valgrind. After a few warnings relating to DNS_Mgr, the first SSL-related error is this: ==22627== Invalid read of size 2 ==22627== at 0x812D018: SSL_ConnectionProxy::NewSSLRecord(SSL_ProxyEndpoint*, int, unsigned char*) (SSLProxy.cc:702) ==22627== by 0x812D404: SSL_ProxyEndpoint::DoDeliver(int, unsigned char*) (TCP_Contents.h:116) ==22627== by 0x812C540: SSL_RecordBuilder::addSegment(unsigned char*, int) (SSLProxy.cc:206) ==22627== by 0x812D38C: SSL_ProxyEndpoint::Deliver(int, int, unsigned char*) (SSLProxy.cc:798) ==22627== by 0x81045A6: TCP_Contents::DeliverBlock(int, int, unsigned char*) (TCP_Contents.cc:397) ==22627== by 0x810395E: TCP_Reassembler::BlockInserted(DataBlock*) (TCP_Contents.cc:215) ==22627== by 0x80DA2A1: Reassembler::NewBlock(double, int, int, unsigned char const*) (Reassem.cc:99) ==22627== by 0x8104269: TCP_Contents::DataSent(double, int, int, unsigned char const*) (TCP_Contents.cc:335) ==22627== Address 0x422469AC is 28 bytes inside a block of size 40 free'd ==22627== at 0x4002B54E: operator delete(void*) (vg_replace_malloc.c:188) ==22627== by 0x8105744: TCP_Endpoint::AddContentsProcessor(TCP_Contents*) (TCP_Endpoint.cc:89) ==22627== by 0x8103D22: TCP_Contents::TCP_Contents(TCP_Endpoint*, int) (TCP_Contents.cc:284) ==22627== by 0x812D213: SSL_ProxyEndpoint::SSL_ProxyEndpoint(TCP_Endpoint*) (SSLProxy.cc:756) ==22627== by 0x812CFEE: SSL_ConnectionProxy::NewSSLRecord(SSL_ProxyEndpoint*, int, unsigned char*) (SSLProxy.cc:686) ==22627== by 0x812D404: SSL_ProxyEndpoint::DoDeliver(int, unsigned char*) (TCP_Contents.h:116) ==22627== by 0x812C540: SSL_RecordBuilder::addSegment(unsigned char*, int) (SSLProxy.cc:206) ==22627== by 0x812D38C: SSL_ProxyEndpoint::Deliver(int, int, unsigned char*) (SSLProxy.cc:798) From aashish at uiuc.edu Thu Mar 3 08:26:00 2005 From: aashish at uiuc.edu (Aashish Sharma) Date: Thu, 03 Mar 2005 10:26:00 -0600 Subject: [Bro] bro email notifications In-Reply-To: <1109377100.2974.250.camel@yaksha.ncsa.uiuc.edu> References: <200502252059.j1PKxpC9009837@jaguar.icir.org> <1109377100.2974.250.camel@yaksha.ncsa.uiuc.edu> Message-ID: <1109867159.19086.1007.camel@yaksha.ncsa.uiuc.edu> Hello All, Any thoughts about this : On Fri, 2005-02-25 at 18:18, Aashish Sharma wrote: > > With the latest release there are two new notice actions, NOTICE_EMAIL and > > NOTICE_PAGE, which you can use for this. > > > > 1) [ From policy/notice.bro ] > global notice_policy: set[notice_policy_item] = { > [$pred(n: notice_info) = { return T; }, > # $result = NOTICE_ALARM_ALWAYS, > $result = NOTICE_EMAIL, > $priority = 0], > } &redef; > > How do I set up various degree's of notifications. For some things I > would like to be paged, others an email and rest just logged. > > $result seems to let me setup only one notice action option here. > > > [ I do see policy/notice.bro has some email parameters settings but > does > > > not seems to be working ] > > 2) > > Can you provide an example that demonstrates it's not working? > > > > However If I understand it correctly, email sending mechanism is defined > here in notice.bro : > > # Variables the control email notification. > global mail_notification = reading_live_traffic() &redef; > global mail_script = "mail_notice.sh" &redef; > global mail_dest = "bro" &redef; > global mail_page_dest = "bro-page" &redef; > > I dont see mail_notice.sh in scripts folder so right now I am not very > sure how bro is going to handle sending email notification and what this > script is intending to do. > > It is going to parse logs periodically and grep for NOTICE_EMAIL and > take action or has some other mechanism is intended ? > > > Thanks, > Aashish > > > > On Fri, 2005-02-25 at 14:59, Vern Paxson wrote: > > > 1) We used to run wots/swatch on bro logs periodically which checks for > > > alert patterns and send an us an email for that particular bro alert > > > _______________________________________________ > Bro mailing list > bro at bro-ids.org > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro From vern at icir.org Sun Mar 6 00:41:39 2005 From: vern at icir.org (Vern Paxson) Date: Sun, 06 Mar 2005 00:41:39 -0800 Subject: [Bro] Memory consumption of a single table entry In-Reply-To: Your message of Tue, 01 Mar 2005 10:21:58 GMT. Message-ID: <200503060841.j268fdqG054448@jaguar.icir.org> > The tables is, as given below, indexed by destination IP and the entries are of > type bool. One entry uses 139 Bytes (= 1340 KB / 9844 entries). > I do not understand why one entry uses that much of memory. Robin Sommer profiled this and the short answer is yes, it winds up taking about that much memory (the exactly value depends on a number of things) :-(. This is clearly unhappily high. It comes from run-time typing information, timestamps used for table expiration, location information for error message output, and hash table buckets, keys & chains. We'll be looking into ways to lower it, but this will take a while. Vern From vern at icir.org Sun Mar 6 00:53:01 2005 From: vern at icir.org (Vern Paxson) Date: Sun, 06 Mar 2005 00:53:01 -0800 Subject: [Bro] bro email, cleartext passwords In-Reply-To: Your message of Fri, 25 Feb 2005 18:18:20 CST. Message-ID: <200503060853.j268r1PY055432@jaguar.icir.org> > Also, It would be nice if we can capture clear text ftp passwords. You can get these using an "account_tried" event handler. See its invocation in ftp.bro. (In fact, come to think of it passwords.bro should be written in terms of account_tried.) > 1) [ From policy/notice.bro ] > global notice_policy: set[notice_policy_item] = { > [$pred(n: notice_info) = { return T; }, > # $result = NOTICE_ALARM_ALWAYS, > $result = NOTICE_EMAIL, > $priority = 0], > } &redef; > > How do I set up various degree's of notifications. For some things I > would like to be paged, others an email and rest just logged. > > $result seems to let me setup only one notice action option here. The key about notice_policy is that it's a *set* of notice_policy_item's, so you can specify a bunch of them. So for example, here's a subset of the notice_policy modifications we use operationally at ICSI: redef notice_policy += { # Ignore sensitive URLs that are out of the FTP directory. # Note that some weird clients generate tons of global/'s # in their fetches, too. [$pred(n: notice_info) = { return n?$URL && n$URL == /^\/?ftp\/(global\/)*etc.*/; }, $result = NOTICE_FILE, $priority = 1], # wyvern generates a lot of these as it tries to look up remote # hosts for processing mail. [$pred(n: notice_info) = { return n$note == DNS::DNS_PTR_Scan && n$src in wyvern.icir.org; }, $result = NOTICE_IGNORE, $priority = 1], # Connections to 2766/tcp ("Solaris listen service") appear to # always be actually due to P2P apps. [$pred(n: notice_info) = { return n$note == SensitiveConnection && n$msg == /Solaris listen service/; }, $result = NOTICE_FILE, $priority = 1], }; > I dont see mail_notice.sh in scripts folder so right now I am not very > sure how bro is going to handle sending email notification and what this > script is intending to do. I double-checked and it's part of the distribution: bro-0.9a8/scripts/mail_notice.sh > It is going to parse logs periodically and grep for NOTICE_EMAIL and > take action or has some other mechanism is intended ? It's different. The NOTICE() function will invoke email_notice() for NOTICE_EMAIL or NOTICE_PAGE, which in turn will execute the script. Vern From rmkml at free.fr Wed Mar 16 00:28:23 2005 From: rmkml at free.fr (rmkml) Date: Wed, 16 Mar 2005 09:28:23 +0100 (CET) Subject: [Bro] event "Invalid HTTP version string" ? Message-ID: Hi, I have this event : WEB-MISC Invalid HTTP Version String since 195.93.102.x (proxy http aol) and look ascii data dump : GET /uship/ HTTP/1.1\r\nVia: HTTP/1.1 (Velocity/1.1.0 [uScMs f p eN:t cCMp s ]), HTTP/1.1 prs-ab06[C35D6626] (Traffic-Server/5.3.6 [uScM])\r\nAccept-Encoding: gzip, deflate\r\nUser-Agent: Mozilla/4.0 (compatible; MSIE 6.0; AOL 9.0; Windows NT 5.1; SV1; .NET CLR 1.1.4322)\r\n...............: ............\r\nAccept-Language: fr\r\n.......: ....:..............................................................\r\nAccept: */*\r\nHost: www.xx-yy.com\r\n\r\n. Regards Rmkml at free.fr From aashish at uiuc.edu Thu Mar 17 15:31:02 2005 From: aashish at uiuc.edu (Aashish Sharma) Date: Thu, 17 Mar 2005 17:31:02 -0600 Subject: [Bro] bro alerts over email Message-ID: <1111102261.5711.11.camel@yaksha.ncsa.uiuc.edu> Hello All : Thanks for the clearing questions in the last email. I need some more clarifications about email alert generations using bro. I thank you for your time and help. I am writing this email in the order alert generation need to be configured on bro (I think). 1) I am unable to redefine variables sensitive_URIs (policy/http-request.bro) and hot_files (policy/ftp.bro) in my site policy file. Right now I am adding all my sensitive_URI's and ftp hot files into the corresponding policy files. export { const sensitive_URIs = [ policy/http-request.bro; lines 9+ ] export { # Indexed by source & destination addresses and the id. const skip_hot: set[addr, addr, string] &redef; const hot_files = [from policy/ftp.bro; lines 12+ ] So, How do I redef these variables (which are 'export { const ' declared in the policy files in my site/policy.bro file ? 2) In-order to send emails from bro I had to comment out the following from notice.bro file : # if ( ! mail_notification ) ----------------------- (2A) # return; # local action = notice_action_filters[n$note](n); --------- (2B) # Choose destination address based on action type. # local destination = (action == NOTICE_EMAIL) ? # mail_dest : mail_page_dest; local destination = mail_dest ; 2A) I think 'if (! mail_notification)' condition is not holding true at all. I see the following definition ../policy/notice.bro:global mail_notification = reading_live_traffic() &redef; and ../policy/bro.bif.bro:global reading_live_traffic: function(): bool; I don't see reading_live_traffic function defined anywhere? Do I need to redef reading_live_traffic() function. If yes, should it be in the site policy file ? Would its value affect other policy files ? (its used in conn.bro, load-level.bro and stats.bro) 2B) local action = notice_action_filters[n$note](n) gives the following error in info.log file and bro stops : 1111094454.266502 /usr/local/bro/policy/notice.bro, line 193 (notice_action_filters[n$note]): run-time error, no such index 1111094454.266502 /usr/local/bro/policy/notice.bro, line 196 (action): run-time error, value used but not set Commenting the action variable makes email work fine but I am not sure how other things would be affected due to this. 3) Finally declaring sensitive_URI's in (1) and commenting (2) I am getting email notifications working on bro. As suggested I am declaring, for example : [$pred(n: notice_info) = { return n?$URL && n$URL == /^.*rootdown.pl.*$/ ; }, $result = NOTICE_EMAIL, $priority = 4], in my site-policy file for getting email/page alert. If I understand it correctly, I have to first put rootdown.pl (etc) in Sensitive_URI list to get bro generate an alert and then declare that particular alert using the above $pred config in my site policy file. Right ? Since this could lead to lot of $pred declearations, Is it possible to have a formation like following for similar category of alerts : /usr/local/bro/site/hail.ncsa.uiuc.edu.bro, line 157 (/^?(^.*rootdown.pl.*$)$?/ || /^?(^.*lads.exe.*$)$?/): error, requires boolean operands which is, obviously, errornous right now. 4) I checked again and mail_notice.sh file comes as part of bro tarball and is available in bro-09a8/scripts folder. However, after running make install-brolite it does not get copied over to /usr/local/bro/scripts. I thought should let you know this. I appriciate all the help here. Thanks a lot. Aashish Sharma From smoogen at lanl.gov Thu Mar 17 17:45:31 2005 From: smoogen at lanl.gov (Stephen J Smoogen) Date: Thu, 17 Mar 2005 18:45:31 -0700 Subject: [Bro] Using other libpcaps and bro-0.9 Message-ID: <423A32BB.9050305@lanl.gov> Hi I am just started with bro to evaluate it against the other tools we have. The first question I have is about using different libpcaps. We have our own fork of libpcap here (Phil Woods code) and I am needing to use it as a comparison with our snort and other tools. How hard is it to compile bro with another version of libpcap :)? -- Stephen John Smoogen | CCN-5 Security Team LANL SIRT Team Lead | SMTP: smoogen at lanl.gov Los Alamos National Laboratory | Voice: 505.664.0645 Ta-03 SM-1498 MS: B255 DP 10S | FAX: 505.665.7793 Los Alamos, NM 87545 | PGR: 505.664.1535 From jrlee at lbl.gov Fri Mar 18 12:54:31 2005 From: jrlee at lbl.gov (Jason Lee (DSD staff)) Date: Fri, 18 Mar 2005 12:54:31 -0800 Subject: [Bro] Using other libpcaps and bro-0.9 In-Reply-To: <423A32BB.9050305@lanl.gov> References: <423A32BB.9050305@lanl.gov> Message-ID: <423B4007.7050804@lbl.gov> Stephen, It should be fairly straight forward to use other libpcaps with bro. There is an option to configure (--disable-localpcap) the will disable including the pcap distributed with bro, and instead will search for a libpcap on the system. I believe that bro will first look for a libpcap directory at the same level as the bro directory, and if it doesn't find one at that level it looks for one installed on the system. There was a bug in the --disable-localpcap, and I'm not sure if the fix is the last release. Let me know if you have any problems, the patch is only a couple of lines. Hope this helps. Cheers, jason Stephen J Smoogen wrote: > > Hi I am just started with bro to evaluate it against the other tools we > have. The first question I have is about using different libpcaps. We > have our own fork of libpcap here (Phil Woods code) and I am needing to > use it as a comparison with our snort and other tools. How hard is it to > compile bro with another version of libpcap :)? > From smoogen at lanl.gov Fri Mar 18 12:55:55 2005 From: smoogen at lanl.gov (Stephen J Smoogen) Date: Fri, 18 Mar 2005 13:55:55 -0700 Subject: [Bro] Using other libpcaps and bro-0.9 In-Reply-To: <423B4007.7050804@lbl.gov> References: <423A32BB.9050305@lanl.gov> <423B4007.7050804@lbl.gov> Message-ID: <423B405B.4090508@lanl.gov> Jason Lee (DSD staff) wrote: > > Stephen, > > It should be fairly straight forward to use other libpcaps > with bro. There is an option to configure (--disable-localpcap) > the will disable including the pcap distributed with bro, and > instead will search for a libpcap on the system. I believe that > bro will first look for a libpcap directory at the same level > as the bro directory, and if it doesn't find one at that level > it looks for one installed on the system. > > There was a bug in the --disable-localpcap, and I'm not sure if > the fix is the last release. Let me know if you have any problems, > the patch is only a couple of lines. > > Hope this helps. > Thanks it does help. I think that I dont have the patch.. it seems to be looking for stuff in the pcap directory.. but I havent looked at it too deeply so I could be off still. > Cheers, > jason > > > Stephen J Smoogen wrote: > >> >> Hi I am just started with bro to evaluate it against the other tools >> we have. The first question I have is about using different libpcaps. >> We have our own fork of libpcap here (Phil Woods code) and I am >> needing to use it as a comparison with our snort and other tools. How >> hard is it to compile bro with another version of libpcap :)? >> > -- Stephen John Smoogen | CCN-5 Security Team LANL SIRT Team Lead | SMTP: smoogen at lanl.gov Los Alamos National Laboratory | Voice: 505.664.0645 Ta-03 SM-1498 MS: B255 DP 10S | FAX: 505.665.7793 Los Alamos, NM 87545 | PGR: 505.664.1535 From berry1.0 at gmail.com Fri Mar 18 19:31:24 2005 From: berry1.0 at gmail.com (Dana Zhang) Date: Sat, 19 Mar 2005 14:31:24 +1100 Subject: [Bro] Bro on other Packet Trace Dumps. Message-ID: Hi, I'm new to bro and what I would like to do is run bro on 38 hours of packet traces that I've aquired from another website. Is there any simple way to do this? I'm a bit confused as how to do this because I don't want to monitor the traffic of my own website/network but analyse data that I extracted from another source. From goeldich at ee.ethz.ch Sat Mar 19 05:11:36 2005 From: goeldich at ee.ethz.ch (=?ISO-8859-1?Q?Christoph_G=F6ldi?=) Date: Sat, 19 Mar 2005 14:11:36 +0100 Subject: [Bro] Bro on other Packet Trace Dumps. In-Reply-To: References: Message-ID: <79252A0CAA9E1B70F06D8C7B@[192.168.0.2]> hi if you have tcpdump files, you can easily do this with the -r flag: > bro -r example.trace brolite see page 9 and the following in the reference manual. have fun christoph --On Samstag, 19. M?rz 2005 14:31 +1100 Dana Zhang wrote: > Hi, I'm new to bro and what I would like to do is run bro on 38 hours > of packet traces that I've aquired from another website. > Is there any simple way to do this? > I'm a bit confused as how to do this because I don't want to monitor > the traffic of my own website/network but analyse data that I > extracted from another source. > _______________________________________________ > Bro mailing list > bro at bro-ids.org > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro From bltierney at lbl.gov Sun Mar 20 14:18:35 2005 From: bltierney at lbl.gov (Brian Tierney) Date: Sun, 20 Mar 2005 14:18:35 -0800 Subject: [Bro] bro alerts over email In-Reply-To: <1111102261.5711.11.camel@yaksha.ncsa.uiuc.edu> References: <1111102261.5711.11.camel@yaksha.ncsa.uiuc.edu> Message-ID: <8cf9a2512e2518e1d6df26dc31f50193@lbl.gov> I can answer question #2A. Someone else will have to handle #1, #2B, and #3. I noticed and fixed the problem you are referring to a couple weeks ago. We changed the beginning of function "email_notice" to be: if ( ! reading_live_traffic() || mail_dest == "" ) return; where mail_dest is defined at the beginning of the file as: global mail_dest = "" &redef; Then to you need to set mail_dest in your site policy file. reading_live_traffic() is defined in bro.bif.bro, but they way it was being used there was a race condition where it was not always being set correctly. On Mar 17, 2005, at 3:31 PM, Aashish Sharma wrote: > > 2) In-order to send emails from bro I had to comment out the following > from notice.bro file : > > # if ( ! mail_notification ) ----------------------- (2A) > # return; > > # local action = notice_action_filters[n$note](n); --------- (2B) > > # Choose destination address based on action type. > # local destination = (action == NOTICE_EMAIL) ? > # mail_dest : mail_page_dest; > local destination = mail_dest ; > > > 2A) I think 'if (! mail_notification)' condition is not holding true at > all. I see the following definition > > ../policy/notice.bro:global mail_notification = reading_live_traffic() > &redef; > > and > > ../policy/bro.bif.bro:global reading_live_traffic: function(): bool; > > I don't see reading_live_traffic function defined anywhere? Do I need > to > redef reading_live_traffic() function. > > If yes, should it be in the site policy file ? Would its value affect > other policy files ? (its used in conn.bro, load-level.bro and > stats.bro) > > From vern at icir.org Sun Mar 20 23:41:23 2005 From: vern at icir.org (Vern Paxson) Date: Sun, 20 Mar 2005 23:41:23 -0800 Subject: [Bro] bro alerts over email In-Reply-To: Your message of Thu, 17 Mar 2005 17:31:02 CST. Message-ID: <200503210741.j2L7fNu9025912@jaguar.icir.org> > 1) I am unable to redefine variables sensitive_URIs > (policy/http-request.bro) and hot_files (policy/ftp.bro) in my site > policy file. These are declared inside module scope, so you need :: to access them. For example: redef HTTP::sensitive_URIs += /rootdown.pl/; > 2B) local action = notice_action_filters[n$note](n) > > gives the following error in info.log file and bro stops : Oops, a bug. Patch appended. > in my site-policy file for getting email/page alert. If I understand it > correctly, I have to first put rootdown.pl (etc) in Sensitive_URI list > to get bro generate an alert and then declare that particular alert > using the above $pred config in my site policy file. Right ? > > Since this could lead to lot of $pred declearations, Is it possible to > have a formation like following for similar category of alerts : > > /usr/local/bro/site/hail.ncsa.uiuc.edu.bro, line 157 > (/^?(^.*rootdown.pl.*$)$?/ || /^?(^.*lads.exe.*$)$?/): error, requires > boolean operands One of the changes already in place for the next release is use of "||" and "&&" for combining patterns, for exactly this sort of reason. Vern diff -Lpolicy/notice.bro -Lpolicy/notice.bro -u -r1.14 -r1.15 --- policy/notice.bro +++ policy/notice.bro @@ -181,13 +181,11 @@ } } -function email_notice(n: notice_info) +function email_notice(n: notice_info, action: NoticeAction) { if ( ! reading_live_traffic() || mail_dest == "" ) return; - local action = notice_action_filters[n$note](n); - # Choose destination address based on action type. local destination = (action == NOTICE_EMAIL) ? mail_dest : mail_page_dest; @@ -311,7 +309,7 @@ if ( action != NOTICE_FILE ) { if ( action == NOTICE_EMAIL || action == NOTICE_PAGE ) - email_notice(n); + email_notice(n, action); if ( use_tagging ) alarm info; From vern at icir.org Sun Mar 20 23:43:29 2005 From: vern at icir.org (Vern Paxson) Date: Sun, 20 Mar 2005 23:43:29 -0800 Subject: [Bro] bro alerts over email In-Reply-To: Your message of Sun, 20 Mar 2005 14:18:35 PST. Message-ID: <200503210743.j2L7hTG6025981@jaguar.icir.org> > reading_live_traffic() is defined in bro.bif.bro, but they way it was > being used there was > a race condition where it was not always being set correctly. Minor clarification: this isn't a race condition in terms of not being deterministic. Rather, the problem is that Bro doesn't know whether it's reading live traffic until it finishes initializing global variables (in particular, the "interfaces" variable); so a call to reading_live_traffic() for a variable's initialization returns F even if later Bro determines it indeed is going to be reading live traffic. Vern From berry1.0 at gmail.com Mon Mar 21 20:18:46 2005 From: berry1.0 at gmail.com (Dana Zhang) Date: Tue, 22 Mar 2005 15:18:46 +1100 Subject: [Bro] Installing Bro Message-ID: I've been having some trouble install bro on my machine. First I tried to install the Stable 0.8 Release. After ./configure, I tried to make it. But in /bro-pub-0.8a88/libedit/history.c there was a compilation error with variable VIS_WHITE. It appears this variable appears only once I haven't been able to locate the source of this variable. Neither can the make file. Then I tried to install the current 0.9 Development Release. The first problem I encountered was when untarring the downloaded file from the bro side, there was an check sum error. That's never a promising sign. Then when configuring, my installation of libpcap is unable to be found. I have already installed it in the ~/bro-0.9a8/aux/libpcap-0.7.2 directory. From jean-philippe.luiggi at sagem.com Mon Mar 21 23:17:40 2005 From: jean-philippe.luiggi at sagem.com (Jean-Philippe LUIGGI) Date: Tue, 22 Mar 2005 08:17:40 +0100 Subject: [Bro] =?ISO-8859-1?Q?R=E9f=2E_=3A_[Bro]_Installing_Bro?= Message-ID: Hello, About the first problem, could you give us the OS you use ? And then the second part, did you download the package from http://www.bro-ids.org or ftp.ee.lbl.gov ? It's currently bro-0.9a8.tar.gz the latest version i think, i just got it a few minutes ago (in order to check) from the ftp site and all the process went smoothly. Best regards. Dana Zhang cc : (ccc : Jean-Philippe LUIGGI/DADM/SAGEM) Objet : [Bro] Installing Bro Envoy? par : bro-admin at ICSI.Berk eley.EDU 22/03/2005 05:18 Veuillez r?pondre ? Dana Zhang Remis le : 22/03/2005 05:20 I've been having some trouble install bro on my machine. First I tried to install the Stable 0.8 Release. After ./configure, I tried to make it. But in /bro-pub-0.8a88/libedit/history.c there was a compilation error with variable VIS_WHITE. It appears this variable appears only once I haven't been able to locate the source of this variable. Neither can the make file. Then I tried to install the current 0.9 Development Release. The first problem I encountered was when untarring the downloaded file from the bro side, there was an check sum error. That's never a promising sign. Then when configuring, my installation of libpcap is unable to be found. I have already installed it in the ~/bro-0.9a8/aux/libpcap-0.7.2 directory. _______________________________________________ Bro mailing list bro at bro-ids.org http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro From berry1.0 at gmail.com Tue Mar 22 02:31:56 2005 From: berry1.0 at gmail.com (Dana Zhang) Date: Tue, 22 Mar 2005 21:31:56 +1100 Subject: [Bro] =?ISO-8859-1?Q?Re:_R=E9f._:_[Bro]_Installing_Bro?= In-Reply-To: References: Message-ID: Thank you for looking at my problem. I was really quite confused as two why I was unable to install so I moved to a different computer/network all together and I had no problems installing version 0.9. I should have changed computers sooner. I downloaded my packages from http://www.bro-ids.org. I didn't have problems untarring the package on the new system. On Tue, 22 Mar 2005 08:17:40 +0100, Jean-Philippe LUIGGI wrote: > Hello, > > About the first problem, could you give us the OS you use ? > And then the second part, did you download the package from > http://www.bro-ids.org or ftp.ee.lbl.gov ? > It's currently bro-0.9a8.tar.gz the latest version i think, i just got it a > few minutes ago (in order to check) from the ftp site and all the process > went smoothly. > > Best regards. > > Dana Zhang > > cc : (ccc : Jean-Philippe LUIGGI/DADM/SAGEM) > Objet : [Bro] Installing Bro > Envoy? par : > bro-admin at ICSI.Berk > eley.EDU > > 22/03/2005 05:18 > Veuillez r?pondre ? > Dana Zhang > Remis le : > 22/03/2005 05:20 > > I've been having some trouble install bro on my machine. > First I tried to install the Stable 0.8 Release. After ./configure, I > tried to make it. But in /bro-pub-0.8a88/libedit/history.c there was a > compilation error with variable VIS_WHITE. It appears this variable > appears only once I haven't been able to locate the source of this > variable. Neither can the make file. > > Then I tried to install the current 0.9 Development Release. The first > problem I encountered was when untarring the downloaded file from the > bro side, there was an check sum error. That's never a promising sign. > Then when configuring, my installation of libpcap is unable to be > found. I have already installed it in the > ~/bro-0.9a8/aux/libpcap-0.7.2 directory. > _______________________________________________ > Bro mailing list > bro at bro-ids.org > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro > > From goeldich at ee.ethz.ch Tue Mar 22 04:54:56 2005 From: goeldich at ee.ethz.ch (Christoph Goeldi) Date: Tue, 22 Mar 2005 12:54:56 +0000 Subject: [Bro] Bro on other Packet Trace Dumps. In-Reply-To: References: <79252A0CAA9E1B70F06D8C7B@192.168.0.2> Message-ID: <1111496096.424015a02f60e@email.ee.ethz.ch> hi dana > Page 9 of the reference manual appears to bea list figures and tables. > I tried to run > > bro -r example.trace brolite > and it should work if I had a tcpdump file. Unfortunately my trace > file are not in tcpdump format. i'm not sure, but i think that tcpdump is the only format at the moment which can be read by bro. what format do you have? maybe there is a converter around... > On page 18 of the Bro user manual, the following command was suggested > for use with a tcpdump file. > > bro -r dumpfile brohost i meant page 17 of the pdf file which is page number 9 in the reference manual. (see the number in the right upper corner) by the way if you have installed bro with the commands "./configure", "make", "make install" and "make install-brolite" or similar you can start it with the command > bro -r dumpfile brolite you have to replace the word "brohost" in the command with the name of the policy file you want to load. read more of it in the user and quick start manuals... cheers christoph > On Sat, 19 Mar 2005 14:11:36 +0100, Christoph G?ldi > wrote: > > hi > > > > if you have tcpdump files, you can easily do this with the -r flag: > > > > > bro -r example.trace brolite > > > > see page 9 and the following in the reference manual. > > > > have fun > > christoph > > > > --On Samstag, 19. M?rz 2005 14:31 +1100 Dana Zhang > > wrote: > > > > > Hi, I'm new to bro and what I would like to do is run bro on 38 hours > > > of packet traces that I've aquired from another website. > > > Is there any simple way to do this? > > > I'm a bit confused as how to do this because I don't want to monitor > > > the traffic of my own website/network but analyse data that I > > > extracted from another source. > > > _______________________________________________ > > > Bro mailing list > > > bro at bro-ids.org > > > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro > > > > > From rpang at CS.Princeton.EDU Tue Mar 22 06:41:04 2005 From: rpang at CS.Princeton.EDU (Ruoming Pang) Date: Tue, 22 Mar 2005 09:41:04 -0500 Subject: [Bro] =?ISO-8859-1?Q?Re:_[Bro]_Re:_R=E9f._:_[Bro]_Installing_Bro?= In-Reply-To: References: Message-ID: <88ca385d8d687ee681d1bdcad69cb0d2@cs.princeton.edu> > so I moved to a different > computer/network all together and I had no problems installing version > 0.9. Hi, What is the OS on the old system? We'd like to understand why it didn't work. Thanks! Ruoming From aashish at uiuc.edu Tue Mar 22 10:37:19 2005 From: aashish at uiuc.edu (aashish at uiuc.edu) Date: Tue, 22 Mar 2005 12:37:19 -0600 Subject: [Bro] bro alerts over email In-Reply-To: <200503210743.j2L7hTG6025981@jaguar.icir.org> Message-ID: <20050322183719.GA25476@uiuc.edu> Mail notification via bro is working just fine (and fast) now. Thanks for all the input and help. I would like to point a few more things which we had to add localy: 1) currently NOTICE_PAGE and NOTICE_EMAIL are independent actions so we had to do minor modifications in notice.bro to be able to send an email as well when NOTICE_PAGE action takes place. I think would be a good idea to have an email sent while NOTICE_PAGE action takes place. 2) Going back to reading_live_traffic()/mail notification issue : Since, > (in particular, the "interfaces" variable); so a call to reading_live_traffic() > for a variable's initialization returns F even if later Bro determines > it indeed is going to be reading live traffic. Not sure why we needed '!' in 'if (! mail_notification)' condition because mail_notification is returning false irrespective of live_traffic capture or a tcpdump reply. The following seems to be working fine : if ( mail_notification || mail_dest == "" || mail_page_dest == "" ) return; Thanks a lot. Aashish Sharma On Sun, Mar 20, 2005 at 11:43:29PM -0800, Vern Paxson wrote: > > reading_live_traffic() is defined in bro.bif.bro, but they way it was > > being used there was > > a race condition where it was not always being set correctly. > > Minor clarification: > this isn't a race condition in terms of not being > deterministic. Rather, the problem is that Bro doesn't know whether it's > reading live traffic until it finishes initializing global variables > (in particular, the "interfaces" variable); so a call to reading_live_traffic() > for a variable's initialization returns F even if later Bro determines > it indeed is going to be reading live traffic. > > Vern > _______________________________________________ > Bro mailing list > bro at bro-ids.org > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro From berry1.0 at gmail.com Tue Mar 22 16:14:17 2005 From: berry1.0 at gmail.com (Dana Zhang) Date: Wed, 23 Mar 2005 11:14:17 +1100 Subject: [Bro] =?ISO-8859-1?Q?Re:_[Bro]_Re:_R=E9f._:_[Bro]_Installing_Bro?= In-Reply-To: <88ca385d8d687ee681d1bdcad69cb0d2@cs.princeton.edu> References: <88ca385d8d687ee681d1bdcad69cb0d2@cs.princeton.edu> Message-ID: Hi, I was on SunOS 5.9. I moved to a Linux 2.4.18-3bigmem. Cheers, Dana On Tue, 22 Mar 2005 09:41:04 -0500, Ruoming Pang wrote: > > so I moved to a different > > computer/network all together and I had no problems installing version > > 0.9. > > Hi, > > What is the OS on the old system? We'd like to understand why it didn't > work. Thanks! > > Ruoming > > From berry1.0 at gmail.com Tue Mar 22 16:14:17 2005 From: berry1.0 at gmail.com (Dana Zhang) Date: Wed, 23 Mar 2005 11:14:17 +1100 Subject: [Bro] =?ISO-8859-1?Q?Re:_[Bro]_Re:_R=E9f._:_[Bro]_Installing_Bro?= In-Reply-To: <88ca385d8d687ee681d1bdcad69cb0d2@cs.princeton.edu> References: <88ca385d8d687ee681d1bdcad69cb0d2@cs.princeton.edu> Message-ID: Hi, I was on SunOS 5.9. I moved to a Linux 2.4.18-3bigmem. Cheers, Dana On Tue, 22 Mar 2005 09:41:04 -0500, Ruoming Pang wrote: > > so I moved to a different > > computer/network all together and I had no problems installing version > > 0.9. > > Hi, > > What is the OS on the old system? We'd like to understand why it didn't > work. Thanks! > > Ruoming > > From vern at icir.org Tue Mar 22 23:54:05 2005 From: vern at icir.org (Vern Paxson) Date: Tue, 22 Mar 2005 23:54:05 -0800 Subject: [Bro] bro alerts over email In-Reply-To: Your message of Tue, 22 Mar 2005 12:37:19 CST. Message-ID: <200503230754.j2N7s5Uo098749@jaguar.icir.org> > 1) currently NOTICE_PAGE and NOTICE_EMAIL are independent actions so we had to do minor modifications in notice.bro > to be able to send an email as well when NOTICE_PAGE action takes place. > > I think would be a good idea to have an email sent while NOTICE_PAGE action takes place. Yes, we agree. I've added this to the to-do list. Not sure how quickly it'l be done, though (since the right way to do it is to allow the user to specify either one, or the other, *or* both, and that sort of flexiblity doesn't fit with the current exclusive-action model). > Not sure why we needed '!' in 'if (! mail_notification)' condition because mail_notification is returning false > irrespective of live_traffic capture or a tcpdump reply. Well, that was a bug, per the earlier discussion. In any case, it's gone with the upcoming 0.9a9 release. Vern From lionel2 at enorth.com.cn Thu Mar 24 12:42:43 2005 From: lionel2 at enorth.com.cn (Lionel Zhou) Date: Fri, 25 Mar 2005 04:42:43 +0800 Subject: [Bro] On Demand Staffing for Information Technology Projects Message-ID: <42432643.3070805@enorth.com.cn> Rent-A-Pro.com offers on demand staffing and service delivery for information technology projects. It is convenient for small businesses as well as individual developers to get temporary help, consulting service and outsourcing service from independent contractors around the world through our site. We provide functions for posting project requirements, bidding for projects and rating between buyers and sellers. Buyers and sellers are protected from fraud by an escrow system and a dispute resolution system. http://www.rent-a-pro.com/ -------------- next part -------------- A non-text attachment was scrubbed... Name: lionel2.vcf Type: text/x-vcard Size: 253 bytes Desc: not available Url : http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20050325/e59f1244/attachment.vcf From berry1.0 at gmail.com Mon Mar 28 00:35:10 2005 From: berry1.0 at gmail.com (Dana Zhang) Date: Mon, 28 Mar 2005 18:35:10 +1000 Subject: [Bro] Bro on other Packet Trace Dumps. In-Reply-To: <1111496096.424015a02f60e@email.ee.ethz.ch> References: <79252A0CAA9E1B70F06D8C7B@192.168.0.2> <1111496096.424015a02f60e@email.ee.ethz.ch> Message-ID: hi Chris, > i'm not sure, but i think that tcpdump is the only format at the moment which > can be read by bro. > what format do you have? maybe there is a converter around... > The current format of my data is just packet headers in binary. I tried to convert to tcpdump format myself. can I confirm that tcpdump format for tcp commections is: src > dst: flags data-seqno ack window urgent options i'm only working with tcp packets. a couple of examples of my packets are as follows 10.0.0.163.1422 > 10.0.0.219.80: . 17193851:17193851(0) ack 1278587442 win 8623 10.0.0.7.1202 > 10.0.0.8.25: P 22414518:22415922(1404) ack 20496183 win 8474 10.0.0.67.4945 > 10.0.0.66.80: S 2222637079:2222637079(0) win 32696 urg 0 10.0.0.11.26159 > 10.0.0.12.25: . 868560419:868561879(1460) ack 1691568355 win 61320 However, when I run this file with bro using > bro -r dumpfile brolite I receive the error problem with trace file dumpfile - bad dump file format. Is there something I missed? Cheers, Dana From goeldich at ee.ethz.ch Mon Mar 28 00:49:01 2005 From: goeldich at ee.ethz.ch (=?ISO-8859-1?Q?Christoph_G=F6ldi?=) Date: Mon, 28 Mar 2005 10:49:01 +0200 Subject: [Bro] Bro on other Packet Trace Dumps. In-Reply-To: References: <79252A0CAA9E1B70F06D8C7B@192.168.0.2> <1111496096.424015a02f60e@email.ee.ethz.ch> Message-ID: hi dana tcpdump is also a binary format. how did you catch your dump? i mean when you catch it with tcpdump you get exactly what you described: packet headers in binary. cheers christoph --On Montag, 28. M?rz 2005 18:35 +1000 Dana Zhang wrote: > hi Chris, > >> i'm not sure, but i think that tcpdump is the only format at the moment >> which can be read by bro. >> what format do you have? maybe there is a converter around... >> > > The current format of my data is just packet headers in binary. I > tried to convert to tcpdump format myself. can I confirm that tcpdump > format for tcp commections is: > src > dst: flags data-seqno ack window urgent options > > i'm only working with tcp packets. > a couple of examples of my packets are as follows > 10.0.0.163.1422 > 10.0.0.219.80: . 17193851:17193851(0) ack 1278587442 > win 8623 10.0.0.7.1202 > 10.0.0.8.25: P 22414518:22415922(1404) ack > 20496183 win 8474 10.0.0.67.4945 > 10.0.0.66.80: S > 2222637079:2222637079(0) win 32696 urg 0 10.0.0.11.26159 > 10.0.0.12.25: > . 868560419:868561879(1460) ack > 1691568355 win 61320 > > However, when I run this file with bro using >> bro -r dumpfile brolite > I receive the error problem with trace file dumpfile - bad dump file > format. > > Is there something I missed? > Cheers, > Dana From berry1.0 at gmail.com Mon Mar 28 00:59:24 2005 From: berry1.0 at gmail.com (Dana Zhang) Date: Mon, 28 Mar 2005 18:59:24 +1000 Subject: [Bro] Bro on other Packet Trace Dumps. In-Reply-To: References: <79252A0CAA9E1B70F06D8C7B@192.168.0.2> <1111496096.424015a02f60e@email.ee.ethz.ch> Message-ID: Hi christoph, are you saying that when this is run: >> bro -r dumpfile brolite dumpfile is a binary file? I thought bro took a tcpdump file and tcpdump outputs files in the format of : src > dst: flags data-seqno ack window urgent options my packets were captured using a DAG2 system. traces are in DAG format, which is a fixed 64 bytes record format with 40 bytes of IP header. I extracted from my binary to make it look like a tcpdump file. cheers, Dana On Mon, 28 Mar 2005 10:49:01 +0200, Christoph G?ldi wrote: > hi dana > > tcpdump is also a binary format. > how did you catch your dump? > i mean when you catch it with tcpdump you get exactly what you described: > packet headers in binary. > > cheers > christoph > > --On Montag, 28. M?rz 2005 18:35 +1000 Dana Zhang > wrote: > > > hi Chris, > > > >> i'm not sure, but i think that tcpdump is the only format at the moment > >> which can be read by bro. > >> what format do you have? maybe there is a converter around... > >> > > > > The current format of my data is just packet headers in binary. I > > tried to convert to tcpdump format myself. can I confirm that tcpdump > > format for tcp commections is: > > src > dst: flags data-seqno ack window urgent options > > > > i'm only working with tcp packets. > > a couple of examples of my packets are as follows > > 10.0.0.163.1422 > 10.0.0.219.80: . 17193851:17193851(0) ack 1278587442 > > win 8623 10.0.0.7.1202 > 10.0.0.8.25: P 22414518:22415922(1404) ack > > 20496183 win 8474 10.0.0.67.4945 > 10.0.0.66.80: S > > 2222637079:2222637079(0) win 32696 urg 0 10.0.0.11.26159 > 10.0.0.12.25: > > . 868560419:868561879(1460) ack > > 1691568355 win 61320 > > > > However, when I run this file with bro using > >> bro -r dumpfile brolite > > I receive the error problem with trace file dumpfile - bad dump file > > format. > > > > Is there something I missed? > > Cheers, > > Dana > > From jp-www at dcs.gla.ac.uk Mon Mar 28 01:40:48 2005 From: jp-www at dcs.gla.ac.uk (Jonathan Paisley) Date: Mon, 28 Mar 2005 10:40:48 +0100 Subject: [Bro] Bro on other Packet Trace Dumps. In-Reply-To: References: <79252A0CAA9E1B70F06D8C7B@192.168.0.2> <1111496096.424015a02f60e@email.ee.ethz.ch> Message-ID: On 28 Mar 2005, at 9:59, Dana Zhang wrote: > my packets were captured using a DAG2 system. traces are in DAG > format, which is a fixed 64 bytes record format with 40 bytes of IP > header. I extracted from my binary to make it look like a tcpdump > file. You can probably use Endace's 'dagconvert' utility to convert from the DAG format to pcap format. pcap format /is/ the tcpdump binary format. You get this if you use the '-w file' option to tcpdump. Otherwise, it just outputs a textual description of the packets. From goeldich at ee.ethz.ch Mon Mar 28 01:52:44 2005 From: goeldich at ee.ethz.ch (=?ISO-8859-1?Q?Christoph_G=F6ldi?=) Date: Mon, 28 Mar 2005 11:52:44 +0200 Subject: [Bro] Bro on other Packet Trace Dumps. In-Reply-To: References: <79252A0CAA9E1B70F06D8C7B@192.168.0.2> <1111496096.424015a02f60e@email.ee.ethz.ch> Message-ID: <5D84748D74FD05FF879D8BFC@[192.168.0.2]> hi dana > are you saying that when this is run: >>> bro -r dumpfile brolite > dumpfile is a binary file? yes it is! > I thought bro took a tcpdump file and > tcpdump outputs files in the format of : > src > dst: flags data-seqno ack window urgent options no. tcpdump files are in a binary format. but when you make it visible with the tcpdump command it looks like what you said. > my packets were captured using a DAG2 system. traces are in DAG > format, which is a fixed 64 bytes record format with 40 bytes of IP > header. I extracted from my binary to make it look like a tcpdump > file. what an exotic format! please, go to http://dag.cs.waikato.ac.nz/, then enter the download section and get the dag-tools. install it and use the dagbpf command. i didn't check this out, i only made the internet searches for you... what i tell you are basics. maybe you have to read first a bit before come and ask again. have fun christoph > On Mon, 28 Mar 2005 10:49:01 +0200, Christoph G?ldi > wrote: >> hi dana >> >> tcpdump is also a binary format. >> how did you catch your dump? >> i mean when you catch it with tcpdump you get exactly what you described: >> packet headers in binary. >> >> cheers >> christoph >> >> --On Montag, 28. M?rz 2005 18:35 +1000 Dana Zhang >> wrote: >> >> > hi Chris, >> > >> >> i'm not sure, but i think that tcpdump is the only format at the >> >> moment which can be read by bro. >> >> what format do you have? maybe there is a converter around... >> >> >> > >> > The current format of my data is just packet headers in binary. I >> > tried to convert to tcpdump format myself. can I confirm that tcpdump >> > format for tcp commections is: >> > src > dst: flags data-seqno ack window urgent options >> > >> > i'm only working with tcp packets. >> > a couple of examples of my packets are as follows >> > 10.0.0.163.1422 > 10.0.0.219.80: . 17193851:17193851(0) ack 1278587442 >> > win 8623 10.0.0.7.1202 > 10.0.0.8.25: P 22414518:22415922(1404) ack >> > 20496183 win 8474 10.0.0.67.4945 > 10.0.0.66.80: S >> > 2222637079:2222637079(0) win 32696 urg 0 10.0.0.11.26159 > >> > 10.0.0.12.25: . 868560419:868561879(1460) ack >> > 1691568355 win 61320 >> > >> > However, when I run this file with bro using >> >> bro -r dumpfile brolite >> > I receive the error problem with trace file dumpfile - bad dump file >> > format. >> > >> > Is there something I missed? >> > Cheers, >> > Dana >> >> From berry1.0 at gmail.com Mon Mar 28 01:58:08 2005 From: berry1.0 at gmail.com (Dana Zhang) Date: Mon, 28 Mar 2005 19:58:08 +1000 Subject: [Bro] Bro on other Packet Trace Dumps. In-Reply-To: References: <79252A0CAA9E1B70F06D8C7B@192.168.0.2> <1111496096.424015a02f60e@email.ee.ethz.ch> Message-ID: On Mon, 28 Mar 2005 10:40:48 +0100, Jonathan Paisley wrote: > > On 28 Mar 2005, at 9:59, Dana Zhang wrote: > > > my packets were captured using a DAG2 system. traces are in DAG > > format, which is a fixed 64 bytes record format with 40 bytes of IP > > header. I extracted from my binary to make it look like a tcpdump > > file. > > You can probably use Endace's 'dagconvert' utility to convert from the > DAG format to pcap format. > > pcap format /is/ the tcpdump binary format. You get this if you use the > '-w file' option to tcpdump. Otherwise, it just outputs a textual > description of the packets. > > I thought I was recreating the textual output that tcpdump would create. I don't understand why bro is telling me there is something wrong with my tcpdump imitation trace file. What exactly should my packet trace file look like? I'm starting to get confused as to what bro accepts. From jp-www at dcs.gla.ac.uk Mon Mar 28 02:09:29 2005 From: jp-www at dcs.gla.ac.uk (Jonathan Paisley) Date: Mon, 28 Mar 2005 11:09:29 +0100 Subject: [Bro] Bro on other Packet Trace Dumps. In-Reply-To: References: <79252A0CAA9E1B70F06D8C7B@192.168.0.2> <1111496096.424015a02f60e@email.ee.ethz.ch> Message-ID: <58dff9ecfb3680a0a37c64c009c9942d@dcs.gla.ac.uk> On 28 Mar 2005, at 10:58, Dana Zhang wrote: > I thought I was recreating the textual output that tcpdump would > create. I don't understand why bro is telling me there is something > wrong with my tcpdump imitation trace file. This text output is _not_ what bro expects. > What exactly should my packet trace file look like? I'm starting to > get confused as to what bro accepts. bro accepts only the tcpdump (aka pcap) _binary_ format. Since you have a DAG format trace, you should just be able to use Endace's 'dagconvert' tool to convert to pcap format. You'd do something like: $ dagconvert -T eth:pcap -i yourfile -o out.pcap $ bro ... -r out.pcap I've assumed above that you've got legacy Ethernet file format. You may have ERF files from your dag capture, in which case you'd use '-T erf:pcap' in the dagconvert command line. Here's the output from dagconvert -h: $ dagconvert -h dagconvert: DAG file conversion utility. Usage: dagconvert [options] -d DAG device name -h display help (this page) -v increase verbosity -i input file -o output file -r N[k|m|g] change output file after N Bytes. k, m, g suffixes for kilobytes, megabytes, gigabytes. -s output snap length -t capture period in seconds -T input and output types (see list of types below) -A output record alignment (ERF only) -V select variable length output (ERF only) -F select fixed length output (ERF only) -G specify GMT offset in seconds (pcap only) -c 0|16|32 specify number of bits in FCS checksum (pcap only) -f comma separated list of filters (see list of filters below) -b specify a BPF style filter Supported types: dag ERF direct from DAG device (input only) erf ERF (extensible record format) file atm legacy ATM file (input only) eth legacy Ethernet file (input only) pos legacy PoS file (input only) null produces no input or output pcap libpcap format file (output only) prt ASCII text packet dump (output only) Supported filters: rx filter out rx errors (link layer) ds filter out ds errors (framing) trunc filter out truncated packets a,b,c,d filter on indicated interface(s) ==== For reference, an extract from the pcap.h header file, which describes the file structure somewhat: struct pcap_file_header { bpf_u_int32 magic; u_short version_major; u_short version_minor; bpf_int32 thiszone; /* gmt to local correction */ bpf_u_int32 sigfigs; /* accuracy of timestamps */ bpf_u_int32 snaplen; /* max length saved portion of each pkt */ bpf_u_int32 linktype; /* data link type (LINKTYPE_*) */ }; /* * Each packet in the dump file is prepended with this generic header. * This gets around the problem of different headers for different * packet interfaces. */ struct pcap_pkthdr { struct timeval ts; /* time stamp */ bpf_u_int32 caplen; /* length of portion present */ bpf_u_int32 len; /* length this packet (off wire) */ }; From aashish at uiuc.edu Wed Mar 30 12:24:21 2005 From: aashish at uiuc.edu (aashish at uiuc.edu) Date: Wed, 30 Mar 2005 14:24:21 -0600 Subject: [Bro] Re: Bro Packet Trace Dumps. In-Reply-To: <5D84748D74FD05FF879D8BFC@[192.168.0.2]> Message-ID: <20050330202421.GA4119@uiuc.edu> This is slightly off topic from the last bro packet trace dump thread. Right now I see bro rolling over bulk trace files as soon as the file size is 2G. (Even though we have large file system support on the os). The issue with this is that all the other log files are also rolled over. I think bro just restarts itself. Is it possible to set up bro to define the size at which bulk trace file should roll over and not have any other log files roll over (even if the bulk trace files roll over at 2G) ? I tried looking in the source and also putting tcpdump like options in the config file for but that does not seem to work. Aashish Sharma