From vern at icir.org Sun Feb 8 14:28:37 2004 From: vern at icir.org (Vern Paxson) Date: Sun, 08 Feb 2004 14:28:37 -0800 Subject: new bro "CURRENT" release - 0.8a70 Message-ID: <200402082228.i18MSbmO046590@jaguar.icir.org> An updated "CURRENT" version of Bro is now available from the usual location: ftp://ftp.ee.lbl.gov/bro-pub-0.8-current.tar.gz This version has a lot of changes, including a much-expanded Bro home page at http://www-nrg.ee.lbl.gov/bro.html, including a "wish list" for Bro development projects at http://www-nrg.ee.lbl.gov/bro-wishlist.html (suggestions/contributions welcomed!) and a powerful new facility for post-filtering alerts. I've appended the changes since the last "CURRENT" version (0.8a58). Vern -+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 0.8a70 Sun Feb 8 14:19:45 PST 2004 - Bro has a new home page at http://www-nrg.ee.lbl.gov/bro.html It includes a "wish list" of Bro development projects: http://www-nrg.ee.lbl.gov/bro-wishlist.html - The "match" expression has been completely overhauled (Umesh Shankar). It now has the syntax: match EXPR1 using EXPR2 Its semantics are complicated, but it's very powerful (see its use for alert filtering below). EXPR1 can have any type T. EXPR2 must be of type "set[R]", where R is a record type. R must have the following fields: $pred - type is "function(T): bool". This is the predicate associated with the record. It is passed in EXPR1's value and returns true or false. $result - can have any type T'. This is the value to use when if $pred returns true for EXPR1. $priority - type must be arithmetic (count, int, double). This is the priority associated with the match of EXPR1 if $pred returns true. The way the expression works is that EXPR1 is evaluated yielding a value V. EXPR2 is then evaluated yielding a set of records whose type includes the above fields. Bro then spins through each of the records in the set and tests whether its $pred predicate holds for V. If so, it records the given $result and the associated $priority. It then returns for the value of the entire expression the $result with the highest $priority. Here's an example. The following script: global match_stuff = { [$pred = function(a: count): bool { return a > 5; }, $result = "it's big", $priority = 2], [$pred = function(a: count): bool { return a > 15; }, $result = "it's really big", $priority = 3], [$pred = function(a: count): bool { return T; }, $result = "default", $priority = 0], }; print match 0 using match_stuff; print match 10 using match_stuff; print match 20 using match_stuff; when executed will print: default it's big it's really big (Note that Bro actually will first evalute $priority before evaluating $pred, and if it already has a better (higher) priority result, it will not bother calling $pred.) - There's a new syntax for designating function values (Umesh Shankar). It currently only works when associating a function value with a record field in a record constructor: [$foo(bad_guy: addr) = { launch_counter_DDOS(bad_guy); return 3.14; }] is equivalent to: [$foo = function(bad_guy: addr): double = { launch_counter_DDOS(bad_guy); return 3.14; }] The goal is to make such functions easier on the eye to express. The changes are (1) no "function" keywork necessary, (2) no function return type necessary (note, it is inferred from the "return" statement in the function body; eventually this will work for all functions, not just those in record constructors), (3) the '=' sign comes after the ')' rather than before the keyword "function". Given this syntax, we can rewrite the initialization of match_stuff in the example above as: global match_stuff = { [$pred(a: count) = { return a > 5; }, $result = "it's big", $priority = 2], [$pred(a: count) = { return a > 15; }, $result = "it's really big", $priority = 3], [$pred(a: count) = { return T; }, $result = "default", $priority = 0], }; - The motivation behind these elaborate new mechanisms is to provide a powerful and streamlined way to filter alerts. According, alert.bro now processes any alerts generated via ALERT() through a new global, alert_policy. alert_policy's type is set[alert_policy_item], where alert_policy_item is: type alert_policy_item: record { result: AlertAction; pred: function(a: alert_info): bool; priority: count; }; The idea is that you specify your alert post-filtering by redef'ing new elements into alert_policy. For example, here are two post-filtering rules used at LBL to weed out uninteresting alerts: # Ignore connections marked as sensitive because they're # NTP to otherwise-sensitive hosts (first clause) or they happen # to involve 2766/tcp (Solaris Listen), which happens frequently # to ftp.ee.lbl.gov if Bro misses the PORT negotiation. [$pred(a: alert_info) = { return a$alert == SensitiveConnection && (a$conn$id$resp_p == 123/udp || # NTP a$msg == /Solaris listen service/); }, $result = ALERT_FILE, $priority = 1], # Ignore sensitive URIs if the request was unsuccessful (code 404, # or not answered.) [$pred(a: alert_info) = { return a$alert == HTTP::HTTP_SensitiveURI && a$msg == /.*((\(404)|(no reply)).*/; }, $result = ALERT_FILE, $priority = 1], These rules are part of: redef alert_policy += { ... these records and others ... }; The idea behind them is to demote certain alerts that would ordinarily be syslog'd (i.e., the associated action is ALERT_LOG_ALWAYS) to instead just be recorded in the alert.$BRO_ID file. Naturally, there are many other possibilities. For example: [$pred(a: alert_info) = { if ( a$alert == FTP::FTP_Sensitive && a$msg == /.*crown_jewels.*/ ) { system("page_the_duty_officer \"crown jewels theft!\""); return T; } else return F; }, $result = ALERT_LOG_ALWAYS, $priority = 1000], would run the program page_the_duty_officer with the argument "crown jewels theft!" if an FTP_Sensitive alert was generated and the log message included the text "crown_jewels". More generally, post-filtering needn't just be about deciding on how the alert is logged; the processing can run programs, update tables, etc., just like any other function call might. - You can use the new function tally_alert_type in an alert_action_filters initialization in order to suppress immediate logging of an alert and instead have Bro generate a summary of how many times the given alert was seen when it exits. You can use another new function, file_alert, to specify an alert_action_filters initialization that causes the alerts to just be written to the alert.$BRO_ID file but not otherwise logged. For example: redef alert_action_filters += { # Just summarize various packet capture glitches. [[RetransmissionInconsistency, ContentGap, DroppedPackets, AckAboveHole]] = tally_alert_type, [RemoteWorm] = file_alert, }; would specify that RetransmissionInconsistency (etc.) alerts should just be reported in the log file (log.$BRO_ID) as a total count, and RemoteWorm should only be put in the alert.$BRO_ID file, but not otherwise logged or counted. You could get the same functionality by writing alert_policy records, but they're quite a bit bulkier than the above. Note that alert_action_filters entries take precedence over alert_policy records, but are evaluated *after* the "match" on alert_policy, so if you have predicates in alert_policy with side-effects (like the invocation of page_the_duty_officer in the example above), those will still happen. - The alert_info record (which is used in calls to ALERT) now has slots for some more additional information: user: string; # can hold an assocaited username filename: string; # an associated filename method: string; # associated HTTP method URL: string; # associated URL n: count; # any associated count/number/status code (These are all &optional, so you don't need to specify them if they're not appropriate.) A number of ALERT's in the default policy scripts have been changed to include these. The intent is to add more such information in the future. Ideally, alert_policy records shouldn't be doing checks like "a$msg == /.*((\(404)|(no reply)).*/" but instead extracting the status code directly from a field of 'a' (which is an alert_info record). - ALERT now fills in the '$id' field in the alert_info record with the $id of the associated connection, if the caller didn't suppy a $id but did supply a $conn. Likewise, it will fill in $src with the $orig_h from $id (after first filling in $id). The net result is that you can rely on $id and $src being set for any alert that has an associated connection. - The HTTP analysis scripts (policy/http*.bro) have been converted to use the "module" facility, similar to how ftp.bro was converted for 0.8a48. This may require changing some of your own scripts, generally just to add "HTTP::" qualifiers. - Now that the variables associated with FTP analysis are part of an "FTP" module, the "ftp_" prefix associated with: ftp_excessive_filename_len ftp_excessive_filename_trunc_len ftp_guest_ids ftp_hot_cmds ftp_hot_files ftp_hot_guest_files ftp_ignore_invalid_PORT ftp_ignore_privileged_PASVs ftp_log ftp_skip_hot has been removed, and these are now called: excessive_filename_len excessive_filename_trunc_len guest_ids hot_cmds hot_files hot_guest_files ignore_invalid_PORT ignore_privileged_PASVs log_file skip_hot To get to them from other scripts, you specify, for example, redef FTP::guest_ids = { .... }; whereas before you had to use: redef FTP::ftp_guest_ids = { .... }; - The new connection logging format introduced in 0.8a57 is now the default, unless you redef the new variable "traditional_conn_format" to be T (Robin Sommer). Connections using unidentified ephemeral ports now have a service of simply "other" rather than other-XXXX. The 'U' connection status flag has been removed (unless you're using traditional_conn_format). - Tables can now be directly indexed by records, and indexing using records is no longer interchangeable with using a corresponding list of indices (Umesh Shankar). This may require adjustments to existing policy scripts. - Hostnames such as www.google.com now have type set[addr] rather than a funky internal list type. - The new function dump_current_packet(file_name: string) dumps a copy of the current packet to the file with the given name, appending it if the file already exists (Robin Sommer). The file is in tcpdump format. A handy use for this is in an event handler for signature_match(), to record packets that match given signatures. - The event new_packet() is invoked for each new packet (Robin Sommer). It currently doesn't provide the packet contents but soon will in a fashion similar to secondary-filter.bro. - "cf -f fmt" specifies a strtime() format. -u specifics UTC time rather than local time (Mark Delow and Craig Leres). cf now has a man page (Craig Leres). - Two new variables, backdoor_ignore_local and backdoor_ignore_remote, can be used to specify backdoor signatures that should be ignored if the server is local/remote. - A bug has been fixed in which a "next" executed in the final iteration of a for loop would mangle the subsequent processing of the outer statements (Chema Gonzalez). - Bug fixes for MIME and Base64 processing (Ruoming Pang). - pcap.bro now builds its filter in the opposite order (restrict_filters first), which can improve packet filtering performance (Robin Sommer). - A bug in &default has been fixed. - Portability for different pcap_compile_nopcap() calling sequences (Chema Gonzalez). - Some tweaks for a minor reduction in memory consumption. - A memory leak for secondary packet filters has been fixed. - The localization of error messages (what script line they correspond to) has been improved. From wct2003jm at comcast.net Tue Feb 10 10:44:45 2004 From: wct2003jm at comcast.net (wct2003jm at comcast.net) Date: Tue, 10 Feb 2004 18:44:45 +0000 Subject: Test suites? Message-ID: <021020041844.8301.15b7@comcast.net> Are there any test suites for Bro? PCAP files to read, scripts to run, and outputs to diff when done? I'm looking at modifying Bro, but would love it if there were some solid, defensible way to detect and inadvertent regressions... Thanks, -wt -- Busily building the unknown. From vern at icir.org Tue Feb 10 22:38:39 2004 From: vern at icir.org (Vern Paxson) Date: Tue, 10 Feb 2004 22:38:39 -0800 Subject: Test suites? In-Reply-To: Your message of Tue, 10 Feb 2004 18:44:45 GMT. Message-ID: <200402110638.i1B6cdmO030379@jaguar.icir.org> > Are there any test suites for Bro? PCAP files to read, scripts to run, > and outputs to diff when done? Unfortunately, no. This is on the new "wish list" at: http://www-nrg.ee.lbl.gov/bro-wishlist.html (which the Bro home page now links to) - Vern From rmkml at wanadoo.fr Sat Feb 14 13:38:13 2004 From: rmkml at wanadoo.fr (rmkml) Date: Sat, 14 Feb 2004 22:38:13 +0100 (CET) Subject: Bro & freebsd Message-ID: Hi, I am new user bro. I have compiled bro 08 current in linux : ok but I have compiled pb with freebsd v4.9R, look my error : ./configure ... make ... gcc -g -O2 -I. -I. -g -O0 -c strlcpy.c ar -r libedit.a chared.o common.o el.o emacs.o fcns.o hist.o history.o key.o map.o parse.o prompt.o read.o refresh.o search.o sig.o term.o tokenizer.o tty.o vi.o help.o fgetln.o readline.o strlcpy.o bison -y -d -t -v builtin-func.y flex -obif_lex.cc builtin-func.l g++ -o bif_lex.o -c bif_lex.cc g++ -o bif_parse.o -c bif_parse.cc y.tab.c: In function `int yyparse()': y.tab.c:1705: syntax error before `goto' *** Error code 1 Possible help me ? Thanks for your reply Regards Rmkml at Wanadoo.fr From vern at icir.org Sat Feb 14 14:12:59 2004 From: vern at icir.org (Vern Paxson) Date: Sat, 14 Feb 2004 14:12:59 -0800 Subject: Bro & freebsd In-Reply-To: Your message of Sat, 14 Feb 2004 22:38:13 +0100. Message-ID: <200402142213.i1EMCxmO077125@jaguar.icir.org> > but I have compiled pb with freebsd v4.9R, This is strange, as Bro is primarily developed under FreeBSD. > bison -y -d -t -v builtin-func.y > flex -obif_lex.cc builtin-func.l > g++ -o bif_lex.o -c bif_lex.cc > g++ -o bif_parse.o -c bif_parse.cc > y.tab.c: In function `int yyparse()': > y.tab.c:1705: syntax error before `goto' Can you please send the y.tab.c file? That's necessary for diagnosing what's going on here. Vern From rmkml at wanadoo.fr Sat Feb 14 14:18:58 2004 From: rmkml at wanadoo.fr (rmkml) Date: Sat, 14 Feb 2004 23:18:58 +0100 (CET) Subject: Bro & freebsd In-Reply-To: <200402142213.i1EMCxmO077125@jaguar.icir.org> References: <200402142213.i1EMCxmO077125@jaguar.icir.org> Message-ID: Hi Vern, oops, I don't find this file on my freebsd box, Thanks again for your help Regards Rmkml at Wanadoo.fr On Sat, 14 Feb 2004, Vern Paxson wrote: > Date: Sat, 14 Feb 2004 14:12:59 -0800 > From: Vern Paxson > To: rmkml > Cc: bro at listserv.lbl.gov > Subject: Re: Bro & freebsd > > > but I have compiled pb with freebsd v4.9R, > > This is strange, as Bro is primarily developed under FreeBSD. > > > bison -y -d -t -v builtin-func.y > > flex -obif_lex.cc builtin-func.l > > g++ -o bif_lex.o -c bif_lex.cc > > g++ -o bif_parse.o -c bif_parse.cc > > y.tab.c: In function `int yyparse()': > > y.tab.c:1705: syntax error before `goto' > > Can you please send the y.tab.c file? That's necessary for diagnosing > what's going on here. > > Vern > From vern at icir.org Sat Feb 14 14:35:38 2004 From: vern at icir.org (Vern Paxson) Date: Sat, 14 Feb 2004 14:35:38 -0800 Subject: Bro & freebsd In-Reply-To: Your message of Sat, 14 Feb 2004 23:18:58 +0100. Message-ID: <200402142235.i1EMZcmO077667@jaguar.icir.org> > I don't find this file on my freebsd box, It is renamed by the Makefile. The Makefile steps will look like: bif_parse.h bif_parse.cc: builtin-func.y bif_arg.h bif_type.def $(YACC) $(YFLAGS) builtin-func.y @mv y.tab.h bif_parse.h @mv y.tab.c bif_parse.cc so you'll find y.tab.c as bif_parse.cc. That said, checking my email archives reveals that this is a FreeBSD 4.9 problem with its bison installation, and not a Bro problem, per the appended. As a work-around, I can send you the bif_parse.cc that's generated using my version of bison - that should build okay for you. Vern Date: Fri, 02 Jan 2004 09:22:46 -0800 From: Vern Paxson Subject: Re: Error compiling bro-pub-0.8a58 To: Richard Bejtlich Well, the problem line is: #if defined (__GNUC_MINOR__) && 2093 <= (__GNUC__ * 1000 + __GNUC_MINOR__) __attribute__ ((__unused__)) #endif but that's not anything from the Bro sources. It must be from your version of bison (1.875), which is much more recent than the version I use. So I'm afraid I can't help you further with untangling it - it's some sort of configuration problem or mismatch w/ gcc version. > I'm interested in Bro because I cite you as an IDS > pioneer in a book I'm writing, and I'd like to briefly > describe how to get Bro working on a FreeBSD system. Great! > I've only dabbled with Bro but I want to give the > reader an easy guide to installation and basic use. > Your install doc is quite thorough, but do you have a > "rapid deployment" guide of sorts you could share? There's a "Getting Started" section in the manual, but currently that's it. Vern From rmkml at wanadoo.fr Sat Feb 14 14:40:41 2004 From: rmkml at wanadoo.fr (rmkml) Date: Sat, 14 Feb 2004 23:40:41 +0100 (CET) Subject: Bro & freebsd In-Reply-To: <200402142235.i1EMZcmO077667@jaguar.icir.org> References: <200402142235.i1EMZcmO077667@jaguar.icir.org> Message-ID: ok good, possible send this file to me ? and possible add this file to bro 08 current ? Regards Rmkml at Wanadoo.fr On Sat, 14 Feb 2004, Vern Paxson wrote: > Date: Sat, 14 Feb 2004 14:35:38 -0800 > From: Vern Paxson > To: rmkml > Cc: bro at listserv.lbl.gov > Subject: Re: Bro & freebsd > > > I don't find this file on my freebsd box, > > It is renamed by the Makefile. The Makefile steps will look like: > > bif_parse.h bif_parse.cc: builtin-func.y bif_arg.h bif_type.def > $(YACC) $(YFLAGS) builtin-func.y > @mv y.tab.h bif_parse.h > @mv y.tab.c bif_parse.cc > > so you'll find y.tab.c as bif_parse.cc. > > That said, checking my email archives reveals that this is a FreeBSD 4.9 > problem with its bison installation, and not a Bro problem, per the appended. > As a work-around, I can send you the bif_parse.cc that's generated using > my version of bison - that should build okay for you. > > Vern > > > Date: Fri, 02 Jan 2004 09:22:46 -0800 > From: Vern Paxson > Subject: Re: Error compiling bro-pub-0.8a58 > To: Richard Bejtlich > > Well, the problem line is: > > #if defined (__GNUC_MINOR__) && 2093 <= (__GNUC__ * 1000 + __GNUC_MINOR__) > __attribute__ ((__unused__)) > #endif > > but that's not anything from the Bro sources. It must be from your version > of bison (1.875), which is much more recent than the version I use. So I'm > afraid I can't help you further with untangling it - it's some sort of > configuration problem or mismatch w/ gcc version. > > > I'm interested in Bro because I cite you as an IDS > > pioneer in a book I'm writing, and I'd like to briefly > > describe how to get Bro working on a FreeBSD system. > > Great! > > > I've only dabbled with Bro but I want to give the > > reader an easy guide to installation and basic use. > > Your install doc is quite thorough, but do you have a > > "rapid deployment" guide of sorts you could share? > > There's a "Getting Started" section in the manual, but currently that's it. > > Vern > From vern at icir.org Sat Feb 14 14:50:12 2004 From: vern at icir.org (Vern Paxson) Date: Sat, 14 Feb 2004 14:50:12 -0800 Subject: Bro & freebsd In-Reply-To: Your message of Sat, 14 Feb 2004 23:40:41 +0100. Message-ID: <200402142250.i1EMoCmO077995@jaguar.icir.org> > possible send this file to me ? I can, but first try the following. Your system may have "byacc" installed on it in addition to bison. If so, try changing YACC = bison -y in the Makefile to YACC = byacc > and possible add this file to bro 08 current ? I am reluctant to add complexity to the Bro distribution in order to work around bugs in various operating system releases (except when they can be incorporated as autoconf elements), as that's the path to ongoing maintenance headaches ... Vern From rmkml at wanadoo.fr Sat Feb 14 15:03:59 2004 From: rmkml at wanadoo.fr (rmkml) Date: Sun, 15 Feb 2004 00:03:59 +0100 (CET) Subject: Bro & freebsd In-Reply-To: <200402142250.i1EMoCmO077995@jaguar.icir.org> References: <200402142250.i1EMoCmO077995@jaguar.icir.org> Message-ID: very thanks Guru, this change has solved pb Regards Rmkml at Wanadoo.fr http://crusoecids.dyndns.org On Sat, 14 Feb 2004, Vern Paxson wrote: > Date: Sat, 14 Feb 2004 14:50:12 -0800 > From: Vern Paxson > To: rmkml > Cc: bro at listserv.lbl.gov > Subject: Re: Bro & freebsd > > > possible send this file to me ? > > I can, but first try the following. Your system may have "byacc" installed > on it in addition to bison. If so, try changing > > YACC = bison -y > > in the Makefile to > > YACC = byacc > > > and possible add this file to bro 08 current ? > > I am reluctant to add complexity to the Bro distribution in order to work > around bugs in various operating system releases (except when they can be > incorporated as autoconf elements), as that's the path to ongoing maintenance > headaches ... > > Vern > From richard_bejtlich at yahoo.com Sat Feb 14 16:35:31 2004 From: richard_bejtlich at yahoo.com (Richard Bejtlich) Date: Sat, 14 Feb 2004 16:35:31 -0800 (PST) Subject: Bro & freebsd In-Reply-To: <200402142250.i1EMoCmO077995@jaguar.icir.org> Message-ID: <20040215003531.12016.qmail@web60808.mail.yahoo.com> --- Vern Paxson wrote: > Your system may have "byacc" installed > on it in addition to bison. If so, try changing > > YACC = bison -y > > in the Makefile to > > YACC = byacc > If you can narrow down how to fix this problem, please send your fix to ports at freebsd.org. The port maintainer can then modify the port to account for this behavior. Thank you, Richard __________________________________ Do you Yahoo!? Yahoo! Finance: Get your refund fast by filing online. http://taxes.yahoo.com/filing.html From rmkml at wanadoo.fr Sun Feb 15 13:38:51 2004 From: rmkml at wanadoo.fr (rmkml) Date: Sun, 15 Feb 2004 22:38:51 +0100 (CET) Subject: syslog events Message-ID: Hi, Possible receive bro events with syslog daemon ? possible coding function to convert all events in same function ? (and on this new function, choice syslog / stdout / stderr ...) Regards Rmkml at Wanadoo.fr From rpang at cs.princeton.edu Sun Feb 15 14:03:52 2004 From: rpang at cs.princeton.edu (Ruoming Pang) Date: Sun, 15 Feb 2004 17:03:52 -0500 Subject: syslog events In-Reply-To: Message-ID: <000b01c3f40f$99a303b0$98bdb48c@CS.Princeton.EDU> > Possible receive bro events with syslog daemon ? Sorry, I don't fully get it -- do you mean generating events from syslog or dumping events to syslog? If it's the latter, a little hack can make it possible -- all events go though EventMgr (see Event.cc). However, you may want to be careful in printing the event parameters, some are large chunks of data, e.g. in event http_entity_data. Ruoming From rmkml at wanadoo.fr Sun Feb 15 14:16:15 2004 From: rmkml at wanadoo.fr (rmkml) Date: Sun, 15 Feb 2004 23:16:15 +0100 (CET) Subject: syslog events In-Reply-To: <000b01c3f40f$99a303b0$98bdb48c@CS.Princeton.EDU> References: <000b01c3f40f$99a303b0$98bdb48c@CS.Princeton.EDU> Message-ID: Thanks Ruoming for quick answers yes my question is how dumping bro events to syslog. I have another question, How receive bro events in centralized console mgmt ? ( syslog / redir stdout / *sql* / bro_tunnel / ... ?) like snort/acid ? Thanks again Regards Rmkml at Wanadoo.fr On Sun, 15 Feb 2004, Ruoming Pang wrote: > Date: Sun, 15 Feb 2004 17:03:52 -0500 > From: Ruoming Pang > To: 'rmkml' , bro at listserv.lbl.gov > Subject: RE: syslog events > > > Possible receive bro events with syslog daemon ? > > Sorry, I don't fully get it -- do you mean generating events from syslog > or dumping events to syslog? > > If it's the latter, a little hack can make it possible -- all events go > though EventMgr (see Event.cc). However, you may want to be careful in > printing the event parameters, some are large chunks of data, e.g. in > event http_entity_data. > > Ruoming > > From vern at icir.org Sun Feb 15 15:24:46 2004 From: vern at icir.org (Vern Paxson) Date: Sun, 15 Feb 2004 15:24:46 -0800 Subject: syslog events In-Reply-To: Your message of Sun, 15 Feb 2004 22:38:51 +0100. Message-ID: <200402152324.i1FNOkmO003603@jaguar.icir.org> > Possible receive bro events with syslog daemon ? That's not the Bro model. Events are meant to be processed by event handlers in your policy scripts. Those handlers might then decide to execute "log" statements (or by calling the ALERT() function, which is specified in policy/alert.bro), though, and *those* are sent via syslog. Note, an event is *NOT* an alert, it's a description of some network activity in security-neutral terms. > possible coding function to convert all events in same function ? > (and on this new function, choice syslog / stdout / stderr ...) Not in a policy script, since events are strongly typed. The tracing facility, "-t tracefile", let's you dump a list of all the events to a given file. Note, on any good-sized network stream, this file gets huge very fast. Vern From David_Sames at NAI.com Wed Feb 18 13:33:15 2004 From: David_Sames at NAI.com (David_Sames at NAI.com) Date: Wed, 18 Feb 2004 16:33:15 -0500 Subject: Minimal Bro? Message-ID: <5856CEA9F0E6244CB2A8F77162041781669E8F@rocexmb1.corp.nai.org> I'm sure someone out there has done some performance profiling on Bro. In just my rudimentary monitoring, I find that the running executable requires anywhere from 5M to 7.2M of memory when running on my local Linux host, looking at just my own network traffic. Size depends on number of policies read. It looks to be fairly CPU-efficient in that it hardly even registers on "top" when monitoring my local net. I'm wondering if this memory footprint coincides with anyone else's observations, and is there a way to reduce that footprint to something smaller. I don't need Bro to have a huge policy, - however, what I want may require all of Bro functionality. I'm just looking to do to attack detection (and hopefully blocking) (like the latest 10 "worm" signatures that may be available). Regards, David L. Sames McAfee Research 15204 Omega Drive Rockville, MD 20850 301.947.7189 | Direct 301.527.0482 | Fax david_sames at nai.com | E-mail | Web From anton at netForensics.com Wed Feb 18 13:45:45 2004 From: anton at netForensics.com (Anton Chuvakin, Ph.D.) Date: Wed, 18 Feb 2004 16:45:45 -0500 (EST) Subject: Minimal Bro? In-Reply-To: <5856CEA9F0E6244CB2A8F77162041781669E8F@rocexmb1.corp.nai.org> References: <5856CEA9F0E6244CB2A8F77162041781669E8F@rocexmb1.corp.nai.org> Message-ID: >In just my rudimentary monitoring, I find that the running executable >requires anywhere from 5M to 7.2M of memory when running on my local Hmm, that sounds suspiciously good. Even with reduce-memory.bro and on not too loaded net, bro never takes less than 20MB in my case and often bloats to 70MB. As for the CPU, it almost never shows up on top of top. -- Anton Chuvakin, Ph.D., GCIA, GCIH - http://www.info-secure.org Author of "Security Warrior" from O'Reilly - http://www.securitywarrior.com Senior Security Analyst Product Management Group netForensics - http://www.netForensics.com From vern at icir.org Thu Feb 19 21:33:55 2004 From: vern at icir.org (Vern Paxson) Date: Thu, 19 Feb 2004 21:33:55 -0800 Subject: Minimal Bro? In-Reply-To: Your message of Wed, 18 Feb 2004 16:33:15 EST. Message-ID: <200402200533.i1K5XtmO022788@jaguar.icir.org> Having a low-footprint version of Bro hasn't been part of our operational needs, so it hasn't received attention. I imagine the current footprint could be diminished a fair amount by compiling out various functionality, and doing so shouldn't be too hard in most cases since quite a bit of it is modularized/layered pretty cleanly. (You can get a sense of where some of the memory is going by using statistics.bro and print-globals.bro) This won't be something for which we'll have cycles available to work on, though. Vern From rmkml at wanadoo.fr Sat Feb 21 13:49:54 2004 From: rmkml at wanadoo.fr (rmkml) Date: Sat, 21 Feb 2004 22:49:54 +0100 (CET) Subject: Strange question Message-ID: Hi, I little tested bro on my fbsd v49R box, I send with tcpreplay on another box (linux) trafic 50Mbits (/8 for MBytes) [Cross cable with two box] and if I send 'systat 1 -vmstat' on freebsd, I read on interrupt fxp1 case : ~6000-6500 interrupt ok is good because 6500 * 8 = ~50Mbits My strange question is, why if (I killed bro) and start tcpdump v381 (or 372 or snort or prelude) I read on interrupt fxp1 case : ~4200 interrupt this is not good because 4200 * 8 = 33.6Mbits I have many repeat this and same results. same results if change sysctl bpf buf. This is not a question to bro, but if anybody to explain me ? I compiled bro 0.8a70 current with libpcap 081. I compiled freebsd kernel with HZ=1000. (my kernel not have polling compiled/enabled, and my fxp1 have uniq irq) Regards Rmkml at Wanadoo.fr From rmkml at wanadoo.fr Sat Feb 21 14:50:25 2004 From: rmkml at wanadoo.fr (rmkml) Date: Sat, 21 Feb 2004 23:50:25 +0100 (CET) Subject: Segmentation fault (core dumped) Message-ID: Hi, I started bro v0.8a70 current : $ /usr/local/bin/bro08a70 -i xl1 -t /crusoe/dataL/bro.log bro.init mt Execution tracing ON. Segmentation fault (core dumped) Program terminated with signal 11, Segmentation fault. (gdb) where #0 0x0808a461 in TraceState::LogTrace () #1 0x080c0814 in BuiltinFunc::Call () #2 0x080b19da in CallExpr::Eval () #3 0x0809da13 in Expr::InitVal () #4 0x0816eead in init_val () #5 0x0816f4f9 in make_var () #6 0x0816f6de in add_global () #7 0x08056a5a in yyparse () #8 0x0804d4b6 in main () #9 0x0804bcee in _start () I started bro on fbsd v4.9R. Regards Rmkml at Wanadoo.fr From vern at icir.org Sun Feb 22 08:31:00 2004 From: vern at icir.org (Vern Paxson) Date: Sun, 22 Feb 2004 08:31:00 -0800 Subject: Strange question In-Reply-To: Your message of Sat, 21 Feb 2004 22:49:54 +0100. Message-ID: <200402221631.i1MGV0mO083591@jaguar.icir.org> > My strange question is, > why if (I killed bro) and start tcpdump v381 (or 372 or snort or prelude) > I read on interrupt fxp1 case : ~4200 interrupt > this is not good because 4200 * 8 = 33.6Mbits You shouldn't gauge it by number of interrupts since it's possible due to processor load that in some cases you get mulitple packets per interface and in others you don't. You could instead look at the statistics reported by libpcap (though these can be untrustworthy) or record the traffic using -w and then count just how many packets were captured. Vern From vern at icir.org Sun Feb 22 08:33:05 2004 From: vern at icir.org (Vern Paxson) Date: Sun, 22 Feb 2004 08:33:05 -0800 Subject: Segmentation fault (core dumped) In-Reply-To: Your message of Sat, 21 Feb 2004 23:50:25 +0100. Message-ID: <200402221633.i1MGX5mO083755@jaguar.icir.org> > $ /usr/local/bin/bro08a70 -i xl1 -t /crusoe/dataL/bro.log bro.init mt > Execution tracing ON. > Segmentation fault (core dumped) Egads, it appears that -t has been seriously broken, no doubt by the recent changes I introduced to how Bro tracks line numbers to identify where script values and functions occur. I'll put this on the near-term to-fix list. Sorry about that. Vern