From bro at tracker.bro-ids.org Mon Oct 1 08:33:33 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Mon, 01 Oct 2012 15:33:33 -0000 Subject: [Bro-Dev] #877: libCurl configure options In-Reply-To: <046.5730894ab0b50cbe92592bc3edadce6c@tracker.bro-ids.org> References: <046.5730894ab0b50cbe92592bc3edadce6c@tracker.bro-ids.org> Message-ID: <061.cc0b7339499c7c05aa8382e082c7e44a@tracker.bro-ids.org> #877: libCurl configure options ----------------------------+------------------------ Reporter: seth | Owner: Type: Merge Request | Status: testing Priority: Normal | Milestone: Bro2.2 Component: Bro | Version: git/master Resolution: | Keywords: ----------------------------+------------------------ Changes (by matthias): * status: new => testing * type: Problem => Merge Request Comment: Since the timeout feature was defunct before, I think we should move forward and merge this patch because it prevents Bro from building on certain platforms. -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Mon Oct 1 08:58:00 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Mon, 01 Oct 2012 15:58:00 -0000 Subject: [Bro-Dev] #889: topic/jsiwek/fix-quoting Message-ID: <048.ac7ed8454063423b64e7ec3cb4223ea1@tracker.bro-ids.org> #889: topic/jsiwek/fix-quoting ---------------------------+------------------------ Reporter: jsiwek | Owner: Type: Merge Request | Status: new Priority: Normal | Milestone: Bro2.2 Component: Bro | Version: git/master Keywords: | ---------------------------+------------------------ This branch in the `cmake` repo fixes a string quoting issue that results in an error when building the `install` target in an environment without the `USER` variable set. All `cmake` submodules in the repo tree should be updated when merging. -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Mon Oct 1 13:16:02 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Mon, 01 Oct 2012 20:16:02 -0000 Subject: [Bro-Dev] #884: Scripting inconsistency in the input framework In-Reply-To: <054.9e9785f366de54bb61f65bfa53416a41@tracker.bro-ids.org> References: <054.9e9785f366de54bb61f65bfa53416a41@tracker.bro-ids.org> Message-ID: <069.3520909c597a67f04b82017a996956f0@tracker.bro-ids.org> #884: Scripting inconsistency in the input framework ---------------------------+------------------------ Reporter: sheharbano.k | Owner: amannb Type: Problem | Status: accepted Priority: Normal | Milestone: Bro2.2 Component: Bro | Version: git/master Resolution: | Keywords: ---------------------------+------------------------ Changes (by amannb): * owner: => amannb * status: new => accepted -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Mon Oct 1 13:21:15 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Mon, 01 Oct 2012 20:21:15 -0000 Subject: [Bro-Dev] #890: known-services hasty service detection Message-ID: <048.b62cc809005a37f75d745f8752fc5238@tracker.bro-ids.org> #890: known-services hasty service detection ---------------------+------------------------ Reporter: jsiwek | Owner: Type: Problem | Status: new Priority: Normal | Milestone: Bro2.2 Component: Bro | Version: git/master Keywords: | ---------------------+------------------------ known-services.bro mostly treats a `protocol_confirmation` like a "service" confirmation, but there's not a strict rule preventing an analyzer from doing a `protocol_confirmation` when only the originator side has been seen so far. Such a rule/convention could be made to fix the problem, or maybe the script could change to be more flexible in how a protocol confirmation gets promoted to a service confirmation like if the user were able to define that in a function per-protocol. Bill Jones was specifically having trouble with Teredo. From the mailing list "I don't think TEREDO is working correctly. It is filling up the known_services.log with entries for local host ports that I know are closed just because there was a TEREDO packet sent to that port." -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Mon Oct 1 13:42:32 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Mon, 01 Oct 2012 20:42:32 -0000 Subject: [Bro-Dev] #884: Scripting inconsistency in the input framework In-Reply-To: <054.9e9785f366de54bb61f65bfa53416a41@tracker.bro-ids.org> References: <054.9e9785f366de54bb61f65bfa53416a41@tracker.bro-ids.org> Message-ID: <069.534afd348d30480594e78c234c60da12@tracker.bro-ids.org> #884: Scripting inconsistency in the input framework ---------------------------+------------------------ Reporter: sheharbano.k | Owner: amannb Type: Problem | Status: accepted Priority: Normal | Milestone: Bro2.2 Component: Bro | Version: git/master Resolution: | Keywords: ---------------------------+------------------------ Comment (by amannb): Sorry, I was not entirely able to reproduce this problem. If a table is read using the input framework and you use your type (IdxIp) as the index type, the resulting table is of type {{{ global tb_ip: table[addr] of count; }}} and not of type {{{ global tb_ip: table[IdxIp] of count; }}} The index record is just there to give the input framework the information about the names of the fields that are present in the input file -- the bro table that is constructed will use an IndexType that contains everything that was contained in the record in the same order. Thus if you have a record like {{{ type testrecord: record { a: addr; b: count; c: string; } }}} the resulting table will be of type {{{ table test [addr, count, string] of count; }}} and not of type {{{ table test[testrecord] of count; }}} However you should not even have been able to load the data into a table with the wrong type -- the input framework should refuse loading data in a table with nonmatching index types. And this seemed to work in my tests. So could you perhaps send me the exact scripts that you are using (or attach them here)? Perhaps something else is going on that I am not thinking of at the moment. -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Mon Oct 1 13:43:01 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Mon, 01 Oct 2012 20:43:01 -0000 Subject: [Bro-Dev] #890: known-services hasty service detection In-Reply-To: <048.b62cc809005a37f75d745f8752fc5238@tracker.bro-ids.org> References: <048.b62cc809005a37f75d745f8752fc5238@tracker.bro-ids.org> Message-ID: <063.9dc61036d45785a3deec8462c6a2951d@tracker.bro-ids.org> #890: known-services hasty service detection ----------------------+------------------------ Reporter: jsiwek | Owner: Type: Problem | Status: new Priority: Normal | Milestone: Bro2.2 Component: Bro | Version: git/master Resolution: | Keywords: ----------------------+------------------------ Comment (by seth): > Bill Jones was specifically having trouble with Teredo. From the mailing > list "I don't think TEREDO is working correctly. It is filling up the > known_services.log with entries for local host ports that I know are > closed just because there was a TEREDO packet sent to that port." In this case, I think that the teredo analyzer is confirming the protocol too early. Eventually we are going to have to address the subtleties in protocol confirmation and protocol violation though, they don't feel expressive enough yet. -- Ticket URL: Bro Tracker Bro Issue Tracker From sheharbano.k at gmail.com Mon Oct 1 14:05:47 2012 From: sheharbano.k at gmail.com (Sheharbano Khattak) Date: Tue, 2 Oct 2012 02:05:47 +0500 Subject: [Bro-Dev] #884: Scripting inconsistency in the input framework In-Reply-To: <069.534afd348d30480594e78c234c60da12@tracker.bro-ids.org> References: <054.9e9785f366de54bb61f65bfa53416a41@tracker.bro-ids.org> <069.534afd348d30480594e78c234c60da12@tracker.bro-ids.org> Message-ID: I have attached the files: config.txt: the file to be read config.bro: the file that reads in info from config.txt (plz change to reflect the path where config.txt is located on your machine) test.bro: the file that makes use of the config info in Config::table_config. (I didn't test this one). Try applying different functions to_* in bro.bif to the values in Config::table_config and see what happens. For me, it generated error. Thanks. On Tue, Oct 2, 2012 at 1:42 AM, Bro Tracker wrote: > #884: Scripting inconsistency in the input framework > ---------------------------+------------------------ > Reporter: sheharbano.k | Owner: amannb > Type: Problem | Status: accepted > Priority: Normal | Milestone: Bro2.2 > Component: Bro | Version: git/master > Resolution: | Keywords: > ---------------------------+------------------------ > > Comment (by amannb): > > Sorry, I was not entirely able to reproduce this problem. > > If a table is read using the input framework and you use your type (IdxIp) > as the index type, the resulting table is of type > > {{{ > global tb_ip: table[addr] of count; > }}} > > and not of type > > {{{ > global tb_ip: table[IdxIp] of count; > }}} > > The index record is just there to give the input framework the information > about the names of the fields that are present in the input file -- the > bro table that is constructed will use an IndexType that contains > everything that was contained in the record in the same order. > > Thus if you have a record like > > {{{ > type testrecord: record { > a: addr; > b: count; > c: string; > } > }}} > > the resulting table will be of type > > {{{ > table test [addr, count, string] of count; > }}} > > and not of type > > {{{ > table test[testrecord] of count; > }}} > > However you should not even have been able to load the data into a table > with the wrong type -- the input framework should refuse loading data in a > table with nonmatching index types. And this seemed to work in my tests. > > So could you perhaps send me the exact scripts that you are using (or > attach them here)? Perhaps something else is going on that I am not > thinking of at the moment. > > -- > Ticket URL: > Bro Tracker > Bro Issue Tracker > -- Sheharbano Khattak http://etheryell.com -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.icsi.berkeley.edu/pipermail/bro-dev/attachments/20121002/e7f6bf34/attachment-0001.html -------------- next part -------------- #separator \x09 #set_separator , #empty_field (empty) #unset_field - #path reporter #fields parameter value #types string string local_net 1.1.1.1/32 th_addr_scan 35 weight_addr_scan 0.8 th_addr_scan_critical 20 weight_addr_scan_critical 1.0 th_port_scan 15 weight_port_scan 0.25 th_low_port_troll 10 weight_low_port_troll 0.5 wnd_addr_scan 5mins wnd_port_scan 5mins scan_evaluation_mode OR th_ssh_login 10 weight_breakin 1.0 wnd_breakin 10mins wnd_exploit 10mins weight_exploit_blacklist_match 0.5 exploit_evaluation_mode OR evaluation_breakin_mode OR th_disguised_exe 1 weight_disguised_exe 0.8 weight_egg_signature_match 1.0 wnd_egg 10mins evaluation_mode OR th_dns_failure 25 weight_dns_failure 0.8 evaluation_mode OR wnd_cnc 5mins weight_cnc_blacklist_match 1.0 weight_cnc_blacklist_dns_match 0.5 weight_cnc_signature_match 0.8 weight_rbn_blacklist_match 0.5 th_sqli_attempt 10 wnd_sqli 5mins weight_sqli 0.5 sqli_evaluation_mode OR th_mx_queries 5 th_smtp 25 weight_spam_failed_mx 1.0 weight_spam_failed_smtp 0.8 wnd_spam 5mins spam_evaluation_mode OR wnd_correlation 12hrs wnd_bot 1day -------------- next part -------------- A non-text attachment was scrubbed... Name: config.bro Type: application/octet-stream Size: 1506 bytes Desc: not available Url : http://mailman.icsi.berkeley.edu/pipermail/bro-dev/attachments/20121002/e7f6bf34/attachment-0002.obj -------------- next part -------------- A non-text attachment was scrubbed... Name: test.bro Type: application/octet-stream Size: 612 bytes Desc: not available Url : http://mailman.icsi.berkeley.edu/pipermail/bro-dev/attachments/20121002/e7f6bf34/attachment-0003.obj From bro at tracker.bro-ids.org Mon Oct 1 14:05:49 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Mon, 01 Oct 2012 21:05:49 -0000 Subject: [Bro-Dev] #884: Scripting inconsistency in the input framework In-Reply-To: <054.9e9785f366de54bb61f65bfa53416a41@tracker.bro-ids.org> References: <054.9e9785f366de54bb61f65bfa53416a41@tracker.bro-ids.org> Message-ID: <069.7cb8c87f18f1621956a05dbb1969b601@tracker.bro-ids.org> #884: Scripting inconsistency in the input framework ---------------------------+------------------------ Reporter: sheharbano.k | Owner: amannb Type: Problem | Status: accepted Priority: Normal | Milestone: Bro2.2 Component: Bro | Version: git/master Resolution: | Keywords: ---------------------------+------------------------ Comment (by sheharbano.k): None Filename config.txt could not be saved, problem: [Errno 13] Permission denied: '/da/trac/projects/bro/attachments/ticket/884' Filename config.bro could not be saved, problem: [Errno 13] Permission denied: '/da/trac/projects/bro/attachments/ticket/884' Filename test.bro could not be saved, problem: [Errno 13] Permission denied: '/da/trac/projects/bro/attachments/ticket/884'\I have attached the files: config.txt: the file to be read config.bro: the file that reads in info from config.txt (plz change to reflect the path where config.txt is located on your machine) test.bro: the file that makes use of the config info in Config::table_config. (I didn't test this one). Try applying different functions to_* in bro.bif to the values in Config::table_config and see what happens. For me, it generated error. Thanks. On Tue, Oct 2, 2012 at 1:42 AM, Bro Tracker wrote: > #884: Scripting inconsistency in the input framework > ---------------------------+------------------------ > Reporter: sheharbano.k | Owner: amannb > Type: Problem | Status: accepted > Priority: Normal | Milestone: Bro2.2 > Component: Bro | Version: git/master > Resolution: | Keywords: > ---------------------------+------------------------ > > Comment (by amannb): > > Sorry, I was not entirely able to reproduce this problem. > > If a table is read using the input framework and you use your type (IdxIp) > as the index type, the resulting table is of type > > {{{ > global tb_ip: table[addr] of count; > }}} > > and not of type > > {{{ > global tb_ip: table[IdxIp] of count; > }}} > > The index record is just there to give the input framework the information > about the names of the fields that are present in the input file -- the > bro table that is constructed will use an IndexType that contains > everything that was contained in the record in the same order. > > Thus if you have a record like > > {{{ > type testrecord: record { > a: addr; > b: count; > c: string; > } > }}} > > the resulting table will be of type > > {{{ > table test [addr, count, string] of count; > }}} > > and not of type > > {{{ > table test[testrecord] of count; > }}} > > However you should not even have been able to load the data into a table > with the wrong type -- the input framework should refuse loading data in a > table with nonmatching index types. And this seemed to work in my tests. > > So could you perhaps send me the exact scripts that you are using (or > attach them here)? Perhaps something else is going on that I am not > thinking of at the moment. > > -- > Ticket URL: > Bro Tracker > Bro Issue Tracker > [attachment:"config.txt"] [attachment:"config.bro"] [attachment:"test.bro"] -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Mon Oct 1 14:47:44 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Mon, 01 Oct 2012 21:47:44 -0000 Subject: [Bro-Dev] #884: Scripting inconsistency in the input framework In-Reply-To: <054.9e9785f366de54bb61f65bfa53416a41@tracker.bro-ids.org> References: <054.9e9785f366de54bb61f65bfa53416a41@tracker.bro-ids.org> Message-ID: <069.60e1a2f6f4d3ab1afbd8c44aca3d9809@tracker.bro-ids.org> #884: Scripting inconsistency in the input framework ---------------------------+------------------------ Reporter: sheharbano.k | Owner: amannb Type: Problem | Status: closed Priority: Normal | Milestone: Bro2.2 Component: Bro | Version: git/master Resolution: Invalid | Keywords: ---------------------------+------------------------ Changes (by amannb): * status: accepted => closed * resolution: => Invalid Comment: After some more discussion - still not able to reproduce, could have been a problem with an early version. Closing ticket for now. -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Tue Oct 2 12:08:20 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Tue, 02 Oct 2012 19:08:20 -0000 Subject: [Bro-Dev] #877: libCurl configure options In-Reply-To: <046.5730894ab0b50cbe92592bc3edadce6c@tracker.bro-ids.org> References: <046.5730894ab0b50cbe92592bc3edadce6c@tracker.bro-ids.org> Message-ID: <061.8cb715c8d1a11946d22ca5359bb669a5@tracker.bro-ids.org> #877: libCurl configure options ----------------------------+------------------------ Reporter: seth | Owner: Type: Merge Request | Status: testing Priority: Normal | Milestone: Bro2.2 Component: Bro | Version: git/master Resolution: | Keywords: ----------------------------+------------------------ Comment (by robin): Merging, but it seems not all of this ticket has been addresses (i.e., the option to disable/relocate lib curl)? -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Tue Oct 2 12:08:58 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Tue, 02 Oct 2012 19:08:58 -0000 Subject: [Bro-Dev] #877: libCurl configure options In-Reply-To: <046.5730894ab0b50cbe92592bc3edadce6c@tracker.bro-ids.org> References: <046.5730894ab0b50cbe92592bc3edadce6c@tracker.bro-ids.org> Message-ID: <061.691b79cd17f2f4bc3e7b50ca240ed89b@tracker.bro-ids.org> #877: libCurl configure options ----------------------+------------------------ Reporter: seth | Owner: Type: Problem | Status: testing Priority: Normal | Milestone: Bro2.2 Component: Bro | Version: git/master Resolution: | Keywords: ----------------------+------------------------ Changes (by robin): * type: Merge Request => Problem -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Tue Oct 2 13:02:47 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Tue, 02 Oct 2012 20:02:47 -0000 Subject: [Bro-Dev] #877: libCurl configure options In-Reply-To: <046.5730894ab0b50cbe92592bc3edadce6c@tracker.bro-ids.org> References: <046.5730894ab0b50cbe92592bc3edadce6c@tracker.bro-ids.org> Message-ID: <061.5d4b9da72ba806c856fcd904a653aed2@tracker.bro-ids.org> #877: libCurl configure options ----------------------+------------------------ Reporter: seth | Owner: Type: Problem | Status: testing Priority: Normal | Milestone: Bro2.2 Component: Bro | Version: git/master Resolution: | Keywords: ----------------------+------------------------ Comment (by matthias): At the moment, disabling ElasticSearch automatically disables the `libcurl` dependency, because the Manager does not include the affected headers. That looks like an acceptable interim solution to me. In general, the ElasticSearch writer and `libcurl` are tightly coupled and would benefit from separation of concerns (i.e., create a separate interface for HTTP communication), but this would fit better in a separate ticket. -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Tue Oct 2 13:12:12 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Tue, 02 Oct 2012 20:12:12 -0000 Subject: [Bro-Dev] #889: topic/jsiwek/fix-quoting In-Reply-To: <048.ac7ed8454063423b64e7ec3cb4223ea1@tracker.bro-ids.org> References: <048.ac7ed8454063423b64e7ec3cb4223ea1@tracker.bro-ids.org> Message-ID: <063.bf8719d1a2652436a9a61925d1657681@tracker.bro-ids.org> #889: topic/jsiwek/fix-quoting ----------------------------+------------------------ Reporter: jsiwek | Owner: robin Type: Merge Request | Status: closed Priority: Normal | Milestone: Bro2.2 Component: Bro | Version: git/master Resolution: fixed | Keywords: ----------------------------+------------------------ Changes (by robin): * owner: => robin * status: new => closed * resolution: => fixed Comment: In [b4b7a384dcb038060f3e33fc5bbd36708e8ff1f5/bro]: {{{ #!CommitTicketReference repository="bro" revision="b4b7a384dcb038060f3e33fc5bbd36708e8ff1f5" Updating submodule(s). [nomail] Closes #889 }}} -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Tue Oct 2 13:19:56 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Tue, 02 Oct 2012 20:19:56 -0000 Subject: [Bro-Dev] #890: known-services hasty service detection In-Reply-To: <048.b62cc809005a37f75d745f8752fc5238@tracker.bro-ids.org> References: <048.b62cc809005a37f75d745f8752fc5238@tracker.bro-ids.org> Message-ID: <063.46044bc8be7fe6979ab62eea26cff578@tracker.bro-ids.org> #890: known-services hasty service detection ----------------------+------------------------ Reporter: jsiwek | Owner: Type: Problem | Status: new Priority: Normal | Milestone: Bro2.2 Component: Bro | Version: git/master Resolution: | Keywords: ----------------------+------------------------ Comment (by jsiwek): In [5f3af9e9ebd474f41d2c20d64cd6ac0a37f75782/bro]: {{{ #!CommitTicketReference repository="bro" revision="5f3af9e9ebd474f41d2c20d64cd6ac0a37f75782" Add new Tunnel::delay_teredo_confirmation option, default to true. This option indicates that the Teredo analyzer should wait until it sees both sides of a connection using a valid Teredo encapsulation before issuing a protocol_confirmation. Previous behavior confirmed on the first instance of a valid encapsulation, which could result in more false positives (and e.g. bogus entries in known-services.log). Addresses #890. }}} -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Tue Oct 2 13:26:52 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Tue, 02 Oct 2012 20:26:52 -0000 Subject: [Bro-Dev] #890: known-services hasty service detection In-Reply-To: <048.b62cc809005a37f75d745f8752fc5238@tracker.bro-ids.org> References: <048.b62cc809005a37f75d745f8752fc5238@tracker.bro-ids.org> Message-ID: <063.ef94e9718ca139d07875ab6e6d6fa670@tracker.bro-ids.org> #890: known-services hasty service detection ----------------------------+------------------------ Reporter: jsiwek | Owner: Type: Merge Request | Status: new Priority: Normal | Milestone: Bro2.2 Component: Bro | Version: git/master Resolution: | Keywords: ----------------------------+------------------------ Changes (by jsiwek): * type: Problem => Merge Request Comment: > In this case, I think that the teredo analyzer is confirming the protocol too early. It was confirming in a way consistent with the DNS analyzer (the other UDP protocol logged in known-services), but I think a protocol_confirmation as a result of a single valid Teredo encapsulation is a lot weaker than one from a parseable DNS message so it probably does make sense to change the Teredo analyzer specifically. Fix is in `topic/jsiwek/delay-teredo-confirm` -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Tue Oct 2 18:53:14 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Wed, 03 Oct 2012 01:53:14 -0000 Subject: [Bro-Dev] #861: Merging DNP3 Analyzer In-Reply-To: <045.d25b5df6a13b96f4b3e3618bd9398e58@tracker.bro-ids.org> References: <045.d25b5df6a13b96f4b3e3618bd9398e58@tracker.bro-ids.org> Message-ID: <060.ccd7df20c14c8a2e4aff64ec31d20b8f@tracker.bro-ids.org> #861: Merging DNP3 Analyzer ---------------------+------------------------ Reporter: hui | Owner: robin Type: Task | Status: assigned Priority: Normal | Milestone: Bro2.2 Component: Bro | Version: git/master Resolution: | Keywords: dnp3 ---------------------+------------------------ Comment (by hui): Hi, Robin, I need to resolve a confusion related to global variable you mentioned in the email. Other comments are being resolved. The DNP 3 trace that I am using usually follow this pattern for short packets: TCP handshake DNP3 request DNP3 response TCP Close Session (For this situation, I don't need the global variable in my codes, a local variable will do) Howerver, for longer packets TCP handshake DNP3 packet #1 DNP3 packet #2 .... DNP3 response TCP Well this looks similar as a single HTTP fragment spanned into several TCP payloads. I am sure that TCP reassembly in Bro will do that. Though I am not sure how TCP reassembly works, my intuition is the length of payload, or other fields to indicate that this is not the last application payload. But things are getting ugly for DNP3 packets. In details, it looks like this: TCP : DNP3 Serial Link Layer : DNP3 Serial Transport Layer : DNP3 Serial Application Layer #1 (292 bytes) TCP : DNP3 Serial Link Layer : DNP3 Serial Transport Layer : DNP3 Serial Application Layer #2 (292 bytes) TCP : DNP3 Serial Link Layer : DNP3 Serial Transport Layer : DNP3 Serial Application Layer #3 ... So what are concatenated are the DNP3 Serial Application Layer #1 + DNP3 Serial Application Layer #2 + DNP3 Serial Application Layer #3 + ....Instead of concatenating application layer payload such as HTTP. In this case, I use a global variable to across different flow to do this job. All these flows are within a single TCP session, so I guess that is why it is working. Fortunately, this long DNP3 packet is very rare, but I do have test pcap. So my understanding is that a analyzer class instance represent a single TCP session including all flows ? Actually I am writing to confirm this understanding. Hope this helps, let me know if there is any question related to this issue. Hui Replying to [comment:3 robin]: > I've merged this with master into the temporary branch {{topic/topic/robin/dnp3-merge}}. > > Hui, a number of points/questions: > > - I've added a set of {{{TODO-Hui}}} throughout the new code. Please take a look and address (just grep for it). > > - I moved the global variables in {{{DNP3.cc}{} into the analyzer class. I'm actually surprised that this has ever worked: it looks like you kept state across flows with a single variable; am I missing something? > > - Please take a look at the new test {{{scripts.base.protocols.dnp3.events}}} and check the output if it matches with what you would expect. > > - Which of the many events did you have data for to test with? I've added the DNP3 traces from Dina, they trigger 11 of the 51 events. Do we have more we can add to the test suite? > > - Please document the events in {{{src/events.bif}}}, similar to how other events are documented. > > - Does DNP3 have cases similar to Modbus where it would make sense to pass arrays of integers (or other elements)? If so, that would be good to do (but I don't know the protocol enough to say more). -- Ticket URL: Bro Tracker Bro Issue Tracker From noreply at bro-ids.org Wed Oct 3 00:00:03 2012 From: noreply at bro-ids.org (Merge Tracker) Date: Wed, 3 Oct 2012 00:00:03 -0700 Subject: [Bro-Dev] [Auto] Merge Status Message-ID: <201210030700.q93703gW011542@bro-ids.icir.org> > Open Merge Requests for Bro2.2 > ============================== Component | Id | Reporter | Owner | Prio | Summary ------------------------------------------------------------------------------------------------------------------ Bro | 890 [1] | jsiwek | | Normal | known-services hasty service detection > Unmerged Fastpath Commits > ========================= Component | Revision | Committer | Date | Summary ------------------------------------------------------------------------------------------------------------------ bro | e93748d | Jon Siwek | 2012-10-02 | Add general FAQ entry about upgrading Bro. [2] [1] #890: http://tracker.bro-ids.org/bro/ticket/890 [2] fastpath: http://tracker.bro-ids.org/bro/changeset/e93748d28b5d1915bda94dc951c42406e0eb2f9e/bro From bro at tracker.bro-ids.org Wed Oct 3 11:44:52 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Wed, 03 Oct 2012 18:44:52 -0000 Subject: [Bro-Dev] #861: Merging DNP3 Analyzer In-Reply-To: <045.d25b5df6a13b96f4b3e3618bd9398e58@tracker.bro-ids.org> References: <045.d25b5df6a13b96f4b3e3618bd9398e58@tracker.bro-ids.org> Message-ID: <060.db377208393206d4700497e04a1d947f@tracker.bro-ids.org> #861: Merging DNP3 Analyzer ---------------------+------------------------ Reporter: hui | Owner: robin Type: Task | Status: assigned Priority: Normal | Milestone: Bro2.2 Component: Bro | Version: git/master Resolution: | Keywords: dnp3 ---------------------+------------------------ Comment (by robin): Replying to [comment:6 hui]: > So my understanding is that a analyzer class instance represent a single TCP session including all flows ? Actually I am writing to confirm this understanding. Yes, the analyzer gets (all) the data from a single TCP session, so if you need to remember things for the (pseudo-)link layer reassembly, you can (and should) do that as part of the analyzer class. One additional note though: I'm now wondering if you need to buffer the data at all. Can you just pass it into the BinPAC analyzer as it comes in? You don't need to have the full PDU assembled before starting to send data in (just like for HTTP, you don't need to have the full HTTP session). -- Ticket URL: Bro Tracker Bro Issue Tracker From robin at icir.org Wed Oct 3 12:19:50 2012 From: robin at icir.org (Robin Sommer) Date: Wed, 3 Oct 2012 12:19:50 -0700 Subject: [Bro-Dev] [Bro-Commits] [git/bro] fastpath: Add general FAQ entry about upgrading Bro. (e93748d) In-Reply-To: <201210022036.q92KaVek001446@bro-ids.icir.org> References: <201210022036.q92KaVek001446@bro-ids.icir.org> Message-ID: <20121003191950.GA25785@icir.org> On Tue, Oct 02, 2012 at 13:36 -0700, Jonathan Siwek wrote: > +What files will get overwritten when upgrading/installing a new Bro version? > +---------------------------------------------------------------------------- > + > +Expect everything except things in ``$prefix/share/bro`` and > +``$prefix/etc`` to be overwritten, but backing up the entire ``$prefix`` > +before upgrading is good practice (``$prefix`` indicating the root of > +where Bro was installed). > + This isn't really right, is it? The scripts under $prefix/share/bro are going to be overwritten except for site/. On the other hand, $prefix/share/logs shouldn't be touched. More generally, because we install into a general $prefix, there's lots of non-Bro stuff there that won't be touched, so maybe it needs a framing like "Of the files that Bro installs, it may override ...". Or maybe turn it around: "The following files/paths won't be touched: ..." Robin -- Robin Sommer * Phone +1 (510) 722-6541 * robin at icir.org ICSI/LBNL * Fax +1 (510) 666-2956 * www.icir.org From bro at tracker.bro-ids.org Wed Oct 3 12:27:03 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Wed, 03 Oct 2012 19:27:03 -0000 Subject: [Bro-Dev] #890: known-services hasty service detection In-Reply-To: <048.b62cc809005a37f75d745f8752fc5238@tracker.bro-ids.org> References: <048.b62cc809005a37f75d745f8752fc5238@tracker.bro-ids.org> Message-ID: <063.2df84253671041d3cf6998c8cf577273@tracker.bro-ids.org> #890: known-services hasty service detection ----------------------------+------------------------ Reporter: jsiwek | Owner: Type: Merge Request | Status: new Priority: Normal | Milestone: Bro2.2 Component: Bro | Version: git/master Resolution: | Keywords: ----------------------------+------------------------ Comment (by robin): Question about the code: {{{Teredo_Analyzer::DeliverPacket()}}} clear the {{{valid_orig/valid_resp}}} flags with every packet. Is that indeed the intent? What I'd expect is that once one ir's set, it stays so for the remainder of the connection? -- Ticket URL: Bro Tracker Bro Issue Tracker From jsiwek at illinois.edu Wed Oct 3 12:48:49 2012 From: jsiwek at illinois.edu (Siwek, Jonathan Luke) Date: Wed, 3 Oct 2012 19:48:49 +0000 Subject: [Bro-Dev] [Bro-Commits] [git/bro] fastpath: Add general FAQ entry about upgrading Bro. (e93748d) In-Reply-To: <20121003191950.GA25785@icir.org> References: <201210022036.q92KaVek001446@bro-ids.icir.org> <20121003191950.GA25785@icir.org> Message-ID: > >> +What files will get overwritten when upgrading/installing a new Bro version? >> +---------------------------------------------------------------------------- >> + >> +Expect everything except things in ``$prefix/share/bro`` and >> +``$prefix/etc`` to be overwritten, but backing up the entire ``$prefix`` >> +before upgrading is good practice (``$prefix`` indicating the root of >> +where Bro was installed). >> + > > This isn't really right, is it? The scripts under $prefix/share/bro > are going to be overwritten except for site/. Oops, I actually did mean to type "$prefix/share/bro/site" and not "$prefix/share/bro", thanks. > On the other hand, > $prefix/share/logs shouldn't be touched. More generally, because we > install into a general $prefix, there's lots of non-Bro stuff there > that won't be touched, so maybe it needs a framing like "Of the files > that Bro installs, it may override ?". The question I was trying to answer was "Of the files that Bro installs, which *won't* it overwrite?". I'll redo the FAQ for that and see what you think. > Or maybe turn it around: "The following files/paths won't be touched: ..." Not sure if people care to know about what happens to most run-time stuff, but I can see mentioning that at least logs don't get touched. Jon From bro at tracker.bro-ids.org Wed Oct 3 13:34:26 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Wed, 03 Oct 2012 20:34:26 -0000 Subject: [Bro-Dev] #890: known-services hasty service detection In-Reply-To: <048.b62cc809005a37f75d745f8752fc5238@tracker.bro-ids.org> References: <048.b62cc809005a37f75d745f8752fc5238@tracker.bro-ids.org> Message-ID: <063.569f7377894ceafeb1173ab129ab697d@tracker.bro-ids.org> #890: known-services hasty service detection ----------------------------+------------------------ Reporter: jsiwek | Owner: Type: Merge Request | Status: new Priority: Normal | Milestone: Bro2.2 Component: Bro | Version: git/master Resolution: | Keywords: ----------------------------+------------------------ Comment (by jsiwek): Replying to [comment:3 robin]: > Question about the code: {{{Teredo_Analyzer::DeliverPacket()}}} clear the {{{valid_orig/valid_resp}}} flags with every packet. Is that indeed the intent? What I'd expect is that once one ir's set, it stays so for the remainder of the connection? Kind of what I was going for was that both the latest packet from orig and latest packet from resp are valid teredo encapsulations, which I thought was a stronger indication that it's actually teredo than just somewhere in the connection one packet from orig and one packet from resp happened to have a valid teredo encapsulation. Do you think that's a good idea or no? -- Ticket URL: Bro Tracker Bro Issue Tracker From dnthayer at illinois.edu Wed Oct 3 14:29:46 2012 From: dnthayer at illinois.edu (Daniel Thayer) Date: Wed, 3 Oct 2012 16:29:46 -0500 Subject: [Bro-Dev] [Bro-Commits] [git/bro] fastpath: Redo the "how to upgrade" FAQ. (06d6277) In-Reply-To: <201210032115.q93LFOIY026129@bro-ids.icir.org> References: <201210032115.q93LFOIY026129@bro-ids.icir.org> Message-ID: <506CAE4A.5020306@illinois.edu> It seems to me that this info should be in the "Installing Bro" document, since upgrading from a previous version will be a very common scenario. On 10/03/2012 04:15 PM, Jonathan Siwek wrote: > Repository : ssh://git at bro-ids.icir.org/bro > > On branch : fastpath > Link : http://tracker.bro-ids.org/bro/changeset/06d6277f0aa97836b9c25f7aa97fdf8549fd7da9/bro > >> --------------------------------------------------------------- > > commit 06d6277f0aa97836b9c25f7aa97fdf8549fd7da9 > Author: Jon Siwek > Date: Wed Oct 3 16:14:52 2012 -0500 > > Redo the "how to upgrade" FAQ. > > >> --------------------------------------------------------------- > > 06d6277f0aa97836b9c25f7aa97fdf8549fd7da9 > doc/faq.rst | 41 +++++++++++++++++++++++++++++++++++------ > 1 files changed, 35 insertions(+), 6 deletions(-) > > diff --git a/doc/faq.rst b/doc/faq.rst > index 1836e5a..2342af1 100644 > --- a/doc/faq.rst > +++ b/doc/faq.rst > @@ -12,13 +12,42 @@ Frequently Asked Questions > Installation and Configuration > ============================== > > -What files will get overwritten when upgrading/installing a new Bro version? > ----------------------------------------------------------------------------- > - > -Expect everything except things in ``$prefix/share/bro`` and > -``$prefix/etc`` to be overwritten, but backing up the entire ``$prefix`` > -before upgrading is good practice (``$prefix`` indicating the root of > -where Bro was installed). > +How do I upgrade to a new version of Bro? > +----------------------------------------- > + > +There's two suggested approaches, either install Bro using the same > +installation prefix directory as before, or pick a new prefix and copy > +local customizations over. > + > +Re-Use Previous Install Prefix > +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ > + > +If you choose to configure and install Bro with the same prefix > +directory as before, local customization and configuration to files in > +``$prefix/share/bro/site`` and ``$prefix/etc`` won't be overwritten > +(``$prefix`` indicating the root of where Bro was installed), but making > +a backup of local changes before proceeding is recommended. Also, logs > +generated at run-time won't be touched by the upgrade. > + > +After upgrading, remember to check ``$prefix/share/bro/site`` and > +``$prefix/etc`` for ``.example`` files, which indicate the > +distribution's version of the file differs from the local one, which may > +include local changes. Review the differences, and make adjustments > +as necessary (for differences that aren't the result of a local change, > +use the new version's). > + > +Pick a New Install prefix > +^^^^^^^^^^^^^^^^^^^^^^^^^ > + > +If you want the install the newer version in a different prefix > +directory than before, you can just copy local customization and > +configuration files from ``$prefix/share/bro/site`` and ``$prefix/etc`` > +to the new location (``$prefix`` indicating the root of where Bro was > +originally installed). Make sure to review the files for difference > +before copying and make adjustments as necessary (for differences that > +aren't the result of a local change, use the new version's). Of > +particular note, the copied version of ``$prefix/etc/broctl.cfg`` is > +likely to need changes to the ``SpoolDir`` and ``LogDir`` settings. > > How can I tune my operating system for best capture performance? > ---------------------------------------------------------------- > From robin at icir.org Wed Oct 3 15:03:24 2012 From: robin at icir.org (Robin Sommer) Date: Wed, 3 Oct 2012 15:03:24 -0700 Subject: [Bro-Dev] [Bro-Commits] [git/bro] fastpath: Redo the "how to upgrade" FAQ. (06d6277) In-Reply-To: <506CAE4A.5020306@illinois.edu> References: <201210032115.q93LFOIY026129@bro-ids.icir.org> <506CAE4A.5020306@illinois.edu> Message-ID: <20121003220324.GC24995@icir.org> On Wed, Oct 03, 2012 at 16:29 -0500, you wrote: > It seems to me that this info should be in the "Installing Bro" > document, since upgrading from a previous version will be a > very common scenario. Good point, maybe we need an "updating" section there. Robin -- Robin Sommer * Phone +1 (510) 722-6541 * robin at icir.org ICSI/LBNL * Fax +1 (510) 666-2956 * www.icir.org From dnthayer at illinois.edu Wed Oct 3 15:06:55 2012 From: dnthayer at illinois.edu (Daniel Thayer) Date: Wed, 3 Oct 2012 17:06:55 -0500 Subject: [Bro-Dev] [Bro-Commits] [git/bro] fastpath: Redo the "how to upgrade" FAQ. (06d6277) In-Reply-To: <20121003220324.GC24995@icir.org> References: <201210032115.q93LFOIY026129@bro-ids.icir.org> <506CAE4A.5020306@illinois.edu> <20121003220324.GC24995@icir.org> Message-ID: <506CB6FF.1010007@illinois.edu> On 10/03/2012 05:03 PM, Robin Sommer wrote: > > On Wed, Oct 03, 2012 at 16:29 -0500, you wrote: > >> It seems to me that this info should be in the "Installing Bro" >> document, since upgrading from a previous version will be a >> very common scenario. > > Good point, maybe we need an "updating" section there. > > Robin > If you want, you could just merge it into master, then I can change it later to point to the "Installing Bro" material (as part of my documentation work for 2.2). -Daniel From bro at tracker.bro-ids.org Wed Oct 3 16:00:28 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Wed, 03 Oct 2012 23:00:28 -0000 Subject: [Bro-Dev] #890: known-services hasty service detection In-Reply-To: <048.b62cc809005a37f75d745f8752fc5238@tracker.bro-ids.org> References: <048.b62cc809005a37f75d745f8752fc5238@tracker.bro-ids.org> Message-ID: <063.db92ee5fff201918ca8551f07900b393@tracker.bro-ids.org> #890: known-services hasty service detection ----------------------------+------------------------ Reporter: jsiwek | Owner: Type: Merge Request | Status: new Priority: Normal | Milestone: Bro2.2 Component: Bro | Version: git/master Resolution: | Keywords: ----------------------------+------------------------ Comment (by robin): On Wed, Oct 03, 2012 at 20:34 -0000, you wrote: > Kind of what I was going for was that both the latest packet from orig and > latest packet from resp are valid teredo encapsulations, which I thought > was a stronger indication that it's actually teredo than just somewhere in > the connection one packet from orig and one packet from resp happened to > have a valid teredo encapsulation. Do you think that's a good idea or no? I see. Yeah, sounds like it should work in this case (it's just an uncommon idiom, that's why I stubled across it.) -- Robin Sommer * Phone +1 (510) 722-6541 * robin at icir.org ICSI/LBNL * Fax +1 (510) 666-2956 * www.icir.org -- Ticket URL: Bro Tracker Bro Issue Tracker From robin at icir.org Wed Oct 3 16:06:44 2012 From: robin at icir.org (Robin Sommer) Date: Wed, 3 Oct 2012 16:06:44 -0700 Subject: [Bro-Dev] [Bro-Commits] [git/bro] fastpath: Redo the "how to upgrade" FAQ. (06d6277) In-Reply-To: <506CB6FF.1010007@illinois.edu> References: <201210032115.q93LFOIY026129@bro-ids.icir.org> <506CAE4A.5020306@illinois.edu> <20121003220324.GC24995@icir.org> <506CB6FF.1010007@illinois.edu> Message-ID: <20121003230644.GG24995@icir.org> On Wed, Oct 03, 2012 at 17:06 -0500, you wrote: > If you want, you could just merge it into master, then I can > change it later to point to the "Installing Bro" material > (as part of my documentation work for 2.2). Sounds good. Btw, it's a bit unfortunate that we have put the FAQ document into the distribution as that means it won't update on the web server until the next release comes out. That's something I think we should chnage in the future: the FAQ shouldn't be tied to a specific version but maintained separately as a web page instead. Robin -- Robin Sommer * Phone +1 (510) 722-6541 * robin at icir.org ICSI/LBNL * Fax +1 (510) 666-2956 * www.icir.org From vladg at cmu.edu Wed Oct 3 16:24:37 2012 From: vladg at cmu.edu (Vlad Grigorescu) Date: Wed, 3 Oct 2012 23:24:37 +0000 Subject: [Bro-Dev] [Bro-Commits] [git/bro] fastpath: Redo the "how to upgrade" FAQ. (06d6277) In-Reply-To: <23298_1349305608_q93N6loY029278_20121003230644.GG24995@icir.org> References: <201210032115.q93LFOIY026129@bro-ids.icir.org> <506CAE4A.5020306@illinois.edu> <20121003220324.GC24995@icir.org> <506CB6FF.1010007@illinois.edu> <23298_1349305608_q93N6loY029278_20121003230644.GG24995@icir.org> Message-ID: <1202BE242E080642B0CD0AD0A03E85524B2BCC@PGH-MSGMB-03.andrew.ad.cmu.edu> Github has an interesting approach to how it generates the optional "Github pages" documentation. It uses a specific branch ("gh-pages") from your repository, to generate them. Assuming I'm understanding this properly, one potential option would be to have a specific branch that the documentation up on the web server is generated from. --Vlad On Oct 3, 2012, at 7:06 PM, Robin Sommer wrote: > > On Wed, Oct 03, 2012 at 17:06 -0500, you wrote: > >> If you want, you could just merge it into master, then I can >> change it later to point to the "Installing Bro" material >> (as part of my documentation work for 2.2). > > Sounds good. > > Btw, it's a bit unfortunate that we have put the FAQ document into the > distribution as that means it won't update on the web server until the > next release comes out. That's something I think we should chnage in > the future: the FAQ shouldn't be tied to a specific version but > maintained separately as a web page instead. > > Robin > > -- > Robin Sommer * Phone +1 (510) 722-6541 * robin at icir.org > ICSI/LBNL * Fax +1 (510) 666-2956 * www.icir.org > _______________________________________________ > bro-dev mailing list > bro-dev at bro-ids.org > http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev From slagell at illinois.edu Wed Oct 3 16:29:13 2012 From: slagell at illinois.edu (Slagell, Adam J) Date: Wed, 3 Oct 2012 23:29:13 +0000 Subject: [Bro-Dev] [Bro-Commits] [git/bro] fastpath: Redo the "how to upgrade" FAQ. (06d6277) In-Reply-To: <20121003230644.GG24995@icir.org> References: <201210032115.q93LFOIY026129@bro-ids.icir.org> <506CAE4A.5020306@illinois.edu> <20121003220324.GC24995@icir.org> <506CB6FF.1010007@illinois.edu>,<20121003230644.GG24995@icir.org> Message-ID: +1 Sent from my iPhone On Oct 3, 2012, at 6:07 PM, "Robin Sommer" wrote: > > Btw, it's a bit unfortunate that we have put the FAQ document into the > distribution as that means it won't update on the web server until the > next release comes out. That's something I think we should chnage in > the future: the FAQ shouldn't be tied to a specific version but > maintained separately as a web page instead. From dnthayer at illinois.edu Wed Oct 3 16:31:22 2012 From: dnthayer at illinois.edu (Daniel Thayer) Date: Wed, 3 Oct 2012 18:31:22 -0500 Subject: [Bro-Dev] [Bro-Commits] [git/bro] fastpath: Redo the "how to upgrade" FAQ. (06d6277) In-Reply-To: <20121003230644.GG24995@icir.org> References: <201210032115.q93LFOIY026129@bro-ids.icir.org> <506CAE4A.5020306@illinois.edu> <20121003220324.GC24995@icir.org> <506CB6FF.1010007@illinois.edu> <20121003230644.GG24995@icir.org> Message-ID: <506CCACA.1070704@illinois.edu> On 10/03/2012 06:06 PM, Robin Sommer wrote: > > On Wed, Oct 03, 2012 at 17:06 -0500, you wrote: > >> If you want, you could just merge it into master, then I can >> change it later to point to the "Installing Bro" material >> (as part of my documentation work for 2.2). > > Sounds good. > > Btw, it's a bit unfortunate that we have put the FAQ document into the > distribution as that means it won't update on the web server until the > next release comes out. That's something I think we should chnage in > the future: the FAQ shouldn't be tied to a specific version but > maintained separately as a web page instead. > > Robin > But don't we have the same issue when, for example, someone corrects an error in the documentation? (the correction won't appear in the docs until the next Bro release) As a sort of workaround, whenever I look at the documentation on the web site, I choose the "Development" option under the "Documentation" tab, so that I always see the most recent version of the docs. -Daniel From noreply at bro-ids.org Thu Oct 4 00:00:02 2012 From: noreply at bro-ids.org (Merge Tracker) Date: Thu, 4 Oct 2012 00:00:02 -0700 Subject: [Bro-Dev] [Auto] Merge Status Message-ID: <201210040700.q94702do021896@bro-ids.icir.org> > Open Merge Requests for Bro2.2 > ============================== Component | Id | Reporter | Owner | Prio | Summary ------------------------------------------------------------------------------------------------------------------ Bro | 890 [1] | jsiwek | | Normal | known-services hasty service detection > Unmerged Fastpath Commits > ========================= Component | Revision | Committer | Date | Summary ------------------------------------------------------------------------------------------------------------------ bro | 06d6277 | Jon Siwek | 2012-10-03 | Redo the "how to upgrade" FAQ. [2] bro | e93748d | Jon Siwek | 2012-10-02 | Add general FAQ entry about upgrading Bro. [3] [1] #890: http://tracker.bro-ids.org/bro/ticket/890 [2] fastpath: http://tracker.bro-ids.org/bro/changeset/06d6277f0aa97836b9c25f7aa97fdf8549fd7da9/bro [3] fastpath: http://tracker.bro-ids.org/bro/changeset/e93748d28b5d1915bda94dc951c42406e0eb2f9e/bro From bro at tracker.bro-ids.org Thu Oct 4 07:55:43 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Thu, 04 Oct 2012 14:55:43 -0000 Subject: [Bro-Dev] #861: Merging DNP3 Analyzer In-Reply-To: <045.d25b5df6a13b96f4b3e3618bd9398e58@tracker.bro-ids.org> References: <045.d25b5df6a13b96f4b3e3618bd9398e58@tracker.bro-ids.org> Message-ID: <060.f6ea70d2aa52042f44d4c2260829587e@tracker.bro-ids.org> #861: Merging DNP3 Analyzer ---------------------+------------------------ Reporter: hui | Owner: robin Type: Task | Status: assigned Priority: Normal | Milestone: Bro2.2 Component: Bro | Version: git/master Resolution: | Keywords: dnp3 ---------------------+------------------------ Comment (by hui): Replying to [comment:7 robin]: > Replying to [comment:6 hui]: > > > So my understanding is that a analyzer class instance represent a single TCP session including all flows ? Actually I am writing to confirm this understanding. > > Yes, the analyzer gets (all) the data from a single TCP session, so if you need to remember things for the (pseudo-)link layer reassembly, you can (and should) do that as part of the analyzer class. > > One additional note though: I'm now wondering if you need to buffer the data at all. Can you just pass it into the BinPAC analyzer as it comes in? You don't need to have the full PDU assembled before starting to send data in (just like for HTTP, you don't need to have the full HTTP session). > Buffering all data can cause the performance issue as well as being attacked by DOS. I think what you said is the "incremental parsing" mentioned in the binpac paper. But actually, I am not quite sure how this is implemented in the binpac. Can you please direct me to some codes that I refer to? I think I can first document this version of the DNP3 analyzer and finish your comments. For this work, I still have to test a little bit. Then we can remove buffering data in the later version of the DNP3 analyzer. Does that sound OK to you? -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Thu Oct 4 08:07:33 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Thu, 04 Oct 2012 15:07:33 -0000 Subject: [Bro-Dev] #861: Merging DNP3 Analyzer In-Reply-To: <045.d25b5df6a13b96f4b3e3618bd9398e58@tracker.bro-ids.org> References: <045.d25b5df6a13b96f4b3e3618bd9398e58@tracker.bro-ids.org> Message-ID: <060.3fb0842db875a9567ed8a18828c64c1d@tracker.bro-ids.org> #861: Merging DNP3 Analyzer ---------------------+------------------------ Reporter: hui | Owner: robin Type: Task | Status: assigned Priority: Normal | Milestone: Bro2.2 Component: Bro | Version: git/master Resolution: | Keywords: dnp3 ---------------------+------------------------ Comment (by seth): > I think what you said is the "incremental parsing" > mentioned in the binpac paper. But actually, I am not quite sure how this > is implemented in the binpac. Can you please direct me to some codes that > I refer to? It's actually not something you even need to worry about. Just instantiate your binpac parser and begin passing data into it as you receive it, the binpac parser will take care of the data even if it doesn't receive the full PDU in one go. -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Thu Oct 4 13:42:20 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Thu, 04 Oct 2012 20:42:20 -0000 Subject: [Bro-Dev] #890: known-services hasty service detection In-Reply-To: <048.b62cc809005a37f75d745f8752fc5238@tracker.bro-ids.org> References: <048.b62cc809005a37f75d745f8752fc5238@tracker.bro-ids.org> Message-ID: <063.c4fc851a252f37b10459b388befab52f@tracker.bro-ids.org> #890: known-services hasty service detection ----------------------------+------------------------ Reporter: jsiwek | Owner: robin Type: Merge Request | Status: closed Priority: Normal | Milestone: Bro2.2 Component: Bro | Version: git/master Resolution: fixed | Keywords: ----------------------------+------------------------ Changes (by robin): * owner: => robin * status: new => closed * resolution: => fixed Comment: In [1290ed6b950ac1be31039bd95d3411c42dafab51/bro]: {{{ #!CommitTicketReference repository="bro" revision="1290ed6b950ac1be31039bd95d3411c42dafab51" Merge remote-tracking branch 'origin/topic/jsiwek/delay-teredo-confirm' * origin/topic/jsiwek/delay-teredo-confirm: Add new Tunnel::delay_teredo_confirmation option, default to true. Closes #890. }}} -- Ticket URL: Bro Tracker Bro Issue Tracker From noreply at bro-ids.org Fri Oct 5 00:00:02 2012 From: noreply at bro-ids.org (Merge Tracker) Date: Fri, 5 Oct 2012 00:00:02 -0700 Subject: [Bro-Dev] [Auto] Merge Status Message-ID: <201210050700.q95702Vp032227@bro-ids.icir.org> > Unmerged Fastpath Commits > ========================= Component | Revision | Committer | Date | Summary ------------------------------------------------------------------------------------------------------------------ bro | 53d9832 | Jon Siwek | 2012-10-04 | Fix a problem with non-manager cluster nodes applying Notice::policy. [1] [1] fastpath: http://tracker.bro-ids.org/bro/changeset/53d9832d5adb526bf80f1d225a13941c7d05bdb2/bro From noreply at bro-ids.org Sat Oct 6 00:00:02 2012 From: noreply at bro-ids.org (Merge Tracker) Date: Sat, 6 Oct 2012 00:00:02 -0700 Subject: [Bro-Dev] [Auto] Merge Status Message-ID: <201210060700.q96702Mp023033@bro-ids.icir.org> > Unmerged Fastpath Commits > ========================= Component | Revision | Committer | Date | Summary ------------------------------------------------------------------------------------------------------------------ bro | 53d9832 | Jon Siwek | 2012-10-04 | Fix a problem with non-manager cluster nodes applying Notice::policy. [1] [1] fastpath: http://tracker.bro-ids.org/bro/changeset/53d9832d5adb526bf80f1d225a13941c7d05bdb2/bro From bro at tracker.bro-ids.org Sat Oct 6 18:30:37 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Sun, 07 Oct 2012 01:30:37 -0000 Subject: [Bro-Dev] #861: Merging DNP3 Analyzer In-Reply-To: <045.d25b5df6a13b96f4b3e3618bd9398e58@tracker.bro-ids.org> References: <045.d25b5df6a13b96f4b3e3618bd9398e58@tracker.bro-ids.org> Message-ID: <060.718dd1efc739944c52780427b80edc1a@tracker.bro-ids.org> #861: Merging DNP3 Analyzer ---------------------+------------------------ Reporter: hui | Owner: robin Type: Task | Status: assigned Priority: Normal | Milestone: Bro2.2 Component: Bro | Version: git/master Resolution: | Keywords: dnp3 ---------------------+------------------------ Comment (by hui): Replying to [comment:12 seth]: > > I think what you said is the "incremental parsing" > > mentioned in the binpac paper. But actually, I am not quite sure how this > > is implemented in the binpac. Can you please direct me to some codes that > > I refer to? > > It's actually not something you even need to worry about. Just instantiate your binpac parser and begin passing data into it as you receive it, the binpac parser will take care of the data even if it doesn't receive the full PDU in one go. Just come up another question. When a HTTP fragment is very long and carried in different network packets, the HTTP binpac analyzer should know the length of the whole fragment when the first application layer trunk is received. Is this correct? The HTTP message contains some field to indicate that length, right? But for DNP3 analyzer, this is not possible. I know the length of the whole logical DNP3 fragment only when the last trunk is received. To better explained TCP : DNP3 Pseudo Data Link Layer (length field is 255) : DNP3 Pseudo Data Transport Layer : DNP3 Pseudo Application Layer #1 TCP : DNP3 Pseudo Data Link Layer (length field is 255) : DNP3 Pseudo Data Transport Layer : DNP3 Pseudo Application Layer #2 .... TCP : DNP3 Pseudo Data Link Layer (length field is x) : DNP3 Pseudo Data Transport Layer : DNP3 Pseudo Application Layer #n So the length field in the Pseudo Data Link does not contain the length of the whole DNP3 fragment, but the length of the trunk following this data link layer. So in order to know the whole length of the DNP3 fragment (in this case is , 255 + 255 + ... + x), all the application layer trunk has to be received. So is there any way to use incremental parsing in the binpac in this case? -- Ticket URL: Bro Tracker Bro Issue Tracker From noreply at bro-ids.org Sun Oct 7 00:00:03 2012 From: noreply at bro-ids.org (Merge Tracker) Date: Sun, 7 Oct 2012 00:00:03 -0700 Subject: [Bro-Dev] [Auto] Merge Status Message-ID: <201210070700.q97703pW019227@bro-ids.icir.org> > Unmerged Fastpath Commits > ========================= Component | Revision | Committer | Date | Summary ------------------------------------------------------------------------------------------------------------------ bro | 53d9832 | Jon Siwek | 2012-10-04 | Fix a problem with non-manager cluster nodes applying Notice::policy. [1] [1] fastpath: http://tracker.bro-ids.org/bro/changeset/53d9832d5adb526bf80f1d225a13941c7d05bdb2/bro From noreply at bro-ids.org Mon Oct 8 00:00:03 2012 From: noreply at bro-ids.org (Merge Tracker) Date: Mon, 8 Oct 2012 00:00:03 -0700 Subject: [Bro-Dev] [Auto] Merge Status Message-ID: <201210080700.q98703jS010033@bro-ids.icir.org> > Unmerged Fastpath Commits > ========================= Component | Revision | Committer | Date | Summary ------------------------------------------------------------------------------------------------------------------ bro | 53d9832 | Jon Siwek | 2012-10-04 | Fix a problem with non-manager cluster nodes applying Notice::policy. [1] [1] fastpath: http://tracker.bro-ids.org/bro/changeset/53d9832d5adb526bf80f1d225a13941c7d05bdb2/bro From bro at tracker.bro-ids.org Mon Oct 8 11:33:37 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Mon, 08 Oct 2012 18:33:37 -0000 Subject: [Bro-Dev] #891: topic/jsiwek/gridftp Message-ID: <048.1baa655406ff9dd5ee0406d71bac3615@tracker.bro-ids.org> #891: topic/jsiwek/gridftp ---------------------------+------------------------ Reporter: jsiwek | Owner: Type: Merge Request | Status: new Priority: Normal | Milestone: Bro2.2 Component: Bro | Version: git/master Keywords: | ---------------------------+------------------------ This branch exists in `bro`, `bro-testing`, and `bro-testing-private` repos and adds support for GridFTP detection. Relevant commits are: [e34f6d9e3b1475828e11b590211311581dd05955/bro] [49b8c7e3909ba0b57019285eaa07022c44f45270/bro] [68aead024ab4a93ac83dc83f5ba61427bd1401e4/bro] Summarized changes are: - Add generic connection polling script: base/protocols/conn/polling.bro - Add GridFTP detection script (mostly deals with GridFTP data channel detection): base/protocols/ftp/gridftp.bro - Add a new support analyzer which looks at FTP requests for AUTH GSSAPI and forwards data to an SSL analyzer instance if it looks like the GSI mechanism. This makes it possible to identify GridFTP control channels. - Change FTP analyzer to no longer skip further analysis of a connection if the server accepts an AUTH request. - SSL client certificates are now tracked/logged by default. -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Mon Oct 8 14:03:40 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Mon, 08 Oct 2012 21:03:40 -0000 Subject: [Bro-Dev] #877: libCurl configure options In-Reply-To: <046.5730894ab0b50cbe92592bc3edadce6c@tracker.bro-ids.org> References: <046.5730894ab0b50cbe92592bc3edadce6c@tracker.bro-ids.org> Message-ID: <061.651f1506c3696cdaaee37ad36f2a51c6@tracker.bro-ids.org> #877: libCurl configure options ----------------------+------------------------ Reporter: seth | Owner: Type: Problem | Status: testing Priority: Normal | Milestone: Bro2.2 Component: Bro | Version: git/master Resolution: | Keywords: ----------------------+------------------------ Comment (by jsiwek): > Merging, but it seems not all of this ticket has been addresses (i.e., the option to disable/relocate lib curl)? I can do the alternate path option, but I'm not sure adding a disable option makes sense right now since it's only used by the elasticsearch writer. It's more useful to add options for ./configure to enable/disable *features*, not just particular dependencies. So the lacking option in this case looks like --disable-elasticsearch. -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Mon Oct 8 14:47:20 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Mon, 08 Oct 2012 21:47:20 -0000 Subject: [Bro-Dev] #877: libCurl configure options In-Reply-To: <046.5730894ab0b50cbe92592bc3edadce6c@tracker.bro-ids.org> References: <046.5730894ab0b50cbe92592bc3edadce6c@tracker.bro-ids.org> Message-ID: <061.e7754fb483815843665997160a616d2e@tracker.bro-ids.org> #877: libCurl configure options ----------------------+------------------------ Reporter: seth | Owner: Type: Problem | Status: testing Priority: Normal | Milestone: Bro2.2 Component: Bro | Version: git/master Resolution: | Keywords: ----------------------+------------------------ Comment (by jsiwek): In [6caeb7d7cfe51b672ed2784877cdc050c41b6d0b/bro]: {{{ #!CommitTicketReference repository="bro" revision="6caeb7d7cfe51b672ed2784877cdc050c41b6d0b" Add --with-curl option to ./configure, addresses #877. }}} -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Mon Oct 8 14:47:20 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Mon, 08 Oct 2012 21:47:20 -0000 Subject: [Bro-Dev] #877: libCurl configure options In-Reply-To: <046.5730894ab0b50cbe92592bc3edadce6c@tracker.bro-ids.org> References: <046.5730894ab0b50cbe92592bc3edadce6c@tracker.bro-ids.org> Message-ID: <061.a62aa9785676c895d76d7e1ac11029b4@tracker.bro-ids.org> #877: libCurl configure options ----------------------+------------------------ Reporter: seth | Owner: Type: Problem | Status: testing Priority: Normal | Milestone: Bro2.2 Component: Bro | Version: git/master Resolution: | Keywords: ----------------------+------------------------ Comment (by jsiwek): In [c69431beac1c636439929e059cda47a4b6bd2f4f/bro]: {{{ #!CommitTicketReference repository="bro" revision="c69431beac1c636439929e059cda47a4b6bd2f4f" Teach --disable-dataseries/--disable-elasticsearch to ./configure. Addresses #877. }}} -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Mon Oct 8 14:51:20 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Mon, 08 Oct 2012 21:51:20 -0000 Subject: [Bro-Dev] #877: libCurl configure options In-Reply-To: <046.5730894ab0b50cbe92592bc3edadce6c@tracker.bro-ids.org> References: <046.5730894ab0b50cbe92592bc3edadce6c@tracker.bro-ids.org> Message-ID: <061.617a87da7791e9c72d97cd89b9463ec6@tracker.bro-ids.org> #877: libCurl configure options ----------------------------+------------------------ Reporter: seth | Owner: Type: Merge Request | Status: testing Priority: Normal | Milestone: Bro2.2 Component: Bro | Version: git/master Resolution: | Keywords: ----------------------------+------------------------ Changes (by matthias): * type: Problem => Merge Request Comment: Replying to [comment:8 jsiwek]: > So the lacking option in this case looks like --disable-elasticsearch. Agreed. Thanks for finishing this ticket appropriately! -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Mon Oct 8 14:55:34 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Mon, 08 Oct 2012 21:55:34 -0000 Subject: [Bro-Dev] #877: libCurl configure options In-Reply-To: <046.5730894ab0b50cbe92592bc3edadce6c@tracker.bro-ids.org> References: <046.5730894ab0b50cbe92592bc3edadce6c@tracker.bro-ids.org> Message-ID: <061.c087f6e8f849845f96c2ca78cd2f24fa@tracker.bro-ids.org> #877: libCurl configure options ----------------------------+------------------------ Reporter: seth | Owner: Type: Merge Request | Status: testing Priority: Normal | Milestone: Bro2.2 Component: Bro | Version: git/master Resolution: | Keywords: ----------------------------+------------------------ Comment (by jsiwek): `topic/jsiwek/config-options` in the `bro` repo has the config options, and it depends on `topic/jsiwek/findlibcurl` in the `cmake` repo. -- Ticket URL: Bro Tracker Bro Issue Tracker From noreply at bro-ids.org Tue Oct 9 00:00:02 2012 From: noreply at bro-ids.org (Merge Tracker) Date: Tue, 9 Oct 2012 00:00:02 -0700 Subject: [Bro-Dev] [Auto] Merge Status Message-ID: <201210090700.q99702B8030557@bro-ids.icir.org> > Open Merge Requests for Bro2.2 > ============================== Component | Id | Reporter | Owner | Prio | Summary ------------------------------------------------------------------------------------------------------------------ Bro | 877 [1] | seth | | Normal | libCurl configure options Bro | 891 [2] | jsiwek | | Normal | topic/jsiwek/gridftp [3] [1] #877: http://tracker.bro-ids.org/bro/ticket/877 [2] #891: http://tracker.bro-ids.org/bro/ticket/891 [3] gridftp: http://tracker.bro-ids.org/bro/changeset?old_path=%2Fbro&old=master&new_path=%2Fbro&new=topic/jsiwek/gridftp From bro at tracker.bro-ids.org Tue Oct 9 11:46:21 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Tue, 09 Oct 2012 18:46:21 -0000 Subject: [Bro-Dev] #892: topic/seth/dns-zero-responses-fix Message-ID: <046.2449a25a42383baa22edeba6b6ecd314@tracker.bro-ids.org> #892: topic/seth/dns-zero-responses-fix ---------------------------+------------------------ Reporter: seth | Owner: robin Type: Merge Request | Status: new Priority: Normal | Milestone: Bro2.2 Component: Bro | Version: git/master Keywords: | ---------------------------+------------------------ This branch is in `bro`, `bro-testing`, and `bro-testing-private` and implements a fix for a small issue in the base DNS script where DNS requests where the response contained zero responses incorrectly represented the situation. -- Ticket URL: Bro Tracker Bro Issue Tracker From noreply at bro-ids.org Wed Oct 10 00:00:02 2012 From: noreply at bro-ids.org (Merge Tracker) Date: Wed, 10 Oct 2012 00:00:02 -0700 Subject: [Bro-Dev] [Auto] Merge Status Message-ID: <201210100700.q9A702ZI005252@bro-ids.icir.org> > Open Merge Requests for Bro2.2 > ============================== Component | Id | Reporter | Owner | Prio | Summary ------------------------------------------------------------------------------------------------------------------ Bro | 877 [1] | seth | | Normal | libCurl configure options Bro | 891 [2] | jsiwek | | Normal | topic/jsiwek/gridftp [3] Bro | 892 [4] | seth | robin | Normal | topic/seth/dns-zero-responses-fix [5] > Unmerged Fastpath Commits > ========================= Component | Revision | Committer | Date | Summary ------------------------------------------------------------------------------------------------------------------ bro | 2efb976 | Bernhard Amann | 2012-10-09 | ...and forgotten debug-output, sorry (was already merged in some other internal repositories before I noticed) [6] bro | 82eaddb | Bernhard Amann | 2012-10-09 | and another bug in the input framework: config table does not work (is not transmitted to the readers) because the initialization was done the wrong way round. [7] [1] #877: http://tracker.bro-ids.org/bro/ticket/877 [2] #891: http://tracker.bro-ids.org/bro/ticket/891 [3] gridftp: http://tracker.bro-ids.org/bro/changeset?old_path=%2Fbro&old=master&new_path=%2Fbro&new=topic/jsiwek/gridftp [4] #892: http://tracker.bro-ids.org/bro/ticket/892 [5] dns-zero-responses-fix: http://tracker.bro-ids.org/bro/changeset?old_path=%2Fbro&old=master&new_path=%2Fbro&new=topic/seth/dns-zero-responses-fix [6] fastpath: http://tracker.bro-ids.org/bro/changeset/2efb976aaf91c99b596f52c220e6d9e3c783573f/bro [7] fastpath: http://tracker.bro-ids.org/bro/changeset/82eaddb160727d4489bd45cf787d2d8a8031db96/bro From bro at tracker.bro-ids.org Wed Oct 10 08:37:19 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Wed, 10 Oct 2012 15:37:19 -0000 Subject: [Bro-Dev] #893: calling event handler via local variable doesn't work Message-ID: <050.4c99cecaadfc5862ed0ad9f548c99994@tracker.bro-ids.org> #893: calling event handler via local variable doesn't work ----------------------+------------------------ Reporter: dnthayer | Owner: Type: Problem | Status: new Priority: Normal | Milestone: Bro2.2 Component: Bro | Version: git/master Keywords: | ----------------------+------------------------ Calling an event handler (using the "event" statement) does not seem to work when the event handler being called is actually a local variable assigned to an existing event handler. There is a test case for this in the file bro/testing/btest/language/event.bro: event e4(num: count) {...} event bro_init() { [...] # Test assigning an event variable to an event local e5: event(num: count); e5 = e4; event e5(6); # TODO: this does not do anything } -- Ticket URL: Bro Tracker Bro Issue Tracker From noreply at bro-ids.org Thu Oct 11 00:00:02 2012 From: noreply at bro-ids.org (Merge Tracker) Date: Thu, 11 Oct 2012 00:00:02 -0700 Subject: [Bro-Dev] [Auto] Merge Status Message-ID: <201210110700.q9B702iu011633@bro-ids.icir.org> > Open Merge Requests for Bro2.2 > ============================== Component | Id | Reporter | Owner | Prio | Summary ------------------------------------------------------------------------------------------------------------------ Bro | 877 [1] | seth | | Normal | libCurl configure options Bro | 891 [2] | jsiwek | | Normal | topic/jsiwek/gridftp [3] Bro | 892 [4] | seth | robin | Normal | topic/seth/dns-zero-responses-fix [5] > Unmerged Fastpath Commits > ========================= Component | Revision | Committer | Date | Summary ------------------------------------------------------------------------------------------------------------------ bro | 2efb976 | Bernhard Amann | 2012-10-09 | ...and forgotten debug-output, sorry (was already merged in some other internal repositories before I noticed) [6] bro | 82eaddb | Bernhard Amann | 2012-10-09 | and another bug in the input framework: config table does not work (is not transmitted to the readers) because the initialization was done the wrong way round. [7] [1] #877: http://tracker.bro-ids.org/bro/ticket/877 [2] #891: http://tracker.bro-ids.org/bro/ticket/891 [3] gridftp: http://tracker.bro-ids.org/bro/changeset?old_path=%2Fbro&old=master&new_path=%2Fbro&new=topic/jsiwek/gridftp [4] #892: http://tracker.bro-ids.org/bro/ticket/892 [5] dns-zero-responses-fix: http://tracker.bro-ids.org/bro/changeset?old_path=%2Fbro&old=master&new_path=%2Fbro&new=topic/seth/dns-zero-responses-fix [6] fastpath: http://tracker.bro-ids.org/bro/changeset/2efb976aaf91c99b596f52c220e6d9e3c783573f/bro [7] fastpath: http://tracker.bro-ids.org/bro/changeset/82eaddb160727d4489bd45cf787d2d8a8031db96/bro From bro at tracker.bro-ids.org Thu Oct 11 09:23:34 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Thu, 11 Oct 2012 16:23:34 -0000 Subject: [Bro-Dev] #894: Merge request for topic/bernhard/input-end-of-data Message-ID: <048.93fe8ff21e78302acee9559acecb6a90@tracker.bro-ids.org> #894: Merge request for topic/bernhard/input-end-of-data ---------------------+------------------------ Reporter: amannb | Owner: Type: Problem | Status: new Priority: Normal | Milestone: Bro2.2 Component: Bro | Version: git/master Keywords: | ---------------------+------------------------ The branch changes the name of the update_finished event to end_of_data. The end_of_data event is now also thrown when using the event read api and it is exposed to the reader threads who can choose to throw it themselves, when using the manual put/get/event_send api. -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Thu Oct 11 09:23:45 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Thu, 11 Oct 2012 16:23:45 -0000 Subject: [Bro-Dev] #894: Merge request for topic/bernhard/input-end-of-data In-Reply-To: <048.93fe8ff21e78302acee9559acecb6a90@tracker.bro-ids.org> References: <048.93fe8ff21e78302acee9559acecb6a90@tracker.bro-ids.org> Message-ID: <063.4eae8f1c97985dc4086e81a3a7fcb582@tracker.bro-ids.org> #894: Merge request for topic/bernhard/input-end-of-data ----------------------------+------------------------ Reporter: amannb | Owner: Type: Merge Request | Status: new Priority: Normal | Milestone: Bro2.2 Component: Bro | Version: git/master Resolution: | Keywords: ----------------------------+------------------------ Changes (by amannb): * type: Problem => Merge Request -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Thu Oct 11 20:01:00 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Fri, 12 Oct 2012 03:01:00 -0000 Subject: [Bro-Dev] #895: Use of &redef attribute at inappropriate times Message-ID: <046.90467779e2f646ebf3d9faac3e1d48cc@tracker.bro-ids.org> #895: Use of &redef attribute at inappropriate times ---------------------+------------------------ Reporter: seth | Owner: Type: Problem | Status: new Priority: Normal | Milestone: Bro2.2 Component: Bro | Version: git/master Keywords: | ---------------------+------------------------ This should result in a parse-time error since it doesn't make any sense: {{{ const foobar = 10 &redef; redef foobar = 20 &redef; }}} -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Thu Oct 11 20:02:57 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Fri, 12 Oct 2012 03:02:57 -0000 Subject: [Bro-Dev] #896: SCP and SFTP rotators printing Message-ID: <046.4440dc9ff4c441e15eb1b217ea187567@tracker.bro-ids.org> #896: SCP and SFTP rotators printing ---------------------+------------------------ Reporter: seth | Owner: Type: Problem | Status: new Priority: Normal | Milestone: Bro2.2 Component: Bro | Version: git/master Keywords: | ---------------------+------------------------ The SCP and SFTP log rotators are causing output on stdout and shouldn't be. I think the output is coming from the scp and sftp clients, maybe they just need a quiet flag? -- Ticket URL: Bro Tracker Bro Issue Tracker From noreply at bro-ids.org Fri Oct 12 00:00:02 2012 From: noreply at bro-ids.org (Merge Tracker) Date: Fri, 12 Oct 2012 00:00:02 -0700 Subject: [Bro-Dev] [Auto] Merge Status Message-ID: <201210120700.q9C702Ii029205@bro-ids.icir.org> > Open Merge Requests for Bro2.2 > ============================== Component | Id | Reporter | Owner | Prio | Summary ------------------------------------------------------------------------------------------------------------------ Bro | 877 [1] | seth | | Normal | libCurl configure options Bro | 891 [2] | jsiwek | | Normal | topic/jsiwek/gridftp [3] Bro | 892 [4] | seth | robin | Normal | topic/seth/dns-zero-responses-fix [5] Bro | 894 [6] | amannb | | Normal | Merge request for topic/bernhard/input-end-of-data [7] > Unmerged Fastpath Commits > ========================= Component | Revision | Committer | Date | Summary ------------------------------------------------------------------------------------------------------------------ bro | 2efb976 | Bernhard Amann | 2012-10-09 | ...and forgotten debug-output, sorry (was already merged in some other internal repositories before I noticed) [8] bro | 82eaddb | Bernhard Amann | 2012-10-09 | and another bug in the input framework: config table does not work (is not transmitted to the readers) because the initialization was done the wrong way round. [9] [1] #877: http://tracker.bro-ids.org/bro/ticket/877 [2] #891: http://tracker.bro-ids.org/bro/ticket/891 [3] gridftp: http://tracker.bro-ids.org/bro/changeset?old_path=%2Fbro&old=master&new_path=%2Fbro&new=topic/jsiwek/gridftp [4] #892: http://tracker.bro-ids.org/bro/ticket/892 [5] dns-zero-responses-fix: http://tracker.bro-ids.org/bro/changeset?old_path=%2Fbro&old=master&new_path=%2Fbro&new=topic/seth/dns-zero-responses-fix [6] #894: http://tracker.bro-ids.org/bro/ticket/894 [7] input-end-of-data: http://tracker.bro-ids.org/bro/changeset?old_path=%2Fbro&old=master&new_path=%2Fbro&new=topic/bernhard/input-end-of-data [8] fastpath: http://tracker.bro-ids.org/bro/changeset/2efb976aaf91c99b596f52c220e6d9e3c783573f/bro [9] fastpath: http://tracker.bro-ids.org/bro/changeset/82eaddb160727d4489bd45cf787d2d8a8031db96/bro From bro at tracker.bro-ids.org Fri Oct 12 07:23:51 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Fri, 12 Oct 2012 14:23:51 -0000 Subject: [Bro-Dev] #897: File extraction oddities Message-ID: <048.72f4d1641f0e9a2944e67eee51d68665@tracker.bro-ids.org> #897: File extraction oddities ------------------------+--------------------- Reporter: sconzo | Type: Problem Status: new | Priority: Normal Milestone: Bro2.2 | Component: Bro Version: git/master | Keywords: ------------------------+--------------------- The HTTP file extraction appears to not work correctly if files are also MD5 sum'd. The filename gets written to disk but the file is empty. However, other files that are not configured to be MD5'd get persisted to disk correctly. Example: When, redef HTTP::extract_file_types += /.*\/.*/; is defined in a .bro script bro extracts all files, except for the windows exes that were MD5'd ---- When, redef HTTP::extract_file_types += /.*\/.*/; redef HTTP::generate_md5 = /NO_FILE_TYPE_EVER/; is defined in a .bro script bro extracts all files including windows exes, but no files are MD5'd. -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Fri Oct 12 08:18:35 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Fri, 12 Oct 2012 15:18:35 -0000 Subject: [Bro-Dev] #895: Use of &redef attribute at inappropriate times In-Reply-To: <046.90467779e2f646ebf3d9faac3e1d48cc@tracker.bro-ids.org> References: <046.90467779e2f646ebf3d9faac3e1d48cc@tracker.bro-ids.org> Message-ID: <061.69847b8514c3aa1e0f97964982136bbc@tracker.bro-ids.org> #895: Use of &redef attribute at inappropriate times ----------------------+------------------------ Reporter: seth | Owner: Type: Problem | Status: new Priority: Normal | Milestone: Bro2.2 Component: Bro | Version: git/master Resolution: | Keywords: ----------------------+------------------------ Comment (by jsiwek): Replying to [ticket:895 seth]: > This should result in a parse-time error since it doesn't make any sense: > > {{{ > const foobar = 10 &redef; > redef foobar = 20 &redef; > }}} Why is an error preferred over a no-op? If `redef` can be used to add (or modify values of?) attributes, then the above is just redundant. -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Fri Oct 12 08:29:20 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Fri, 12 Oct 2012 15:29:20 -0000 Subject: [Bro-Dev] #895: Use of &redef attribute at inappropriate times In-Reply-To: <046.90467779e2f646ebf3d9faac3e1d48cc@tracker.bro-ids.org> References: <046.90467779e2f646ebf3d9faac3e1d48cc@tracker.bro-ids.org> Message-ID: <061.78c12d0d710b370b48fc554ae422e0f8@tracker.bro-ids.org> #895: Use of &redef attribute at inappropriate times ----------------------+------------------------ Reporter: seth | Owner: Type: Problem | Status: new Priority: Normal | Milestone: Bro2.2 Component: Bro | Version: git/master Resolution: | Keywords: ----------------------+------------------------ Comment (by seth): > Why is an error preferred over a no-op? If `redef` can be used to add (or > modify values of?) attributes, then the above is just redundant. I guess it's sort of a wash. When I filed the ticket I was a little worried about unintended side effects being caused by it. Do you suppose that's a possibility? I guess if it's not a concern then we can just close this ticket. -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Fri Oct 12 08:53:36 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Fri, 12 Oct 2012 15:53:36 -0000 Subject: [Bro-Dev] #895: Use of &redef attribute at inappropriate times In-Reply-To: <046.90467779e2f646ebf3d9faac3e1d48cc@tracker.bro-ids.org> References: <046.90467779e2f646ebf3d9faac3e1d48cc@tracker.bro-ids.org> Message-ID: <061.bf04a7e756ad12ea93c473b71af3abd7@tracker.bro-ids.org> #895: Use of &redef attribute at inappropriate times ----------------------+------------------------ Reporter: seth | Owner: Type: Problem | Status: new Priority: Normal | Milestone: Bro2.2 Component: Bro | Version: git/master Resolution: | Keywords: ----------------------+------------------------ Comment (by jsiwek): > I guess it's sort of a wash. When I filed the ticket I was a little worried about unintended side effects being caused by it. Do you suppose that's a possibility? I guess if it's not a concern then we can just close this ticket. At a glance, it looked harmless, but having some more through tests of attributes and redef statement interaction that could verify that could be a to-do item. -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Fri Oct 12 09:26:18 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Fri, 12 Oct 2012 16:26:18 -0000 Subject: [Bro-Dev] #898: Confusion over the accept_input field in communication code Message-ID: <046.19a23c542bf4d8411d7c77416d007d0e@tracker.bro-ids.org> #898: Confusion over the accept_input field in communication code ---------------------+------------------------ Reporter: seth | Owner: Type: Problem | Status: new Priority: Normal | Milestone: Bro2.2 Component: Bro | Version: git/master Keywords: | ---------------------+------------------------ The communication framework configuration include a field named accept_input which isn't named well and seems to be over used. The result of it is that state is synchronized between the manager and workers which shouldn't be happening. This will require some discussion before removing the functionality though because pushing data from the manager to workers can be useful on clusters where data is read with the input framework and needs to be distributed to the workers. -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Fri Oct 12 09:49:18 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Fri, 12 Oct 2012 16:49:18 -0000 Subject: [Bro-Dev] #892: topic/seth/dns-zero-responses-fix In-Reply-To: <046.2449a25a42383baa22edeba6b6ecd314@tracker.bro-ids.org> References: <046.2449a25a42383baa22edeba6b6ecd314@tracker.bro-ids.org> Message-ID: <061.1d319c8ab40f1c0f474b2285dcc85d97@tracker.bro-ids.org> #892: topic/seth/dns-zero-responses-fix ----------------------------+------------------------ Reporter: seth | Owner: robin Type: Merge Request | Status: closed Priority: Normal | Milestone: Bro2.2 Component: Bro | Version: git/master Resolution: fixed | Keywords: ----------------------------+------------------------ Changes (by robin): * status: new => closed * resolution: => fixed Comment: In [b1b1991cc742cd1707c91c4f07b6d31e73719c87/bro]: {{{ #!CommitTicketReference repository="bro" revision="b1b1991cc742cd1707c91c4f07b6d31e73719c87" Merge remote-tracking branch 'origin/topic/seth/dns-zero-responses-fix' * origin/topic/seth/dns-zero-responses-fix: Fix for DNS log problem when a DNS response is seen with 0 RRs. Closes #892. }}} -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Fri Oct 12 09:49:18 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Fri, 12 Oct 2012 16:49:18 -0000 Subject: [Bro-Dev] #894: Merge request for topic/bernhard/input-end-of-data In-Reply-To: <048.93fe8ff21e78302acee9559acecb6a90@tracker.bro-ids.org> References: <048.93fe8ff21e78302acee9559acecb6a90@tracker.bro-ids.org> Message-ID: <063.79cd2c1282ee7a483c64df228897da3d@tracker.bro-ids.org> #894: Merge request for topic/bernhard/input-end-of-data ----------------------------+------------------------ Reporter: amannb | Owner: robin Type: Merge Request | Status: closed Priority: Normal | Milestone: Bro2.2 Component: Bro | Version: git/master Resolution: fixed | Keywords: ----------------------------+------------------------ Changes (by robin): * owner: => robin * status: new => closed * resolution: => fixed Comment: In [503412e472e33c6ae3c0a0ff5c8bacbd230a5189/bro]: {{{ #!CommitTicketReference repository="bro" revision="503412e472e33c6ae3c0a0ff5c8bacbd230a5189" Merge remote-tracking branch 'origin/topic/bernhard/input-end-of-data' * origin/topic/bernhard/input-end-of-data: rename the update_finished event to end_of_data and make it fire in more cases. Closes #894. }}} -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Fri Oct 12 09:49:18 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Fri, 12 Oct 2012 16:49:18 -0000 Subject: [Bro-Dev] #877: libCurl configure options In-Reply-To: <046.5730894ab0b50cbe92592bc3edadce6c@tracker.bro-ids.org> References: <046.5730894ab0b50cbe92592bc3edadce6c@tracker.bro-ids.org> Message-ID: <061.5a376703c3ab5857e24db873e63e4309@tracker.bro-ids.org> #877: libCurl configure options ----------------------------+------------------------ Reporter: seth | Owner: robin Type: Merge Request | Status: closed Priority: Normal | Milestone: Bro2.2 Component: Bro | Version: git/master Resolution: fixed | Keywords: ----------------------------+------------------------ Changes (by robin): * owner: => robin * status: testing => closed * resolution: => fixed Comment: In [c24fa2cd0146f122a0d5bd4fd443aa1a2e6cf863/bro]: {{{ #!CommitTicketReference repository="bro" revision="c24fa2cd0146f122a0d5bd4fd443aa1a2e6cf863" Merge remote-tracking branch 'origin/topic/jsiwek/config-options' * origin/topic/jsiwek/config-options: Teach --disable-dataseries/--disable-elasticsearch to ./configure. Add --with-curl option to ./configure, addresses #877. Closes #877. }}} -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Fri Oct 12 10:37:33 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Fri, 12 Oct 2012 17:37:33 -0000 Subject: [Bro-Dev] #891: topic/jsiwek/gridftp In-Reply-To: <048.1baa655406ff9dd5ee0406d71bac3615@tracker.bro-ids.org> References: <048.1baa655406ff9dd5ee0406d71bac3615@tracker.bro-ids.org> Message-ID: <063.be553f2150add89689630423313c52dc@tracker.bro-ids.org> #891: topic/jsiwek/gridftp ----------------------------+------------------------ Reporter: jsiwek | Owner: Type: Merge Request | Status: new Priority: Normal | Milestone: Bro2.2 Component: Bro | Version: git/master Resolution: | Keywords: ----------------------------+------------------------ Comment (by robin): Good job, I like seeing the analyzer infrastructure being used as it was intended :) Questions: - The extensions to the SSL analysis store cert/chain information for all SSL conntections now. Could that have a significant impact on memory or CPU? - in {{{ftp/gridftp.bro}}} {{{ event ssl_established(c: connection) &priority=5 ????????{ ????????# Add service label to control channels. ????????if ( "FTP" in c$service ) ????????????????add c$service["gridftp"]; ????????} }}} Can that condition really only be true for GridFTP sessions? - in {{{ssl/main.bro}}} {{{ c$ssl = [$ts=network_time(), $uid=c$uid, $id=c$id, $cert_chain=vector(), $client_cert_chain=vector()]; }}} Is that initialization with {{{vector()}}} still needed? I thought we had fixed that a while ago so that containers in records get initialized to an empty instance? -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Fri Oct 12 10:41:17 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Fri, 12 Oct 2012 17:41:17 -0000 Subject: [Bro-Dev] #895: Use of &redef attribute at inappropriate times In-Reply-To: <046.90467779e2f646ebf3d9faac3e1d48cc@tracker.bro-ids.org> References: <046.90467779e2f646ebf3d9faac3e1d48cc@tracker.bro-ids.org> Message-ID: <061.32753ab6ff990b4fd7e374a37946cdce@tracker.bro-ids.org> #895: Use of &redef attribute at inappropriate times ----------------------+------------------------ Reporter: seth | Owner: Type: Problem | Status: new Priority: Normal | Milestone: Bro2.2 Component: Bro | Version: git/master Resolution: | Keywords: ----------------------+------------------------ Comment (by dnthayer): Replying to [comment:2 jsiwek]: > > > I guess it's sort of a wash. When I filed the ticket I was a little worried about unintended side effects being caused by it. Do you suppose that's a possibility? I guess if it's not a concern then we can just close this ticket. > > At a glance, it looked harmless, but having some more through tests of attributes and redef statement interaction that could verify that could be a to-do item. It's been on my to-do list for a while now (I plan to write a complete set of tests for all attributes). -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Fri Oct 12 10:41:44 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Fri, 12 Oct 2012 17:41:44 -0000 Subject: [Bro-Dev] #891: topic/jsiwek/gridftp In-Reply-To: <048.1baa655406ff9dd5ee0406d71bac3615@tracker.bro-ids.org> References: <048.1baa655406ff9dd5ee0406d71bac3615@tracker.bro-ids.org> Message-ID: <063.6a54d645872aa48cf77066a856080ed6@tracker.bro-ids.org> #891: topic/jsiwek/gridftp ----------------------------+------------------------ Reporter: jsiwek | Owner: Type: Merge Request | Status: new Priority: Normal | Milestone: Bro2.2 Component: Bro | Version: git/master Resolution: | Keywords: ----------------------------+------------------------ Comment (by seth): > - in {{{ftp/gridftp.bro}}} > > {{{ > event ssl_established(c: connection) &priority=5 > ????????{ > ????????# Add service label to control channels. > ????????if ( "FTP" in c$service ) > ????????????????add c$service["gridftp"]; > ????????} > }}} > > Can that condition really only be true for GridFTP sessions? oh, FTP has a starttls command, doesn't it? -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Fri Oct 12 10:49:31 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Fri, 12 Oct 2012 17:49:31 -0000 Subject: [Bro-Dev] #895: Use of &redef attribute at inappropriate times In-Reply-To: <046.90467779e2f646ebf3d9faac3e1d48cc@tracker.bro-ids.org> References: <046.90467779e2f646ebf3d9faac3e1d48cc@tracker.bro-ids.org> Message-ID: <061.bc3aa5512c00d59ba2f24544f9a5752f@tracker.bro-ids.org> #895: Use of &redef attribute at inappropriate times ----------------------+------------------------ Reporter: seth | Owner: Type: Problem | Status: new Priority: Normal | Milestone: Bro2.2 Component: Bro | Version: git/master Resolution: | Keywords: ----------------------+------------------------ Comment (by seth): > It's been on my to-do list for a while now (I plan to write a complete set > of tests for all attributes). If you'd rather track this work through another ticket or something, feel free to close this one. -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Fri Oct 12 11:45:09 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Fri, 12 Oct 2012 18:45:09 -0000 Subject: [Bro-Dev] #891: topic/jsiwek/gridftp In-Reply-To: <048.1baa655406ff9dd5ee0406d71bac3615@tracker.bro-ids.org> References: <048.1baa655406ff9dd5ee0406d71bac3615@tracker.bro-ids.org> Message-ID: <063.a234d62b35712c1d17cdf68fc7967ffb@tracker.bro-ids.org> #891: topic/jsiwek/gridftp ----------------------------+------------------------ Reporter: jsiwek | Owner: Type: Merge Request | Status: new Priority: Normal | Milestone: Bro2.2 Component: Bro | Version: git/master Resolution: | Keywords: ----------------------------+------------------------ Comment (by jsiwek): > - The extensions to the SSL analysis store cert/chain information for all SSL conntections now. Could that have a significant impact on memory or CPU? The storing of server certs was there previously and nothing should have changed regarding that to support gridftp. The presence of client certs shouldn't be the common case for SSL and by default no validation is performed on them. > - in {{{ftp/gridftp.bro}}} > {{{ > event ssl_established(c: connection) &priority=5 >????????{ >????????# Add service label to control channels. >????????if ( "FTP" in c$service ) >????????????????add c$service["gridftp"]; >????????} > }}} > > Can that condition really only be true for GridFTP sessions? Currently, I think so, but I could probably add some logic to further qualify that for only FTP sessions that used AUTH GSSAPI. > - in {{{ssl/main.bro}}} > > {{{ c$ssl = [$ts=network_time(), $uid=c$uid, $id=c$id,$cert_chain=vector(), $client_cert_chain=vector()]; }}} > > Is that initialization with {{{vector()}}} still needed? I thought we had > fixed that a while ago so that containers in records get initialized to an > empty instance? If I remember right, it was necessary, but I could have just been copying what had already existed for $cert_chain. > Comment (by seth): > oh, FTP has a starttls command, doesn't it? I think there is an AUTH TLS method, but that wouldn't automatically get an SSL analyzer attached to the connection. It would need the same explicit internal support as gridftp (attaching the support analyzer to decode the ADAT exchanges over FTP before forwarding for SSL analysis.) -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Fri Oct 12 11:50:44 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Fri, 12 Oct 2012 18:50:44 -0000 Subject: [Bro-Dev] #891: topic/jsiwek/gridftp In-Reply-To: <048.1baa655406ff9dd5ee0406d71bac3615@tracker.bro-ids.org> References: <048.1baa655406ff9dd5ee0406d71bac3615@tracker.bro-ids.org> Message-ID: <063.0489fded0b8ec7702e0a7f8225872484@tracker.bro-ids.org> #891: topic/jsiwek/gridftp ----------------------------+------------------------ Reporter: jsiwek | Owner: Type: Merge Request | Status: new Priority: Normal | Milestone: Bro2.2 Component: Bro | Version: git/master Resolution: | Keywords: ----------------------------+------------------------ Comment (by robin): On Fri, Oct 12, 2012 at 18:45 -0000, you wrote: > Currently, I think so, but I could probably add some logic to further > qualify that for only FTP sessions that used AUTH GSSAPI. That would be good I think. > I think there is an AUTH TLS method, but that wouldn't automatically get > an SSL analyzer attached to the connection. It would need the same > explicit internal support as gridftp (attaching the support analyzer to > decode the ADAT exchanges over FTP before forwarding for SSL analysis.) Maybe something for the nice-to-have todo list. -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Fri Oct 12 12:04:33 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Fri, 12 Oct 2012 19:04:33 -0000 Subject: [Bro-Dev] #891: topic/jsiwek/gridftp In-Reply-To: <048.1baa655406ff9dd5ee0406d71bac3615@tracker.bro-ids.org> References: <048.1baa655406ff9dd5ee0406d71bac3615@tracker.bro-ids.org> Message-ID: <063.cb142a0b693cf55dad624c8c56d45041@tracker.bro-ids.org> #891: topic/jsiwek/gridftp ----------------------------+------------------------ Reporter: jsiwek | Owner: Type: Merge Request | Status: new Priority: Normal | Milestone: Bro2.2 Component: Bro | Version: git/master Resolution: | Keywords: ----------------------------+------------------------ Comment (by slagell): Replying to [comment:3 robin]: > > > I think there is an AUTH TLS method, but that wouldn't automatically get > > an SSL analyzer attached to the connection. It would need the same > > explicit internal support as gridftp (attaching the support analyzer to > > decode the ADAT exchanges over FTP before forwarding for SSL analysis.) > > Maybe something for the nice-to-have todo list. Probably in the nice-to-have category. P.S. Damn, Tracker is slow sometimes. -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Fri Oct 12 12:10:19 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Fri, 12 Oct 2012 19:10:19 -0000 Subject: [Bro-Dev] #891: topic/jsiwek/gridftp In-Reply-To: <048.1baa655406ff9dd5ee0406d71bac3615@tracker.bro-ids.org> References: <048.1baa655406ff9dd5ee0406d71bac3615@tracker.bro-ids.org> Message-ID: <063.d7d9203e9ee43da3a7aa580cf3a50f78@tracker.bro-ids.org> #891: topic/jsiwek/gridftp ----------------------------+------------------------ Reporter: jsiwek | Owner: Type: Merge Request | Status: new Priority: Normal | Milestone: Bro2.2 Component: Bro | Version: git/master Resolution: | Keywords: ----------------------------+------------------------ Comment (by seth): > The storing of server certs was there previously and nothing should have > changed regarding that to support gridftp. The presence of client certs > shouldn't be the common case for SSL and by default no validation is > performed on them. I think I have the c$ssl record being deleted after the ssl_established event anyway so it's a very temporary memory use. -- Ticket URL: Bro Tracker Bro Issue Tracker From james.swaro at gmail.com Fri Oct 12 13:00:09 2012 From: james.swaro at gmail.com (James Swaro) Date: Fri, 12 Oct 2012 16:00:09 -0400 Subject: [Bro-Dev] network_time Message-ID: As I am aware, Bro has steadily moved toward a multi-threaded approach to logging at the least. For an analyzer that I've been developing as part of my research, I am curious to know if network_time remains coherent with the network time given in pcap files and live capture. If not, is there a more accurate variable available? Of note: I've never really observed a discrepancy between the pcap files and reported network time through the event system. Gilbert Clark and I had a small discussion on this and I feel that from what I've seen in the source code, network_time is likely fine, but I thought I'd get the answer from the folk who know the source quite a bit better than I do. Best, -- James Swaro* * Internetworking Research Group Ohio University -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.icsi.berkeley.edu/pipermail/bro-dev/attachments/20121012/b1339ea1/attachment.html From seth at icir.org Fri Oct 12 13:09:56 2012 From: seth at icir.org (Seth Hall) Date: Fri, 12 Oct 2012 16:09:56 -0400 Subject: [Bro-Dev] network_time In-Reply-To: References: Message-ID: <7E843AC6-0CDB-48E7-B42E-8F2B96F1D7F7@icir.org> On Oct 12, 2012, at 4:00 PM, James Swaro wrote: > As I am aware, Bro has steadily moved toward a multi-threaded approach to logging at the least. For an analyzer that I've been developing as part of my research, I am curious to know if network_time remains coherent with the network time given in pcap files and live capture. If not, is there a more accurate variable available? There are several things going into this from what I understand. network_time is obviously driven by the packet timestamps so it's going to represent the most recent packet received. Do to stream reassembly and it's interaction with the event queue it's possible that there are misrepresentations in the value returned from network_time(). If you use current_time(), that's going to get the current clock time but that could also be misleading due to latency in the event queue. The best answer I have is like most things in Bro, you have to transition your mind away from packets and if what you are trying to do is intrinsically some packet level thing Bro might not be the right too (unless you are working in the core where you can absolutely do packet level work). On the upside, Bro does flush it's event queue after every single packet so it's not like you are going to see large latency due to the event queue. Bro's event queue and stream parsing were not threaded either. Only the logging and input frameworks which almost certainly won't affect anything you are doing. .Seth -- Seth Hall International Computer Science Institute (Bro) because everyone has a network http://www.bro-ids.org/ From bro at tracker.bro-ids.org Fri Oct 12 14:48:57 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Fri, 12 Oct 2012 21:48:57 -0000 Subject: [Bro-Dev] #895: Use of &redef attribute at inappropriate times In-Reply-To: <046.90467779e2f646ebf3d9faac3e1d48cc@tracker.bro-ids.org> References: <046.90467779e2f646ebf3d9faac3e1d48cc@tracker.bro-ids.org> Message-ID: <061.ce18062b40245211bed57944dfe4f977@tracker.bro-ids.org> #895: Use of &redef attribute at inappropriate times ----------------------+------------------------ Reporter: seth | Owner: Type: Problem | Status: new Priority: Normal | Milestone: Bro2.2 Component: Bro | Version: git/master Resolution: | Keywords: ----------------------+------------------------ Comment (by vern): A couple of minor comments: (1) While it does appear to be unneeded, one could imagine a variant &noredef which means "do not allow any further redef'ing". I'm not sure whether that's something we will ever wind up considering, though. (2) In general, tests of attributes are a Good Thing. But for whether this particular construct does anything at all, the direct way to tell is of course inspecting the code rather than trying to observe it externally. -- Ticket URL: Bro Tracker Bro Issue Tracker From noreply at bro-ids.org Sat Oct 13 00:00:03 2012 From: noreply at bro-ids.org (Merge Tracker) Date: Sat, 13 Oct 2012 00:00:03 -0700 Subject: [Bro-Dev] [Auto] Merge Status Message-ID: <201210130700.q9D703xe028241@bro-ids.icir.org> > Open Merge Requests for Bro2.2 > ============================== Component | Id | Reporter | Owner | Prio | Summary ------------------------------------------------------------------------------------------------------------------ Bro | 891 [1] | jsiwek | | Normal | topic/jsiwek/gridftp [2] > Unmerged Fastpath Commits > ========================= Component | Revision | Committer | Date | Summary ------------------------------------------------------------------------------------------------------------------ bro | 15033a2 | Daniel Thayer | 2012-10-12 | Allow faster rebuilds in certain cases [3] [1] #891: http://tracker.bro-ids.org/bro/ticket/891 [2] gridftp: http://tracker.bro-ids.org/bro/changeset?old_path=%2Fbro&old=master&new_path=%2Fbro&new=topic/jsiwek/gridftp [3] fastpath: http://tracker.bro-ids.org/bro/changeset/15033a2b626ea8d932ca6743587ead0b2bd70688/bro From noreply at bro-ids.org Sun Oct 14 00:00:03 2012 From: noreply at bro-ids.org (Merge Tracker) Date: Sun, 14 Oct 2012 00:00:03 -0700 Subject: [Bro-Dev] [Auto] Merge Status Message-ID: <201210140700.q9E703qZ000643@bro-ids.icir.org> > Open Merge Requests for Bro2.2 > ============================== Component | Id | Reporter | Owner | Prio | Summary ------------------------------------------------------------------------------------------------------------------ Bro | 891 [1] | jsiwek | | Normal | topic/jsiwek/gridftp [2] > Unmerged Fastpath Commits > ========================= Component | Revision | Committer | Date | Summary ------------------------------------------------------------------------------------------------------------------ bro | 15033a2 | Daniel Thayer | 2012-10-12 | Allow faster rebuilds in certain cases [3] [1] #891: http://tracker.bro-ids.org/bro/ticket/891 [2] gridftp: http://tracker.bro-ids.org/bro/changeset?old_path=%2Fbro&old=master&new_path=%2Fbro&new=topic/jsiwek/gridftp [3] fastpath: http://tracker.bro-ids.org/bro/changeset/15033a2b626ea8d932ca6743587ead0b2bd70688/bro From bro at tracker.bro-ids.org Sun Oct 14 13:08:16 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Sun, 14 Oct 2012 20:08:16 -0000 Subject: [Bro-Dev] #891: topic/jsiwek/gridftp In-Reply-To: <048.1baa655406ff9dd5ee0406d71bac3615@tracker.bro-ids.org> References: <048.1baa655406ff9dd5ee0406d71bac3615@tracker.bro-ids.org> Message-ID: <063.d5d0d5a5063a6b21b54e1930587efae8@tracker.bro-ids.org> #891: topic/jsiwek/gridftp ----------------------------+------------------------ Reporter: jsiwek | Owner: Type: Merge Request | Status: new Priority: Normal | Milestone: Bro2.2 Component: Bro | Version: git/master Resolution: | Keywords: ----------------------------+------------------------ Comment (by robin): Ok, storing the client side information sounds fine. -- Ticket URL: Bro Tracker Bro Issue Tracker From noreply at bro-ids.org Mon Oct 15 00:00:02 2012 From: noreply at bro-ids.org (Merge Tracker) Date: Mon, 15 Oct 2012 00:00:02 -0700 Subject: [Bro-Dev] [Auto] Merge Status Message-ID: <201210150700.q9F702As026769@bro-ids.icir.org> > Open Merge Requests for Bro2.2 > ============================== Component | Id | Reporter | Owner | Prio | Summary ------------------------------------------------------------------------------------------------------------------ Bro | 891 [1] | jsiwek | | Normal | topic/jsiwek/gridftp [2] [1] #891: http://tracker.bro-ids.org/bro/ticket/891 [2] gridftp: http://tracker.bro-ids.org/bro/changeset?old_path=%2Fbro&old=master&new_path=%2Fbro&new=topic/jsiwek/gridftp From bro at tracker.bro-ids.org Mon Oct 15 06:39:23 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Mon, 15 Oct 2012 13:39:23 -0000 Subject: [Bro-Dev] #891: topic/jsiwek/gridftp In-Reply-To: <048.1baa655406ff9dd5ee0406d71bac3615@tracker.bro-ids.org> References: <048.1baa655406ff9dd5ee0406d71bac3615@tracker.bro-ids.org> Message-ID: <063.0fe010129d956b4ba19cd297d1899afb@tracker.bro-ids.org> #891: topic/jsiwek/gridftp ---------------------+------------------------ Reporter: jsiwek | Owner: Type: Task | Status: new Priority: Normal | Milestone: Bro2.2 Component: Bro | Version: git/master Resolution: | Keywords: ---------------------+------------------------ Changes (by jsiwek): * type: Merge Request => Task Comment: Ok, will change how "gridftp" is added to c$service. -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Mon Oct 15 11:04:52 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Mon, 15 Oct 2012 18:04:52 -0000 Subject: [Bro-Dev] #899: Make the archive log compression configurable Message-ID: <048.fa2bccdfd788af68c4f45fb3911686c9@tracker.bro-ids.org> #899: Make the archive log compression configurable ------------------------+---------------------------------- Reporter: justin | Type: Feature Request Status: new | Priority: Low Milestone: Bro2.2 | Component: BroControl Version: git/master | Keywords: log archive compress ------------------------+---------------------------------- This makes the currently harcoded 'gzip -9' configurable. -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Tue Oct 16 08:57:01 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Tue, 16 Oct 2012 15:57:01 -0000 Subject: [Bro-Dev] #888: Bro crashes on IPv6 subnet [::FFFF:0:0]/96 In-Reply-To: <054.da51ec8ed0c0114143b25b45dd4f9e5a@tracker.bro-ids.org> References: <054.da51ec8ed0c0114143b25b45dd4f9e5a@tracker.bro-ids.org> Message-ID: <069.0fa629f8f5c784a7a84bbc66a25a6e1b@tracker.bro-ids.org> #888: Bro crashes on IPv6 subnet [::FFFF:0:0]/96 ---------------------------+------------------------ Reporter: sheharbano.k | Owner: Type: Problem | Status: new Priority: High | Milestone: Bro2.2 Component: Bro | Version: git/master Resolution: | Keywords: ---------------------------+------------------------ Comment (by jsiwek): Replying to [ticket:888 sheharbano.k]: > Bro cannot handle the subnet [::FFFF:0:0]/96 (IPv4 mapped IPv6 addresses-sec 2.2 of rfc5156). The error message says: > internal error: Bad IPAddr(v4) IPPrefix length : 96 > Aborted > > Works fine on other subnets. > > --------------------------------- > test.bro > --------------------------------- > > event bro_init() > { > local snet = [::FFFF:0:0]/96; > #local snet = [1::]/96; > print snet; > } The problem here is that the internal representation of an IPv4 address uses the IPv4-mapped IPv6 representation and that the parser doesn't have a rule to make literal constants of type subnet. Rather, the subnet value is the result of a division expression with a left operand being a literal constant of addr type and the right operand being one of the integral types. In this case, the address is taken as an IPv4 addr and doesn't care to remember that it came from the mapped representation, so the max mask length would be 32 and 96 is exceeds that. I'm thinking the fix is to teach the parser rules to make subnet literal constants (I have to add these rules to the signature parser anyway), but I don't know if that helps you do what you were expecting to do with the `[::FFFF:0:0]/96` subnet. If the intention is to just detect if an address is within the IPv4 space, then that fix is ok (or you could instead use the `is_v4_addr()` BIF or just the `0.0.0.0/0` subnet). If the intention is to see whether an IPv4-mapped IPv6 address was used on the wire (I think that's generally considered a no-no), that fix doesn't help, and using such a subnet wouldn't be the way to do it (instead you might be able to check for an IPv6 packet header that contains "IPv4" src/dst fields). So I think adding the parser rules is fine regardless of the intention, but more discussion/work is necessary if the second usage of that subnet is what is wanted. Thoughts? -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Tue Oct 16 11:15:41 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Tue, 16 Oct 2012 18:15:41 -0000 Subject: [Bro-Dev] #900: reference versions of startup scripts Message-ID: <051.f3a13383cd869aa4caeeb6c8347ff7c9@tracker.bro-ids.org> #900: reference versions of startup scripts ------------------------+--------------------- Reporter: scampbell | Type: Problem Status: new | Priority: Normal Milestone: Bro2.2 | Component: bro-aux Version: git/master | Keywords: ------------------------+--------------------- Would be handy to include a pair of general startup scripts for linux and bsd. I realize this is a lesser problem in the big scheme of bro, but standardizing on these things can be quite nice. cheers -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Wed Oct 17 09:13:42 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Wed, 17 Oct 2012 16:13:42 -0000 Subject: [Bro-Dev] #880: Cannot do signature matching for ICMP payload In-Reply-To: <054.d1302df7172c22b24b96560b2aef73c6@tracker.bro-ids.org> References: <054.d1302df7172c22b24b96560b2aef73c6@tracker.bro-ids.org> Message-ID: <069.d415ff8630bdc48e590bbba44c6db5b7@tracker.bro-ids.org> #880: Cannot do signature matching for ICMP payload ------------------------------+------------------------ Reporter: sheharbano.k | Owner: Type: Feature Request | Status: new Priority: Normal | Milestone: Bro2.2 Component: Bro | Version: git/master Resolution: | Keywords: ------------------------------+------------------------ Comment (by jsiwek): In [e835a55229315f61e6994811b0eb6423f14c905a/bro]: {{{ #!CommitTicketReference repository="bro" revision="e835a55229315f61e6994811b0eb6423f14c905a" Add IPv6 support to signature header conditions. - "src-ip" and "dst-ip" conditions can now use IPv6 addresses/subnets. They must be written in colon-hexadecimal representation and enclosed in square brackets (e.g. [fe80::1]). Addresses #774. - "icmp6" is now a valid protocol for use with "ip-proto" and "header" conditions. This allows signatures to be written that can match against ICMPv6 payloads. Addresses #880. - "ip6" is now a valid protocol for use with the "header" condition. (also the "ip-proto" condition, but it results in a no-op in that case since signatures apply only to the inner-most IP packet when packets are tunneled). This allows signatures to match specifically against IPv6 packets (whereas "ip" only matches against IPv4 packets). - "ip-proto" conditions can now match against IPv6 packets. Before, IPv6 packets were just silently ignored which meant DPD based on signatures did not function for IPv6 -- protocol analyzers would only get attached to a connection over IPv6 based on the well-known ports set in the "dpd_config" table. }}} -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Wed Oct 17 09:13:43 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Wed, 17 Oct 2012 16:13:43 -0000 Subject: [Bro-Dev] #774: IPv6 in signatures In-Reply-To: <046.e3fc459c715e558f55a8d0d7807eaeac@tracker.bro-ids.org> References: <046.e3fc459c715e558f55a8d0d7807eaeac@tracker.bro-ids.org> Message-ID: <061.d136fcfec43a86902b0e560d2b8b3761@tracker.bro-ids.org> #774: IPv6 in signatures ----------------------+------------------------ Reporter: seth | Owner: Type: Problem | Status: new Priority: Low | Milestone: Bro2.2 Component: Bro | Version: git/master Resolution: | Keywords: ipv6 ----------------------+------------------------ Comment (by jsiwek): In [e835a55229315f61e6994811b0eb6423f14c905a/bro]: {{{ #!CommitTicketReference repository="bro" revision="e835a55229315f61e6994811b0eb6423f14c905a" Add IPv6 support to signature header conditions. - "src-ip" and "dst-ip" conditions can now use IPv6 addresses/subnets. They must be written in colon-hexadecimal representation and enclosed in square brackets (e.g. [fe80::1]). Addresses #774. - "icmp6" is now a valid protocol for use with "ip-proto" and "header" conditions. This allows signatures to be written that can match against ICMPv6 payloads. Addresses #880. - "ip6" is now a valid protocol for use with the "header" condition. (also the "ip-proto" condition, but it results in a no-op in that case since signatures apply only to the inner-most IP packet when packets are tunneled). This allows signatures to match specifically against IPv6 packets (whereas "ip" only matches against IPv4 packets). - "ip-proto" conditions can now match against IPv6 packets. Before, IPv6 packets were just silently ignored which meant DPD based on signatures did not function for IPv6 -- protocol analyzers would only get attached to a connection over IPv6 based on the well-known ports set in the "dpd_config" table. }}} -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Wed Oct 17 09:24:45 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Wed, 17 Oct 2012 16:24:45 -0000 Subject: [Bro-Dev] #901: topic/jsiwek/ipv6-sigs Message-ID: <048.15f84e594f19e020748d3a81c88c465f@tracker.bro-ids.org> #901: topic/jsiwek/ipv6-sigs ---------------------------+------------------------ Reporter: jsiwek | Owner: Type: Merge Request | Status: new Priority: Normal | Milestone: Bro2.2 Component: Bro | Version: git/master Keywords: | ---------------------------+------------------------ This branch is just in the `bro` repo and adds IPv6 support to signature header conditions. See commit message of [e835a55229315f61e6994811b0eb6423f14c905a/bro] for details. I think merging this would close #774 and #880. -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Wed Oct 17 09:53:15 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Wed, 17 Oct 2012 16:53:15 -0000 Subject: [Bro-Dev] #902: Error loading __load__.bro/main.bro if dir containing these scripts is not in the site directory Message-ID: <049.68992989cc6a92f7bf297f8f889fde62@tracker.bro-ids.org> #902: Error loading __load__.bro/main.bro if dir containing these scripts is not in the site directory ------------------------+--------------------- Reporter: aashish | Type: Problem Status: new | Priority: Normal Milestone: Bro2.2 | Component: Bro Version: git/master | Keywords: ------------------------+--------------------- __load__.bro and main.bro won't load if the folder containing these is not in the standard site directory even though this folders location is specified via SitePolicyPath. Explanation: I encountered this problem while trying to get ssn-exposure running. @load ssn-exposure in local.bro works just fine if folder (ssn-exposure) is in the site directory. However if this same folder (ssn-exposure) is moved to a different/non- standard directory (eg. /usr/local/my-scripts) @load ssn-exposure would give errors and not load; even though /usr/local/my-scripts is added to SitePolicyPath in broctl config and other standalone scripts within my- scripts folder load properly. ssn-exposure (https://github.com/sethhall/ssn-exposure) is an example. This seems to be a general issue with any similar folder encapsulated scripts. -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Wed Oct 17 10:16:24 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Wed, 17 Oct 2012 17:16:24 -0000 Subject: [Bro-Dev] #891: topic/jsiwek/gridftp In-Reply-To: <048.1baa655406ff9dd5ee0406d71bac3615@tracker.bro-ids.org> References: <048.1baa655406ff9dd5ee0406d71bac3615@tracker.bro-ids.org> Message-ID: <063.3ece2dcc164b78e8141f532b3cc40d75@tracker.bro-ids.org> #891: topic/jsiwek/gridftp ---------------------+------------------------ Reporter: jsiwek | Owner: Type: Task | Status: new Priority: Normal | Milestone: Bro2.2 Component: Bro | Version: git/master Resolution: | Keywords: ---------------------+------------------------ Comment (by jsiwek): In [18f84275793e3eb50f0fb23c9978621cddac4117/bro]: {{{ #!CommitTicketReference repository="bro" revision="18f84275793e3eb50f0fb23c9978621cddac4117" Change how "gridftp" gets added to service field of connection records. In addition to checking for a finished SSL handshake over an FTP connection, it now also requires that the SSL handshake occurs after the FTP client requested AUTH GSSAPI, more specifically identifying the characteristics of GridFTP control channels. Addresses #891. }}} -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Wed Oct 17 10:17:31 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Wed, 17 Oct 2012 17:17:31 -0000 Subject: [Bro-Dev] #891: topic/jsiwek/gridftp In-Reply-To: <048.1baa655406ff9dd5ee0406d71bac3615@tracker.bro-ids.org> References: <048.1baa655406ff9dd5ee0406d71bac3615@tracker.bro-ids.org> Message-ID: <063.4a97054ca76f999250c779720cd3fbb1@tracker.bro-ids.org> #891: topic/jsiwek/gridftp ----------------------------+------------------------ Reporter: jsiwek | Owner: Type: Merge Request | Status: new Priority: Normal | Milestone: Bro2.2 Component: Bro | Version: git/master Resolution: | Keywords: ----------------------------+------------------------ Changes (by jsiwek): * type: Task => Merge Request Comment: Replying to [comment:7 jsiwek]: > Ok, will change how "gridftp" is added to c$service. Did that in a commit to `topic/jsiwek/gridftp` in `bro` repo. -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Wed Oct 17 12:54:48 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Wed, 17 Oct 2012 19:54:48 -0000 Subject: [Bro-Dev] #902: Error loading __load__.bro/main.bro if dir containing these scripts is not in the site directory In-Reply-To: <049.68992989cc6a92f7bf297f8f889fde62@tracker.bro-ids.org> References: <049.68992989cc6a92f7bf297f8f889fde62@tracker.bro-ids.org> Message-ID: <064.cd56e4c624b0c637b1144b0070c0f9dc@tracker.bro-ids.org> #902: Error loading __load__.bro/main.bro if dir containing these scripts is not in the site directory ----------------------+------------------------ Reporter: aashish | Owner: Type: Problem | Status: new Priority: Normal | Milestone: Bro2.2 Component: Bro | Version: git/master Resolution: | Keywords: ----------------------+------------------------ Comment (by jsiwek): In [f71f50d9d14d43ff547c0891377d0d630d0b61b3/broctl]: {{{ #!CommitTicketReference repository="broctl" revision="f71f50d9d14d43ff547c0891377d0d630d0b61b3" Fix `broctl install` to now also copy subdirs of dirs in SitePolicyPath. Paths in SitePolicyPath were expected to be a flat directory of bro scripts, so adding a new path to the default SitePolicyPath and then adding bro script "module" directories to it would not be correctly installed by BroControl. Instead of silently ignoring subdirs inside a path in SitePolicyPath, the install command now recursively copies them to the right place. Addresses #902. }}} -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Wed Oct 17 12:56:47 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Wed, 17 Oct 2012 19:56:47 -0000 Subject: [Bro-Dev] #902: Error loading __load__.bro/main.bro if dir containing these scripts is not in the site directory In-Reply-To: <049.68992989cc6a92f7bf297f8f889fde62@tracker.bro-ids.org> References: <049.68992989cc6a92f7bf297f8f889fde62@tracker.bro-ids.org> Message-ID: <064.84564c92bc38fe90d1e3df9cc1914e5e@tracker.bro-ids.org> #902: Error loading __load__.bro/main.bro if dir containing these scripts is not in the site directory ----------------------------+------------------------ Reporter: aashish | Owner: Type: Merge Request | Status: new Priority: Normal | Milestone: Bro2.2 Component: Bro | Version: git/master Resolution: | Keywords: ----------------------------+------------------------ Changes (by jsiwek): * type: Problem => Merge Request Comment: Think I fixed this with `topic/jsiwek/install-custom-site-dirs` in `broctl` repo. -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Wed Oct 17 18:24:54 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Thu, 18 Oct 2012 01:24:54 -0000 Subject: [Bro-Dev] #903: -b turns off -f Message-ID: <046.26a7f9b91032b96af44682bb790af096@tracker.bro-ids.org> #903: -b turns off -f ---------------------+------------------------ Reporter: vern | Owner: Type: Problem | Status: new Priority: Normal | Milestone: Bro2.2 Component: Bro | Version: git/master Keywords: | ---------------------+------------------------ Running with -b (bare bones) disables processing by -f. Boy did this take me a long time to figure out :-(. Reproduce using the appended trace. Invoking with '''-e 'event connection_established(c:connection) { print "yep"; }'''' will print "yep". Invoking with that plus '''-f 'not tcp'''' won't print anything. But invoking with '''-f 'not tcp' -b''' ''does'' print "yep". -- Ticket URL: Bro Tracker Bro Issue Tracker From noreply at bro-ids.org Thu Oct 18 00:00:03 2012 From: noreply at bro-ids.org (Merge Tracker) Date: Thu, 18 Oct 2012 00:00:03 -0700 Subject: [Bro-Dev] [Auto] Merge Status Message-ID: <201210180700.q9I703lN020217@bro-ids.icir.org> > Open Merge Requests for Bro2.2 > ============================== Component | Id | Reporter | Owner | Prio | Summary ------------------------------------------------------------------------------------------------------------------ Bro | 891 [1] | jsiwek | | Normal | topic/jsiwek/gridftp [2] Bro | 901 [3] | jsiwek | | Normal | topic/jsiwek/ipv6-sigs [4] Bro | 902 [5] | aashish | | Normal | Error loading __load__.bro/main.bro if dir containing these scripts is not in the site directory [1] #891: http://tracker.bro-ids.org/bro/ticket/891 [2] gridftp: http://tracker.bro-ids.org/bro/changeset?old_path=%2Fbro&old=master&new_path=%2Fbro&new=topic/jsiwek/gridftp [3] #901: http://tracker.bro-ids.org/bro/ticket/901 [4] ipv6-sigs: http://tracker.bro-ids.org/bro/changeset?old_path=%2Fbro&old=master&new_path=%2Fbro&new=topic/jsiwek/ipv6-sigs [5] #902: http://tracker.bro-ids.org/bro/ticket/902 From bro at tracker.bro-ids.org Thu Oct 18 09:21:01 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Thu, 18 Oct 2012 16:21:01 -0000 Subject: [Bro-Dev] #902: Error loading __load__.bro/main.bro if dir containing these scripts is not in the site directory In-Reply-To: <049.68992989cc6a92f7bf297f8f889fde62@tracker.bro-ids.org> References: <049.68992989cc6a92f7bf297f8f889fde62@tracker.bro-ids.org> Message-ID: <064.7ed5097d1a98ca139b0870ecc05a8b22@tracker.bro-ids.org> #902: Error loading __load__.bro/main.bro if dir containing these scripts is not in the site directory ----------------------------+------------------------ Reporter: aashish | Owner: Type: Merge Request | Status: new Priority: Normal | Milestone: Bro2.2 Component: BroControl | Version: git/master Resolution: | Keywords: ----------------------------+------------------------ Changes (by dnthayer): * component: Bro => BroControl -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Thu Oct 18 14:19:29 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Thu, 18 Oct 2012 21:19:29 -0000 Subject: [Bro-Dev] #899: Make the archive log compression configurable In-Reply-To: <048.fa2bccdfd788af68c4f45fb3911686c9@tracker.bro-ids.org> References: <048.fa2bccdfd788af68c4f45fb3911686c9@tracker.bro-ids.org> Message-ID: <063.43e817abf557f9e6f0d573fa2c8fb90a@tracker.bro-ids.org> #899: Make the archive log compression configurable ----------------------------+---------------------------------- Reporter: justin | Owner: Type: Merge Request | Status: new Priority: Low | Milestone: Bro2.2 Component: BroControl | Version: git/master Resolution: | Keywords: log archive compress ----------------------------+---------------------------------- Changes (by dnthayer): * type: Feature Request => Merge Request Comment: Applied the patch in branch topic/dnthayer/compress-options, and improved the documentation of this feature. -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Thu Oct 18 23:09:06 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Fri, 19 Oct 2012 06:09:06 -0000 Subject: [Bro-Dev] #904: topic/seth/strptime Message-ID: <046.a1e4c8c1ef538d3000129d961cb7ce5a@tracker.bro-ids.org> #904: topic/seth/strptime ---------------------------+------------------------ Reporter: seth | Owner: robin Type: Merge Request | Status: new Priority: Normal | Milestone: Bro2.2 Component: Bro | Version: git/master Keywords: | ---------------------------+------------------------ Added a new strptime BiF to support parsing arbitrary textual time representations into Bro time values. -- Ticket URL: Bro Tracker Bro Issue Tracker From noreply at bro-ids.org Fri Oct 19 00:00:02 2012 From: noreply at bro-ids.org (Merge Tracker) Date: Fri, 19 Oct 2012 00:00:02 -0700 Subject: [Bro-Dev] [Auto] Merge Status Message-ID: <201210190700.q9J702oR001656@bro-ids.icir.org> > Open Merge Requests for Bro2.2 > ============================== Component | Id | Reporter | Owner | Prio | Summary ------------------------------------------------------------------------------------------------------------------ Bro | 891 [1] | jsiwek | | Normal | topic/jsiwek/gridftp [2] Bro | 901 [3] | jsiwek | | Normal | topic/jsiwek/ipv6-sigs [4] Bro | 904 [5] | seth | robin | Normal | topic/seth/strptime [6] BroControl | 899 [7] | justin | | Low | Make the archive log compression configurable BroControl | 902 [8] | aashish | | Normal | Error loading __load__.bro/main.bro if dir containing these scripts is not in the site directory [1] #891: http://tracker.bro-ids.org/bro/ticket/891 [2] gridftp: http://tracker.bro-ids.org/bro/changeset?old_path=%2Fbro&old=master&new_path=%2Fbro&new=topic/jsiwek/gridftp [3] #901: http://tracker.bro-ids.org/bro/ticket/901 [4] ipv6-sigs: http://tracker.bro-ids.org/bro/changeset?old_path=%2Fbro&old=master&new_path=%2Fbro&new=topic/jsiwek/ipv6-sigs [5] #904: http://tracker.bro-ids.org/bro/ticket/904 [6] strptime: http://tracker.bro-ids.org/bro/changeset?old_path=%2Fbro&old=master&new_path=%2Fbro&new=topic/seth/strptime [7] #899: http://tracker.bro-ids.org/bro/ticket/899 [8] #902: http://tracker.bro-ids.org/bro/ticket/902 From bro at tracker.bro-ids.org Fri Oct 19 12:11:17 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Fri, 19 Oct 2012 19:11:17 -0000 Subject: [Bro-Dev] #690: GTP de-tunneling In-Reply-To: <047.bf6216f5c5ffabe44fd63ab9aa939d04@tracker.bro-ids.org> References: <047.bf6216f5c5ffabe44fd63ab9aa939d04@tracker.bro-ids.org> Message-ID: <062.0ead9e1307d85b6fe53b0ec99f8bf9fe@tracker.bro-ids.org> #690: GTP de-tunneling ----------------------+------------------------ Reporter: robin | Owner: Type: Problem | Status: new Priority: Normal | Milestone: 2.1 Component: Bro | Version: git/master Resolution: | Keywords: ----------------------+------------------------ Comment (by jsiwek): In [9edbf3e53c849635de62042afc82c0c5d0e128fc/bro]: {{{ #!CommitTicketReference repository="bro" revision="9edbf3e53c849635de62042afc82c0c5d0e128fc" Add GPRS Tunnelling Protocol (GTPv1) decapsulation. This currently supports automatic decapsulation of GTP-U packets on UDP port 2152. The GTPv1 headers for such tunnels can be inspected by handling the "gtpv1_g_pdu_packet" event, which has a parameter of type "gtpv1_hdr". Analyzer and test cases are derived from submissions by Carsten Langer. Addresses #690. }}} -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Fri Oct 19 13:48:48 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Fri, 19 Oct 2012 20:48:48 -0000 Subject: [Bro-Dev] #876: BroControl "diag" emails In-Reply-To: <046.b720fcd2aac59166bd3a426e1051d076@tracker.bro-ids.org> References: <046.b720fcd2aac59166bd3a426e1051d076@tracker.bro-ids.org> Message-ID: <061.7596f67ed8b91c7b9bd522d59c95ea32@tracker.bro-ids.org> #876: BroControl "diag" emails ------------------------------+------------------------ Reporter: seth | Owner: dnthayer Type: Feature Request | Status: new Priority: Normal | Milestone: Bro2.2 Component: BroControl | Version: git/master Resolution: | Keywords: ------------------------------+------------------------ Comment (by dnthayer): In [2c3754ba655b91d5ff96efab3263638c6fe576fb/broctl]: {{{ #!CommitTicketReference repository="broctl" revision="2c3754ba655b91d5ff96efab3263638c6fe576fb" Add forwarding message to crash emails Add a message at the top of broctl-generated crash report emails that explains how to submit the crash report to a mailing list address. Addresses #876. }}} -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Fri Oct 19 15:03:17 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Fri, 19 Oct 2012 22:03:17 -0000 Subject: [Bro-Dev] #876: BroControl "diag" emails In-Reply-To: <046.b720fcd2aac59166bd3a426e1051d076@tracker.bro-ids.org> References: <046.b720fcd2aac59166bd3a426e1051d076@tracker.bro-ids.org> Message-ID: <061.4e6de8816fba4260d10f2c451bc79944@tracker.bro-ids.org> #876: BroControl "diag" emails ----------------------------+------------------------ Reporter: seth | Owner: dnthayer Type: Merge Request | Status: new Priority: Normal | Milestone: Bro2.2 Component: BroControl | Version: git/master Resolution: | Keywords: ----------------------------+------------------------ Changes (by dnthayer): * type: Feature Request => Merge Request Comment: In branch topic/dnthayer/forward-crash-email, added a message that is added to the top of each crash report email. -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Fri Oct 19 15:45:32 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Fri, 19 Oct 2012 22:45:32 -0000 Subject: [Bro-Dev] #891: topic/jsiwek/gridftp In-Reply-To: <048.1baa655406ff9dd5ee0406d71bac3615@tracker.bro-ids.org> References: <048.1baa655406ff9dd5ee0406d71bac3615@tracker.bro-ids.org> Message-ID: <063.fb0a79c18b81d09fb3c8ffa9a85abad6@tracker.bro-ids.org> #891: topic/jsiwek/gridftp ----------------------------+------------------------ Reporter: jsiwek | Owner: robin Type: Merge Request | Status: closed Priority: Normal | Milestone: Bro2.2 Component: Bro | Version: git/master Resolution: fixed | Keywords: ----------------------------+------------------------ Changes (by robin): * owner: => robin * status: new => closed * resolution: => fixed Comment: In [c7b39efe858665f6aebc2cd554077ecd2a72bf03/bro]: {{{ #!CommitTicketReference repository="bro" revision="c7b39efe858665f6aebc2cd554077ecd2a72bf03" Merge remote-tracking branch 'origin/topic/jsiwek/gridftp' * origin/topic/jsiwek/gridftp: Change how "gridftp" gets added to service field of connection records. Closes #891. }}} -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Fri Oct 19 15:45:32 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Fri, 19 Oct 2012 22:45:32 -0000 Subject: [Bro-Dev] #880: Cannot do signature matching for ICMP payload In-Reply-To: <054.d1302df7172c22b24b96560b2aef73c6@tracker.bro-ids.org> References: <054.d1302df7172c22b24b96560b2aef73c6@tracker.bro-ids.org> Message-ID: <069.c8b9a832d0ca2cbea1662e860c2e3f26@tracker.bro-ids.org> #880: Cannot do signature matching for ICMP payload ------------------------------+------------------------ Reporter: sheharbano.k | Owner: robin Type: Feature Request | Status: closed Priority: Normal | Milestone: Bro2.2 Component: Bro | Version: git/master Resolution: fixed | Keywords: ------------------------------+------------------------ Changes (by robin): * owner: => robin * status: new => closed * resolution: => fixed Comment: In [b9811e87e51e3e5235103af4e282a3ff6317ff5a/bro]: {{{ #!CommitTicketReference repository="bro" revision="b9811e87e51e3e5235103af4e282a3ff6317ff5a" Merge remote-tracking branch 'origin/topic/jsiwek/ipv6-sigs' * origin/topic/jsiwek/ipv6-sigs: Add IPv6 support to signature header conditions. Closes #774. Closes #880. }}} -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Fri Oct 19 15:45:32 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Fri, 19 Oct 2012 22:45:32 -0000 Subject: [Bro-Dev] #774: IPv6 in signatures In-Reply-To: <046.e3fc459c715e558f55a8d0d7807eaeac@tracker.bro-ids.org> References: <046.e3fc459c715e558f55a8d0d7807eaeac@tracker.bro-ids.org> Message-ID: <061.df15236f4eae4e812e9e7ba015f42846@tracker.bro-ids.org> #774: IPv6 in signatures ----------------------+------------------------ Reporter: seth | Owner: robin Type: Problem | Status: closed Priority: Low | Milestone: Bro2.2 Component: Bro | Version: git/master Resolution: fixed | Keywords: ipv6 ----------------------+------------------------ Changes (by robin): * owner: => robin * status: new => closed * resolution: => fixed Comment: In [b9811e87e51e3e5235103af4e282a3ff6317ff5a/bro]: {{{ #!CommitTicketReference repository="bro" revision="b9811e87e51e3e5235103af4e282a3ff6317ff5a" Merge remote-tracking branch 'origin/topic/jsiwek/ipv6-sigs' * origin/topic/jsiwek/ipv6-sigs: Add IPv6 support to signature header conditions. Closes #774. Closes #880. }}} -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Fri Oct 19 15:45:32 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Fri, 19 Oct 2012 22:45:32 -0000 Subject: [Bro-Dev] #904: topic/seth/strptime In-Reply-To: <046.a1e4c8c1ef538d3000129d961cb7ce5a@tracker.bro-ids.org> References: <046.a1e4c8c1ef538d3000129d961cb7ce5a@tracker.bro-ids.org> Message-ID: <061.3296ecaf632ba1ba85cbf6477661f556@tracker.bro-ids.org> #904: topic/seth/strptime ----------------------------+------------------------ Reporter: seth | Owner: robin Type: Merge Request | Status: closed Priority: Normal | Milestone: Bro2.2 Component: Bro | Version: git/master Resolution: fixed | Keywords: ----------------------------+------------------------ Changes (by robin): * status: new => closed * resolution: => fixed Comment: In [5716545cfa8f67e35aa3e9b43e138d7ff184bf4e/bro]: {{{ #!CommitTicketReference repository="bro" revision="5716545cfa8f67e35aa3e9b43e138d7ff184bf4e" Merge remote-tracking branch 'origin/topic/seth/strptime' * origin/topic/seth/strptime: Added a BiF to wrap the strptime function. Closes #904. }}} -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Fri Oct 19 15:45:46 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Fri, 19 Oct 2012 22:45:46 -0000 Subject: [Bro-Dev] #899: Make the archive log compression configurable In-Reply-To: <048.fa2bccdfd788af68c4f45fb3911686c9@tracker.bro-ids.org> References: <048.fa2bccdfd788af68c4f45fb3911686c9@tracker.bro-ids.org> Message-ID: <063.dbfe5af0b788f3f47ce05ebcca2e4e4c@tracker.bro-ids.org> #899: Make the archive log compression configurable ----------------------------+---------------------------------- Reporter: justin | Owner: robin Type: Merge Request | Status: closed Priority: Low | Milestone: Bro2.2 Component: BroControl | Version: git/master Resolution: fixed | Keywords: log archive compress ----------------------------+---------------------------------- Changes (by robin): * owner: => robin * status: new => closed * resolution: => fixed Comment: In [3d7c2d61e63d7d51c455363cdfe9373e4e680b89/broctl]: {{{ #!CommitTicketReference repository="broctl" revision="3d7c2d61e63d7d51c455363cdfe9373e4e680b89" Merge remote-tracking branch 'origin/topic/dnthayer/compress-options' * origin/topic/dnthayer/compress-options: Fix docs and comments related to log compression Add log compress cmd and extension options Closes #899. }}} -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Fri Oct 19 15:45:46 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Fri, 19 Oct 2012 22:45:46 -0000 Subject: [Bro-Dev] #902: Error loading __load__.bro/main.bro if dir containing these scripts is not in the site directory In-Reply-To: <049.68992989cc6a92f7bf297f8f889fde62@tracker.bro-ids.org> References: <049.68992989cc6a92f7bf297f8f889fde62@tracker.bro-ids.org> Message-ID: <064.1c6974f1280047a51c91f42cce83d4aa@tracker.bro-ids.org> #902: Error loading __load__.bro/main.bro if dir containing these scripts is not in the site directory ----------------------------+------------------------ Reporter: aashish | Owner: robin Type: Merge Request | Status: closed Priority: Normal | Milestone: Bro2.2 Component: BroControl | Version: git/master Resolution: fixed | Keywords: ----------------------------+------------------------ Changes (by robin): * owner: => robin * status: new => closed * resolution: => fixed Comment: In [fd0e7e0b0cf50131efaf536a5683266cfe169455/broctl]: {{{ #!CommitTicketReference repository="broctl" revision="fd0e7e0b0cf50131efaf536a5683266cfe169455" Merge remote-tracking branch 'origin/topic/jsiwek/install-custom-site- dirs' * origin/topic/jsiwek/install-custom-site-dirs: Fix `broctl install` to now also copy subdirs of dirs in SitePolicyPath. Closes #902 }}} -- Ticket URL: Bro Tracker Bro Issue Tracker From noreply at bro-ids.org Sat Oct 20 00:00:03 2012 From: noreply at bro-ids.org (Merge Tracker) Date: Sat, 20 Oct 2012 00:00:03 -0700 Subject: [Bro-Dev] [Auto] Merge Status Message-ID: <201210200700.q9K703C4008948@bro-ids.icir.org> > Open Merge Requests for Bro2.2 > ============================== Component | Id | Reporter | Owner | Prio | Summary ------------------------------------------------------------------------------------------------------------------ Bro | 901 [1] | jsiwek | | Normal | topic/jsiwek/ipv6-sigs [2] BroControl | 876 [3] | seth | dnthayer | Normal | "BroControl ""diag"" emails" [1] #901: http://tracker.bro-ids.org/bro/ticket/901 [2] ipv6-sigs: http://tracker.bro-ids.org/bro/changeset?old_path=%2Fbro&old=master&new_path=%2Fbro&new=topic/jsiwek/ipv6-sigs [3] #876: http://tracker.bro-ids.org/bro/ticket/876 From noreply at bro-ids.org Sun Oct 21 00:00:04 2012 From: noreply at bro-ids.org (Merge Tracker) Date: Sun, 21 Oct 2012 00:00:04 -0700 Subject: [Bro-Dev] [Auto] Merge Status Message-ID: <201210210700.q9L704NQ002203@bro-ids.icir.org> > Open Merge Requests for Bro2.2 > ============================== Component | Id | Reporter | Owner | Prio | Summary ------------------------------------------------------------------------------------------------------------------ Bro | 901 [1] | jsiwek | | Normal | topic/jsiwek/ipv6-sigs [2] BroControl | 876 [3] | seth | dnthayer | Normal | "BroControl ""diag"" emails" [1] #901: http://tracker.bro-ids.org/bro/ticket/901 [2] ipv6-sigs: http://tracker.bro-ids.org/bro/changeset?old_path=%2Fbro&old=master&new_path=%2Fbro&new=topic/jsiwek/ipv6-sigs [3] #876: http://tracker.bro-ids.org/bro/ticket/876 From noreply at bro-ids.org Mon Oct 22 00:00:02 2012 From: noreply at bro-ids.org (Merge Tracker) Date: Mon, 22 Oct 2012 00:00:02 -0700 Subject: [Bro-Dev] [Auto] Merge Status Message-ID: <201210220700.q9M702Mw006224@bro-ids.icir.org> > Open Merge Requests for Bro2.2 > ============================== Component | Id | Reporter | Owner | Prio | Summary ------------------------------------------------------------------------------------------------------------------ Bro | 901 [1] | jsiwek | | Normal | topic/jsiwek/ipv6-sigs [2] BroControl | 876 [3] | seth | dnthayer | Normal | "BroControl ""diag"" emails" [1] #901: http://tracker.bro-ids.org/bro/ticket/901 [2] ipv6-sigs: http://tracker.bro-ids.org/bro/changeset?old_path=%2Fbro&old=master&new_path=%2Fbro&new=topic/jsiwek/ipv6-sigs [3] #876: http://tracker.bro-ids.org/bro/ticket/876 From bro at tracker.bro-ids.org Mon Oct 22 00:51:54 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Mon, 22 Oct 2012 07:51:54 -0000 Subject: [Bro-Dev] #905: Need tcp_max_initial_window to be redefinable Message-ID: <049.1c47ee8c0297ed5eea803cd6d7d4cac1@tracker.bro-ids.org> #905: Need tcp_max_initial_window to be redefinable ------------------------+--------------------- Reporter: carsten | Type: Problem Status: new | Priority: Normal Milestone: Bro2.2 | Component: Bro Version: git/master | Keywords: ------------------------+--------------------- In my scripts, I must redefine tcp_max_initial_window from scripts/base /init-bare.bro. Currently I have to patch the source for doing it, as the constant is not redefinable from a script. The fun thing is: the documentation even says "Set to zero to turn off this determination.", which you currently cannot do from scripts. Please change inside scripts/base/init-bare.bro from const tcp_max_initial_window = 4096; to const tcp_max_initial_window = 4096 &redef; -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Mon Oct 22 00:55:51 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Mon, 22 Oct 2012 07:55:51 -0000 Subject: [Bro-Dev] #906: Need tcp_max_above_hole_without_any_acks to be redefinable Message-ID: <049.4c53fed260631285c48c0adbb9443e3f@tracker.bro-ids.org> #906: Need tcp_max_above_hole_without_any_acks to be redefinable ------------------------+--------------------- Reporter: carsten | Type: Problem Status: new | Priority: Normal Milestone: Bro2.2 | Component: Bro Version: git/master | Keywords: ------------------------+--------------------- In my scripts, I must redefine tcp_max_above_hole_without_any_acks from scripts/base/init-bare.bro. Currently I have to patch the source for doing it, as the constant is not redefinable from a script. The fun thing is: the documentation even says "If set to zero, then we don?t ever give up.", which you currently cannot do from scripts. Please change inside scripts/base/init-bare.bro from const tcp_max_above_hole_without_any_acks = 4096; to const tcp_max_above_hole_without_any_acks = 4096 &redef; -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Mon Oct 22 00:58:24 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Mon, 22 Oct 2012 07:58:24 -0000 Subject: [Bro-Dev] #907: Need tcp_excessive_data_without_further_acks to be redefinable Message-ID: <049.b80fe7a7a38edfc9c4be0b6e74c3725c@tracker.bro-ids.org> #907: Need tcp_excessive_data_without_further_acks to be redefinable ------------------------+--------------------- Reporter: carsten | Type: Problem Status: new | Priority: Normal Milestone: Bro2.2 | Component: Bro Version: git/master | Keywords: ------------------------+--------------------- In my scripts, I must redefine tcp_excessive_data_without_further_acks from scripts/base/init-bare.bro. Currently I have to patch the source for doing it, as the constant is not redefinable from a script. The fun thing is: the documentation even says "If set to zero, then we don?t ever give up.", which you currently cannot do from scripts. Please change inside scripts/base/init-bare.bro from const tcp_excessive_data_without_further_acks = 10 * 1024 * 1024; to const tcp_excessive_data_without_further_acks = 10 * 1024 * 1024 &redef; -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Mon Oct 22 08:02:42 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Mon, 22 Oct 2012 15:02:42 -0000 Subject: [Bro-Dev] #901: topic/jsiwek/ipv6-sigs In-Reply-To: <048.15f84e594f19e020748d3a81c88c465f@tracker.bro-ids.org> References: <048.15f84e594f19e020748d3a81c88c465f@tracker.bro-ids.org> Message-ID: <063.51fac26ee0f63e96f9c49d2a45037e6e@tracker.bro-ids.org> #901: topic/jsiwek/ipv6-sigs -----------------------------+------------------------ Reporter: jsiwek | Owner: Type: Merge Request | Status: closed Priority: Normal | Milestone: Bro2.2 Component: Bro | Version: git/master Resolution: Solved/Applied | Keywords: -----------------------------+------------------------ Changes (by jsiwek): * status: new => closed * resolution: => Solved/Applied Comment: Looks like this got merged, let me know if there was anything needing further work. -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Mon Oct 22 08:24:51 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Mon, 22 Oct 2012 15:24:51 -0000 Subject: [Bro-Dev] #905: Need tcp_max_initial_window to be redefinable In-Reply-To: <049.1c47ee8c0297ed5eea803cd6d7d4cac1@tracker.bro-ids.org> References: <049.1c47ee8c0297ed5eea803cd6d7d4cac1@tracker.bro-ids.org> Message-ID: <064.5ed023a4ef5222160500b03189b5fabc@tracker.bro-ids.org> #905: Need tcp_max_initial_window to be redefinable ----------------------------+------------------------ Reporter: carsten | Owner: robin Type: Merge Request | Status: assigned Priority: Normal | Milestone: Bro2.2 Component: Bro | Version: git/master Resolution: | Keywords: ----------------------------+------------------------ Changes (by robin): * owner: => robin * status: new => assigned * type: Problem => Merge Request Comment: Looks like an oversight, will fix it. -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Mon Oct 22 08:25:48 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Mon, 22 Oct 2012 15:25:48 -0000 Subject: [Bro-Dev] #906: Need tcp_max_above_hole_without_any_acks to be redefinable In-Reply-To: <049.4c53fed260631285c48c0adbb9443e3f@tracker.bro-ids.org> References: <049.4c53fed260631285c48c0adbb9443e3f@tracker.bro-ids.org> Message-ID: <064.2c48e72dacdbd3834ded8bf5dcf29c87@tracker.bro-ids.org> #906: Need tcp_max_above_hole_without_any_acks to be redefinable ----------------------------+------------------------ Reporter: carsten | Owner: robin Type: Merge Request | Status: assigned Priority: Normal | Milestone: Bro2.2 Component: Bro | Version: git/master Resolution: | Keywords: ----------------------------+------------------------ Changes (by robin): * owner: => robin * status: new => assigned * type: Problem => Merge Request -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Mon Oct 22 08:26:02 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Mon, 22 Oct 2012 15:26:02 -0000 Subject: [Bro-Dev] #907: Need tcp_excessive_data_without_further_acks to be redefinable In-Reply-To: <049.b80fe7a7a38edfc9c4be0b6e74c3725c@tracker.bro-ids.org> References: <049.b80fe7a7a38edfc9c4be0b6e74c3725c@tracker.bro-ids.org> Message-ID: <064.ef04f6446087cdcba92e22d39297d62b@tracker.bro-ids.org> #907: Need tcp_excessive_data_without_further_acks to be redefinable ----------------------------+------------------------ Reporter: carsten | Owner: robin Type: Merge Request | Status: assigned Priority: Normal | Milestone: Bro2.2 Component: Bro | Version: git/master Resolution: | Keywords: ----------------------------+------------------------ Changes (by robin): * owner: => robin * status: new => assigned * type: Problem => Merge Request -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Mon Oct 22 08:31:39 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Mon, 22 Oct 2012 15:31:39 -0000 Subject: [Bro-Dev] #907: Need tcp_excessive_data_without_further_acks to be redefinable In-Reply-To: <049.b80fe7a7a38edfc9c4be0b6e74c3725c@tracker.bro-ids.org> References: <049.b80fe7a7a38edfc9c4be0b6e74c3725c@tracker.bro-ids.org> Message-ID: <064.41c453db69a80604da3c645fdd746400@tracker.bro-ids.org> #907: Need tcp_excessive_data_without_further_acks to be redefinable ----------------------------+------------------------ Reporter: carsten | Owner: robin Type: Merge Request | Status: assigned Priority: Normal | Milestone: Bro2.2 Component: Bro | Version: git/master Resolution: | Keywords: ----------------------------+------------------------ Comment (by seth): > Please change inside scripts/base/init-bare.bro from > const tcp_excessive_data_without_further_acks = 10 * 1024 * 1024; > to > const tcp_excessive_data_without_further_acks = 10 * 1024 * 1024 > &redef; We should probably do a quick run through of that script when merging this to make sure there aren't any other non-redefable things in there. -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Mon Oct 22 09:18:39 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Mon, 22 Oct 2012 16:18:39 -0000 Subject: [Bro-Dev] #908: Need for function bytestring_to_double Message-ID: <049.360fac6a490c92ac55b12cf22ed72a91@tracker.bro-ids.org> #908: Need for function bytestring_to_double ------------------------+----------------------------- Reporter: carsten | Type: Feature Request Status: new | Priority: Normal Milestone: Bro2.2 | Component: Bro Version: git/master | Keywords: ------------------------+----------------------------- Hi, for my analysis I need a function to interpret a 8-byte bytestring from network traffic as a double value. Such function is currently missing from bro.bif. The attached addition to bro.bif works for me. Would be nice if you could include it in the regular release. -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Mon Oct 22 10:06:50 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Mon, 22 Oct 2012 17:06:50 -0000 Subject: [Bro-Dev] #690: GTP de-tunneling In-Reply-To: <047.bf6216f5c5ffabe44fd63ab9aa939d04@tracker.bro-ids.org> References: <047.bf6216f5c5ffabe44fd63ab9aa939d04@tracker.bro-ids.org> Message-ID: <062.5ed4de091845f3a5f2f6ac991d9b9c40@tracker.bro-ids.org> #690: GTP de-tunneling ----------------------------+------------------------ Reporter: robin | Owner: Type: Merge Request | Status: new Priority: Normal | Milestone: 2.1 Component: Bro | Version: git/master Resolution: | Keywords: ----------------------------+------------------------ Changes (by jsiwek): * type: Problem => Merge Request Comment: `topic/jsiwek/gtp` -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Mon Oct 22 14:04:23 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Mon, 22 Oct 2012 21:04:23 -0000 Subject: [Bro-Dev] #888: Bro crashes on IPv6 subnet [::FFFF:0:0]/96 In-Reply-To: <054.da51ec8ed0c0114143b25b45dd4f9e5a@tracker.bro-ids.org> References: <054.da51ec8ed0c0114143b25b45dd4f9e5a@tracker.bro-ids.org> Message-ID: <069.ecd2d1fb25a97f8d5f4f295918277a6d@tracker.bro-ids.org> #888: Bro crashes on IPv6 subnet [::FFFF:0:0]/96 ---------------------------+------------------------ Reporter: sheharbano.k | Owner: Type: Problem | Status: new Priority: High | Milestone: Bro2.2 Component: Bro | Version: git/master Resolution: | Keywords: ---------------------------+------------------------ Comment (by jsiwek): In [46d225cc5bd4f2cba953a14925432080615d948e/bro]: {{{ #!CommitTicketReference repository="bro" revision="46d225cc5bd4f2cba953a14925432080615d948e" Add parsing rules for IPv4/IPv6 subnet literal constants, addresses #888 This fixes specifying IPv4 subnets in IPv4-mapped-IPv6 format with a mask length relative to the 128 bits of the mapped IPv6 address. }}} -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Mon Oct 22 14:06:45 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Mon, 22 Oct 2012 21:06:45 -0000 Subject: [Bro-Dev] #888: Bro crashes on IPv6 subnet [::FFFF:0:0]/96 In-Reply-To: <054.da51ec8ed0c0114143b25b45dd4f9e5a@tracker.bro-ids.org> References: <054.da51ec8ed0c0114143b25b45dd4f9e5a@tracker.bro-ids.org> Message-ID: <069.73da2da96b7e4278caaf96bf25445d40@tracker.bro-ids.org> #888: Bro crashes on IPv6 subnet [::FFFF:0:0]/96 ----------------------------+------------------------ Reporter: sheharbano.k | Owner: Type: Merge Request | Status: new Priority: High | Milestone: Bro2.2 Component: Bro | Version: git/master Resolution: | Keywords: ----------------------------+------------------------ Changes (by jsiwek): * type: Problem => Merge Request Comment: Proposed fix is in `topic/jsiwek/subnet-literal-const`. -- Ticket URL: Bro Tracker Bro Issue Tracker From vern at icir.org Mon Oct 22 21:41:02 2012 From: vern at icir.org (Vern Paxson) Date: Mon, 22 Oct 2012 21:41:02 -0700 Subject: [Bro-Dev] Bro history Message-ID: <20121023044102.7BB842C4003@rock.ICSI.Berkeley.EDU> Luckily, looks like someone has already written this up for us: http://mnstream.blogspot.com/2008/03/history-of-bro-part-one-past-attire.html - Vern From vallentin at icir.org Mon Oct 22 23:36:04 2012 From: vallentin at icir.org (Matthias Vallentin) Date: Mon, 22 Oct 2012 23:36:04 -0700 Subject: [Bro-Dev] Bro history In-Reply-To: <20121023044102.7BB842C4003@rock.ICSI.Berkeley.EDU> References: <20121023044102.7BB842C4003@rock.ICSI.Berkeley.EDU> Message-ID: > Luckily, looks like someone has already written this up for us: These Bro's Are Not Berkeley Enough [1]. Matthias [1] http://youtu.be/UysN1vMkmZE From noreply at bro-ids.org Tue Oct 23 00:00:04 2012 From: noreply at bro-ids.org (Merge Tracker) Date: Tue, 23 Oct 2012 00:00:04 -0700 Subject: [Bro-Dev] [Auto] Merge Status Message-ID: <201210230700.q9N704UP013279@bro-ids.icir.org> > Open Merge Requests for Bro2.2 > ============================== Component | Id | Reporter | Owner | Prio | Summary ------------------------------------------------------------------------------------------------------------------ Bro | 888 [1] | sheharbano.k | | High | Bro crashes on IPv6 subnet [::FFFF:0:0]/96 Bro | 905 [2] | carsten | robin | Normal | Need tcp_max_initial_window to be redefinable Bro | 906 [3] | carsten | robin | Normal | Need tcp_max_above_hole_without_any_acks to be redefinable Bro | 907 [4] | carsten | robin | Normal | Need tcp_excessive_data_without_further_acks to be redefinable BroControl | 876 [5] | seth | dnthayer | Normal | "BroControl ""diag"" emails" [1] #888: http://tracker.bro-ids.org/bro/ticket/888 [2] #905: http://tracker.bro-ids.org/bro/ticket/905 [3] #906: http://tracker.bro-ids.org/bro/ticket/906 [4] #907: http://tracker.bro-ids.org/bro/ticket/907 [5] #876: http://tracker.bro-ids.org/bro/ticket/876 From bro at tracker.bro-ids.org Tue Oct 23 07:35:42 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Tue, 23 Oct 2012 14:35:42 -0000 Subject: [Bro-Dev] #690: GTP de-tunneling In-Reply-To: <047.bf6216f5c5ffabe44fd63ab9aa939d04@tracker.bro-ids.org> References: <047.bf6216f5c5ffabe44fd63ab9aa939d04@tracker.bro-ids.org> Message-ID: <062.539ef614615dde682141070dd9b8852a@tracker.bro-ids.org> #690: GTP de-tunneling ---------------------+------------------------ Reporter: robin | Owner: jsiwek Type: Task | Status: assigned Priority: Normal | Milestone: 2.1 Component: Bro | Version: git/master Resolution: | Keywords: ---------------------+------------------------ Changes (by jsiwek): * owner: => jsiwek * status: new => assigned * type: Merge Request => Task Comment: Carsten reported a problem with this, so going to resolve that first. -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Tue Oct 23 15:52:56 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Tue, 23 Oct 2012 22:52:56 -0000 Subject: [Bro-Dev] #908: Need for function bytestring_to_double In-Reply-To: <049.360fac6a490c92ac55b12cf22ed72a91@tracker.bro-ids.org> References: <049.360fac6a490c92ac55b12cf22ed72a91@tracker.bro-ids.org> Message-ID: <064.1de95ce4f4d23f5f0be7d822899af594@tracker.bro-ids.org> #908: Need for function bytestring_to_double ------------------------------+------------------------ Reporter: carsten | Owner: Type: Feature Request | Status: new Priority: Normal | Milestone: Bro2.2 Component: Bro | Version: git/master Resolution: | Keywords: ------------------------------+------------------------ Comment (by robin): Hmm ... Is this portable ...? -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Tue Oct 23 22:15:23 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Wed, 24 Oct 2012 05:15:23 -0000 Subject: [Bro-Dev] #909: topic/seth/pppoe Message-ID: <046.f8b83e5ff2311a31a95d50ab6029f35c@tracker.bro-ids.org> #909: topic/seth/pppoe ---------------------------+------------------------ Reporter: seth | Owner: robin Type: Merge Request | Status: new Priority: Normal | Milestone: Bro2.2 Component: Bro | Version: git/master Keywords: | ---------------------------+------------------------ Support for IP in PPPoE along with a test. -- Ticket URL: Bro Tracker Bro Issue Tracker From noreply at bro-ids.org Wed Oct 24 00:00:03 2012 From: noreply at bro-ids.org (Merge Tracker) Date: Wed, 24 Oct 2012 00:00:03 -0700 Subject: [Bro-Dev] [Auto] Merge Status Message-ID: <201210240700.q9O703vC006476@bro-ids.icir.org> > Open Merge Requests for Bro2.2 > ============================== Component | Id | Reporter | Owner | Prio | Summary ------------------------------------------------------------------------------------------------------------------ Bro | 888 [1] | sheharbano.k | | High | Bro crashes on IPv6 subnet [::FFFF:0:0]/96 Bro | 905 [2] | carsten | robin | Normal | Need tcp_max_initial_window to be redefinable Bro | 906 [3] | carsten | robin | Normal | Need tcp_max_above_hole_without_any_acks to be redefinable Bro | 907 [4] | carsten | robin | Normal | Need tcp_excessive_data_without_further_acks to be redefinable Bro | 909 [5] | seth | robin | Normal | topic/seth/pppoe [6] BroControl | 876 [7] | seth | dnthayer | Normal | "BroControl ""diag"" emails" [1] #888: http://tracker.bro-ids.org/bro/ticket/888 [2] #905: http://tracker.bro-ids.org/bro/ticket/905 [3] #906: http://tracker.bro-ids.org/bro/ticket/906 [4] #907: http://tracker.bro-ids.org/bro/ticket/907 [5] #909: http://tracker.bro-ids.org/bro/ticket/909 [6] pppoe: http://tracker.bro-ids.org/bro/changeset?old_path=%2Fbro&old=master&new_path=%2Fbro&new=topic/seth/pppoe [7] #876: http://tracker.bro-ids.org/bro/ticket/876 From bro at tracker.bro-ids.org Wed Oct 24 15:42:58 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Wed, 24 Oct 2012 22:42:58 -0000 Subject: [Bro-Dev] #908: Need for function bytestring_to_double In-Reply-To: <049.360fac6a490c92ac55b12cf22ed72a91@tracker.bro-ids.org> References: <049.360fac6a490c92ac55b12cf22ed72a91@tracker.bro-ids.org> Message-ID: <064.b09a4074bddd7f398ac191f99246f88e@tracker.bro-ids.org> #908: Need for function bytestring_to_double ------------------------------+------------------------------- Reporter: carsten | Owner: Type: Feature Request | Status: needs information Priority: Normal | Milestone: Bro2.2 Component: Bro | Version: git/master Resolution: | Keywords: ------------------------------+------------------------------- Changes (by robin): * status: new => needs information Comment: Asking more precisely: is the binary double representation portable across platforms? -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Wed Oct 24 16:37:09 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Wed, 24 Oct 2012 23:37:09 -0000 Subject: [Bro-Dev] #910: topic/dnthayer/installing-bro Message-ID: <050.bd46a17ef7eb423a5b5209b5d9a0fd99@tracker.bro-ids.org> #910: topic/dnthayer/installing-bro ---------------------------+------------------------ Reporter: dnthayer | Owner: Type: Merge Request | Status: new Priority: Normal | Milestone: Bro2.2 Component: Bro | Version: git/master Keywords: | ---------------------------+------------------------ This branch reorganizes the documentation about installing Bro. The install info has been consolidated into one document (instead of two), the upgrade info has been moved from the FAQ to a section in the install doc, and the "upgrading from 1.5 to 2.0" document has been updated (and renamed) to also include 2.0 to 2.1 upgrade info. -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Wed Oct 24 17:02:53 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Thu, 25 Oct 2012 00:02:53 -0000 Subject: [Bro-Dev] #888: Bro crashes on IPv6 subnet [::FFFF:0:0]/96 In-Reply-To: <054.da51ec8ed0c0114143b25b45dd4f9e5a@tracker.bro-ids.org> References: <054.da51ec8ed0c0114143b25b45dd4f9e5a@tracker.bro-ids.org> Message-ID: <069.4a481cbe7353b8d9c68036ea7887e7fd@tracker.bro-ids.org> #888: Bro crashes on IPv6 subnet [::FFFF:0:0]/96 ----------------------------+------------------------ Reporter: sheharbano.k | Owner: robin Type: Merge Request | Status: closed Priority: High | Milestone: Bro2.2 Component: Bro | Version: git/master Resolution: fixed | Keywords: ----------------------------+------------------------ Changes (by robin): * owner: => robin * status: new => closed * resolution: => fixed Comment: In [7ddbca8b3513207e5153fca0d9d9ddd6f4338f30/bro]: {{{ #!CommitTicketReference repository="bro" revision="7ddbca8b3513207e5153fca0d9d9ddd6f4338f30" Merge remote-tracking branch 'origin/topic/jsiwek/subnet-literal-const' * origin/topic/jsiwek/subnet-literal-const: Add parsing rules for IPv4/IPv6 subnet literal constants, addresses #888 Closes #888. }}} -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Wed Oct 24 17:02:53 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Thu, 25 Oct 2012 00:02:53 -0000 Subject: [Bro-Dev] #906: Need tcp_max_above_hole_without_any_acks to be redefinable In-Reply-To: <049.4c53fed260631285c48c0adbb9443e3f@tracker.bro-ids.org> References: <049.4c53fed260631285c48c0adbb9443e3f@tracker.bro-ids.org> Message-ID: <064.30d4cdcd294f47119cdcb6566fd287d3@tracker.bro-ids.org> #906: Need tcp_max_above_hole_without_any_acks to be redefinable ----------------------------+------------------------ Reporter: carsten | Owner: robin Type: Merge Request | Status: closed Priority: Normal | Milestone: Bro2.2 Component: Bro | Version: git/master Resolution: fixed | Keywords: ----------------------------+------------------------ Changes (by robin): * status: assigned => closed * resolution: => fixed Comment: In [ae38aad2bbba572f5a3750d730064ef7b5525c9a/bro]: {{{ #!CommitTicketReference repository="bro" revision="ae38aad2bbba572f5a3750d730064ef7b5525c9a" Adding missing &redef for some TCP options. Reported by Carsten Langer. Closes #905. Closes #906. Closes #907. }}} -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Wed Oct 24 17:02:53 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Thu, 25 Oct 2012 00:02:53 -0000 Subject: [Bro-Dev] #907: Need tcp_excessive_data_without_further_acks to be redefinable In-Reply-To: <049.b80fe7a7a38edfc9c4be0b6e74c3725c@tracker.bro-ids.org> References: <049.b80fe7a7a38edfc9c4be0b6e74c3725c@tracker.bro-ids.org> Message-ID: <064.d3cc4801e19dbbd29edf756ea2138ed4@tracker.bro-ids.org> #907: Need tcp_excessive_data_without_further_acks to be redefinable ----------------------------+------------------------ Reporter: carsten | Owner: robin Type: Merge Request | Status: closed Priority: Normal | Milestone: Bro2.2 Component: Bro | Version: git/master Resolution: fixed | Keywords: ----------------------------+------------------------ Changes (by robin): * status: assigned => closed * resolution: => fixed Comment: In [ae38aad2bbba572f5a3750d730064ef7b5525c9a/bro]: {{{ #!CommitTicketReference repository="bro" revision="ae38aad2bbba572f5a3750d730064ef7b5525c9a" Adding missing &redef for some TCP options. Reported by Carsten Langer. Closes #905. Closes #906. Closes #907. }}} -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Wed Oct 24 17:02:54 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Thu, 25 Oct 2012 00:02:54 -0000 Subject: [Bro-Dev] #909: topic/seth/pppoe In-Reply-To: <046.f8b83e5ff2311a31a95d50ab6029f35c@tracker.bro-ids.org> References: <046.f8b83e5ff2311a31a95d50ab6029f35c@tracker.bro-ids.org> Message-ID: <061.fe374d2c516a83848eb2e2fcda553d96@tracker.bro-ids.org> #909: topic/seth/pppoe ----------------------------+------------------------ Reporter: seth | Owner: robin Type: Merge Request | Status: closed Priority: Normal | Milestone: Bro2.2 Component: Bro | Version: git/master Resolution: fixed | Keywords: ----------------------------+------------------------ Changes (by robin): * status: new => closed * resolution: => fixed Comment: In [9ec2bfe59fc9962bd72e78cbf32a4165e41ebc2f/bro]: {{{ #!CommitTicketReference repository="bro" revision="9ec2bfe59fc9962bd72e78cbf32a4165e41ebc2f" Merge remote-tracking branch 'origin/topic/seth/pppoe' Looks like the code in this function is getting ready for some refactoring. Leaving that for another time though ... Closes #909. * origin/topic/seth/pppoe: Adding a test for PPPoE support. Adding PPPoE support to Bro. }}} -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Wed Oct 24 17:02:53 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Thu, 25 Oct 2012 00:02:53 -0000 Subject: [Bro-Dev] #905: Need tcp_max_initial_window to be redefinable In-Reply-To: <049.1c47ee8c0297ed5eea803cd6d7d4cac1@tracker.bro-ids.org> References: <049.1c47ee8c0297ed5eea803cd6d7d4cac1@tracker.bro-ids.org> Message-ID: <064.9d47d1177efcfd5ea49760cf553da829@tracker.bro-ids.org> #905: Need tcp_max_initial_window to be redefinable ----------------------------+------------------------ Reporter: carsten | Owner: robin Type: Merge Request | Status: closed Priority: Normal | Milestone: Bro2.2 Component: Bro | Version: git/master Resolution: fixed | Keywords: ----------------------------+------------------------ Changes (by robin): * status: assigned => closed * resolution: => fixed Comment: In [ae38aad2bbba572f5a3750d730064ef7b5525c9a/bro]: {{{ #!CommitTicketReference repository="bro" revision="ae38aad2bbba572f5a3750d730064ef7b5525c9a" Adding missing &redef for some TCP options. Reported by Carsten Langer. Closes #905. Closes #906. Closes #907. }}} -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Wed Oct 24 17:03:10 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Thu, 25 Oct 2012 00:03:10 -0000 Subject: [Bro-Dev] #876: BroControl "diag" emails In-Reply-To: <046.b720fcd2aac59166bd3a426e1051d076@tracker.bro-ids.org> References: <046.b720fcd2aac59166bd3a426e1051d076@tracker.bro-ids.org> Message-ID: <061.b1c935859b78139c49b67fad81bc2482@tracker.bro-ids.org> #876: BroControl "diag" emails ----------------------------+------------------------ Reporter: seth | Owner: dnthayer Type: Merge Request | Status: closed Priority: Normal | Milestone: Bro2.2 Component: BroControl | Version: git/master Resolution: fixed | Keywords: ----------------------------+------------------------ Changes (by robin): * status: new => closed * resolution: => fixed Comment: In [8c53c57ebf16f5aaf34052eab3b02be75774cd75/broctl]: {{{ #!CommitTicketReference repository="broctl" revision="8c53c57ebf16f5aaf34052eab3b02be75774cd75" Merge remote-tracking branch 'origin/topic/dnthayer/forward-crash-email' * origin/topic/dnthayer/forward-crash-email: Add forwarding message to crash emails Closes #876. }}} -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Wed Oct 24 23:57:22 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Thu, 25 Oct 2012 06:57:22 -0000 Subject: [Bro-Dev] #908: Need for function bytestring_to_double In-Reply-To: <049.360fac6a490c92ac55b12cf22ed72a91@tracker.bro-ids.org> References: <049.360fac6a490c92ac55b12cf22ed72a91@tracker.bro-ids.org> Message-ID: <064.208f51e1fe19f8c534728eeeceacfa83@tracker.bro-ids.org> #908: Need for function bytestring_to_double ------------------------------+------------------------------- Reporter: carsten | Owner: Type: Feature Request | Status: needs information Priority: Normal | Milestone: Bro2.2 Component: Bro | Version: git/master Resolution: | Keywords: ------------------------------+------------------------------- Comment (by carsten): Well, I'm no expert in portability, but I think it should be portable. Reasoning: - there is a check that the length of bytestring equals the length of the internal representation of double in "if ( s->Len() != sizeof(d))". - the bit-pattern is first translated from network to host order before it is interpreted as double, to avoid invalid bitpatterns as may occur in ntohd(). - the bitpattern interpretation for a double is defined in IEEE 754, http://en.wikipedia.org/wiki/Double-precision_floating-point_format and I just *expect* every compiler on every platform to adhere to this spec. -- Ticket URL: Bro Tracker Bro Issue Tracker From noreply at bro-ids.org Thu Oct 25 00:00:05 2012 From: noreply at bro-ids.org (Merge Tracker) Date: Thu, 25 Oct 2012 00:00:05 -0700 Subject: [Bro-Dev] [Auto] Merge Status Message-ID: <201210250700.q9P705g9015855@bro-ids.icir.org> > Open Merge Requests for Bro2.2 > ============================== Component | Id | Reporter | Owner | Prio | Summary ------------------------------------------------------------------------------------------------------------------ Bro | 910 [1] | dnthayer | | Normal | topic/dnthayer/installing-bro [2] [1] #910: http://tracker.bro-ids.org/bro/ticket/910 [2] installing-bro: http://tracker.bro-ids.org/bro/changeset?old_path=%2Fbro&old=master&new_path=%2Fbro&new=topic/dnthayer/installing-bro From bro at tracker.bro-ids.org Thu Oct 25 00:02:36 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Thu, 25 Oct 2012 07:02:36 -0000 Subject: [Bro-Dev] #908: Need for function bytestring_to_double In-Reply-To: <049.360fac6a490c92ac55b12cf22ed72a91@tracker.bro-ids.org> References: <049.360fac6a490c92ac55b12cf22ed72a91@tracker.bro-ids.org> Message-ID: <064.83ea8d77eceb1f1f6834e2f87d47beff@tracker.bro-ids.org> #908: Need for function bytestring_to_double ------------------------------+------------------------------- Reporter: carsten | Owner: Type: Feature Request | Status: needs information Priority: Normal | Milestone: Bro2.2 Component: Bro | Version: git/master Resolution: | Keywords: ------------------------------+------------------------------- Comment (by carsten): another argument: I analyze traces with many double values, coming from many different OS platforms. With the provided patch I always get reasonable values. That tells me that the different OS platforms encode the double all in the same way, so I think also the decoding should work on all OS platforms. -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Thu Oct 25 08:55:52 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Thu, 25 Oct 2012 15:55:52 -0000 Subject: [Bro-Dev] #908: Need for function bytestring_to_double In-Reply-To: <049.360fac6a490c92ac55b12cf22ed72a91@tracker.bro-ids.org> References: <049.360fac6a490c92ac55b12cf22ed72a91@tracker.bro-ids.org> Message-ID: <064.172783142f93edfa7976c6d4e7898186@tracker.bro-ids.org> #908: Need for function bytestring_to_double ------------------------------+------------------------------- Reporter: carsten | Owner: Type: Feature Request | Status: needs information Priority: Normal | Milestone: Bro2.2 Component: Bro | Version: git/master Resolution: | Keywords: ------------------------------+------------------------------- Comment (by jsiwek): > - there is a check that the length of bytestring equals the length of the internal representation of double in "if ( s->Len() != sizeof(d))". Maybe it also needs a check that `sizeof(uint64) == sizeof(double)` since the bytes are copied to `uint64` storage first. > - the bit-pattern is first translated from network to host order before it is interpreted as double, to avoid invalid bitpatterns as may occur in ntohd(). I don't get that entirely, I would have thought doing a `memcpy` in to `double` storage and then using `ntohd` would work, but maybe not. > - the bitpattern interpretation for a double is defined in IEEE 754, http://en.wikipedia.org/wiki/Double-precision_floating-point_format and I just *expect* every compiler on every platform to adhere to this spec. Yeah, I think in practice if you need to do byteswapping on floating point data types, treating it like an integer of same width might be ok. More on that: http://en.wikipedia.org/wiki/Endianness#Floating- point_and_endianness {{{Because there have been many floating point formats with no "network" standard representation for them, there is no formal standard for transferring floating point values between heterogeneous systems. Theoretically, this means that even standard IEEE floating point data written by one machine might not be readable by another. However, on modern standard computers (i.e., implementing IEEE 754), one may in practice safely assume that the endianness is the same for floating point numbers as for integers, making the conversion straightforward regardless of data type.}}} Otherwise, I still wonder about the second bullet above and whether some type casts can be avoided. And maybe the name should be `network_bytestring_to_host_double` to be clear there may be a byteswap operation in it. Or just make separate BIFs for byte swapping bytestrings of various lengths and also separate bytestring to other data type conversion BIFs. -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Thu Oct 25 13:18:50 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Thu, 25 Oct 2012 20:18:50 -0000 Subject: [Bro-Dev] #908: Need for function bytestring_to_double In-Reply-To: <049.360fac6a490c92ac55b12cf22ed72a91@tracker.bro-ids.org> References: <049.360fac6a490c92ac55b12cf22ed72a91@tracker.bro-ids.org> Message-ID: <064.bda57cf14de1a31f9d1c7e145de454e6@tracker.bro-ids.org> #908: Need for function bytestring_to_double ------------------------------+------------------------------- Reporter: carsten | Owner: Type: Feature Request | Status: needs information Priority: Normal | Milestone: Bro2.2 Component: Bro | Version: git/master Resolution: | Keywords: ------------------------------+------------------------------- Comment (by dnthayer): In [f6caa565bd01f2513f4358621fdf4ffe4fe3edad/bro]: {{{ #!CommitTicketReference repository="bro" revision="f6caa565bd01f2513f4358621fdf4ffe4fe3edad" Add a new BIF "bytestring_to_double" Addresses #908. }}} -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Thu Oct 25 17:14:29 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Fri, 26 Oct 2012 00:14:29 -0000 Subject: [Bro-Dev] #908: Need for function bytestring_to_double In-Reply-To: <049.360fac6a490c92ac55b12cf22ed72a91@tracker.bro-ids.org> References: <049.360fac6a490c92ac55b12cf22ed72a91@tracker.bro-ids.org> Message-ID: <064.f2a42b4c8051f18a8330aef3a33ffa20@tracker.bro-ids.org> #908: Need for function bytestring_to_double ------------------------------+------------------------------- Reporter: carsten | Owner: Type: Feature Request | Status: needs information Priority: Normal | Milestone: Bro2.2 Component: Bro | Version: git/master Resolution: | Keywords: ------------------------------+------------------------------- Comment (by matthias): Replying to [comment:7 jsiwek]: > > - there is a check that the length of bytestring equals the length of the internal representation of double in "if ( s->Len() != sizeof(d))". > > Maybe it also needs a check that `sizeof(uint64) == sizeof(double)` since the bytes are copied to `uint64` storage first. I'm not entirely convinced that this is the right solution. In memory, a double is indeed 64 bits, however, double-precision registers are 80 bits on x86 [1]. Consequently, if a compiler decides to hold a double purely in a register, you will see truncation. Matthias [1] http://en.wikipedia.org/wiki/Extended_precision -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Thu Oct 25 18:06:56 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Fri, 26 Oct 2012 01:06:56 -0000 Subject: [Bro-Dev] #911: SRV replies don't get processed by DNS analyzer Message-ID: <046.078ef418dae255df319137a91c87e994@tracker.bro-ids.org> #911: SRV replies don't get processed by DNS analyzer ---------------------+------------------------ Reporter: vern | Owner: Type: Problem | Status: new Priority: Normal | Milestone: Bro2.2 Component: Bro | Version: git/master Keywords: | ---------------------+------------------------ The event engine doesn't appear to generate {{{dns_SRV_reply}}} in some cases, as indicated by running on the attached trace. I've tried this with both the default DNS analysis and my own custom analysis (that uses -b to not run other stuff) and have confirmed that the reply event isn't getting generated, even though there aren't any checksum issues or such. -- Ticket URL: Bro Tracker Bro Issue Tracker From noreply at bro-ids.org Fri Oct 26 00:00:05 2012 From: noreply at bro-ids.org (Merge Tracker) Date: Fri, 26 Oct 2012 00:00:05 -0700 Subject: [Bro-Dev] [Auto] Merge Status Message-ID: <201210260700.q9Q705x5003896@bro-ids.icir.org> > Open Merge Requests for Bro2.2 > ============================== Component | Id | Reporter | Owner | Prio | Summary ------------------------------------------------------------------------------------------------------------------ Bro | 910 [1] | dnthayer | | Normal | topic/dnthayer/installing-bro [2] [1] #910: http://tracker.bro-ids.org/bro/ticket/910 [2] installing-bro: http://tracker.bro-ids.org/bro/changeset?old_path=%2Fbro&old=master&new_path=%2Fbro&new=topic/dnthayer/installing-bro From hlin33 at illinois.edu Fri Oct 26 11:15:51 2012 From: hlin33 at illinois.edu (Hui Lin (Hugo) ) Date: Fri, 26 Oct 2012 13:15:51 -0500 Subject: [Bro-Dev] #861: Merging DNP3 Analyzer In-Reply-To: <193a78cb56ba4628bc4e562519e1dfde@CITESHT3.ad.uillinois.edu> References: <045.d25b5df6a13b96f4b3e3618bd9398e58@tracker.bro-ids.org> <193a78cb56ba4628bc4e562519e1dfde@CITESHT3.ad.uillinois.edu> Message-ID: HI, Robin, Can you please take a look at current version of the codes. I added documentation on the analyzer. The work that left undone so far is (1) so far, we support logic DNP3 application layer fragment as long as 65536. (2) documentation on protocol validation policy, the policy that checks some rules defined by DNP3 protocols Best, Hui Lin On Sat, Oct 6, 2012 at 8:30 PM, Bro Tracker wrote: > #861: Merging DNP3 Analyzer > ---------------------+------------------------ > Reporter: hui | Owner: robin > Type: Task | Status: assigned > Priority: Normal | Milestone: Bro2.2 > Component: Bro | Version: git/master > Resolution: | Keywords: dnp3 > ---------------------+------------------------ > > Comment (by hui): > > Replying to [comment:12 seth]: > > > I think what you said is the "incremental parsing" > > > mentioned in the binpac paper. But actually, I am not quite sure how > this > > > is implemented in the binpac. Can you please direct me to some codes > that > > > I refer to? > > > > It's actually not something you even need to worry about. Just > instantiate your binpac parser and begin passing data into it as you > receive it, the binpac parser will take care of the data even if it > doesn't receive the full PDU in one go. > > Just come up another question. When a HTTP fragment is very long and > carried in different network packets, the HTTP binpac analyzer should know > the length of the whole fragment when the first application layer trunk is > received. Is this correct? The HTTP message contains some field to > indicate that length, right? > > But for DNP3 analyzer, this is not possible. I know the length of the > whole logical DNP3 fragment only when the last trunk is received. To > better explained > > TCP : DNP3 Pseudo Data Link Layer (length field is 255) : DNP3 Pseudo Data > Transport Layer : DNP3 Pseudo Application Layer #1 > TCP : DNP3 Pseudo Data Link Layer (length field is 255) : DNP3 Pseudo Data > Transport Layer : DNP3 Pseudo Application Layer #2 > .... > TCP : DNP3 Pseudo Data Link Layer (length field is x) : DNP3 Pseudo Data > Transport Layer : DNP3 Pseudo Application Layer #n > > So the length field in the Pseudo Data Link does not contain the length of > the whole DNP3 fragment, but the length of the trunk following this data > link layer. So in order to know the whole length of the DNP3 fragment (in > this case is , 255 + 255 + ... + x), all the application layer trunk has > to be received. So is there any way to use incremental parsing in the > binpac in this case? > > -- > Ticket URL: > Bro Tracker > Bro Issue Tracker > -- Hui Lin PhD Candidate, Research Assistant Electrical and Computer Engineering Department University of Illinois at Urbana-Champaign -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.icsi.berkeley.edu/pipermail/bro-dev/attachments/20121026/c4d5eb29/attachment.html From bro at tracker.bro-ids.org Fri Oct 26 11:16:16 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Fri, 26 Oct 2012 18:16:16 -0000 Subject: [Bro-Dev] #861: Merging DNP3 Analyzer In-Reply-To: <045.d25b5df6a13b96f4b3e3618bd9398e58@tracker.bro-ids.org> References: <045.d25b5df6a13b96f4b3e3618bd9398e58@tracker.bro-ids.org> Message-ID: <060.407e5966448f98eb9ea5085e0dcae172@tracker.bro-ids.org> #861: Merging DNP3 Analyzer ---------------------+------------------------ Reporter: hui | Owner: robin Type: Task | Status: assigned Priority: Normal | Milestone: Bro2.2 Component: Bro | Version: git/master Resolution: | Keywords: dnp3 ---------------------+------------------------ Comment (by hui): HI, Robin, Can you please take a look at current version of the codes. I added documentation on the analyzer. The work that left undone so far is (1) so far, we support logic DNP3 application layer fragment as long as 65536. (2) documentation on protocol validation policy, the policy that checks some rules defined by DNP3 protocols Best, Hui Lin On Sat, Oct 6, 2012 at 8:30 PM, Bro Tracker wrote: > #861: Merging DNP3 Analyzer > ---------------------+------------------------ > Reporter: hui | Owner: robin > Type: Task | Status: assigned > Priority: Normal | Milestone: Bro2.2 > Component: Bro | Version: git/master > Resolution: | Keywords: dnp3 > ---------------------+------------------------ > > Comment (by hui): > > Replying to [comment:12 seth]: > > > I think what you said is the "incremental parsing" > > > mentioned in the binpac paper. But actually, I am not quite sure how > this > > > is implemented in the binpac. Can you please direct me to some codes > that > > > I refer to? > > > > It's actually not something you even need to worry about. Just > instantiate your binpac parser and begin passing data into it as you > receive it, the binpac parser will take care of the data even if it > doesn't receive the full PDU in one go. > > Just come up another question. When a HTTP fragment is very long and > carried in different network packets, the HTTP binpac analyzer should know > the length of the whole fragment when the first application layer trunk is > received. Is this correct? The HTTP message contains some field to > indicate that length, right? > > But for DNP3 analyzer, this is not possible. I know the length of the > whole logical DNP3 fragment only when the last trunk is received. To > better explained > > TCP : DNP3 Pseudo Data Link Layer (length field is 255) : DNP3 Pseudo Data > Transport Layer : DNP3 Pseudo Application Layer #1 > TCP : DNP3 Pseudo Data Link Layer (length field is 255) : DNP3 Pseudo Data > Transport Layer : DNP3 Pseudo Application Layer #2 > .... > TCP : DNP3 Pseudo Data Link Layer (length field is x) : DNP3 Pseudo Data > Transport Layer : DNP3 Pseudo Application Layer #n > > So the length field in the Pseudo Data Link does not contain the length of > the whole DNP3 fragment, but the length of the trunk following this data > link layer. So in order to know the whole length of the DNP3 fragment (in > this case is , 255 + 255 + ... + x), all the application layer trunk has > to be received. So is there any way to use incremental parsing in the > binpac in this case? > > -- > Ticket URL: > Bro Tracker > Bro Issue Tracker > -- Ticket URL: Bro Tracker Bro Issue Tracker From noreply at bro-ids.org Sat Oct 27 00:00:05 2012 From: noreply at bro-ids.org (Merge Tracker) Date: Sat, 27 Oct 2012 00:00:05 -0700 Subject: [Bro-Dev] [Auto] Merge Status Message-ID: <201210270700.q9R705Tc025826@bro-ids.icir.org> > Open Merge Requests for Bro2.2 > ============================== Component | Id | Reporter | Owner | Prio | Summary ------------------------------------------------------------------------------------------------------------------ Bro | 910 [1] | dnthayer | | Normal | topic/dnthayer/installing-bro [2] [1] #910: http://tracker.bro-ids.org/bro/ticket/910 [2] installing-bro: http://tracker.bro-ids.org/bro/changeset?old_path=%2Fbro&old=master&new_path=%2Fbro&new=topic/dnthayer/installing-bro From noreply at bro-ids.org Sun Oct 28 00:00:06 2012 From: noreply at bro-ids.org (Merge Tracker) Date: Sun, 28 Oct 2012 00:00:06 -0700 Subject: [Bro-Dev] [Auto] Merge Status Message-ID: <201210280700.q9S7069w019479@bro-ids.icir.org> > Open Merge Requests for Bro2.2 > ============================== Component | Id | Reporter | Owner | Prio | Summary ------------------------------------------------------------------------------------------------------------------ Bro | 910 [1] | dnthayer | | Normal | topic/dnthayer/installing-bro [2] [1] #910: http://tracker.bro-ids.org/bro/ticket/910 [2] installing-bro: http://tracker.bro-ids.org/bro/changeset?old_path=%2Fbro&old=master&new_path=%2Fbro&new=topic/dnthayer/installing-bro From bro at tracker.bro-ids.org Sun Oct 28 12:23:13 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Sun, 28 Oct 2012 19:23:13 -0000 Subject: [Bro-Dev] #908: Need for function bytestring_to_double In-Reply-To: <049.360fac6a490c92ac55b12cf22ed72a91@tracker.bro-ids.org> References: <049.360fac6a490c92ac55b12cf22ed72a91@tracker.bro-ids.org> Message-ID: <064.b448b474c0b0ae05756f90c96a245dc8@tracker.bro-ids.org> #908: Need for function bytestring_to_double ------------------------------+------------------------------- Reporter: carsten | Owner: Type: Feature Request | Status: needs information Priority: Normal | Milestone: Bro2.2 Component: Bro | Version: git/master Resolution: | Keywords: ------------------------------+------------------------------- Comment (by carsten): Replying to [comment:7 jsiwek]: > > - the bit-pattern is first translated from network to host order before it is interpreted as double, to avoid invalid bitpatterns as may occur in ntohd(). > > I don't get that entirely, I would have thought doing a `memcpy` in to `double` storage and then using `ntohd` would work, but maybe not. > I once observed a problem when assinging a still-to-be-swapped bitpattern to a double before swapping it from network to host order. The problem vanished when using the workaround with uint64. My interpretation of the problem was the following: Not all 2 * * 64 possible bitpatterns have an unambiguous meaning according to IEEE 754, see the NaN ans SNaN patterns and - depending on the implememtation - the subnormal patterns in http://www.cs.berkeley.edu/~wkahan/ieee754status/IEEE754.PDF. That could mean that there may be a valid bitpattern for a double, in network order, which is an ambiguous pattern if interpreted (without swapping) in host order, if network and host oder differ by swapping. The memcpy would copy the bitpattern unchanged to the double variable, but the next access (for the reason of swapping it) to the double variable having this bitpattern which could be unusual to the platform accessing it, could result in a "correction", i.e. change of the unusual bitpattern to the "normal" bitpattern representing the same informaation (e.g. NaN). That would mean that the bitpattern is changed before swapping, and after swapping would result in a different number than was originally put on the network. Unfortunately, I have not captured the situation I observed and thus I cannot use it for a test case. At that time I was happy to have solved the problem with using the uint64-workaround. If the above was true and could be reproduced, then this also would mean that the current ntohd() function is flawed, as it assings a to-be-swapped bitpattern to double first before swapping. Now that we start a discussion about it, I tried to artificially recreate the problem by creating supposed problematic bitpatterns and swapping them back and forth. However, I was not able to recreate the change of bitpattern that I though to have observed before. Bottom line: I currently see no more reason anymore to use the uint64 workaround and therfore the patch proposed by Daniel should work fine. I'll test it myself. > Otherwise, I still wonder about the second bullet above and whether some type casts can be avoided. And maybe the name should be `network_bytestring_to_host_double` to be clear there may be a byteswap operation in it. Or just make separate BIFs for byte swapping bytestrings of various lengths and also separate bytestring to other data type conversion BIFs. Fine with me, chose whatever name seems appropriate. -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Sun Oct 28 12:25:15 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Sun, 28 Oct 2012 19:25:15 -0000 Subject: [Bro-Dev] #908: Need for function bytestring_to_double In-Reply-To: <049.360fac6a490c92ac55b12cf22ed72a91@tracker.bro-ids.org> References: <049.360fac6a490c92ac55b12cf22ed72a91@tracker.bro-ids.org> Message-ID: <064.910605f3a7c1f0eb4f5d87d03a6889ec@tracker.bro-ids.org> #908: Need for function bytestring_to_double ------------------------------+------------------------------- Reporter: carsten | Owner: Type: Feature Request | Status: needs information Priority: Normal | Milestone: Bro2.2 Component: Bro | Version: git/master Resolution: | Keywords: ------------------------------+------------------------------- Comment (by carsten): Replying to [comment:9 matthias]: > I'm not entirely convinced that this is the right solution. In memory, a double is indeed 64 bits, however, double-precision registers are 80 bits on x86 [1]. Consequently, if a compiler decides to hold a double purely in a register, you will see truncation. > > [1] http://en.wikipedia.org/wiki/Extended_precision But the situation at hand is that we have already a 64-bit representation within a network trace, and try to bring that back to a double value. If there was trunkation, it happened while putting the value to the network, not when we read it from network. -- Ticket URL: Bro Tracker Bro Issue Tracker From noreply at bro-ids.org Mon Oct 29 00:00:03 2012 From: noreply at bro-ids.org (Merge Tracker) Date: Mon, 29 Oct 2012 00:00:03 -0700 Subject: [Bro-Dev] [Auto] Merge Status Message-ID: <201210290700.q9T703Kr030367@bro-ids.icir.org> > Open Merge Requests for Bro2.2 > ============================== Component | Id | Reporter | Owner | Prio | Summary ------------------------------------------------------------------------------------------------------------------ Bro | 910 [1] | dnthayer | | Normal | topic/dnthayer/installing-bro [2] [1] #910: http://tracker.bro-ids.org/bro/ticket/910 [2] installing-bro: http://tracker.bro-ids.org/bro/changeset?old_path=%2Fbro&old=master&new_path=%2Fbro&new=topic/dnthayer/installing-bro From bro at tracker.bro-ids.org Mon Oct 29 02:35:56 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Mon, 29 Oct 2012 09:35:56 -0000 Subject: [Bro-Dev] #912: Need option for ASCII logging writer to only print CSV header line Message-ID: <049.d710ed16fabc938ed6d83c3181873a7a@tracker.bro-ids.org> #912: Need option for ASCII logging writer to only print CSV header line ------------------------+------------------------------- Reporter: carsten | Type: Feature Request Status: new | Priority: Normal Milestone: Bro2.2 | Component: Bro Version: git/master | Keywords: logging csv ascii ------------------------+------------------------------- The new logging framework is a big step forward, and the current Bro 2.1 ASCII writer gives a lot of meta information to allow unambiguity and script based post-processing. In addition, I like to have a simplified logging format which I can just open in any spreadsheet program like Excel as a CSV file, i.e. only 1 header row, and no prepending "#fields" entry. The attached diffs change 3 source files to add an option "only_single_header_row" to the ascii writer to achieve this simplified output. Please consider to bring it into next release. -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Mon Oct 29 06:03:42 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Mon, 29 Oct 2012 13:03:42 -0000 Subject: [Bro-Dev] #908: Need for function bytestring_to_double In-Reply-To: <049.360fac6a490c92ac55b12cf22ed72a91@tracker.bro-ids.org> References: <049.360fac6a490c92ac55b12cf22ed72a91@tracker.bro-ids.org> Message-ID: <064.83f92c540bb687b74ceccac3f76b7d42@tracker.bro-ids.org> #908: Need for function bytestring_to_double ------------------------------+------------------------------- Reporter: carsten | Owner: Type: Feature Request | Status: needs information Priority: Normal | Milestone: Bro2.2 Component: Bro | Version: git/master Resolution: | Keywords: ------------------------------+------------------------------- Comment (by carsten): Replying to [comment:8 dnthayer]: > In [f6caa565bd01f2513f4358621fdf4ffe4fe3edad/bro]: > {{{ > #!CommitTicketReference repository="bro" revision="f6caa565bd01f2513f4358621fdf4ffe4fe3edad" > Add a new BIF "bytestring_to_double" > > Addresses #908. > }}} I tested Daniel's version of the patch against real network data containing ca. 300,000 double values, and it gives the same results as my uint64-workaround patch. So I'm fine using Daniel's patch. Thanks -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Mon Oct 29 09:11:51 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Mon, 29 Oct 2012 16:11:51 -0000 Subject: [Bro-Dev] #913: Option to specify interface in bro-aux/rst Message-ID: <052.98b639aa8cf2ab48dded52b0217ab42a@tracker.bro-ids.org> #913: Option to specify interface in bro-aux/rst ------------------------+------------------------------- Reporter: grigorescu | Type: Merge Request Status: new | Priority: Normal Milestone: Bro2.2 | Component: bro-aux Version: git/master | Keywords: aux rst interface ------------------------+------------------------------- I needed to be able to set the interface used to inject RST packets with the rst aux utility. I made a couple of changes to add this ability to rst. Please see attached patch. -- Ticket URL: Bro Tracker Bro Issue Tracker From noreply at bro-ids.org Tue Oct 30 00:00:06 2012 From: noreply at bro-ids.org (Merge Tracker) Date: Tue, 30 Oct 2012 00:00:06 -0700 Subject: [Bro-Dev] [Auto] Merge Status Message-ID: <201210300700.q9U706qF005107@bro-ids.icir.org> > Open Merge Requests for Bro2.2 > ============================== Component | Id | Reporter | Owner | Prio | Summary ------------------------------------------------------------------------------------------------------------------ Bro | 910 [1] | dnthayer | | Normal | topic/dnthayer/installing-bro [2] bro-aux | 913 [3] | grigorescu | | Normal | Option to specify interface in bro-aux/rst [1] #910: http://tracker.bro-ids.org/bro/ticket/910 [2] installing-bro: http://tracker.bro-ids.org/bro/changeset?old_path=%2Fbro&old=master&new_path=%2Fbro&new=topic/dnthayer/installing-bro [3] #913: http://tracker.bro-ids.org/bro/ticket/913 From bro at tracker.bro-ids.org Tue Oct 30 07:53:34 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Tue, 30 Oct 2012 14:53:34 -0000 Subject: [Bro-Dev] #912: Need option for ASCII logging writer to only print CSV header line In-Reply-To: <049.d710ed16fabc938ed6d83c3181873a7a@tracker.bro-ids.org> References: <049.d710ed16fabc938ed6d83c3181873a7a@tracker.bro-ids.org> Message-ID: <064.02fdfffd13b71f21c644c2aeb2b2b9da@tracker.bro-ids.org> #912: Need option for ASCII logging writer to only print CSV header line ------------------------------+------------------------------- Reporter: carsten | Owner: Type: Feature Request | Status: new Priority: Normal | Milestone: Bro2.2 Component: Bro | Version: git/master Resolution: | Keywords: logging csv ascii ------------------------------+------------------------------- Comment (by seth): I think the idea makes sense, but I'm not sure that it should be a "global-only" option. It should probably be turned into a filter specific option with that global default value you included. -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Tue Oct 30 08:17:49 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Tue, 30 Oct 2012 15:17:49 -0000 Subject: [Bro-Dev] #912: Need option for ASCII logging writer to only print CSV header line In-Reply-To: <049.d710ed16fabc938ed6d83c3181873a7a@tracker.bro-ids.org> References: <049.d710ed16fabc938ed6d83c3181873a7a@tracker.bro-ids.org> Message-ID: <064.e2137682e4bce9d927a0d3e5d9b68e2b@tracker.bro-ids.org> #912: Need option for ASCII logging writer to only print CSV header line ------------------------------+------------------------------- Reporter: carsten | Owner: Type: Feature Request | Status: new Priority: Normal | Milestone: Bro2.2 Component: Bro | Version: git/master Resolution: | Keywords: logging csv ascii ------------------------------+------------------------------- Comment (by carsten): oh, yes, I would prefer a per-filter-option, but didn't think of an easy way to achieve it. -----Original Message----- From: ext Bro Tracker [mailto:bro at tracker.bro-ids.org] Sent: 2012-10-30 15:54 To: Langer, Carsten (NSN - DE/Duesseldorf) Cc: bro-dev at bro-ids.org Subject: Re: #912: Need option for ASCII logging writer to only print CSV header line #912: Need option for ASCII logging writer to only print CSV header line ------------------------------+------------------------------- Reporter: carsten | Owner: Type: Feature Request | Status: new Priority: Normal | Milestone: Bro2.2 Component: Bro | Version: git/master Resolution: | Keywords: logging csv ascii ------------------------------+------------------------------- Comment (by seth): I think the idea makes sense, but I'm not sure that it should be a "global-only" option. It should probably be turned into a filter specific option with that global default value you included. -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Tue Oct 30 08:32:18 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Tue, 30 Oct 2012 15:32:18 -0000 Subject: [Bro-Dev] #912: Need option for ASCII logging writer to only print CSV header line In-Reply-To: <049.d710ed16fabc938ed6d83c3181873a7a@tracker.bro-ids.org> References: <049.d710ed16fabc938ed6d83c3181873a7a@tracker.bro-ids.org> Message-ID: <064.f52189a772be20e5ca99c5e2745e89d6@tracker.bro-ids.org> #912: Need option for ASCII logging writer to only print CSV header line ------------------------------+------------------------------- Reporter: carsten | Owner: Type: Feature Request | Status: new Priority: Normal | Milestone: Bro2.2 Component: Bro | Version: git/master Resolution: | Keywords: logging csv ascii ------------------------------+------------------------------- Comment (by robin): > oh, yes, I would prefer a per-filter-option, but didn't think of an easy > way to achieve it. Filters have a "config" table that can pass writer-specific information to the backend. -- Robin Sommer * Phone +1 (510) 722-6541 * robin at icir.org ICSI/LBNL * Fax +1 (510) 666-2956 * www.icir.org -- Ticket URL: Bro Tracker Bro Issue Tracker From vladg at cmu.edu Tue Oct 30 09:49:54 2012 From: vladg at cmu.edu (Vlad Grigorescu) Date: Tue, 30 Oct 2012 16:49:54 +0000 Subject: [Bro-Dev] DNS TXT Queries and the Cache File In-Reply-To: <352_1346341138_q7UFcvg7017598_20120830153851.GA89651@icir.org> References: <1202BE242E080642B0CD0AD0A03E855234C4F1@PGH-MSGMB-03.andrew.ad.cmu.edu> <503F3AB2.4040900@illinois.edu> <352_1346341138_q7UFcvg7017598_20120830153851.GA89651@icir.org> Message-ID: <1202BE242E080642B0CD0AD0A03E8552593265@PGH-MSGMB-03.andrew.ad.cmu.edu> Sorry for the huge delay in getting this out there - it just fell on the back burner. I've put my code up at . The changes weren't terribly significant. It adds lookup_hostname_txt: > when (local result = lookup_hostname_txt("733a48a9cb49651d72fe824ca91e8d00.malware.hash.cymru.com")) > print result; Please let me know if anyone sees any issues. There is a save TXT function, but there is no capability to read the data back from a file, as I mentioned. If someone wants to take a stab to getting that working properly, please feel free. Otherwise, let me know and I'll remove the save function. Thanks, --Vlad On Aug 30, 2012, at 11:38 AM, Robin Sommer wrote: > Cool, thanks for working on this, Vlad. > > On Thu, Aug 30, 2012 at 05:04 -0500, you wrote: > >> As the previous poor soul to touch that code, I wouldn't mind looking at >> what you've got so far and then attempting to add the caching support. > > If the caching is trikcy to get in (or makes the code even worse ...), > we can indeed skip it. The main reason for having the caching at all > is DNS names embedded in scripts (e.g., code of the form "set[addr] = > { foo.bar }"). Bro looks these up once at startup and that can > potentially take a while if there are a lot or responses are coming in > slowly. So what one can do is "prime" the cache first, so that the > next time Bro starts up, it doesn't need to do the lookups. That was > more important in the Old Days though when people restarted Bro once a > day to flush state and that had to be fast. > > This is all not relevant to TXT records. And, in fact, I've already > been wondering if we can get rid of the cache altogether to simplify > the DNS code. > > Robin > > -- > Robin Sommer * Phone +1 (510) 722-6541 * robin at icir.org > ICSI/LBNL * Fax +1 (510) 666-2956 * www.icir.org > _______________________________________________ > bro-dev mailing list > bro-dev at bro-ids.org > http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev From bro at tracker.bro-ids.org Tue Oct 30 14:49:31 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Tue, 30 Oct 2012 21:49:31 -0000 Subject: [Bro-Dev] #824: Default the connection and alarm summaries to once per day In-Reply-To: <056.dadb50a21009b083c39e1a280f60c9e9@tracker.bro-ids.org> References: <056.dadb50a21009b083c39e1a280f60c9e9@tracker.bro-ids.org> Message-ID: <071.efa69c325b0d72ee37eba10ab772137d@tracker.bro-ids.org> #824: Default the connection and alarm summaries to once per day -----------------------------+------------------------ Reporter: Tyler.Schoenke | Owner: Type: Merge Request | Status: new Priority: Normal | Milestone: Bro2.2 Component: Bro | Version: git/master Resolution: | Keywords: -----------------------------+------------------------ Changes (by dnthayer): * type: Feature Request => Merge Request Comment: In branch topic/dnthayer/alarms-mail (in both "bro" and "broctl" repos), added a separate option to specify the alarm summary mail interval, and set it to 24hrs by default. Added a corresponding option in broctl (also with default value of 24hrs). -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Tue Oct 30 15:59:52 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Tue, 30 Oct 2012 22:59:52 -0000 Subject: [Bro-Dev] #913: Option to specify interface in bro-aux/rst In-Reply-To: <052.98b639aa8cf2ab48dded52b0217ab42a@tracker.bro-ids.org> References: <052.98b639aa8cf2ab48dded52b0217ab42a@tracker.bro-ids.org> Message-ID: <067.06b5d81fe278498822238f22be912ea8@tracker.bro-ids.org> #913: Option to specify interface in bro-aux/rst ----------------------------+------------------------------- Reporter: grigorescu | Owner: Type: Merge Request | Status: new Priority: Normal | Milestone: Bro2.2 Component: bro-aux | Version: git/master Resolution: | Keywords: aux rst interface ----------------------------+------------------------------- Comment (by dnthayer): In [6783966d0c15f589d0e3bc6252e0d213751fe05f/bro-aux]: {{{ #!CommitTicketReference repository="bro-aux" revision="6783966d0c15f589d0e3bc6252e0d213751fe05f" Fix a typo Changed sizeof to strlen. Addresses #913. }}} -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Tue Oct 30 16:04:39 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Tue, 30 Oct 2012 23:04:39 -0000 Subject: [Bro-Dev] #913: Option to specify interface in bro-aux/rst In-Reply-To: <052.98b639aa8cf2ab48dded52b0217ab42a@tracker.bro-ids.org> References: <052.98b639aa8cf2ab48dded52b0217ab42a@tracker.bro-ids.org> Message-ID: <067.93d4cbd7681a56b4e9c8a40c1a764913@tracker.bro-ids.org> #913: Option to specify interface in bro-aux/rst ----------------------------+------------------------------- Reporter: grigorescu | Owner: Type: Merge Request | Status: new Priority: Normal | Milestone: Bro2.2 Component: bro-aux | Version: git/master Resolution: | Keywords: aux rst interface ----------------------------+------------------------------- Comment (by dnthayer): Branch topic/dnthayer/rst-interface contains the patch and fix. -- Ticket URL: Bro Tracker Bro Issue Tracker From noreply at bro-ids.org Wed Oct 31 00:00:04 2012 From: noreply at bro-ids.org (Merge Tracker) Date: Wed, 31 Oct 2012 00:00:04 -0700 Subject: [Bro-Dev] [Auto] Merge Status Message-ID: <201210310700.q9V704LU005400@bro-ids.icir.org> > Open Merge Requests for Bro2.2 > ============================== Component | Id | Reporter | Owner | Prio | Summary ------------------------------------------------------------------------------------------------------------------ Bro | 824 [1] | Tyler.Schoenke | | Normal | Default the connection and alarm summaries to once per day Bro | 910 [2] | dnthayer | | Normal | topic/dnthayer/installing-bro [3] bro-aux | 913 [4] | grigorescu | | Normal | Option to specify interface in bro-aux/rst [1] #824: http://tracker.bro-ids.org/bro/ticket/824 [2] #910: http://tracker.bro-ids.org/bro/ticket/910 [3] installing-bro: http://tracker.bro-ids.org/bro/changeset?old_path=%2Fbro&old=master&new_path=%2Fbro&new=topic/dnthayer/installing-bro [4] #913: http://tracker.bro-ids.org/bro/ticket/913 From bro at tracker.bro-ids.org Wed Oct 31 01:54:59 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Wed, 31 Oct 2012 08:54:59 -0000 Subject: [Bro-Dev] #912: Need option for ASCII logging writer to only print CSV header line In-Reply-To: <049.d710ed16fabc938ed6d83c3181873a7a@tracker.bro-ids.org> References: <049.d710ed16fabc938ed6d83c3181873a7a@tracker.bro-ids.org> Message-ID: <064.95b13c32ca0e94408547e95aa1809e0e@tracker.bro-ids.org> #912: Need option for ASCII logging writer to only print CSV header line ------------------------------+------------------------------- Reporter: carsten | Owner: Type: Feature Request | Status: new Priority: Normal | Milestone: Bro2.2 Component: Bro | Version: git/master Resolution: | Keywords: logging csv ascii ------------------------------+------------------------------- Comment (by carsten): ok, I see the idea of using "config" table. But I checked the source code and it is not obvious to me how to use it. It seems the feature of the config table is there, but none of the current writers actually use it, so I'm missing a template from where I could learn. a) Could you please give a short code snippet on how best to check if a config option is set to a specific value, like "if option_key is in config table and option_value equals xyz then ...". I'm not sure if one must check the type of option key/value as well before using it. b) Do you have a naming scheme in mind on how to name options and values? If this will be the first implementation, it will set a kind of standard for others to follow. I would currently think to follow Bro Scripting Conventions, for the key use Variable Naming, i.e. Lower-cased with underscores, for values use Enum Naming, i.e. Upper-cased with underscores. The example would be: key: only_single_header_row, value: T or F. -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Wed Oct 31 06:10:34 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Wed, 31 Oct 2012 13:10:34 -0000 Subject: [Bro-Dev] #914: topic/seth/intel-framework Message-ID: <046.6acf3055033d7e1af81faf954effa510@tracker.bro-ids.org> #914: topic/seth/intel-framework ---------------------------+------------------------ Reporter: seth | Owner: robin Type: Merge Request | Status: new Priority: Normal | Milestone: Bro2.2 Component: Bro | Version: git/master Keywords: | ---------------------------+------------------------ This is ready for an initial merge. It's fully functional and being tested on live traffic at a few sites now. There are a couple of core features missing still: - Intelligence handling policy. - Whitelisting (like marking data as a false positive) -- Ticket URL: Bro Tracker Bro Issue Tracker From robin at icir.org Wed Oct 31 08:33:14 2012 From: robin at icir.org (Robin Sommer) Date: Wed, 31 Oct 2012 08:33:14 -0700 Subject: [Bro-Dev] #914: topic/seth/intel-framework In-Reply-To: <046.6acf3055033d7e1af81faf954effa510@tracker.bro-ids.org> References: <046.6acf3055033d7e1af81faf954effa510@tracker.bro-ids.org> Message-ID: <20121031153314.GE30039@icir.org> On Wed, Oct 31, 2012 at 13:10 -0000, you wrote: > This is ready for an initial merge. Yippee! :) Robin -- Robin Sommer * Phone +1 (510) 722-6541 * robin at icir.org ICSI/LBNL * Fax +1 (510) 666-2956 * www.icir.org From bro at tracker.bro-ids.org Wed Oct 31 08:51:05 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Wed, 31 Oct 2012 15:51:05 -0000 Subject: [Bro-Dev] #912: Need option for ASCII logging writer to only print CSV header line In-Reply-To: <049.d710ed16fabc938ed6d83c3181873a7a@tracker.bro-ids.org> References: <049.d710ed16fabc938ed6d83c3181873a7a@tracker.bro-ids.org> Message-ID: <064.973d4979e9977c0f99460821f9a5c58d@tracker.bro-ids.org> #912: Need option for ASCII logging writer to only print CSV header line ------------------------------+------------------------------- Reporter: carsten | Owner: Type: Feature Request | Status: new Priority: Normal | Milestone: Bro2.2 Component: Bro | Version: git/master Resolution: | Keywords: logging csv ascii ------------------------------+------------------------------- Comment (by robin): On Wed, Oct 31, 2012 at 08:54 -0000, you wrote: > But I checked the source code and it is not obvious to me how to use it. > It seems the feature of the config table is there, but none of the current > writers actually use it Almost true. :) The (dummy) {{{None}}} write actually prints out the values for debugging; they show up in the {{{WriterInfo::config}} table, see {{{logging/writers/None.cc}}}, lines 24-25. > b) Do you have a naming scheme in mind on how to name options and values? Not really (but I appreciate you asking!). As these are writer-specific values, we don't need to standardize them much but what you suggest sounds good: lower-cased with underscores, T/F for boolean values. Bernhard, please chime in if you have thoughts (he's using the equivalent mechanism for Input Framework readers). Robin -- Ticket URL: Bro Tracker Bro Issue Tracker From seth at icir.org Wed Oct 31 11:30:44 2012 From: seth at icir.org (Seth Hall) Date: Wed, 31 Oct 2012 14:30:44 -0400 Subject: [Bro-Dev] #914: topic/seth/intel-framework In-Reply-To: <20121031153314.GE30039@icir.org> References: <046.6acf3055033d7e1af81faf954effa510@tracker.bro-ids.org> <20121031153314.GE30039@icir.org> Message-ID: <96505D89-A4A5-43D4-9F35-5E1C788EE744@icir.org> On Oct 31, 2012, at 11:33 AM, Robin Sommer wrote: > Yippee! :) I know, finally. :) Go ahead and delete that branch after you merge it. I'll do the next steps in new topic branches after it's merged. I'm finishing up the mod bus analyzer and analysis scripts now. It's almost done (or at least as done as it's getting right now). .Seth -- Seth Hall International Computer Science Institute (Bro) because everyone has a network http://www.bro-ids.org/ From bro at tracker.bro-ids.org Wed Oct 31 14:35:51 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Wed, 31 Oct 2012 21:35:51 -0000 Subject: [Bro-Dev] #910: topic/dnthayer/installing-bro In-Reply-To: <050.bd46a17ef7eb423a5b5209b5d9a0fd99@tracker.bro-ids.org> References: <050.bd46a17ef7eb423a5b5209b5d9a0fd99@tracker.bro-ids.org> Message-ID: <065.64b367a174cca2770fa660ed27161fb5@tracker.bro-ids.org> #910: topic/dnthayer/installing-bro ----------------------------+------------------------ Reporter: dnthayer | Owner: Type: Merge Request | Status: new Priority: Normal | Milestone: Bro2.2 Component: Bro | Version: git/master Resolution: | Keywords: ----------------------------+------------------------ Comment (by robin): I've merged this but it seems we have now duplicated the 2.0->2.1 upgrade information (in upgrading.rst and in NEWS). We should consolidate the two (didn't compare if they have different things). If the plan going forward is to have everything in upgrading.rst, then NEWS should just point there. -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Wed Oct 31 14:36:20 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Wed, 31 Oct 2012 21:36:20 -0000 Subject: [Bro-Dev] #910: topic/dnthayer/installing-bro In-Reply-To: <050.bd46a17ef7eb423a5b5209b5d9a0fd99@tracker.bro-ids.org> References: <050.bd46a17ef7eb423a5b5209b5d9a0fd99@tracker.bro-ids.org> Message-ID: <065.e15c4d4313057b73e504afd24ab43e05@tracker.bro-ids.org> #910: topic/dnthayer/installing-bro -----------------------+------------------------ Reporter: dnthayer | Owner: dnthayer Type: Problem | Status: assigned Priority: Normal | Milestone: Bro2.2 Component: Bro | Version: git/master Resolution: | Keywords: -----------------------+------------------------ Changes (by robin): * owner: => dnthayer * status: new => assigned * type: Merge Request => Problem -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Wed Oct 31 14:53:14 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Wed, 31 Oct 2012 21:53:14 -0000 Subject: [Bro-Dev] #910: topic/dnthayer/installing-bro In-Reply-To: <050.bd46a17ef7eb423a5b5209b5d9a0fd99@tracker.bro-ids.org> References: <050.bd46a17ef7eb423a5b5209b5d9a0fd99@tracker.bro-ids.org> Message-ID: <065.5a5b0e58fc72c7ad03df70018ca497fe@tracker.bro-ids.org> #910: topic/dnthayer/installing-bro -----------------------+------------------------ Reporter: dnthayer | Owner: dnthayer Type: Problem | Status: assigned Priority: Normal | Milestone: Bro2.2 Component: Bro | Version: git/master Resolution: | Keywords: -----------------------+------------------------ Comment (by dnthayer): Replying to [comment:1 robin]: > I've merged this but it seems we have now duplicated the 2.0->2.1 upgrade information (in upgrading.rst and in NEWS). We should consolidate the two (didn't compare if they have different things). If the plan going forward is to have everything in upgrading.rst, then NEWS should just point there. I don't see any conflict between the two docs. The purpose of the upgrade.rst is to give a list of version-specific (e.g., 2.0 to 2.1) potential upgrade problems (things that could break a user's existing scripts). Such problems could include renamed Bro script variables or broctl options. Or, "configure" options that are now removed or have been renamed (such as the change involving the perftools options). The NEWS is a list of all changes, including new functionality (such new functionality will not break user's existing scripts, that's why those changes are not in upgrade.rst). The aim of upgrade.rst is to be as practical and concise as possible for users who are in a hurry to upgrade an existing installation. -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Wed Oct 31 16:21:37 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Wed, 31 Oct 2012 23:21:37 -0000 Subject: [Bro-Dev] #914: topic/seth/intel-framework In-Reply-To: <046.6acf3055033d7e1af81faf954effa510@tracker.bro-ids.org> References: <046.6acf3055033d7e1af81faf954effa510@tracker.bro-ids.org> Message-ID: <061.086948885e4a2d6c886de74fe6681113@tracker.bro-ids.org> #914: topic/seth/intel-framework ----------------------------+------------------------ Reporter: seth | Owner: robin Type: Merge Request | Status: new Priority: Normal | Milestone: Bro2.2 Component: Bro | Version: git/master Resolution: | Keywords: ----------------------------+------------------------ Comment (by robin): Good stuff. Questions: - why not load the various seen() handlers in base/* rather than policy/*? Isn't part of the beauty here that it will just find stuff once intel data has been loaded? - cluster.bro: {{{initial_sync}}} never gets reset; that doesn't seem to work if I restart everything except the manager, right? - {{{match_no_items}}} is not a very intuitive name imo :) - didn't you have some initial documentation as well, or do I misremember that? - Should scripts/policy/protocols/http/detect-intel.bro looks go now? Likewise, there are old tests in {{{scripts/base/frameworks/intel/}}} that use {{{Intel::matcher}}} -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Wed Oct 31 16:22:44 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Wed, 31 Oct 2012 23:22:44 -0000 Subject: [Bro-Dev] #914: topic/seth/intel-framework In-Reply-To: <046.6acf3055033d7e1af81faf954effa510@tracker.bro-ids.org> References: <046.6acf3055033d7e1af81faf954effa510@tracker.bro-ids.org> Message-ID: <061.c22b04021010b3df9119c442815b8494@tracker.bro-ids.org> #914: topic/seth/intel-framework ----------------------------+------------------------ Reporter: seth | Owner: robin Type: Merge Request | Status: new Priority: Normal | Milestone: Bro2.2 Component: Bro | Version: git/master Resolution: | Keywords: ----------------------------+------------------------ Comment (by robin): The latter point leads to something more general: is there a mix-up with the tests in some form? They actually all fail for me: {{{ scripts.base.frameworks.intel.cluster-transparency ... failed % 'btest-diff manager-1/intel.log' failed unexpectedly (exit code 1) % cat .diag == File =============================== #separator \x09 #set_separator , #empty_field (empty) #unset_field - #path intel #open 2012-10-31-23-21-25 #fields ts uid id.orig_h id.orig_p id.resp_h id.resp_p seen.host seen.str seen.str_type seen.where sources #types time string addr port addr port addr string enum enum table[string] 1351725685.541586 - - - - - 123.123.123.123 - - Intel::IN_ANYWHERE worker-1 #close 2012-10-31-23-21-35 == Diff =============================== --- /tmp/test-diff.5238.manager-1.intel.log.baseline.tmp 2012-10-31 23:21:39.234980964 +0000 +++ /tmp/test-diff.5238.manager-1.intel.log.tmp 2012-10-31 23:21:39.247981478 +0000 @@ -4,7 +4,7 @@ #unset_field - #path intel #open XXXX-XX-XX-XX-XX-XX -#fields ts uid id.orig_h id.orig_p id.resp_h id.resp_p seen.host seen.str seen.str_type seen.where -#types time string addr port addr port addr string enum enum -XXXXXXXXXX.XXXXXX - - - - - 123.123.123.123 - - Intel::IN_ANYWHERE +#fields ts uid id.orig_h id.orig_p id.resp_h id.resp_p seen.host seen.str seen.str_type seen.where sources +#types time string addr port addr port addr string enum enum table[string] +XXXXXXXXXX.XXXXXX - - - - - 123.123.123.123 - - Intel::IN_ANYWHERE worker-1 #close XXXX-XX-XX-XX-XX-XX ======================================= % cat .stderr <<< [5133] BROPATH=.:/home/robin/bro/master/scripts:/home/robin/bro/master/scripts/policy:/home/robin/bro/master/scripts/site:/home/robin/bro/master/build/src:.. CLUSTER_NODE=manager-1 bro /da/home/robin/bro/master/testing/btest/.tmp/scripts.base.frameworks.intel .cluster-transparency/cluster-transparency.bro >>> <<< [5147] BROPATH=.:/home/robin/bro/master/scripts:/home/robin/bro/master/scripts/policy:/home/robin/bro/master/scripts/site:/home/robin/bro/master/build/src:.. CLUSTER_NODE=worker-1 bro /da/home/robin/bro/master/testing/btest/.tmp/scripts.base.frameworks.intel .cluster-transparency/cluster-transparency.bro >>> <<< [5159] BROPATH=.:/home/robin/bro/master/scripts:/home/robin/bro/master/scripts/policy:/home/robin/bro/master/scripts/site:/home/robin/bro/master/build/src:.. CLUSTER_NODE=worker-2 bro /da/home/robin/bro/master/testing/btest/.tmp/scripts.base.frameworks.intel .cluster-transparency/cluster-transparency.bro >>> [ 25%] scripts.base.frameworks.intel.input-and-match ... failed % 'bro /da/home/robin/bro/master/testing/btest/.tmp/scripts.base.frameworks.intel .input-and-match/input-and-match.bro >out' failed unexpectedly (exit code 1) % cat .stderr error in /da/home/robin/bro/master/testing/btest/.tmp/scripts.base.frameworks.intel .input-and-match/input-and-match.bro, line 11: unknown identifier Intel::matcher, at or near "Intel::matcher" [ 50%] scripts.base.frameworks.intel.item-merge ... failed % 'bro /da/home/robin/bro/master/testing/btest/.tmp/scripts.base.frameworks.intel .item-merge/item-merge.bro >out' failed unexpectedly (exit code 1) % cat .stderr error in /da/home/robin/bro/master/testing/btest/.tmp/scripts.base.frameworks.intel .item-merge/item-merge.bro, line 6: unknown identifier Intel::MALICIOUS, at or near "Intel::MALICIOUS" [ 75%] scripts.base.frameworks.intel.matching ... failed % 'bro /da/home/robin/bro/master/testing/btest/.tmp/scripts.base.frameworks.intel.matching/matching.bro >out' failed unexpectedly (exit code 1) % cat .stderr error in /da/home/robin/bro/master/testing/btest/.tmp/scripts.base.frameworks.intel.matching/matching.bro, line 7: unknown identifier Intel::MALICIOUS, at or near "Intel::MALICIOUS" 4 of 4 tests failed }}} -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Wed Oct 31 17:23:19 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Thu, 01 Nov 2012 00:23:19 -0000 Subject: [Bro-Dev] #914: topic/seth/intel-framework In-Reply-To: <046.6acf3055033d7e1af81faf954effa510@tracker.bro-ids.org> References: <046.6acf3055033d7e1af81faf954effa510@tracker.bro-ids.org> Message-ID: <061.23cad5c650a8c4de443f4a04184a7772@tracker.bro-ids.org> #914: topic/seth/intel-framework ----------------------+------------------------ Reporter: seth | Owner: seth Type: Problem | Status: assigned Priority: Normal | Milestone: Bro2.2 Component: Bro | Version: git/master Resolution: | Keywords: ----------------------+------------------------ Changes (by robin): * owner: robin => seth * status: new => assigned * type: Merge Request => Problem Comment: The coverage tests fail, too. -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Wed Oct 31 17:30:21 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Thu, 01 Nov 2012 00:30:21 -0000 Subject: [Bro-Dev] #824: Default the connection and alarm summaries to once per day In-Reply-To: <056.dadb50a21009b083c39e1a280f60c9e9@tracker.bro-ids.org> References: <056.dadb50a21009b083c39e1a280f60c9e9@tracker.bro-ids.org> Message-ID: <071.eb86c1b47c7a7bd68a54b7333b0ab3d3@tracker.bro-ids.org> #824: Default the connection and alarm summaries to once per day -----------------------------+------------------------ Reporter: Tyler.Schoenke | Owner: robin Type: Merge Request | Status: closed Priority: Normal | Milestone: Bro2.2 Component: Bro | Version: git/master Resolution: fixed | Keywords: -----------------------------+------------------------ Changes (by robin): * owner: => robin * status: new => closed * resolution: => fixed Comment: In [b85baed95e7b8ba258fbe7c171c3981ee4b6506a/bro]: {{{ #!CommitTicketReference repository="bro" revision="b85baed95e7b8ba258fbe7c171c3981ee4b6506a" Merge remote-tracking branch 'origin/topic/dnthayer/alarms-mail' * origin/topic/dnthayer/alarms-mail: Rename the alarm summary mail variable Change alarm summary email interval to 24hrs Closes #824. }}} -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Wed Oct 31 17:30:21 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Thu, 01 Nov 2012 00:30:21 -0000 Subject: [Bro-Dev] #910: topic/dnthayer/installing-bro In-Reply-To: <050.bd46a17ef7eb423a5b5209b5d9a0fd99@tracker.bro-ids.org> References: <050.bd46a17ef7eb423a5b5209b5d9a0fd99@tracker.bro-ids.org> Message-ID: <065.580c984ade42e18e3c7b17a0a3700437@tracker.bro-ids.org> #910: topic/dnthayer/installing-bro -----------------------+------------------------ Reporter: dnthayer | Owner: dnthayer Type: Problem | Status: closed Priority: Normal | Milestone: Bro2.2 Component: Bro | Version: git/master Resolution: fixed | Keywords: -----------------------+------------------------ Changes (by robin): * status: assigned => closed * resolution: => fixed Comment: In [55006240ae9b8fadcc678f3d24fe76dc8d22fc31/bro]: {{{ #!CommitTicketReference repository="bro" revision="55006240ae9b8fadcc678f3d24fe76dc8d22fc31" Merge remote-tracking branch 'origin/topic/dnthayer/installing-bro' * origin/topic/dnthayer/installing-bro: Add version-specific upgrade info Move upgrade instructions to INSTALL document Reorganize install documentation Closes #910. }}} -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Wed Oct 31 17:30:36 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Thu, 01 Nov 2012 00:30:36 -0000 Subject: [Bro-Dev] #913: Option to specify interface in bro-aux/rst In-Reply-To: <052.98b639aa8cf2ab48dded52b0217ab42a@tracker.bro-ids.org> References: <052.98b639aa8cf2ab48dded52b0217ab42a@tracker.bro-ids.org> Message-ID: <067.562333ba4d45e95e4ae9ee11d07be83b@tracker.bro-ids.org> #913: Option to specify interface in bro-aux/rst ----------------------------+------------------------------- Reporter: grigorescu | Owner: robin Type: Merge Request | Status: closed Priority: Normal | Milestone: Bro2.2 Component: bro-aux | Version: git/master Resolution: fixed | Keywords: aux rst interface ----------------------------+------------------------------- Changes (by robin): * owner: => robin * status: new => closed * resolution: => fixed Comment: In [b02cb54fc79319d8574f474c927715fb3c20187e/bro-aux]: {{{ #!CommitTicketReference repository="bro-aux" revision="b02cb54fc79319d8574f474c927715fb3c20187e" Merge remote-tracking branch 'origin/topic/dnthayer/rst-interface' * origin/topic/dnthayer/rst-interface: Fix a typo Apply patch to add interface option to rst Closes #913. }}} -- Ticket URL: Bro Tracker Bro Issue Tracker From robin at icir.org Wed Oct 31 18:17:01 2012 From: robin at icir.org (Robin Sommer) Date: Wed, 31 Oct 2012 18:17:01 -0700 Subject: [Bro-Dev] [Bro-Commits-Internal] CompileDebug - Build # 235 - Failure! In-Reply-To: <657613901.1.1351730282106.JavaMail.jenkins@brotestbed.ncsa.illinois.edu> References: <657613901.1.1351730282106.JavaMail.jenkins@brotestbed.ncsa.illinois.edu> Message-ID: <20121101011701.GA37734@icir.org> On Wed, Oct 31, 2012 at 19:38 -0500, jenkins at brotestbed.ncsa.illinois.edu wrote: > /home/jenkins/workspace/CompileDebug/bro/aux/bro-aux/rst/rst.c:364: error: 'SO_BINDTODEVICE' undeclared (first use in this function) Yeah, I was afraid that this could cause trouble. Any way to make that portable? Robin -- Robin Sommer * Phone +1 (510) 722-6541 * robin at icir.org ICSI/LBNL * Fax +1 (510) 666-2956 * www.icir.org From bro at tracker.bro-ids.org Wed Oct 31 19:17:35 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Thu, 01 Nov 2012 02:17:35 -0000 Subject: [Bro-Dev] #914: topic/seth/intel-framework In-Reply-To: <046.6acf3055033d7e1af81faf954effa510@tracker.bro-ids.org> References: <046.6acf3055033d7e1af81faf954effa510@tracker.bro-ids.org> Message-ID: <061.04d5370010fcae6ba02b9d6e30148181@tracker.bro-ids.org> #914: topic/seth/intel-framework ----------------------+------------------------ Reporter: seth | Owner: seth Type: Problem | Status: assigned Priority: Normal | Milestone: Bro2.2 Component: Bro | Version: git/master Resolution: | Keywords: ----------------------+------------------------ Comment (by seth): > - why not load the various seen() handlers in base/* rather than policy/*? > Isn't part of the beauty here that it will just find stuff once intel data > has been loaded? Sending data to the Intel::seen function has implicit overhead and I wanted it to be optional if someone chooses to send data to it. My personal rule is that loading a framework from base/ shouldn't have any implicit overhead (other than loading the code into memory). We probably should go ahead and add that policy load into the local.bro script though since that would give back the magic. :) > - cluster.bro: {{{initial_sync}}} never gets reset; that doesn't seem to > work if I restart everything except the manager, right? Hm, that variable isn't in the code anymore (it was old). > - {{{match_no_items}}} is not a very intuitive name imo :) It's internal to the framework for doing minimal data matches. Users should never know about it. > - didn't you have some initial documentation as well, or do I misremember > that? doc/intel.rst > - Should scripts/policy/protocols/http/detect-intel.bro looks go now? > Likewise, there are old tests in {{{scripts/base/frameworks/intel/}}} that > use {{{Intel::matcher}}} Did you update? I removed all of those tests. I think you may be trying to merge an older version of my code. -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Wed Oct 31 20:26:27 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Thu, 01 Nov 2012 03:26:27 -0000 Subject: [Bro-Dev] #914: topic/seth/intel-framework In-Reply-To: <046.6acf3055033d7e1af81faf954effa510@tracker.bro-ids.org> References: <046.6acf3055033d7e1af81faf954effa510@tracker.bro-ids.org> Message-ID: <061.d608c601b3c1fdec6b7fdc2b30dbbb47@tracker.bro-ids.org> #914: topic/seth/intel-framework ----------------------+------------------------ Reporter: seth | Owner: seth Type: Problem | Status: assigned Priority: Normal | Milestone: Bro2.2 Component: Bro | Version: git/master Resolution: | Keywords: ----------------------+------------------------ Comment (by robin): On Thu, Nov 01, 2012 at 02:17 -0000, you wrote: > Did you update? I removed all of those tests. Hmmm .... Looks like I may have indeed a mixup of two versions here. Not sure how I did that, will try to get that worked out tomorrow. Robin -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Wed Oct 31 20:38:14 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Thu, 01 Nov 2012 03:38:14 -0000 Subject: [Bro-Dev] #915: topic/seth/modbus-merge Message-ID: <046.7fd5131d44e94e3cc695acf8bde185db@tracker.bro-ids.org> #915: topic/seth/modbus-merge ---------------------------+------------------------ Reporter: seth | Owner: robin Type: Merge Request | Status: new Priority: Normal | Milestone: Bro2.2 Component: Bro | Version: git/master Keywords: | ---------------------------+------------------------ This branch is ready for evaluation and merging. -- Ticket URL: Bro Tracker Bro Issue Tracker