From Tyler.Schoenke at colorado.edu Thu Dec 2 08:00:03 2010 From: Tyler.Schoenke at colorado.edu (Tyler T. Schoenke) Date: Thu, 02 Dec 2010 09:00:03 -0700 Subject: [Bro] scan.bro and missing log entries Message-ID: <4CF7C283.7010208@colorado.edu> I've been seeing AddressScan alerts, but when I check conn.log, I can't find the corresponding entries. I got an alert yesterday about a 5060/udp scan hitting 100 hosts. Below are the conn.log, flowscan, and notice.log for the entire day matching the IP and port. conn.log: Dec 1 11:27:45 0.000000 172.21.210.116 151.32.190.137 other 51272 5060 udp 101 ? S0 L flowscan: 12/01 11:27:45 172.21.210.116 151.32.190.137 17 51272 5060 1 129 notice.log: Dec 1 11:27:45 no=AddressScan na=NOTICE_EMAIL es=w5 sa=172.21.210.116 p=5060/udp num=100 msg=172.21.210.116\ has\ scanned\ 100\ hosts\ (5060/udp) tag=@62-7ba5-3df5e6 As you can see, at 11:27, Bro thinks 100 hosts were scanned on 5060/udp. But the conn.log and flowscan data only show one host being scanned. Any ideas why this alert thinks 100 hosts are being hit when it is one host with a single SYN? Tyler -- -- Tyler Schoenke Network Security Analyst IT Security Office University of Colorado - Boulder From vern at icir.org Thu Dec 2 08:13:06 2010 From: vern at icir.org (Vern Paxson) Date: Thu, 02 Dec 2010 08:13:06 -0800 Subject: [Bro] scan.bro and missing log entries In-Reply-To: <4CF7C283.7010208@colorado.edu> (Thu, 02 Dec 2010 09:00:03 MST). Message-ID: <20101202161306.2970B369FC7@taffy.ICSI.Berkeley.EDU> > I've been seeing AddressScan alerts, but when I check conn.log, I can't > find the corresponding entries. In general with these sorts of problems, it helps hugely if you can supply a trace that reproduces the problem, and also summarize the command line / analysis you're using. Vern From Tyler.Schoenke at colorado.edu Thu Dec 2 08:27:19 2010 From: Tyler.Schoenke at colorado.edu (Tyler T. Schoenke) Date: Thu, 02 Dec 2010 09:27:19 -0700 Subject: [Bro] scan.bro and missing log entries In-Reply-To: <20101202161306.2970B369FC7@taffy.ICSI.Berkeley.EDU> References: <20101202161306.2970B369FC7@taffy.ICSI.Berkeley.EDU> Message-ID: <4CF7C8E7.6030906@colorado.edu> On 12/02/2010 09:13 AM, Vern Paxson wrote: > In general with these sorts of problems, it helps hugely if you can supply > a trace that reproduces the problem, and also summarize the command line / > analysis you're using. > Any suggestions on how to grab a trace of these events? They are fairly random and infrequent. I've been thinking about running TimeMachine, but haven't had time to play with it. Tyler From JAzoff at uamail.albany.edu Thu Dec 2 08:38:08 2010 From: JAzoff at uamail.albany.edu (Justin Azoff) Date: Thu, 2 Dec 2010 11:38:08 -0500 Subject: [Bro] scan.bro and missing log entries In-Reply-To: <4CF7C283.7010208@colorado.edu> References: <4CF7C283.7010208@colorado.edu> Message-ID: <20101202163808.GI22071@datacomm.albany.edu> On Thu, Dec 02, 2010 at 11:00:03AM -0500, Tyler T. Schoenke wrote: > As you can see, at 11:27, Bro thinks 100 hosts were scanned on > 5060/udp. But the conn.log and flowscan data only show one host being > scanned. Any ideas why this alert thinks 100 hosts are being hit when > it is one host with a single SYN? Well if it was a udp scan for sip servers, there wouldn't be any SYN packets.. does conn.log normally record udp streams? -- -- Justin Azoff -- Network Security & Performance Analyst From gregor at icir.org Thu Dec 2 09:30:16 2010 From: gregor at icir.org (Gregor Maier) Date: Thu, 02 Dec 2010 09:30:16 -0800 Subject: [Bro] scan.bro and missing log entries In-Reply-To: <4CF7C283.7010208@colorado.edu> References: <4CF7C283.7010208@colorado.edu> Message-ID: <4CF7D7A8.1040008@icir.org> > As you can see, at 11:27, Bro thinks 100 hosts were scanned on > 5060/udp. But the conn.log and flowscan data only show one host being > scanned. Any ideas why this alert thinks 100 hosts are being hit when > it is one host with a single SYN? It might be the ConnectionCompressor, but I'm not 100% sure what the conn.log semantics are when the ConnectionCompressor is used. Robin, will know this. FYI, from ConnCompressor.cc: // The basic model of the compressor is to wait for an answer before // instantiating full connection state. Until we see a reply, only a minimal // amount of state is stored. This has some consequences: // // - We try to mimic TCP.cc as close as possible, but this works only to a // certain degree; e.g., we don't consider any of the wait-a-bit-after- // the-connection-has-been-closed timers. That means we will get differences // in connection semantics if the compressor is turned on. On the other // hand, these differences will occur only for not well-established // sessions, and experience shows that for these kinds of connections // semantics are ill-defined in any case. // // - If an originator sends multiple different packets before we see a reply, // we lose the information about additional packets (more precisely, we // merge the packet headers into one). In particular, we lose any payload. // This is a major problem if we see only one direction of a connection. // When analyzing only SYN/FIN/RSTs this leads to differences if we miss // the SYN/ACK. // // To avoid losing payload, there is the option cc_instantiate_on_data: // if enabled and the originator sends a non-control packet after the // initial packet, we instantiate full connection state. // // - We lose some of the information contained in initial packets (e.g., most // IP/TCP options and any payload). If you depend on them, you don't // want to use the compressor. // // Optionally, the compressor can take care only of initial SYNs and // instantiate full connection state for all other connection setups. // To enable, set cc_handle_only_syns to true. // // - The compressor may handle refused connections (i.e., initial packets // followed by RST from responder) itself. Again, this leads to differences // from default TCP processing and is therefore turned off by default. // To enable, set cc_handle_resets to true. // // - We don't match signatures on connections which are completely handled // by the compressor. Matching would require significant additional state // w/o being very helpful. // // - Trace rewriting doesn't work if the compressor is turned on (this is // not a conceptual problem, but simply not implemented). -- Gregor Maier gregor at icir.org Int. Computer Science Institute (ICSI) gregor at icsi.berkeley.edu 1947 Center St., Ste. 600 http://www.icir.org/gregor/ Berkeley, CA 94704 USA From vern at icir.org Thu Dec 2 10:09:41 2010 From: vern at icir.org (Vern Paxson) Date: Thu, 02 Dec 2010 10:09:41 -0800 Subject: [Bro] scan.bro and missing log entries In-Reply-To: <4CF7C8E7.6030906@colorado.edu> (Thu, 02 Dec 2010 09:27:19 MST). Message-ID: <20101202180941.322A6369FC7@taffy.ICSI.Berkeley.EDU> > Any suggestions on how to grab a trace of these events? They are fairly > random and infrequent. The usual way is to run bro with -w trace to generate a trace file of the traffic it analyzes. I sometimes run with (separate) full packet recording using tcpdump, because -w files don't always include everything Bro captured (there are mechanisms to not record some packets to it in an attempt to save space). As you note, the Time Machine is another possibility. Finally, Justin's observation about UDP is a good one. What flags and analyzer scripts are you using when running Bro? Vern From Tyler.Schoenke at colorado.edu Thu Dec 2 13:11:00 2010 From: Tyler.Schoenke at colorado.edu (Tyler T. Schoenke) Date: Thu, 02 Dec 2010 14:11:00 -0700 Subject: [Bro] scan.bro and missing log entries In-Reply-To: <20101202180941.322A6369FC7@taffy.ICSI.Berkeley.EDU> References: <20101202180941.322A6369FC7@taffy.ICSI.Berkeley.EDU> Message-ID: <4CF80B64.3040206@colorado.edu> On 12/02/2010 11:09 AM, Vern Paxson wrote: > > The usual way is to run bro with -w trace to generate a trace file of the > traffic it analyzes. I sometimes run with (separate) full packet recording > using tcpdump, because -w files don't always include everything Bro captured > (there are mechanisms to not record some packets to it in an attempt to > save space). > I am running a cluster on a span port that is receiving upwards of 1 Gbps. I'm guessing the -w would quickly fill my disk. I guess I should try to recreate the traffic myself. > Finally, Justin's observation about UDP is a good one. What flags and > analyzer scripts are you using when running Bro? > > I wasn't thinking about UDP not having a handshake. I saw the S0 and assumed that means SYN. I see that just means a connection attempt. It appears that conn.log is logging UDP streams. If there were 100+ scans from that IP address, those should have shown up in conn.log, right? I'm running a majority of the default scripts that are included in the default cluster configuration, Seth's scripts, some of my own, and a few others that I've collected. I have the capture filter set to ip. Tyler From robin at icir.org Thu Dec 2 13:28:53 2010 From: robin at icir.org (Robin Sommer) Date: Thu, 2 Dec 2010 13:28:53 -0800 Subject: [Bro] scan.bro and missing log entries In-Reply-To: <4CF7C283.7010208@colorado.edu> References: <4CF7C283.7010208@colorado.edu> Message-ID: <20101202212853.GH19523@icir.org> On Thu, Dec 02, 2010 at 09:00 -0700, Tyler T. Schoenke wrote: > As you can see, at 11:27, Bro thinks 100 hosts were scanned on > 5060/udp. Actually it means that 100 hosts have been scanned and the *last* attempt triggering the alert was on port 506 (not necessarily all). When you were checking conn.log, did you filter for all connections involving that IP or just those on port 5060? Robin -- Robin Sommer * Phone +1 (510) 722-6541 * robin at icir.org ICSI/LBNL * Fax +1 (510) 666-2956 * www.icir.org From robin at icir.org Thu Dec 2 13:31:36 2010 From: robin at icir.org (Robin Sommer) Date: Thu, 2 Dec 2010 13:31:36 -0800 Subject: [Bro] scan.bro and missing log entries In-Reply-To: <4CF7D7A8.1040008@icir.org> References: <4CF7C283.7010208@colorado.edu> <4CF7D7A8.1040008@icir.org> Message-ID: <20101202213136.GI19523@icir.org> On Thu, Dec 02, 2010 at 09:30 -0800, Gregor Maier wrote: > It might be the ConnectionCompressor, The connection compressor may be blamed for some issues, but not for everything. :-) Even if the compressor kicks in, the connections will still show up in conn.log, and they will be counted by the scan detector. Robin -- Robin Sommer * Phone +1 (510) 722-6541 * robin at icir.org ICSI/LBNL * Fax +1 (510) 666-2956 * www.icir.org From Tyler.Schoenke at colorado.edu Thu Dec 2 14:07:09 2010 From: Tyler.Schoenke at colorado.edu (Tyler T. Schoenke) Date: Thu, 02 Dec 2010 15:07:09 -0700 Subject: [Bro] scan.bro and missing log entries In-Reply-To: <20101202212853.GH19523@icir.org> References: <4CF7C283.7010208@colorado.edu> <20101202212853.GH19523@icir.org> Message-ID: <4CF8188D.1090400@colorado.edu> On 12/02/2010 02:28 PM, Robin Sommer wrote: > Actually it means that 100 hosts have been scanned and the *last* > attempt triggering the alert was on port 506 (not necessarily all). > When you were checking conn.log, did you filter for all connections > involving that IP or just those on port 5060? > > That would explain it. I'm guessing this machine was some sort of software like P2P or Skype. Is there a way to change the scanner so it only fires alerts when 100 hosts have been scanned on a single port? It seems P2P type applications tend to fire a lot of scan notifications. The other ones I see a lot are the Apple servers. Maybe people connecting to them for updates? Tyler From robin at icir.org Fri Dec 3 16:21:16 2010 From: robin at icir.org (Robin Sommer) Date: Fri, 3 Dec 2010 16:21:16 -0800 Subject: [Bro] scan.bro and missing log entries In-Reply-To: <4CF8188D.1090400@colorado.edu> References: <4CF7C283.7010208@colorado.edu> <20101202212853.GH19523@icir.org> <4CF8188D.1090400@colorado.edu> Message-ID: <20101204002116.GB70534@icir.org> On Thu, Dec 02, 2010 at 15:07 -0700, you wrote: > That would explain it. I'm guessing this machine was some sort of > software like P2P or Skype. Is there a way to change the scanner so it > only fires alerts when 100 hosts have been scanned on a single port? No, the script doesn't provide that currently. The problem is that it would require quite a bit more state to keep. I know that it would be useful though, others have been running into similar problems already. Perhaps we should think about adding that. Robin -- Robin Sommer * Phone +1 (510) 722-6541 * robin at icir.org ICSI/LBNL * Fax +1 (510) 666-2956 * www.icir.org From seth at icir.org Fri Dec 3 17:28:09 2010 From: seth at icir.org (Seth Hall) Date: Fri, 3 Dec 2010 20:28:09 -0500 Subject: [Bro] scan.bro and missing log entries In-Reply-To: <20101204002116.GB70534@icir.org> References: <4CF7C283.7010208@colorado.edu> <20101202212853.GH19523@icir.org> <4CF8188D.1090400@colorado.edu> <20101204002116.GB70534@icir.org> Message-ID: <3FA59EBF-FF00-43E2-BA9C-70DC7B506AB7@icir.org> On Dec 3, 2010, at 7:21 PM, Robin Sommer wrote: > No, the script doesn't provide that currently. The problem is that > it would require quite a bit more state to keep. I know that it > would be useful though, others have been running into similar > problems already. Perhaps we should think about adding that. Another option in this case is to improve the Skype analyzer so that Bro can recognize all Skype packets and not count them in scan detection. .Seth From estrada.veronica at gmail.com Sun Dec 5 02:06:52 2010 From: estrada.veronica at gmail.com (Veronica Estrada) Date: Sun, 5 Dec 2010 19:06:52 +0900 Subject: [Bro] Fragmentation and TCP overlapping Issues In-Reply-To: References: Message-ID: Hello, Sorry for sending again these questions to the list but I haven't received any answer to my old post in bro list. Besides, I want to add more details. We try to understand the behavior of BRO regarding fragmentation and other related issues. I am validating results with another IDS (Snort) and results are surprisingly different, but to understand the differences we need answers to the questions in e-mail below. For example, check question 7: in our traces, there is a file that contains corrupted headers and several packets. Snort output is different whether we use the original file or a truncated version: Original file (contains corrupted headers): DISCARD: 87.9% TCP: 11.751% OTHER: 1379 Total fragments: 36 TCP Overlaps: 1494 ALERTS: 1836 Processed packets: 3,771,244 (discard: 3311766) Same file but truncated since the point where wireshark triggers "corrupted header" and cannot read anymore DISCARD: 0 % TCP: 94% OTHER: 22819 Total fragments: 1072 (fragments reassembled 522) TCP Overlaps: 34580 ALERTS: 116877 Processed packets: 3,795,710 Original file 4Gb, Truncated file 3.9 Gb BRO output is identical for both files. Moreover, I cannot check how many discard packets or how BRO deals with the corrupted header packets. Regarding question 1 below, I am using libpcap 1.0.0. Thank you in advance, Veronica Estrada On Thu, Nov 25, 2010 at 12:25 AM, Veronica Estrada < estrada.veronica at gmail.com> wrote: > > Hello everyone, > > First of all, tomorrow is thanksgiving and I would like to thank all of you > for all the feedback I've always received to my posts. > > I continue with my research on anomalies, now focus on evasion techniques, > and I need to ask you some help to understand how BRO deals with > fragmentation and TCP overlapping issues. For reference, I am using Bro > 1.5.1 in offline analysis. > > > 1. Although I am loading "frag", I am not receiving any event related with > fragmentation. > > What could be wrong? libpcap library? my BRO version? > > > 2. What are the possible events triggered by weird analyzer related with > tcp overlapping? (because I am not getting any of them although I think I > should see them on my trace) > > > 3. TCP overlapping problems may generate "partial_ftp_request", > "partial_RPC_request" or other partial events? and also confuse BRO on how > the connection should be flagged? For example a connection with flag "S0", > no reply seen could be related with TCP overlapping problems? > > > 4. How does BRO perform TCP reassembly? I mean, is the traffic on ALL ports > reassembled? Is there any way to apply a default policy for doing TCP > reassembly? Like Policy First or Last or Unix? > > > 5. There is an "active mapping" function to improve TCP reassembly. Can we > define the host profile database without this active function? > > > 6. Can we configure the size of the reassembly buffer? I read in historical > msg (from 2006) there wasn't such config and BRO presented a vulnerability > against an adversary trying to exhaust memory, is this a current > possibility? > > > 7. By doing offline analysis, I understood that BRO will analyze all the > packets without loss even if the CPU is running at 100%. Still, I need > information about dropping packets for other reasons. For example, if BRO > encounters TCP overlapping, Does it drop all the packets? Choose some of > them? Are these actions log somewhere? The same with fragmentations issues. > Where can I check the portion of fragments that where reassembled? how many > frames discarded, etc? > > > 8. I am not seeing any difference in bro logs when I analyze 2 pcap files. > One file contains some malformed packet at the end and wireshark says "the > packet is bigger than 65535", the other pcap file is the same file but > truncated using editcap to avoid this "malformed packet" (if I check the hex > using hd, the part truncated represents 850MB ). All the logs of BRO when > input is one file or the other are identical. Is this the expected result? > > > Veronica Estrada > > Nakao Lab. Network System Research Group > > University of Tokyo > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20101205/a42d95c6/attachment.html From vern at icir.org Sun Dec 5 21:38:54 2010 From: vern at icir.org (Vern Paxson) Date: Sun, 05 Dec 2010 21:38:54 -0800 Subject: [Bro] Fragmentation and TCP overlapping Issues In-Reply-To: (Thu, 25 Nov 2010 00:25:36 +0900). Message-ID: <20101206053854.5B3CF36A038@taffy.ICSI.Berkeley.EDU> > 1. Although I am loading "frag", I am not receiving any event related with > fragmentation. > > What could be wrong? libpcap library? my BRO version? As usual, it helps a great deal if you include a trace and the command line you're using. One possibility is that you're trying to analyze UDP fragments, since the filter included by frag.bro only analyzes TCP fragments (to avoid heavy load from NFS traffic; this is vestigial, and makes sense to remove). > 2. What are the possible events triggered by weird analyzer related with tc= > p > overlapping? The analyzer doesn't look for overlap per se (which is not a TCP spec violation, and occurs for benign reasons; see our IEEE S&P 2008 paper). It looks for inconsistent TCP byte streams. For those, it doesn't use the "weird" interface but generates a distinct event, rexmit_inconsistency. > (because I am not getting any of them although I think I should > see them on my trace) Again, including a trace and the command line you're using to analyze it is far and away the best way to get help for problems like this. > 3. TCP overlapping problems may generate "partial_ftp_request", No. In this case, "partial" means that the connection was torn down in the middle of a request. It doesn't have anything to do with TCP overlap. > 4. How does BRO perform TCP reassembly? I mean, is the traffic on ALL ports > reassembled? Traffic to ports for which there's an analyzer that uses the byte stream. You can also contro lthis using tcp_reassembler_ports_orig and tcp_reassembler_ports_resp. > Is there any way to apply a default policy for doing TCP > reassembly? Like Policy First or Last or Unix No, this is not supported. > 5. There is an "active mapping" function to improve TCP reassembly. Can we > define the host profile database without this active function? No, that's the only way; and it is no longer supported. > 6. Can we configure the size of the reassembly buffer? I read in historical > msg (from 2006) there wasn't such config and BRO presented a vulnerability > against an adversary trying to exhaust memory, is this a current > possibility? See bro.init for the comments discussing tcp_max_above_hole_without_any_acks and tcp_excessive_data_without_further_acks. > 7. By doing offline analysis, I understood that BRO will analyze all the > packets without loss even if the CPU is running at 100%. Still, I need > information about dropping packets for other reasons. For example, if BRO > encounters TCP overlapping, Does it drop all the packets? No. It always delivers the byte stream as soon as it is in sequence. Subsequent overlapping-but-inconsistent packets generate the event mentioned above. > Choose some of > them? Always the first seen. > Are these actions log somewhere? I'm not sure what there is to log. Overlapping that's consistent is not any sort of necessarily peculiar phenomenon. Inconsistent overlapping leads to events. > The same with fragmentations issues. > Where can I check the portion of fragments that where reassembled? No, this isn't instrumented. > how many > frames discarded, etc? I don't know what you mean by discarded here. > 8. I am not seeing any difference in bro logs when I analyze 2 pcap files. Again, you need to send along traces and command lines. Vern From estrada.veronica at gmail.com Mon Dec 6 06:04:18 2010 From: estrada.veronica at gmail.com (Veronica Estrada) Date: Mon, 6 Dec 2010 23:04:18 +0900 Subject: [Bro] Fragmentation and TCP overlapping Issues In-Reply-To: <20101206053854.5B3CF36A038@taffy.ICSI.Berkeley.EDU> References: <20101206053854.5B3CF36A038@taffy.ICSI.Berkeley.EDU> Message-ID: Thank you very much! This information helps me a lot. Regarding fragmentation (question 1), I am running bro with this command: bro $files todai-nets -f "tcp or udp or icmp" dpd_conn_logs=T dpd detect-protocols dyn-disable detect-protocols-http proxy ssh irc-bot brolite print-globals capture-loss The variable $files contains 300 consecutive captured files (pcap) of 4 Gb each. In this experiment I am not getting any "fragment" event. On the contrary, fragmentation events appeared when bro was run using a subset of the aforementioned files. I don't have a record about the Bro parameters set for this case. Basically the difference here is how the input data to Bro was previously processed. In this case, we aggregated these small pcap files in bigger ones (80Gb). Every 20 files we applied ipsumdump to generate a bigger pcap file and feed Bro system so now, Bro triggers this type of events: excessively_small_fragment, fragment_inconsistency, fragment_overlap, fragment_size_inconsistency, fragment_with_DF What would be the reason for not getting these fragment events in the first experiment? Regarding the other questions about overlapping, I am preparing an e-mail with detail of the traces with problems. I cannot publish this on the list. Thank you again, Veronica On Mon, Dec 6, 2010 at 2:38 PM, Vern Paxson wrote: > > 1. Although I am loading "frag", I am not receiving any event related > with > > fragmentation. > > > > What could be wrong? libpcap library? my BRO version? > > As usual, it helps a great deal if you include a trace and the command > line you're using. > > One possibility is that you're trying to analyze UDP fragments, since the > filter included by frag.bro only analyzes TCP fragments (to avoid heavy > load from NFS traffic; this is vestigial, and makes sense to remove). > > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20101206/5e262660/attachment.html From vern at icir.org Mon Dec 6 09:51:44 2010 From: vern at icir.org (Vern Paxson) Date: Mon, 06 Dec 2010 09:51:44 -0800 Subject: [Bro] Fragmentation and TCP overlapping Issues In-Reply-To: (Mon, 06 Dec 2010 23:04:18 +0900). Message-ID: <20101206175144.938D036A41F@taffy.ICSI.Berkeley.EDU> > Regarding fragmentation (question 1), I am running bro with this command: > > bro $files todai-nets -f "tcp or udp or icmp" dpd_conn_logs=T dpd > detect-protocols dyn-disable detect-protocols-http proxy ssh irc-bot brolite > print-globals capture-loss What happens when you use bro ... -f "tcp or udp or icmp or (ip[6:2] & 0x3fff != 0)" ... ? > The variable $files contains 300 consecutive captured files (pcap) of 4 Gb > each. In this experiment I am not getting any "fragment" event. Again, we really need trace snippets to diagnose problems like these. You should extract a small subset of the trace that you believe should cause behavior different from what Bro does. Vern From estrada.veronica at gmail.com Tue Dec 7 05:23:29 2010 From: estrada.veronica at gmail.com (Veronica Estrada) Date: Tue, 7 Dec 2010 22:23:29 +0900 Subject: [Bro] Fragmentation and TCP overlapping Issues In-Reply-To: References: <20101206175144.938D036A41F@taffy.ICSI.Berkeley.EDU> Message-ID: Thank you very much for the help. Some feedback: 1. After including the filter -f "tcp or udp or icmp or (ip[6:2] & 0x3fff != 0)" the fragmentations events appeared. Now, I am re-processing the files but loading only weird and matching unfragmented datagrams only to accelerate the process. Command: bro -r $files -f (ip[6:2] & 0x3fff != 0)" weird All fragment events are handle by flow weird. When is invoked flow weird handler? 2. Related with a retransmission inconsistency. I found the remix inconsistency events in the notice file. 4. "Traffic to ports for which there's an analyzer that uses the byte stream. You ?can also control this using tcp_reassembler_ports_orig and tcp_reassembler_ports_resp." How can I redef these variables? ?I tried to redef this variables on my start policy but all I get are errors ((port and 21): error, arithmetic mixed with non-arithmetic). const tcp_reassembler_ports_orig: set[port] = {} &redef; const tcp_reassembler_ports_resp: set[port] = {} &redef; Could you please illustrate us with an example, for instance include port 21 & 80 in the tcp_reassebler_ports_orig? From sstattla at gmail.com Tue Dec 7 09:11:38 2010 From: sstattla at gmail.com (Sunjeet Singh) Date: Tue, 07 Dec 2010 09:11:38 -0800 Subject: [Bro] Software frontend Message-ID: <4CFE6ACA.60704@gmail.com> Hi, Does any one have a Click or other software frontend implementation that splits traffic to different nodes (and not cores)? Thank you, Sunjeet Singh From sstattla at gmail.com Tue Dec 7 09:14:50 2010 From: sstattla at gmail.com (Sunjeet Singh) Date: Tue, 07 Dec 2010 09:14:50 -0800 Subject: [Bro] Software frontend In-Reply-To: <4CFE6ACA.60704@gmail.com> References: <4CFE6ACA.60704@gmail.com> Message-ID: <4CFE6B8A.6040908@gmail.com> Sorry, forgot to add- Linux based. Sunjeet On 10-12-07 9:11 AM, Sunjeet Singh wrote: > Hi, > > Does any one have a Click or other software frontend implementation > that splits traffic to different nodes (and not cores)? > > Thank you, > Sunjeet Singh > > From Tyler.Schoenke at colorado.edu Tue Dec 7 10:05:09 2010 From: Tyler.Schoenke at colorado.edu (Tyler T. Schoenke) Date: Tue, 07 Dec 2010 11:05:09 -0700 Subject: [Bro] Software frontend In-Reply-To: <4CFE6ACA.60704@gmail.com> References: <4CFE6ACA.60704@gmail.com> Message-ID: <4CFE7755.5090103@colorado.edu> Is this what you are looking for? http://www.bro-ids.org/wiki/index.php/ClusterFrontendClickModularRouter Tyler -- Tyler Schoenke Network Security Analyst IT Security Office University of Colorado - Boulder On 12/7/10 10:11 AM, Sunjeet Singh wrote: > Hi, > > Does any one have a Click or other software frontend implementation that > splits traffic to different nodes (and not cores)? > > Thank you, > Sunjeet Singh > > > _______________________________________________ > Bro mailing list > bro at bro-ids.org > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro > From sstattla at gmail.com Tue Dec 7 10:09:50 2010 From: sstattla at gmail.com (Sunjeet Singh) Date: Tue, 07 Dec 2010 10:09:50 -0800 Subject: [Bro] Software frontend In-Reply-To: <4CFE7755.5090103@colorado.edu> References: <4CFE6ACA.60704@gmail.com> <4CFE7755.5090103@colorado.edu> Message-ID: <4CFE786E.9090300@gmail.com> Thanks Tyler. From my understanding, this would be used to split traffic across cores on the same worker machine. Can this be extended to get what I want- split traffic from the fronted (which will be running this Click daemon) to workers running on different machines? Thanks, Sunjeet On 10-12-07 10:05 AM, Tyler T. Schoenke wrote: > Is this what you are looking for? > > http://www.bro-ids.org/wiki/index.php/ClusterFrontendClickModularRouter > > Tyler > > -- > Tyler Schoenke > Network Security Analyst > IT Security Office > University of Colorado - Boulder > > > On 12/7/10 10:11 AM, Sunjeet Singh wrote: >> Hi, >> >> Does any one have a Click or other software frontend implementation that >> splits traffic to different nodes (and not cores)? >> >> Thank you, >> Sunjeet Singh >> >> >> _______________________________________________ >> Bro mailing list >> bro at bro-ids.org >> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro >> From Tyler.Schoenke at colorado.edu Tue Dec 7 10:38:47 2010 From: Tyler.Schoenke at colorado.edu (Tyler T. Schoenke) Date: Tue, 07 Dec 2010 11:38:47 -0700 Subject: [Bro] Software frontend In-Reply-To: <4CFE786E.9090300@gmail.com> References: <4CFE6ACA.60704@gmail.com> <4CFE7755.5090103@colorado.edu> <4CFE786E.9090300@gmail.com> Message-ID: <4CFE7F37.9080408@colorado.edu> You are correct, this only splits traffic across workers on the same machine. I've investigated, but haven't had time to test splitting traffic across workers on different machines. You should be able to tweak the linked config a little by removing the tapX lines and redirecting the my_switch outputs to the various physical interfaces. For example: my_switch[0] -> Queue -> eth1; #(repeat for eth2... ethX) I haven't tried this, but it should work. This software-based load balancing will only work for smallish amounts of traffic. If you are trying to feed upwards of 1 Gbps, the user mode Click will probably choke. I started to investigate using kernel mode Click with the RouteBricks code to improve performance, but got stuck at a kernel panic and didn't have time to pursue it further. For that, you need a multi-core Nehalem server with Intel 10Gbps 82598EB cards. The best solution is probably to buy a hardware load balancer like the cPacket cFlow device. Currently, they have a 10Gbps version, but heard they are working on a 40Gbps version. Other people have used Cisco routers, or other hardware load balancers. It would be nice to find a low-cost and effective software-based load balancer, but I haven't seen anything yet. Right now, I am using Click! and dropping a significant fraction of our traffic to cope with the limitations of running the software load balancer and workers on one multi-core mid-range server. Tyler On 12/7/10 11:09 AM, Sunjeet Singh wrote: > Thanks Tyler. From my understanding, this would be used to split traffic > across cores on the same worker machine. > > Can this be extended to get what I want- split traffic from the fronted > (which will be running this Click daemon) to workers running on > different machines? > > Thanks, > Sunjeet > > > On 10-12-07 10:05 AM, Tyler T. Schoenke wrote: >> Is this what you are looking for? >> >> http://www.bro-ids.org/wiki/index.php/ClusterFrontendClickModularRouter >> >> Tyler >> >> -- >> Tyler Schoenke >> Network Security Analyst >> IT Security Office >> University of Colorado - Boulder >> >> >> On 12/7/10 10:11 AM, Sunjeet Singh wrote: >>> Hi, >>> >>> Does any one have a Click or other software frontend implementation that >>> splits traffic to different nodes (and not cores)? >>> >>> Thank you, >>> Sunjeet Singh >>> >>> >>> _______________________________________________ >>> Bro mailing list >>> bro at bro-ids.org >>> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro >>> From sstattla at gmail.com Tue Dec 7 12:10:22 2010 From: sstattla at gmail.com (Sunjeet Singh) Date: Tue, 07 Dec 2010 12:10:22 -0800 Subject: [Bro] Software frontend In-Reply-To: <4CFE7F37.9080408@colorado.edu> References: <4CFE6ACA.60704@gmail.com> <4CFE7755.5090103@colorado.edu> <4CFE786E.9090300@gmail.com> <4CFE7F37.9080408@colorado.edu> Message-ID: <4CFE94AE.3040301@gmail.com> Thanks again, Tyler for your reply. Right now, Software-based load balancing is my only option. And I'm not too worried about the 1Gbps performance. Why, you might ask, do I want to deploy a cluster in the first place? Here's why- I am trying to implement the Bro cluster on an Amazon EC2 cloud- just for fun and in an effort to learn about cloud computing. I am interested in seeing what challenges arise from porting a cluster-based implementation to a cloud. I have the manager, proxy and worker set up. But I need a front-end to make use of more than one worker machines. So I'm not worried about the front-end bottleneck, I just want to get the architecture running. Now, considering the Click implementation that you described- > You are correct, this only splits traffic across workers on the same > machine. I've investigated, but haven't had time to test splitting > traffic across workers on different machines. You should be able to > tweak the linked config a little by removing the tapX lines and > redirecting the my_switch outputs to the various physical interfaces. > For example: > > my_switch[0] -> Queue -> eth1; #(repeat for eth2... ethX) Even if this does forward the packet to the eth1 interface (sending out of eth1 I assume), we haven't done the part where the packet goes from the interface to the right worker machine (which is done by rewriting the MAC address on the packet I suppose). > It would be nice to find a low-cost and effective software-based load > balancer, but I haven't seen anything yet. Right now, I am using > Click! and dropping a significant fraction of our traffic to cope with > the limitations of running the software load balancer and workers on > one multi-core mid-range server. > Thank you for sharing your experience. This helps a long way. Sunjeet From Tyler.Schoenke at colorado.edu Tue Dec 7 12:34:54 2010 From: Tyler.Schoenke at colorado.edu (Tyler T. Schoenke) Date: Tue, 07 Dec 2010 13:34:54 -0700 Subject: [Bro] Software frontend In-Reply-To: <4CFE94AE.3040301@gmail.com> References: <4CFE6ACA.60704@gmail.com> <4CFE7755.5090103@colorado.edu> <4CFE786E.9090300@gmail.com> <4CFE7F37.9080408@colorado.edu> <4CFE94AE.3040301@gmail.com> Message-ID: <4CFE9A6E.9080506@colorado.edu> > Even if this does forward the packet to the eth1 interface (sending out > of eth1 I assume), we haven't done the part where the packet goes from > the interface to the right worker machine (which is done by rewriting > the MAC address on the packet I suppose). > I had written a config to do that, but never tested it. Here are the basics. AddressInfo(mymac 10.0.0.1/8 1:1:1:1:1:1); AddressInfo(worker1 10.0.0.2/8 2:2:2:2:2:2); AddressInfo(worker2 10.0.0.3/8 3:3:3:3:3:3); my_switch :: HashSwitch(26, 8); FromDevice(eth1, PROMISC true, BURST 8 ) -> my_switch; todevice1::ToDevice(eth2, ALLOW_NONEXISTENT true); todevice2::ToDevice(eth3, ALLOW_NONEXISTENT true); //example: my_switch[0] -> EtherEncap(0x0800, 1:1:1:1:1:1, 2:2:2:2:2:2) -> Queue -> ToDevice(eth2, ALLOW_NONEXISTENT true); my_switch[0] -> EtherEncap(0x0800, mymac, worker1) -> Queue -> todevice1; my_switch[1] -> EtherEncap(0x0800, mymac, worker2) -> Queue -> todevice2; or if you just want to bypass the MAC rewrite to test that traffic is being load balanced: my_switch[0] -> Queue -> todevice1; mymac would be the MAC of the interface receiving the traffic, worker1 and worker2 need to be set to the MAC of the worker machines. I was hoping this would take in the traffic, load-balance, rewrite the MAC and send it out to several interfaces. I think this is where I was getting a kernel crash and didn't have time to upgrade the kernel. I think it was on CentOS 5.3, and I saw patch references to the kernel error I was getting. Tyler From sstattla at gmail.com Tue Dec 7 13:50:36 2010 From: sstattla at gmail.com (Sunjeet Singh) Date: Tue, 07 Dec 2010 13:50:36 -0800 Subject: [Bro] Software frontend In-Reply-To: <4CFE9A6E.9080506@colorado.edu> References: <4CFE6ACA.60704@gmail.com> <4CFE7755.5090103@colorado.edu> <4CFE786E.9090300@gmail.com> <4CFE7F37.9080408@colorado.edu> <4CFE94AE.3040301@gmail.com> <4CFE9A6E.9080506@colorado.edu> Message-ID: <4CFEAC2C.8030006@gmail.com> Hi Tyler, Can you please help me troubleshoot here? I did what you said (on Linux so some function option-parameters are gone), and here's what my click script looks like (currently testing with one frontend machine and one worker machine)- AddressInfo(mymac ); AddressInfo(worker1 ); AddressInfo(worker2 ); AddressInfo(worker3 ); my_switch :: HashSwitch(26, 8); FromDevice(eth4, PROMISC true) -> my_switch; todevice1 :: ToDevice(eth0); todevice2 :: ToDevice(eth1); todevice3 :: ToDevice(eth2); my_switch[0] -> EtherEncap(0x0800, mymac, worker1) -> Queue -> todevice1; my_switch[1] -> EtherEncap(0x0800, mymac, worker2) -> Queue -> todevice2; my_switch[2] -> EtherEncap(0x0800, mymac, worker3) -> Queue -> todevice3; When I run the script with the command "sudo click try.click", it starts executing and gives no messages. To test it, I used tcpdump to first see if any of the interfaces on worker1 is receiving any traffic -> No. Then I checked if any traffic is going out of eth0, eth1 or eth2 on frontend -> No. Checked if eth4 is receiving the packets I sent through tcpreplay -> Yes. How can I go about debugging this? Thanks, Sunjeet On 10-12-07 12:34 PM, Tyler T. Schoenke wrote: > >> Even if this does forward the packet to the eth1 interface (sending out >> of eth1 I assume), we haven't done the part where the packet goes from >> the interface to the right worker machine (which is done by rewriting >> the MAC address on the packet I suppose). >> > I had written a config to do that, but never tested it. Here are the > basics. > > AddressInfo(mymac 10.0.0.1/8 1:1:1:1:1:1); > AddressInfo(worker1 10.0.0.2/8 2:2:2:2:2:2); > AddressInfo(worker2 10.0.0.3/8 3:3:3:3:3:3); > > my_switch :: HashSwitch(26, 8); > > FromDevice(eth1, PROMISC true, BURST 8 ) -> my_switch; > todevice1::ToDevice(eth2, ALLOW_NONEXISTENT true); > todevice2::ToDevice(eth3, ALLOW_NONEXISTENT true); > > > //example: my_switch[0] -> EtherEncap(0x0800, 1:1:1:1:1:1, > 2:2:2:2:2:2) -> Queue -> ToDevice(eth2, ALLOW_NONEXISTENT true); > my_switch[0] -> EtherEncap(0x0800, mymac, worker1) -> Queue -> todevice1; > my_switch[1] -> EtherEncap(0x0800, mymac, worker2) -> Queue -> todevice2; > > or if you just want to bypass the MAC rewrite to test that traffic is > being load balanced: > > my_switch[0] -> Queue -> todevice1; > > mymac would be the MAC of the interface receiving the traffic, worker1 > and worker2 need to be set to the MAC of the worker machines. I was > hoping this would take in the traffic, load-balance, rewrite the MAC > and send it out to several interfaces. I think this is where I was > getting a kernel crash and didn't have time to upgrade the kernel. I > think it was on CentOS 5.3, and I saw patch references to the kernel > error I was getting. > > Tyler > > From Tyler.Schoenke at colorado.edu Tue Dec 7 14:20:20 2010 From: Tyler.Schoenke at colorado.edu (Tyler T. Schoenke) Date: Tue, 07 Dec 2010 15:20:20 -0700 Subject: [Bro] Software frontend In-Reply-To: <4CFEAC2C.8030006@gmail.com> References: <4CFE6ACA.60704@gmail.com> <4CFE7755.5090103@colorado.edu> <4CFE786E.9090300@gmail.com> <4CFE7F37.9080408@colorado.edu> <4CFE94AE.3040301@gmail.com> <4CFE9A6E.9080506@colorado.edu> <4CFEAC2C.8030006@gmail.com> Message-ID: <4CFEB324.9010108@colorado.edu> Hi Sunjeet, I have hardly worked with Click, and don't have experience with troubleshooting it. Your best bet is to sign up with their mailing list at the bottom of the page on the following link. http://read.cs.ucla.edu/click/ If you get it working, please follow up with the working configuration. Tyler On 12/7/10 2:50 PM, Sunjeet Singh wrote: > Hi Tyler, > > Can you please help me troubleshoot here? I did what you said (on Linux > so some function option-parameters are gone), and here's what my click > script looks like (currently testing with one frontend machine and one > worker machine)- > > AddressInfo(mymac ); > AddressInfo(worker1 ); > AddressInfo(worker2 ); > AddressInfo(worker3 ); > > my_switch :: HashSwitch(26, 8); > > FromDevice(eth4, PROMISC true) -> my_switch; > todevice1 :: ToDevice(eth0); > todevice2 :: ToDevice(eth1); > todevice3 :: ToDevice(eth2); > > my_switch[0] -> EtherEncap(0x0800, mymac, worker1) -> Queue -> todevice1; > my_switch[1] -> EtherEncap(0x0800, mymac, worker2) -> Queue -> todevice2; > my_switch[2] -> EtherEncap(0x0800, mymac, worker3) -> Queue -> todevice3; > > > When I run the script with the command "sudo click try.click", it starts > executing and gives no messages. To test it, > I used tcpdump to first see if any of the interfaces on worker1 is > receiving any traffic -> No. > Then I checked if any traffic is going out of eth0, eth1 or eth2 on > frontend -> No. > Checked if eth4 is receiving the packets I sent through tcpreplay -> Yes. > > How can I go about debugging this? > > Thanks, > Sunjeet > > > On 10-12-07 12:34 PM, Tyler T. Schoenke wrote: >>> Even if this does forward the packet to the eth1 interface (sending out >>> of eth1 I assume), we haven't done the part where the packet goes from >>> the interface to the right worker machine (which is done by rewriting >>> the MAC address on the packet I suppose). >>> >> I had written a config to do that, but never tested it. Here are the >> basics. >> >> AddressInfo(mymac 10.0.0.1/8 1:1:1:1:1:1); >> AddressInfo(worker1 10.0.0.2/8 2:2:2:2:2:2); >> AddressInfo(worker2 10.0.0.3/8 3:3:3:3:3:3); >> >> my_switch :: HashSwitch(26, 8); >> >> FromDevice(eth1, PROMISC true, BURST 8 ) -> my_switch; >> todevice1::ToDevice(eth2, ALLOW_NONEXISTENT true); >> todevice2::ToDevice(eth3, ALLOW_NONEXISTENT true); >> >> >> //example: my_switch[0] -> EtherEncap(0x0800, 1:1:1:1:1:1, >> 2:2:2:2:2:2) -> Queue -> ToDevice(eth2, ALLOW_NONEXISTENT true); >> my_switch[0] -> EtherEncap(0x0800, mymac, worker1) -> Queue -> todevice1; >> my_switch[1] -> EtherEncap(0x0800, mymac, worker2) -> Queue -> todevice2; >> >> or if you just want to bypass the MAC rewrite to test that traffic is >> being load balanced: >> >> my_switch[0] -> Queue -> todevice1; >> >> mymac would be the MAC of the interface receiving the traffic, worker1 >> and worker2 need to be set to the MAC of the worker machines. I was >> hoping this would take in the traffic, load-balance, rewrite the MAC >> and send it out to several interfaces. I think this is where I was >> getting a kernel crash and didn't have time to upgrade the kernel. I >> think it was on CentOS 5.3, and I saw patch references to the kernel >> error I was getting. >> >> Tyler >> >> From estrada.veronica at gmail.com Wed Dec 8 00:59:46 2010 From: estrada.veronica at gmail.com (Veronica Estrada) Date: Wed, 8 Dec 2010 17:59:46 +0900 Subject: [Bro] Fragmentation and TCP overlapping Issues In-Reply-To: <20101206053854.5B3CF36A038@taffy.ICSI.Berkeley.EDU> References: <20101206053854.5B3CF36A038@taffy.ICSI.Berkeley.EDU> Message-ID: 8. By the way, before sending you the trace we did more research and we suspect that the problem is generated by the machine used for the capture. So we are going to check with the vendor. Thanks for the help. On Mon, Dec 6, 2010 at 2:38 PM, Vern Paxson wrote: >> 1. Although I am loading "frag", I am not receiving any event related with >> fragmentation. >> >> What could be wrong? libpcap library? my BRO version? > > As usual, it helps a great deal if you include a trace and the command > line you're using. > > One possibility is that you're trying to analyze UDP fragments, since the > filter included by frag.bro only analyzes TCP fragments (to avoid heavy > load from NFS traffic; this is vestigial, and makes sense to remove). > >> 2. What are the possible events triggered by weird analyzer related with tc= >> p >> overlapping? > > The analyzer doesn't look for overlap per se (which is not a TCP spec > violation, and occurs for benign reasons; see our IEEE S&P 2008 paper). > It looks for inconsistent TCP byte streams. ?For those, it doesn't use the > "weird" interface but generates a distinct event, rexmit_inconsistency. > >> (because I am not getting any of them although I think I should >> see them on my trace) > > Again, including a trace and the command line you're using to analyze > it is far and away the best way to get help for problems like this. > >> 3. TCP overlapping problems may generate "partial_ftp_request", > > No. ?In this case, "partial" means that the connection was torn down > in the middle of a request. ?It doesn't have anything to do with TCP > overlap. > >> 4. How does BRO perform TCP reassembly? I mean, is the traffic on ALL ports >> reassembled? > > Traffic to ports for which there's an analyzer that uses the byte stream. > You ?can also contro lthis using tcp_reassembler_ports_orig and > tcp_reassembler_ports_resp. > >> Is there any way to apply a default policy for doing TCP >> reassembly? Like Policy First or Last or Unix > > No, this is not supported. > >> 5. There is an "active mapping" function to improve TCP reassembly. Can we >> define the host profile database without this active function? > > No, that's the only way; and it is no longer supported. > >> 6. Can we configure the size of the reassembly buffer? I read in historical >> msg (from 2006) there wasn't such config and BRO presented a vulnerability >> against an adversary trying to exhaust memory, is this a current >> possibility? > > See bro.init for the comments discussing tcp_max_above_hole_without_any_acks > and tcp_excessive_data_without_further_acks. > >> 7. By doing offline analysis, I understood that BRO will analyze all the >> packets without loss even if the CPU is running at 100%. Still, I need >> information about dropping packets for other reasons. For example, if BRO >> encounters TCP overlapping, Does it drop all the packets? > > No. ?It always delivers the byte stream as soon as it is in sequence. > Subsequent overlapping-but-inconsistent packets generate the event > mentioned above. > >> Choose some of >> them? > > Always the first seen. > >> Are these actions log somewhere? > > I'm not sure what there is to log. ?Overlapping that's consistent is not > any sort of necessarily peculiar phenomenon. ?Inconsistent overlapping leads > to events. > >> The same with fragmentations issues. >> Where can I check the portion of fragments that where reassembled? > > No, this isn't instrumented. > >> how many >> frames discarded, etc? > > I don't know what you mean by discarded here. > >> 8. I am not seeing any difference in bro logs when I analyze 2 pcap files. > > Again, you need to send along traces and command lines. > > ? ? ? ? ? ? ? ?Vern > From seth at icir.org Wed Dec 8 13:45:25 2010 From: seth at icir.org (Seth Hall) Date: Wed, 8 Dec 2010 16:45:25 -0500 Subject: [Bro] Bro-Dev mailing list Message-ID: <9F9D5681-9461-49AD-87F1-0953BA2CC6EE@icir.org> Hi all, I wanted to send out a quick note about the public availability of the bro-dev mailing list. As some of you know, this list has existed in some form for a long time but it was by invitation only. The list is being opened to everyone to help introduce new developers into the fold and make visible the development process for Bro. I'd encourage everyone that is interested in developing scripts with Bro to join this list because beginning next month I'm going to be starting bi-weekly script review periods. On Mondays and Wednesdays for a currently undefined time period I'll be sending my thoughts on a several existing Bro scripts to the mailing list and inviting everyone to give their own comments about the scripts and potentially their associated analyzers. The goal is to have a clean and documented policy script directory for the 1.6 release along with adding lots of interesting detections out of the box! Even if you don't have comments about what you'd like to see scripts do, it could be worthwhile lurking to see the direction that Bro is moving. One final comment I wanted to make is that the list is kind of high volume at the moment due to our ticket tracker sending activity emails to the list and generally a lot of email activity, so subscriber beware (and prepare to filter!). :) Here's the list interface page: http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev See you on the bro-dev list! .Seth From vern at icir.org Wed Dec 8 21:06:43 2010 From: vern at icir.org (Vern Paxson) Date: Wed, 08 Dec 2010 21:06:43 -0800 Subject: [Bro] Fragmentation and TCP overlapping Issues In-Reply-To: (Tue, 07 Dec 2010 22:23:29 +0900). Message-ID: <20101209050643.C342236A017@taffy.ICSI.Berkeley.EDU> > All fragment events are handle by flow weird. When is invoked flow > weird handler? It's only used for packets that are so broken that Bro can't reliably associate them with a connection. > How can I redef these variables? I tried to redef this variables on > my start policy but all I get are errors ((port and 21): error, If you want TCP port 21 then you specify it as "21/tcp", not just "21". Vern From estrada.veronica at gmail.com Wed Dec 8 22:03:21 2010 From: estrada.veronica at gmail.com (Veronica Estrada) Date: Thu, 9 Dec 2010 15:03:21 +0900 Subject: [Bro] Fragmentation and TCP overlapping Issues In-Reply-To: <20101209050643.C342236A017@taffy.ICSI.Berkeley.EDU> References: <20101209050643.C342236A017@taffy.ICSI.Berkeley.EDU> Message-ID: And now it works! :) On Thu, Dec 9, 2010 at 2:06 PM, Vern Paxson wrote: > > All fragment events are handle by flow weird. When is invoked flow > > weird handler? > > It's only used for packets that are so broken that Bro can't reliably > associate them with a connection. > > > How can I redef these variables? I tried to redef this variables on > > my start policy but all I get are errors ((port and 21): error, > > If you want TCP port 21 then you specify it as "21/tcp", not just "21". > > Vern > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20101209/8ae06253/attachment.html From sstattla at gmail.com Thu Dec 9 11:03:56 2010 From: sstattla at gmail.com (Sunjeet Singh) Date: Thu, 09 Dec 2010 11:03:56 -0800 Subject: [Bro] Software frontend In-Reply-To: <4CFEB324.9010108@colorado.edu> References: <4CFE6ACA.60704@gmail.com> <4CFE7755.5090103@colorado.edu> <4CFE786E.9090300@gmail.com> <4CFE7F37.9080408@colorado.edu> <4CFE94AE.3040301@gmail.com> <4CFE9A6E.9080506@colorado.edu> <4CFEAC2C.8030006@gmail.com> <4CFEB324.9010108@colorado.edu> Message-ID: <4D01281C.1060001@gmail.com> Good news is that I got it to work for a simple case- 1 Frontend node and 1 Worker node. So the job is use Click to simply relay all traffic entering Frontend to a Worker node. Here are the steps- 1. Enterprise traffic comes in at Frontend's interface eth0 2. Source MAC Address of the packet is changed to Frontend's eth3's MAC address 3. Destination MAC Address of packet is changed to Worker's eth3's MAC address 4. Packet is sent to Frontend's eth3 for delivery. The Click script looks like- FromDevice(eth0, PROMISC true) -> StoreEtherAddress(an:on:ny:mi:ze:d1, OFFSET src) -> StoreEtherAddress(an:on:ny:mi:ze:d2, OFFSET dst) -> Queue -> ToDevice(eth3); To debug, I simply used tcpdump to check at interfaces eth0 and eth3 on Frontend, and interface eth3 (with -e flag for displaying MAC address) to check if the packets were reaching. For multiple workers, one can use a Hash function (as described earlier in this thread) to calculate the destination Worker MAC address. Sunjeet On 10-12-07 2:20 PM, Tyler T. Schoenke wrote: > Hi Sunjeet, > > I have hardly worked with Click, and don't have experience with > troubleshooting it. Your best bet is to sign up with their mailing > list at the bottom of the page on the following link. > http://read.cs.ucla.edu/click/ > > If you get it working, please follow up with the working configuration. > > Tyler > > On 12/7/10 2:50 PM, Sunjeet Singh wrote: >> Hi Tyler, >> >> Can you please help me troubleshoot here? I did what you said (on Linux >> so some function option-parameters are gone), and here's what my click >> script looks like (currently testing with one frontend machine and one >> worker machine)- >> >> AddressInfo(mymac ); >> AddressInfo(worker1 ); >> AddressInfo(worker2 ); >> AddressInfo(worker3 ); >> >> my_switch :: HashSwitch(26, 8); >> >> FromDevice(eth4, PROMISC true) -> my_switch; >> todevice1 :: ToDevice(eth0); >> todevice2 :: ToDevice(eth1); >> todevice3 :: ToDevice(eth2); >> >> my_switch[0] -> EtherEncap(0x0800, mymac, worker1) -> Queue -> >> todevice1; >> my_switch[1] -> EtherEncap(0x0800, mymac, worker2) -> Queue -> >> todevice2; >> my_switch[2] -> EtherEncap(0x0800, mymac, worker3) -> Queue -> >> todevice3; >> >> >> When I run the script with the command "sudo click try.click", it starts >> executing and gives no messages. To test it, >> I used tcpdump to first see if any of the interfaces on worker1 is >> receiving any traffic -> No. >> Then I checked if any traffic is going out of eth0, eth1 or eth2 on >> frontend -> No. >> Checked if eth4 is receiving the packets I sent through tcpreplay -> >> Yes. >> >> How can I go about debugging this? >> >> Thanks, >> Sunjeet >> >> >> On 10-12-07 12:34 PM, Tyler T. Schoenke wrote: >>>> Even if this does forward the packet to the eth1 interface (sending >>>> out >>>> of eth1 I assume), we haven't done the part where the packet goes from >>>> the interface to the right worker machine (which is done by rewriting >>>> the MAC address on the packet I suppose). >>>> >>> I had written a config to do that, but never tested it. Here are the >>> basics. >>> >>> AddressInfo(mymac 10.0.0.1/8 1:1:1:1:1:1); >>> AddressInfo(worker1 10.0.0.2/8 2:2:2:2:2:2); >>> AddressInfo(worker2 10.0.0.3/8 3:3:3:3:3:3); >>> >>> my_switch :: HashSwitch(26, 8); >>> >>> FromDevice(eth1, PROMISC true, BURST 8 ) -> my_switch; >>> todevice1::ToDevice(eth2, ALLOW_NONEXISTENT true); >>> todevice2::ToDevice(eth3, ALLOW_NONEXISTENT true); >>> >>> >>> //example: my_switch[0] -> EtherEncap(0x0800, 1:1:1:1:1:1, >>> 2:2:2:2:2:2) -> Queue -> ToDevice(eth2, ALLOW_NONEXISTENT true); >>> my_switch[0] -> EtherEncap(0x0800, mymac, worker1) -> Queue -> >>> todevice1; >>> my_switch[1] -> EtherEncap(0x0800, mymac, worker2) -> Queue -> >>> todevice2; >>> >>> or if you just want to bypass the MAC rewrite to test that traffic is >>> being load balanced: >>> >>> my_switch[0] -> Queue -> todevice1; >>> >>> mymac would be the MAC of the interface receiving the traffic, worker1 >>> and worker2 need to be set to the MAC of the worker machines. I was >>> hoping this would take in the traffic, load-balance, rewrite the MAC >>> and send it out to several interfaces. I think this is where I was >>> getting a kernel crash and didn't have time to upgrade the kernel. I >>> think it was on CentOS 5.3, and I saw patch references to the kernel >>> error I was getting. >>> >>> Tyler >>> >>> From robin at icir.org Fri Dec 10 04:27:34 2010 From: robin at icir.org (Robin Sommer) Date: Fri, 10 Dec 2010 04:27:34 -0800 Subject: [Bro] Software frontend In-Reply-To: <4D01281C.1060001@gmail.com> References: <4CFE6ACA.60704@gmail.com> <4CFE7755.5090103@colorado.edu> <4CFE786E.9090300@gmail.com> <4CFE7F37.9080408@colorado.edu> <4CFE94AE.3040301@gmail.com> <4CFE9A6E.9080506@colorado.edu> <4CFEAC2C.8030006@gmail.com> <4CFEB324.9010108@colorado.edu> <4D01281C.1060001@gmail.com> Message-ID: <20101210122734.GP15294@icir.org> Sorry for coming late to the party but we actually have Click code for this (including hashing): http://www.icir.org/robin/tmp/cluster-click.tgz There's no dcoumentation and it hasn't been used in a while, but should be pretty straight-forward to figure out. Robin -- Robin Sommer * Phone +1 (510) 722-6541 * robin at icir.org ICSI/LBNL * Fax +1 (510) 666-2956 * www.icir.org From seth at icir.org Fri Dec 10 06:11:39 2010 From: seth at icir.org (Seth Hall) Date: Fri, 10 Dec 2010 09:11:39 -0500 Subject: [Bro] Software frontend In-Reply-To: <20101210122734.GP15294@icir.org> References: <4CFE6ACA.60704@gmail.com> <4CFE7755.5090103@colorado.edu> <4CFE786E.9090300@gmail.com> <4CFE7F37.9080408@colorado.edu> <4CFE94AE.3040301@gmail.com> <4CFE9A6E.9080506@colorado.edu> <4CFEAC2C.8030006@gmail.com> <4CFEB324.9010108@colorado.edu> <4D01281C.1060001@gmail.com> <20101210122734.GP15294@icir.org> Message-ID: On Dec 10, 2010, at 7:27 AM, Robin Sommer wrote: > There's no dcoumentation and it hasn't been used in a while, but > should be pretty straight-forward to figure out. OSU has been running this code for years and it has been working well, they have 3 click boxes there now. One big problem is that it doesn't support IPv6 traffic but you can deal with that in your click configuration by redirecting IPv6 traffic "manually" if you need it. .Seth From sstattla at gmail.com Fri Dec 10 10:37:15 2010 From: sstattla at gmail.com (Sunjeet Singh) Date: Fri, 10 Dec 2010 10:37:15 -0800 Subject: [Bro] Software frontend In-Reply-To: <20101210122734.GP15294@icir.org> References: <4CFE6ACA.60704@gmail.com> <4CFE7755.5090103@colorado.edu> <4CFE786E.9090300@gmail.com> <4CFE7F37.9080408@colorado.edu> <4CFE94AE.3040301@gmail.com> <4CFE9A6E.9080506@colorado.edu> <4CFEAC2C.8030006@gmail.com> <4CFEB324.9010108@colorado.edu> <4D01281C.1060001@gmail.com> <20101210122734.GP15294@icir.org> Message-ID: <4D02735B.7080104@gmail.com> Oh, cool, thanks for sharing! New challenge when deploying cluster onto cloud- Just rewriting the MAC addresses does not deliver packet to destination because every two machines on the cloud are not on the same physical network. Network devices between the two discard the packet. Solution? Encapsulate the packet in some new packet- implies extra overhead. => Points to the bigger problem of Software load-balancing in a cloud environment. This just-for-fun experiment is turned out to be more fun than I thought it would. Sunjeet On 10-12-10 4:27 AM, Robin Sommer wrote: > Sorry for coming late to the party but we actually have Click code > for this (including hashing): > > http://www.icir.org/robin/tmp/cluster-click.tgz > > There's no dcoumentation and it hasn't been used in a while, but > should be pretty straight-forward to figure out. > > Robin > From seth at icir.org Fri Dec 10 11:32:09 2010 From: seth at icir.org (Seth Hall) Date: Fri, 10 Dec 2010 14:32:09 -0500 Subject: [Bro] Software frontend In-Reply-To: <4D02735B.7080104@gmail.com> References: <4CFE6ACA.60704@gmail.com> <4CFE7755.5090103@colorado.edu> <4CFE786E.9090300@gmail.com> <4CFE7F37.9080408@colorado.edu> <4CFE94AE.3040301@gmail.com> <4CFE9A6E.9080506@colorado.edu> <4CFEAC2C.8030006@gmail.com> <4CFEB324.9010108@colorado.edu> <4D01281C.1060001@gmail.com> <20101210122734.GP15294@icir.org> <4D02735B.7080104@gmail.com> Message-ID: <7F91403A-1FA5-42BE-9D8A-7026EB85FEE4@icir.org> On Dec 10, 2010, at 1:37 PM, Sunjeet Singh wrote: > => Points to the bigger problem of Software load-balancing in a cloud > environment. > > This just-for-fun experiment is turned out to be more fun than I thought > it would. I know this is really just an experiment, but I'm interested in where you are going with this. Are you thinking that you would use this for processing really large tracefiles on a cloud infrastructure or are you thinking of live traffic? If live traffic, what traffic would you be sniffing in that context? Definitely an interesting idea though. .Seth From sstattla at gmail.com Fri Dec 10 11:57:46 2010 From: sstattla at gmail.com (Sunjeet Singh) Date: Fri, 10 Dec 2010 11:57:46 -0800 Subject: [Bro] Software frontend In-Reply-To: <7F91403A-1FA5-42BE-9D8A-7026EB85FEE4@icir.org> References: <4CFE6ACA.60704@gmail.com> <4CFE7755.5090103@colorado.edu> <4CFE786E.9090300@gmail.com> <4CFE7F37.9080408@colorado.edu> <4CFE94AE.3040301@gmail.com> <4CFE9A6E.9080506@colorado.edu> <4CFEAC2C.8030006@gmail.com> <4CFEB324.9010108@colorado.edu> <4D01281C.1060001@gmail.com> <20101210122734.GP15294@icir.org> <4D02735B.7080104@gmail.com> <7F91403A-1FA5-42BE-9D8A-7026EB85FEE4@icir.org> Message-ID: <4D02863A.60700@gmail.com> > Definitely an interesting idea though. Thank you. Like you mention, there's really multiple directions in which this can go. For passive analysis and for live traffic, where you're sending traffic from your enterprise into the cloud for analysis, there would be significant price involved if all packets were sent as-is. One can imagine a more optimal setting where event-analysis can be done locally and only the batched events are sent to the event-handler stage that runs on the cloud. Another idea and the one that I have in mind is that everything runs on the cloud, even your enterprise. This makes much more sense. A cloud provider can have a Bro Instance (like the existing Snort instance http://www.snort.org/news/2010/07/07/snort-now-available-on-the-amazon-cloud/ ) sitting in front of their cloud network or simply cloud web server. Sunjeet From seth at icir.org Fri Dec 10 12:17:30 2010 From: seth at icir.org (Seth Hall) Date: Fri, 10 Dec 2010 15:17:30 -0500 Subject: [Bro] Software frontend In-Reply-To: <4D02863A.60700@gmail.com> References: <4CFE6ACA.60704@gmail.com> <4CFE7755.5090103@colorado.edu> <4CFE786E.9090300@gmail.com> <4CFE7F37.9080408@colorado.edu> <4CFE94AE.3040301@gmail.com> <4CFE9A6E.9080506@colorado.edu> <4CFEAC2C.8030006@gmail.com> <4CFEB324.9010108@colorado.edu> <4D01281C.1060001@gmail.com> <20101210122734.GP15294@icir.org> <4D02735B.7080104@gmail.com> <7F91403A-1FA5-42BE-9D8A-7026EB85FEE4@icir.org> <4D02863A.60700@gmail.com> Message-ID: On Dec 10, 2010, at 2:57 PM, Sunjeet Singh wrote: > One can imagine a more optimal setting where event-analysis can be done locally and only the batched events are sent to the event-handler stage that runs on the cloud. This is likely to just cause more overhead than it's worth. > A cloud provider can have a Bro Instance I can imagine doing this. I may look into it at some point too. .Seth From sstattla at gmail.com Fri Dec 10 12:37:55 2010 From: sstattla at gmail.com (Sunjeet Singh) Date: Fri, 10 Dec 2010 12:37:55 -0800 Subject: [Bro] Software frontend In-Reply-To: References: <4CFE6ACA.60704@gmail.com> <4CFE7755.5090103@colorado.edu> <4CFE786E.9090300@gmail.com> <4CFE7F37.9080408@colorado.edu> <4CFE94AE.3040301@gmail.com> <4CFE9A6E.9080506@colorado.edu> <4CFEAC2C.8030006@gmail.com> <4CFEB324.9010108@colorado.edu> <4D01281C.1060001@gmail.com> <20101210122734.GP15294@icir.org> <4D02735B.7080104@gmail.com> <7F91403A-1FA5-42BE-9D8A-7026EB85FEE4@icir.org> <4D02863A.60700@gmail.com> Message-ID: <4D028FA3.2090704@gmail.com> On 10-12-10 12:17 PM, Seth Hall wrote: > On Dec 10, 2010, at 2:57 PM, Sunjeet Singh wrote: > >> One can imagine a more optimal setting where event-analysis can be done locally and only the batched events are sent to the event-handler stage that runs on the cloud. > This is likely to just cause more overhead than it's worth. > >> A cloud provider can have a Bro Instance > I can imagine doing this. I may look into it at some point too. > > .Seth Well, if people are going to be looking at this space, I'd quickly like to summarize the information the I partly gained from this forum or otherwise learnt from the challenges that I ran into- The Frontend Load Balancer remains the bottleneck- In a cloud setting, frontend remains the non-scalable part of the existing cluster architecture. With the option of hardware lond balancing gone in the Cloud, software load-balancing will have to incur some overhead. You can't modify only MAC address (the packet will get dropped before reaching Worker) and you can't modify both MAC and IP (you need the original IPs (duh)). You need to either encapsulate the packet yourself (user-level or kernel-level but still processing overhead and requires de-capsulation code at the receiver side) or configure a cluster Virtual Private Cloud and subnet in which case the Cloud is doing the encapsulation for you. How well does this scale? From seth at icir.org Fri Dec 10 12:53:55 2010 From: seth at icir.org (Seth Hall) Date: Fri, 10 Dec 2010 15:53:55 -0500 Subject: [Bro] Software frontend In-Reply-To: <4D028FA3.2090704@gmail.com> References: <4CFE6ACA.60704@gmail.com> <4CFE7755.5090103@colorado.edu> <4CFE786E.9090300@gmail.com> <4CFE7F37.9080408@colorado.edu> <4CFE94AE.3040301@gmail.com> <4CFE9A6E.9080506@colorado.edu> <4CFEAC2C.8030006@gmail.com> <4CFEB324.9010108@colorado.edu> <4D01281C.1060001@gmail.com> <20101210122734.GP15294@icir.org> <4D02735B.7080104@gmail.com> <7F91403A-1FA5-42BE-9D8A-7026EB85FEE4@icir.org> <4D02863A.60700@gmail.com> <4D028FA3.2090704@gmail.com> Message-ID: <7EF9C12E-412C-4E36-BAB0-2CF29BAD9D92@icir.org> On Dec 10, 2010, at 3:37 PM, Sunjeet Singh wrote: > Virtual Private Cloud and subnet in which case the Cloud is doing the encapsulation for you. How well does this scale? I would go this way if the cloud providers offer it. The more you move towards being able to control the layer-2 aspects of the network, the closer you are to being able to easily do load balancing. If you are given control over VLANs within your cloud, you could still use the cluster-click package then Robin sent the link to earlier because it has the capability of distributing packets by appending vlan headers to them. .Seth From sstattla at gmail.com Fri Dec 10 12:57:22 2010 From: sstattla at gmail.com (Sunjeet Singh) Date: Fri, 10 Dec 2010 12:57:22 -0800 Subject: [Bro] Software frontend In-Reply-To: <7EF9C12E-412C-4E36-BAB0-2CF29BAD9D92@icir.org> References: <4CFE6ACA.60704@gmail.com> <4CFE7755.5090103@colorado.edu> <4CFE786E.9090300@gmail.com> <4CFE7F37.9080408@colorado.edu> <4CFE94AE.3040301@gmail.com> <4CFE9A6E.9080506@colorado.edu> <4CFEAC2C.8030006@gmail.com> <4CFEB324.9010108@colorado.edu> <4D01281C.1060001@gmail.com> <20101210122734.GP15294@icir.org> <4D02735B.7080104@gmail.com> <7F91403A-1FA5-42BE-9D8A-7026EB85FEE4@icir.org> <4D02863A.60700@gmail.com> <4D028FA3.2090704@gmail.com> <7EF9C12E-412C-4E36-BAB0-2CF29BAD9D92@icir.org> Message-ID: <4D029432.8030608@gmail.com> Amazon's EC2 cloud has ec2-create-VPC and ec2-create-subnet commands in it's API. Sunjeet On 10-12-10 12:53 PM, Seth Hall wrote: > On Dec 10, 2010, at 3:37 PM, Sunjeet Singh wrote: > >> Virtual Private Cloud and subnet in which case the Cloud is doing the encapsulation for you. How well does this scale? > I would go this way if the cloud providers offer it. The more you move towards being able to control the layer-2 aspects of the network, the closer you are to being able to easily do load balancing. If you are given control over VLANs within your cloud, you could still use the cluster-click package then Robin sent the link to earlier because it has the capability of distributing packets by appending vlan headers to them. > > .Seth From sstattla at gmail.com Mon Dec 13 14:44:59 2010 From: sstattla at gmail.com (Sunjeet Singh) Date: Mon, 13 Dec 2010 14:44:59 -0800 Subject: [Bro] PF_RING for Bro Message-ID: <4D06A1EB.4070807@gmail.com> Hi, I am looking at increasing the performance of Bro by making use of PF_RING and TNAPI, at the kernel and driver level in Linux. These existing tools from http://www.ntop.org/news.php allow multi-threaded applications to drastically increase performance by making use of functionality that exists in several network cards today. While PF_RING, TNAPI and other network-card functionality allow for phenomenal speed-up, they have to be tuned right or else they can lead to a negative effect on performance. So I'm trying to determine how to tune them to meet Bro's needs. As Multi-threaded Bro is not ready yet, I am trying to think of a SIMPLE application that can best characterize the Bro workload. The design criteria that I have for this application so far is- 1. Send every packet belonging to a particular connection to the same core. 2. Introduce a small wait on every packet to simulate Bro's processing of that packet? What will be a good value? I'd be very interested to have everyone's opinion on this. I really am looking for a very simple algorithm that will be a right approximation. Thank you, Sunjeet Singh From sstattla at gmail.com Mon Dec 13 14:50:11 2010 From: sstattla at gmail.com (Sunjeet Singh) Date: Mon, 13 Dec 2010 14:50:11 -0800 Subject: [Bro] PF_RING for Bro In-Reply-To: <4D06A1EB.4070807@gmail.com> References: <4D06A1EB.4070807@gmail.com> Message-ID: <4D06A323.3090200@gmail.com> Alternatively, can I have the source code for multi-threaded Bro? : ) Sunjeet On 10-12-13 2:44 PM, Sunjeet Singh wrote: > Hi, > > I am looking at increasing the performance of Bro by making use of > PF_RING and TNAPI, at the kernel and driver level in Linux. These > existing tools from http://www.ntop.org/news.php allow multi-threaded > applications to drastically increase performance by making use of > functionality that exists in several network cards today. > > While PF_RING, TNAPI and other network-card functionality allow for > phenomenal speed-up, they have to be tuned right or else they can lead > to a negative effect on performance. So I'm trying to determine how to > tune them to meet Bro's needs. > > As Multi-threaded Bro is not ready yet, I am trying to think of a > SIMPLE application that can best characterize the Bro workload. The > design criteria that I have for this application so far is- > > 1. Send every packet belonging to a particular connection to the same > core. > 2. Introduce a small wait on every packet to simulate Bro's processing > of that packet? What will be a good value? > > I'd be very interested to have everyone's opinion on this. I really am > looking for a very simple algorithm that will be a right approximation. > > > Thank you, > Sunjeet Singh > > From j.sentier206 at laposte.net Mon Dec 13 23:53:39 2010 From: j.sentier206 at laposte.net (j.sentier206) Date: Tue, 14 Dec 2010 08:53:39 +0100 (CET) Subject: [Bro] Bro patch Message-ID: <6798166.318995.1292313219531.JavaMail.www@wwinf8214> Hi Bro people, I submit you a small patch. For ICMP, I wanted to access the payload of the packets. For SMB, I fixed a small bug and added the processid field. Hope you find it useful. Cheers, Julien Sentier Une messagerie gratuite, garantie ? vie et des services en plus, ?a vous tente ? Je cr?e ma bo?te mail www.laposte.net Une messagerie gratuite, garantie ? vie et des services en plus, ?a vous tente ? Je cr?e ma bo?te mail www.laposte.net -------------- next part -------------- An embedded and charset-unspecified text was scrubbed... Name: ICMP-SMB.diff Url: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20101214/d6b23395/attachment.ksh From seth at icir.org Tue Dec 14 07:28:16 2010 From: seth at icir.org (Seth Hall) Date: Tue, 14 Dec 2010 10:28:16 -0500 Subject: [Bro] Bro patch In-Reply-To: <6798166.318995.1292313219531.JavaMail.www@wwinf8214> References: <6798166.318995.1292313219531.JavaMail.www@wwinf8214> Message-ID: <2FF8D82B-B08A-4AA5-B82D-2CEBA8746CB2@icir.org> On Dec 14, 2010, at 2:53 AM, j.sentier206 wrote: > For ICMP, I wanted to access the payload of the packets. This absolutely makes sense. I'll file a ticket with the changes to the ICMP analyzer. We'll have to discuss and see if it's something that we want to add. I can see why you'd want it though. > For SMB, I fixed a small bug and added the processid field. Unfortunately, the SMB analyzer does have several bugs that prevent it from working but it looks like you caught all of them. For the 1.6 release, we're hoping to have high quality SMB and SMB2 analyzers but they're being completely rewritten. If you have a further interest in working on SMB+SMB2 analyzers, please let us know, we can point you in the right direction. Thanks, .Seth From j.sentier206 at laposte.net Tue Dec 14 09:05:55 2010 From: j.sentier206 at laposte.net (j.sentier206) Date: Tue, 14 Dec 2010 18:05:55 +0100 (CET) Subject: [Bro] Bro patch In-Reply-To: <2FF8D82B-B08A-4AA5-B82D-2CEBA8746CB2@icir.org> References: <6798166.318995.1292313219531.JavaMail.www@wwinf8214> <2FF8D82B-B08A-4AA5-B82D-2CEBA8746CB2@icir.org> Message-ID: <25883256.89921.1292346355556.JavaMail.www@wwinf8403> > > On Dec 14, 2010, at 2:53 AM, j.sentier206 wrote: > > > For ICMP, I wanted to access the payload of the packets. > > This absolutely makes sense. I'll file a ticket with the changes to the ICMP analyzer. We'll have to discuss and see if it's something that we want to add. I can see why you'd want it though. Thank you for the ticket. I hope I will be able to access ICMP payloads in bro 1.6. > > > For SMB, I fixed a small bug and added the processid field. > > Unfortunately, the SMB analyzer does have several bugs that prevent it from working but it looks like you caught all of them. For the 1.6 release, we're hoping to have high quality SMB and SMB2 analyzers but they're being completely rewritten. If you have a further interest in working on SMB+SMB2 analyzers, please let us know, we can point you in the right direction. > I just happened to have an error with SMB on some pcap. I made a quick fix so that Bro could finish analyzing the file. I am sorry I will not have the time to work on the new analyzers Une messagerie gratuite, garantie ? vie et des services en plus, ?a vous tente ? Je cr?e ma bo?te mail www.laposte.net From sstattla at gmail.com Wed Dec 15 13:41:36 2010 From: sstattla at gmail.com (Sunjeet Singh) Date: Wed, 15 Dec 2010 13:41:36 -0800 Subject: [Bro] Update on using PF_RING/TNAPI with Bro Message-ID: <4D093610.4060303@gmail.com> Hi, I have managed to get TNAPI/PF_RING configured and working with PF_RING-aware libpcap. http://www.ntop.org/TNAPI.html Looks like this will be very well suited to the Multiprocessing version of Bro. 1. At the device driver level, RSS functionality (also, Flow Director in Intel) allows packets multiplexed to different Receiver Queues (and also allows packets belonging to a particular connection be sent to the same RX_Queue) on an I/OAT-supporting network card. 2. By virtue of TNAPI, these multiple RX_Queues get polled concurrently (by one kernel thread per queue), and sent to PF_RING (along with information about which queue the packet came from). 3. PF_RING provides a user API which can be used by user-applications like Bro to directly read from the multiple RX_Queues of a network interface by using notation like eth0 at 1, eth0 at 2, etc. for RX_Queues 1 and 2 belonging to interface eth0. By assigning one thread to one RX_Queue, we ensure that all packets from one connection are being processed by the same core. PF_RING and TNAPI can be used to drastically improve the performance of any multiprocessing application, but need to be properly tuned and used by the application. Performance stems from the fact that for Bro, the packets can bypass the kernel's network stack altogether; one thread polling per RX_Queue thanks to TNAPI; and PF_RING avoiding the mmap from Kernel space to User space by directly copying payloads from RX_Queue rings. Configuration wise, it took a bit of work to change Bro's configure files to use a PF_RING-aware libpcap instead of the libpcap that Bro ships with. When running TNAPI and PF_RING, there is a clear performance improvement in the kernel's ability to receive packets at a higher packet rate (results on the TNAPI website, I also verified). But using PF_RING with the existing Bro leads to a performance degradation of Bro because Bro runs on one user-thread, and when all these packets reach user-space on different user-threads, they need to be processed by the core that is running Bro. But from my knowledge on TNAPI/PF_RING and intuition, multi-threaded Bro can be adapted to PF_RING and will lead to huge gains in performance. Here's the summary of results of a brief experiment that I performed on a 8-core Intel Xeon with32 GB RAM running on Linux and with an Intel 82598EB 10Gbps ethernet card: Goal: Compare conventional Bro installation against Bro with TNAPI and PF_RING (I called it Bro-Ring) Conclusion: Bro-Ring shows a performance drop. Observations: The values in the table show for varying packet-rates, how many packets were accepted by the machine running Bro (rest were lost). Packets/sec Bro-Ring Bro 34000 1368791 1368003 50000 1368546 1367707 65,000 1368614 120000 1224761 130000 1168734 166000 596667 170000 561702 171000 681104 173000 618100 175000 740137 178000 864706 210000 753700 215000 728450 230000 494637 240000 636287 (Note: there was a difference in tcpreplay's input parameter packet-rate and the actual packet rate achieved, so I could not supply exact values for packet rate) Sunjeet Singh -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20101215/e75e9ee8/attachment.html From sstattla at gmail.com Wed Dec 15 13:50:41 2010 From: sstattla at gmail.com (Sunjeet Singh) Date: Wed, 15 Dec 2010 13:50:41 -0800 Subject: [Bro] Update on using PF_RING/TNAPI with Bro In-Reply-To: <4D093610.4060303@gmail.com> References: <4D093610.4060303@gmail.com> Message-ID: <4D093831.4040202@gmail.com> The table got a bit messed up, here are the values again- Packets/sec Bro-Ring Bro 34000 13687911 36800 350000 13685461 367707 65000 1368614 - 120000 - 1224761 130000 - 1168734 166000 596667 - 170000 561702 - 171000 681104 - 173000 618100 - 175000 740137 - 178000 864706 - 210000 - 753700 215000 - 728450 230000 494637 - 240000 - 636287 Sunjeet On 10-12-15 1:41 PM, Sunjeet Singh wrote: > Hi, > > I have managed to get TNAPI/PF_RING configured and working with > PF_RING-aware libpcap. http://www.ntop.org/TNAPI.html > Looks like this will be very well suited to the Multiprocessing > version of Bro. > > 1. At the device driver level, RSS functionality (also, Flow Director > in Intel) allows packets multiplexed to different Receiver Queues (and > also allows packets belonging to a particular connection be sent to > the same RX_Queue) on an I/OAT-supporting network card. > 2. By virtue of TNAPI, these multiple RX_Queues get polled > concurrently (by one kernel thread per queue), and sent to PF_RING > (along with information about which queue the packet came from). > 3. PF_RING provides a user API which can be used by user-applications > like Bro to directly read from the multiple RX_Queues of a network > interface by using notation like eth0 at 1, eth0 at 2, etc. for RX_Queues 1 > and 2 belonging to interface eth0. By assigning one thread to one > RX_Queue, we ensure that all packets from one connection are being > processed by the same core. > > > PF_RING and TNAPI can be used to drastically improve the performance > of any multiprocessing application, but need to be properly tuned and > used by the application. Performance stems from the fact that for Bro, > the packets can bypass the kernel's network stack altogether; one > thread polling per RX_Queue thanks to TNAPI; and PF_RING avoiding the > mmap from Kernel space to User space by directly copying payloads from > RX_Queue rings. > > > Configuration wise, it took a bit of work to change Bro's configure > files to use a PF_RING-aware libpcap instead of the libpcap that Bro > ships with. When running TNAPI and PF_RING, there is a clear > performance improvement in the kernel's ability to receive packets at > a higher packet rate (results on the TNAPI website, I also verified). > But using PF_RING with the existing Bro leads to a performance > degradation of Bro because Bro runs on one user-thread, and when all > these packets reach user-space on different user-threads, they need to > be processed by the core that is running Bro. But from my knowledge on > TNAPI/PF_RING and intuition, multi-threaded Bro can be adapted to > PF_RING and will lead to huge gains in performance. > > Here's the summary of results of a brief experiment that I performed > on a 8-core Intel Xeon with32 GB RAM running on Linux and with an > Intel 82598EB 10Gbps ethernet card: > > Goal: Compare conventional Bro installation against Bro with TNAPI and > PF_RING (I called it Bro-Ring) > Conclusion: Bro-Ring shows a performance drop. > Observations: The values in the table show for varying packet-rates, > how many packets were accepted by the machine running Bro (rest were > lost). > > Packets/sec Bro-Ring Bro > 34000 1368791 1368003 > 50000 1368546 1367707 > 65,000 1368614 > 120000 > 1224761 > 130000 > 1168734 > 166000 596667 > 170000 561702 > 171000 681104 > 173000 618100 > 175000 740137 > 178000 864706 > 210000 > 753700 > 215000 > 728450 > 230000 494637 > 240000 > 636287 > > > (Note: there was a difference in tcpreplay's input parameter > packet-rate and the actual packet rate achieved, so I could not supply > exact values for packet rate) > > > Sunjeet Singh -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20101215/c0ea3cef/attachment.html From JAzoff at uamail.albany.edu Wed Dec 15 14:00:17 2010 From: JAzoff at uamail.albany.edu (Justin Azoff) Date: Wed, 15 Dec 2010 17:00:17 -0500 Subject: [Bro] Update on using PF_RING/TNAPI with Bro In-Reply-To: <4D093610.4060303@gmail.com> References: <4D093610.4060303@gmail.com> Message-ID: <20101215220017.GD22071@datacomm.albany.edu> On Wed, Dec 15, 2010 at 04:41:36PM -0500, Sunjeet Singh wrote: > 3. PF_RING provides a user API which can be used by user-applications > like Bro to directly read from the multiple RX_Queues of a network > interface by using notation like eth0 at 1, eth0 at 2, etc. for RX_Queues 1 > and 2 belonging to interface eth0. ... > But using PF_RING with the existing Bro leads to a performance > degradation of Bro because Bro runs on one user-thread, and when all > these packets reach user-space on different user-threads, they need to > be processed by the core that is running Bro. Why are you only running one bro process? You can setup a single node bro cluster and run multiple bro processes, each listening on one of eth0 at 1, eth0 at 2.. -- -- Justin Azoff -- Network Security & Performance Analyst From sstattla at gmail.com Wed Dec 15 14:11:18 2010 From: sstattla at gmail.com (Sunjeet Singh) Date: Wed, 15 Dec 2010 14:11:18 -0800 Subject: [Bro] Update on using PF_RING/TNAPI with Bro In-Reply-To: <20101215220017.GD22071@datacomm.albany.edu> References: <4D093610.4060303@gmail.com> <20101215220017.GD22071@datacomm.albany.edu> Message-ID: <4D093D06.2090606@gmail.com> > Why are you only running one bro process? You can setup a single node > bro cluster and run multiple bro processes, each listening on one of > eth0 at 1, eth0 at 2.. > Yes, that's a great idea. But I'm not sure how Bro would handle manager-proxy-worker communication between different RX_Queues instead of different interfaces. Can't be as simple as writing eth0 at 1, etc. in the cluster's node.cfg file. Maybe some changes to Bro code? Sunjeet From JAzoff at uamail.albany.edu Wed Dec 15 19:14:47 2010 From: JAzoff at uamail.albany.edu (Justin Azoff) Date: Wed, 15 Dec 2010 22:14:47 -0500 Subject: [Bro] Update on using PF_RING/TNAPI with Bro In-Reply-To: <4D093D06.2090606@gmail.com> References: <4D093610.4060303@gmail.com> <20101215220017.GD22071@datacomm.albany.edu> <4D093D06.2090606@gmail.com> Message-ID: <20101216031447.GE22071@datacomm.albany.edu> On Wed, Dec 15, 2010 at 05:11:18PM -0500, Sunjeet Singh wrote: > Yes, that's a great idea. But I'm not sure how Bro would handle > manager-proxy-worker communication between different RX_Queues instead > of different interfaces. Can't be as simple as writing eth0 at 1, etc. in > the cluster's node.cfg file. Maybe some changes to Bro code? > > Sunjeet Putting eth0 at 1,2,3,4 ec. in node.cfg should work just fine. no changes to bro are needed, but you may have to rebuild bro with ./configure --enable-cluster... the config I use with click just has: [manager] type=manager host=10.10.1.12 [proxy-1] type=proxy host=10.10.1.12 [worker-1] type=worker host=10.10.1.12 interface=tap0 [worker-2] type=worker host=10.10.1.12 interface=tap1 [worker-3] type=worker host=10.10.1.12 interface=tap2 [worker-4] type=worker host=10.10.1.12 interface=tap3 -- -- Justin Azoff -- Network Security & Performance Analyst From sstattla at gmail.com Thu Dec 16 17:30:40 2010 From: sstattla at gmail.com (Sunjeet Singh) Date: Thu, 16 Dec 2010 17:30:40 -0800 Subject: [Bro] Update on using PF_RING/TNAPI with Bro In-Reply-To: <20101216031447.GE22071@datacomm.albany.edu> References: <4D093610.4060303@gmail.com> <20101215220017.GD22071@datacomm.albany.edu> <4D093D06.2090606@gmail.com> <20101216031447.GE22071@datacomm.albany.edu> Message-ID: <4D0ABD40.40108@gmail.com> Okay, if this works, I don't think you'll see a gain in performance. To leverage performance in this case, lets say you have 4 cores, Core 0 running Bro Manager, Core 1 running Bro Proxy, Core 2 running Bro Worker1, Core 3 running Bro Worker2. For max performance (and really any performance gain) through cache localization, you'd want all traffic input to Bro to go to either Core2 or Core3; and both of these cores coupled to one RX_Queue each. You will somehow need to at the driver layer, split the traffic coming from the wire to go to one of these queues (intelligently to send packets sharing state to the same RX_Queue). This has to be done at the RSS, and I have no idea of how to do this on my network card- Intel 82598EB. (You couldn't use Click to do this because you want to do it at the driver level). What do you think? Sunjeet On 10-12-15 7:14 PM, Justin Azoff wrote: > On Wed, Dec 15, 2010 at 05:11:18PM -0500, Sunjeet Singh wrote: >> Yes, that's a great idea. But I'm not sure how Bro would handle >> manager-proxy-worker communication between different RX_Queues instead >> of different interfaces. Can't be as simple as writing eth0 at 1, etc. in >> the cluster's node.cfg file. Maybe some changes to Bro code? >> >> Sunjeet > Putting eth0 at 1,2,3,4 ec. in node.cfg should work just fine. > no changes to bro are needed, but you may have to rebuild bro with > ./configure --enable-cluster... > > the config I use with click just has: > > [manager] > type=manager > host=10.10.1.12 > > [proxy-1] > type=proxy > host=10.10.1.12 > > [worker-1] > type=worker > host=10.10.1.12 > interface=tap0 > > [worker-2] > type=worker > host=10.10.1.12 > interface=tap1 > > [worker-3] > type=worker > host=10.10.1.12 > interface=tap2 > > [worker-4] > type=worker > host=10.10.1.12 > interface=tap3 > > From robin at icir.org Fri Dec 17 02:01:32 2010 From: robin at icir.org (Robin Sommer) Date: Fri, 17 Dec 2010 02:01:32 -0800 Subject: [Bro] PF_RING for Bro In-Reply-To: <4D06A323.3090200@gmail.com> References: <4D06A1EB.4070807@gmail.com> <4D06A323.3090200@gmail.com> Message-ID: <20101217100132.GO96720@icir.org> (Note, I haven't caught up with the follow-up trehad yet). >> 1. Send every packet belonging to a particular connection to the same >> core. >> 2. Introduce a small wait on every packet to simulate Bro's processing >> of that packet? What will be a good value? There probably isn't a good value. :-) I think I'd start with two things: (1) Just doing nothing in the threads, and (2) Accessing all the packets' bytes in some form within the threads, perhaps by calculating a simple checksum. That should give you a good baseline, including an approximation of memory effects in (2). It will be interesting to see the differece between (1) and (2), and also between them and a pure software version (copy the packets from a master thread to child threads "manually", as the current prototype is doing). On Mon, Dec 13, 2010 at 14:50 -0800, you wrote: > Alternatively, can I have the source code for multi-threaded Bro? : ) You can, I'll send you another mail about that. Robin -- Robin Sommer * Phone +1 (510) 722-6541 * robin at icir.org ICSI/LBNL * Fax +1 (510) 666-2956 * www.icir.org From buptmazhen at gmail.com Thu Dec 23 00:12:05 2010 From: buptmazhen at gmail.com (=?GB2312?B?wu3V8ShaaGVuIE1hKQ==?=) Date: Thu, 23 Dec 2010 16:12:05 +0800 Subject: [Bro] weird: spontaneous_FIN problem for HTTP log Message-ID: Hi! I've captured a HTTP packet file using tcpdump. But when I read it with bro, it shows message as following and I can't get log file: 1271639268.624587 weird: spontaneous_FIN 1271639268.624655 weird: spontaneous_FIN 1271639268.624759 weird: spontaneous_FIN 1271639277.565623 weird: above_hole_data_without_any_acks 1271639281.963865 weird: spontaneous_FIN 1271639282.625769 weird: above_hole_data_without_any_acks 1271639283.776172 weird: spontaneous_FIN My command is "bro -r XXX.trace http-reply http-header". I can get log file if I use "bro -r XXX.trace mt", but the output log file is not http information and is not what I want. Will anyone help me to fix this? Thanks! -- Best regards! Eric,Ma Tsinghua University, Beijing Tel?15210836318 -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20101223/6cd1b758/attachment.html From vern at icir.org Thu Dec 23 09:06:15 2010 From: vern at icir.org (Vern Paxson) Date: Thu, 23 Dec 2010 09:06:15 -0800 Subject: [Bro] weird: spontaneous_FIN problem for HTTP log In-Reply-To: (Thu, 23 Dec 2010 16:12:05 +0800). Message-ID: <20101223170615.8556236A58D@taffy.ICSI.Berkeley.EDU> > My command is "bro -r XXX.trace http-reply http-header". I can get log file > if I use "bro -r XXX.trace mt", but the output log file is not http > information and is not what I want. Are you sure the trace has packet payloads? The other question is whether it's unidirectional, as above_hole_data_without_any_acks indicates the analyzer is only seeing one side of a connection (or is processing a trace with heavy measurement loss). If the above aren't the problem, then please send a snippet to me (or the list) so we can see about reproducing what's going on. Vern