[Bro] Bro Traffic analysis flow shunting questions

[Azoff, Justin S]

> On Feb 20, 2018, at 12:46 PM, Drew Dixon <dwdixon at umich.edu> wrote:
> 
> Hello,
> 
> I have some questions regarding implementing shunting of traffic that has no analysis value and what the recommended approach is for doing so with a large bro cluster that will be analyzing very high volumes of traffic.
> 
> I am aware of the Bro net-control framework and am wondering if this is where I should start and what should be leveraged to begin implement shunting with the latest version of bro?  
> 
> - In addition to "ordinary" elephant flows, I'm also interested in shunting out large video streaming services (netflix, hulu, prime video, hbogo, vimeo, etc.) which do not have analysis value, first I'm wondering would any of this get picked up in the bulk transfer/flow detection already built into the existing conn-bulk.bro script detection?

No, since those are not really bulk flows.  According to netflix a stream is "up to 3 GB per hour for HD and 7 GB per hour for Ultra HD" which works out to 7-15mbit.  Dumbno is really intended to filter flows that are 1000mbit+.  The kind of flows that would completely overwhelm the NICs in your sensors.  Video streaming is more of a "death by a thousand paper cuts" kind of thing.

> - Is the use of dumbno with bro-react/conn-bulk.bro for detection and shunting of bulk transfers/elephant flows relevant still considering bro-netcontrol exists?  Are these viewed as plugins/backends for bro-netcontrol?  I believe bro-netcontrol also has some shunting (built in?) is any of dumbno/conn-bulk.bro built into bro-netcontrol already, or is further configuration required to setup dumbno/conn-bulk.bro with net-control?

bro-react predates netcontrol and hasn't been updated to work with the netcontrol framework.  Well, updated is really the wrong word, since there's not really anything to update.. The whole thing is one function and one hook, shorter than this email. If I get a chance I should be able to build a bro package that implements a native dumbno client in C++ and hook it into the netcontrol api.  You'd still need something like conn-bulk to flag the elephant flows.

> - If the answer to first bullet point the above is no, is anyone already shunting this type of video streaming traffic and is there a bro script out there anyone could share so we aren't reinventing the wheel?  Assuming nothing exists, would it be plausible to leverage the ssl.log domain name metadata to do the detection piece for this, then pulling out the IP/Port info to insert shunting ACL's via the packet broker API?  Would extending dumbno/bro-react/conn-bulk.bro to support this (assuming it doesn't catch this traffic already) be something desirable?  Also, would this be a good approach for detecting large video streaming traffic flows, or would there be a better approach?  

The best way to detect video services would be using ssl events and match on the SNI hostname.

The problem you'll run into is that depending on which Arista you have, you can only have a certain number of ACL entries.  Shunting every single netflix video flow individually will likely max out the TCAM on the switch.  And because of things like https://en.wikipedia.org/wiki/Dynamic_Adaptive_Streaming_over_HTTP I don't think netflix streams you a 6GB video file in one request, I think the client pulls down multiple smaller chunks across multiple connections.

It would likely be a much better idea to just identity what the upstream netflix CDN hosts are for your network[1] and filter them manually.  You could possibly use sumstats to identify them automatically, but that would filter things individually and you could end up filtering every address in a /24 using 255 rules which wouldn't be very efficient.


There's also the skip_further_processing function that tells bro to not do anything with packets that are part of a particular connection, which can also help if you are low on resources.


[1]:

$ grep nflxvideo.net ssl.log |cut -f 5|sort|uniq  -c|sort -nr|head
   8332 216.171.0.106
    189 198.38.109.241
    162 198.38.108.206
    161 198.38.109.202
    130 198.38.109.200
    113 198.38.109.220
    102 198.38.108.208
     98 198.38.108.205
     93 198.38.108.162

$ grep nflxvideo.net ssl.log |cut -f 5|cut -d . -f 1-3|sort|uniq  -c|sort -nr
   8388 216.171.0
   2346 198.38.108
   2107 198.38.109
     44 ...

So, 1 host and 2 /24s account for most of the netflix cdn, 3 acls would catch almost all of it.  Filtering individual flows, dumbno would have tried to shunt 10,000+ connections. If things were tweaked to ignore the ports and just filter on src+dst, that's still hundreds of individual acls.

— 
Justin Azoff