[Bro-Dev] New http policy scripts

[Seth Hall]

On Aug 4, 2011, at 11:42 PM, Gregor Maier wrote:

> It appears that per default protocols/http now does analyzer 
> HTTP-headers and HTTP-payload. Both of them are quite expensive in terms 
> of CPU time. Particularly body analysis.
> 
> I would opt to *not* include those if just protocols/http is loaded 

I knew that someone would make this argument eventually. :)  

There are two parts to my argument in favor of including both by default.  

By removing some of the analysis like that by default, you are basically taking a runtime optimization step as you pointed out since it does certain cause overhead to do everything.  The problem is that it would make the usage of Bro more obtuse for users since it would be a singular optimization specifically for a type of traffic that just happens to be prevalent on most networks. I think that in 99% of cases, people want everything anyway (people running Bro on live traffic for operational security purposes at least).  That has been my experience.

The other side of this is the http_body events.  I don't like how I'm doing that either, but it's a stopgap until we have the more general file analyzer that would do everything i'm doing in the base http analysis scripts internally (identifications, hashing, extraction, etc).

I do agree that I'm doing some pretty egregious stuff in some of those scripts from an optimization perspective, but I think that optimization attempts in Bro scripts have led to incredibly convoluted scripts and dependency chains.  I'm going to press instead for optimizations that allow the scripts to remain well structured.  For instance, what about just disabling the http_header or http_data events if you don't want those done?  This should already be do-able with the disable_event_group like this:
	
	disable_event_group("http-body");
	disable_event_group("http-header");

  .Seth 

--
Seth Hall
International Computer Science Institute
(Bro) because everyone has a network
http://www.bro-ids.org/