[Bro-Dev] Script reorg proposal

[Jonathan Siwek]

>>> having to go from-all-to-nothing. Then we could also just use a single
>>> (base) directory for all the scripts instead of bro/base and bro/policy
>>> (which kinda irks me).
>> 
>> One of the main motivations for the proposed reorg is actually to
>> split the two apart. Currently, they are all inside one directory
>> hierarchy, but that makes it hard to get a good picture of what's
>> there.
> 
> I think that needs to be solved by documentation. Have some (short) doc 
> that lists everything that's @load'able and briefly describes what it 
> does.(1)

My preference would be for the "optionality" (or probably even functionality) of a script/package to be implied by where it lives in a directory hierarchy.  The documentation would naturally also reflect this without extra work (probably).

The difference is that if everything lives under a common hierarchy, I 'm forced to consult some additional documentation/manifest in order to understand what parts are optional instead of just being able to know when I'm poking around in a shell.

> So this top-level.bro is basically the same as my default.bro except it 
> lives in base/, right? So what's the advantage? That it's easy to see 
> what's loaded by default based on the directory where the script is?

Yeah, that's my impression of what the main advantage is.

> If that's indeed the case and you want to do the split into two 
> directories, I would still advocate to do top-level.bro the way I 
> described my default.bro, i.e., but it in /site and add comments to make 
> it "user serviceable" :-)

I think this is what the site/local.bro currently does, and I generally like that way of doing it.

> Another question would be whether one would split protocol analysis 
> between base and policy? E.g., is there going to be "base/http/" and 
> "policy/http" and when I load the first as package I get the basic and 
> when I also load "policy/http" as package I get the heavier analysis or 
> would I also cherry-pick additional features in "policy/http/*"?

Yes, both packages could exist, base/http for basic analysis and policy/http for advanced.  In the later case, my feeling is that cherry-picking is always allowed/encouraged, but if we allow policy/http to be loaded as a whole package, we might need some convention that makes that behavior well-defined or inferable from the naming/hierarchy for the cases where a package contains related-but-conflicting scripts that shouldn't be loaded together and thus require cherry-picking.  If that situation occurs, my suggestion would be to use an opt/ subdir of a given package to reflect such add-on scripts that require the user to do a little thinking before loading it.

- Jon