[Bro] intel.log file stops getting generated.

[Azoff, Justin S]

> On Jan 25, 2017, at 4:43 PM, fatema bannatwala <fatema.bannatwala at gmail.com> wrote:
> 
> Alrighty, yeah was looking into how to configure the script according to the environment.
> It appears that we have to define the list of allocated subnets in the network,
> as landmine works on watching connections which are not in allocated subnets.
> 
> Defining the allocated subnets is a pain, have a whole lot list of subnets that are allocated and 
> have just couple of subnets that constitute the darknet, hence was tweaking around the scripts to change that setting
> from defining allocated subnets to rather defining un-allocated subnets, which is much easier. 

That part is optional(but extremely useful).  I'm glad you brought this up, the darknet configuration problem is something I've been thinking about how to fix:

* Some people define darknet as NOT allocated.
* Some people know exactly which subnets are dark.

I did write a version of the darknet code that auto-tunes itself based on allocated subnets, it's part of my scan code:

https://gist.github.com/JustinAzoff/80f97af4f4fbb91ae26492b919a50434

One can let it run for a while, and then do a

    broctl print Site::used_address_space manager

to dump out what it figures out as active, and then put it in a file that does

    @load ./dark-nets
    redef Site::used_address_space = {
    ...
    }

It's not perfect but it's a start.  broker with its persistent data store support may be what is needed to make it more useful.

The only issue is it doesn't support something like a honey net that does technically exist: the auto tuning code will flag it as an allocated subnet.  I need to work out how it should be overridden in cases like that.

Aside from the auto detection the function just comes down to

    return (a in local_nets && a !in used_address_space);

In your case you want this instead

    return (a in dark_address_space);

so I think the simplest thing that may possibly work for everyone is something like 

    global dark_address_space: set[subnet] &redef;

and change the is_darknet logic to be


    if(|dark_address_space|)
        return (a in dark_address_space);
    else
        return (a in local_nets && a !in used_address_space);

Or maybe just

    return (a in local_nets && (a in dark_address_space || a !in used_address_space);

but I could see a different user wanting this instead:

    return (a in local_nets && a in dark_address_space && a !in used_address_space);

for the use case of "dark_address_space is my darknet subnets, but something may be allocated without us knowing, so double check!"

I haven't quite figured this out yet.. Maybe the answer is that there isn't a one size fits all implementation and I just need to have 4 is_darknet functions depending on how people want it to work:


    return (a in dark_address_space);  #mode=darknet

    return (a in local_nets && a !in used_address_space); #mode=not_allocated

    return (a in local_nets && (a in dark_address_space || a !in used_address_space); #mode=darknet_or_not_allocated

    return (a in local_nets && a in dark_address_space && a !in used_address_space); #mode=darknet_and_not_allocated

actually, now that I finally wrote all this out, I see that it's just the 4 combinations of 2 boolean flags.

-- 
- Justin Azoff