[Bro] Tables and Strings

[Brian Chilton]

All,
 
Thanks for the insight, what I am looking at doing is setting a notice log every time we Post to a website when there is no referrer.  I believe trimming down to just see this type of info would be less of a load then if we looked at every post to a site we had never visited before.  Would there be a large performance impact if I decided to use the input framework with this idea? It seems like that is the logical choice for this project, just not sure on how to edit lists within the input framework on the fly.     


On Wednesday, May 28, 2014 1:35 PM, Josh Liburdi <liburdi.joshua at gmail.com> wrote:
  


Yep, there are quite a few attributes that can make them manageable ... lots of good information on this page: http://www.bro.org/sphinx/scripts/builtins.html 
Looks like I made a mistake in the script I shared ... if you want to generate a notice when a new website is found, then the notice should go under the if statement where the value is added to the set/table websites. If you have a master list (set) of websites you'd like to manage separately from a dynamic set/table, then you can add an if statement to check if the value is or isn't in that list as well ... 

const master_list: set[string] = { "google.com", "evil.com" };
{ ... }
if ( name == "HOST" )

  if ( value in master_list )

    #do something

- Josh



On Wed, May 28, 2014 at 2:28 PM, anthony kasza <anthony.kasza at gmail.com> wrote:

Tables also have nice key/value expire functionality that can help curb their size.
>-AK
>On May 28, 2014 11:22 AM, "Josh Liburdi" <liburdi.joshua at gmail.com> wrote:
>
>Brian,
>>
>>
>>Taking a guess, your difficulty might be stemming from trying to use a table type when instead you may want to use a set type. In the Bro language, a table always has an index and yield, but a set is just a list ... if you only want a list of sites users are visiting (no indexes or yields, only a list), then the set type is what you should use. Here's a simple example (hopefully the script formatting comes through correctly): 
>>
>>
>>global websites: set[string];
>>
>>
>>
>>event http_header(c: connection, is_orig: bool, name: string, value: string)
>>{
>>if ( name == "HOST" ) 
>>  if ( value !in websites )
>>    add websites[value];
>>}
>>
>>
>>event bro_done()
>>{
>>print websites;
>>}
>>
>>
>>Keep in mind that in production, the websites set is likely to grow enormously and could cause memory issues. For local testing with pcap, the above script will print the seen host values to stdout when Bro finishes. Alternatively, if you want a table of the originating hosts by the websites they connected to ...  
>>
>>
>>global websites: table[string] of set[addr];
>>
>>
>>event http_header(c: connection, is_orig: bool, name: string, value: string)
>>{
>>if ( name == "HOST" ) 
>>  {
>>  if ( value !in websites )
>>    websites[value] = set();
>>  if ( value in websites )
>>    {
>>    add websites[value][c$id$orig_h];
>>    print "found a new website! "+ value;  
>>    # ^^^ could raise notice here instead
>>    }
>>  }
>>}
>>
>>
>>event bro_done()
>>{
>>print websites;
>>}
>>
>>
>>Same caveat as before regarding the size of the table and running in production, but the above table contains indexes of websites with yields that are sets of the originating hosts who connected to those sites; the table is printed to stdout when Bro finishes. If you want to take this to production, you'd likely be better off writing the websites set/table to a file and re-reading it with the input framework ... I haven't had a need to explore doing that yet, so I don't have much experience there. 
>>
>>
>>- Josh
>>
>>
>>
>>On Wed, May 28, 2014 at 10:03 AM, Brian Chilton <chilton.brian at yahoo.com> wrote:
>>
>>all,
>>>
>>>I will apologize up front for my lack of knowledge in this subject but after 3 weekends of 8 to 12 hours searching I have officially hit the end of the road so I am reaching out to the community hoping you all might have some answers.  What I'm trying to do is simple in context I just don't know the language good enough to do it here is the logic.  
>>>
>>>if (http connection established and method is post) 
>>>check to see have we visited this site before (compare against master list (or table))
>>>if visited this site before
>>>------ignore connection 
>>>if site is newly visited
>>>------add site to list or table, and alert 
>>>
>>>really simple in logic but for the life of me I cannot figure out how to add to a list or table after comparing to that table.   Hopefully I explained this well enough, but if I didn't please let me know and I will try my best to explain it better. 
>>>
>>>
>>>thanks,
>>>
>>>Brian, 
>>>
>>>_______________________________________________
>>>Bro mailing list
>>>bro at bro-ids.org
>>>http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro
>>>
>> 
>>_______________________________________________
>>Bro mailing list
>>bro at bro-ids.org
>>http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro
>> 
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20140529/23baeb5c/attachment.html