[Bro-Dev] [JIRA] (BIT-1182) Input-framework thread spwan

Aashish Sharma (JIRA) jira at bro-tracker.atlassian.net
Mon Mar 2 11:42:00 PST 2015


    [ https://bro-tracker.atlassian.net/browse/BIT-1182?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=19901#comment-19901 ] 

Aashish Sharma commented on BIT-1182:
-------------------------------------

(Mostly FYI) 

I encountered this issue again, while feeding a file with 30-40,000 IP's where about 10K or so change periodically. 

In FreeBSD, bro would die because of limit of 1500 on 

kern.threads.max_threads_per_proc

After changing kern.threads.max_threads_per_proc=12000 bro runs and doesn't die, but won't log until input-framework events are finished ! 


> Input-framework thread spwan
> ----------------------------
>
>                 Key: BIT-1182
>                 URL: https://bro-tracker.atlassian.net/browse/BIT-1182
>             Project: Bro Issue Tracker
>          Issue Type: Problem
>          Components: Bro
>    Affects Versions: 2.2
>            Reporter: Aashish Sharma
>              Labels: input-framework
>
> Using the mode REREAD, I noticed that input-framework spawns a thread for every add/change/delete for the elements in the feed file. 
> this is a VERY desired feature and powerful capability and works quite well in general settings. 
> Since, all the changes in a file spawns a thread to process for: EVENT_NEW, EVENT_CHANGED, EVENT_REMOVED, If there are lets say 5000 Changes in the file, there would be 5000 threads spawned at the same time. this is still alright and system can handle load and processing is done in a few seconds.
> However, if I include a when statement along with exec framework usage to execute an action in Input::EVENT_NEW, Input::EVENT_CHANGED or Input::EVENT_REMOVED - all threads spawned together freezes bro from processing any packets at all. 
> It would be nice if we can serialize this thread creation and spawn only a few at a time. This way we can spread out the increased load over next N mins instead of freezing bro to a standstill. 
> (As always, please let me know if you want code to be able to re-produce this issue). 



--
This message was sent by Atlassian JIRA
(v6.4-OD-15-055#64014)


More information about the bro-dev mailing list