[Bro] connecting to bro with broccoli

[Seth Hall]

On Aug 31, 2014, at 2:31 PM, daniel nagar <dngr7512 at gmail.com> wrote:

> I was sending out many HTTP requests which causes raising of many events per request/response

Generally, I wouldn't recommend sending around protocol based events.  Sending anything with a connection record that needs serialized and deserialized is probably not a good idea. 

Why are you sending so much data by the way?  You may have approached the problem with a suboptimal design.

> I've figured out the memory expansion problem, it seems that the "ChunkQueue" in "ChunkedIO" does not have a limit and I was sending events at higher speeds than my broccoli client could process so the queue just kept growing.

I was sort of curious if that's what was going on.  Nice to have an answer to that. :)

> This is a temporary fix in my opinion, a more robust communication framework is needed such as using an external queue (such as ActiveMQ / ZeroMQ) for transferring events/chunks. 

There is already major overhaul of Bro's communication system underway.

  .Seth

--
Seth Hall
International Computer Science Institute
(Bro) because everyone has a network
http://www.bro.org/

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 495 bytes
Desc: Message signed with OpenPGP using GPGMail
Url : http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20140901/cd132e92/attachment.bin