[Bro-Dev] #1038: Input framework memory usage issue with huge files
Bro Tracker
bro at tracker.bro.org
Mon Jul 15 14:15:57 PDT 2013
#1038: Input framework memory usage issue with huge files
------------------------+------------------------
Reporter: scampbell | Owner: amannb
Type: Problem | Status: accepted
Priority: Medium | Milestone: Bro2.2
Component: Bro | Version: git/master
Resolution: | Keywords:
------------------------+------------------------
Comment (by scampbell):
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
Thanks - I will try to get at it tonight!
The workaround to the "reading huge files" problem seems like a good
one. Now that you mention it, I did notice that before as well and
the description also explains a number of peculiar behaviors that I
have been seeing with the input framework.
cheers,
scott
On 7/15/13 4:00 PM, Bro Tracker wrote:
> #1038: Input framework memory usage issue with huge files
> ------------------------+------------------------ Reporter:
> scampbell | Owner: amannb Type: Problem | Status:
> accepted Priority: Medium | Milestone: Bro2.2 Component:
> Bro | Version: git/master Resolution: |
> Keywords: ------------------------+------------------------ Changes
> (by amannb):
>
> * priority: High => Medium
>
>
> Comment:
>
> Fastpath contains a fix for the memory leak -- could you check if
> that fixes your problem?
>
> However, there is another issue with the input framework at the
> moment. While testing the memory leak, I noticed that Bro memory
> usage will spike while reading huge files. The reason for this is,
> that the input framework will add data to the
> child-thread->main-thread queue much faster than it is being read,
> which can mean that for a file of several hundred Megabytes,
> several GB of main memory are being held.
>
> Robin and I think that solution for this is probably to make the
> child- thread->main-thread queue blocking once it reaches a certain
> size.
>
-----BEGIN PGP SIGNATURE-----
Version: GnuPG/MacGPG2 v2.0.19 (Darwin)
Comment: GPGTools - http://gpgtools.org
Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/
iEYEARECAAYFAlHkZbsACgkQK2Plq8B7ZBwnAwCgkBY5pbIeZSRLjH5+b9RnLkTD
8rAAn1SXnTYipsAsdMX870e5TAX8OqrJ
=J563
-----END PGP SIGNATURE-----
--
Ticket URL: <http://tracker.bro.org/bro/ticket/1038#comment:7>
Bro Tracker <http://tracker.bro.org/bro>
Bro Issue Tracker
More information about the bro-dev
mailing list