[Bro] question: memory requirements of tables

Holger Dreger hdreger at net.in.tum.de
Tue Feb 15 10:17:39 PST 2005


I put this back onto the mailing list, since I think we found again an
interesting memory issue...

On Tue, Feb 15, 2005 at 01:56:56PM +0000, Christoph Goeldi wrote:
> i have no clue what most of the output of the profiling script means,
> but this is interesting:
> 
> the test_table really seems to decrease its memory usage massively.
> but why not proportionally to the containing entries?
> when i have a lot of tables it is really a problem, that each table
> consumes 3 MB even if there are (almost) no entries in it anymore...

we just quickly checked the code: tables are only resized when they
grow beyond a certain threshold. They are not resized when enough
elements are deleted. So far this was not a problem since there are
usually a few tables that grow large (and usually remain large). You
run into problems as soon as you have tables that grow and shrink only
once in their lifetime. 

> your right, there is a second problem:
> even if the tables decrease its memory consumption massively bro does not
> seem to deallocate this memory space.
> 
> have a look at this log file from the profiling script:
>
> ..... 
> > 1108468702.534722 Memory: total=28449K total_adj=27700K malloced: 4842K
> .....

This line from the profiling.log file indicates that the system
accounts 28M memory allocated by the bro process (field total). On
the other hand the malloc library claims bro actually uses only 5M
(field malloced). The difference is memory that is not (yet) returned
to the system (by the malloc lib!) but will be reused as soon as you
reallocate memory. On some (all?) systems you can somehow configure
the malloc lib to return freed memory to the system.

	Holger



More information about the Bro mailing list