[Bro-Dev] [JIRA] (BIT-1215) bro-cut should be rewritten in C for speed and to not depend on gawk

[Robin Sommer]

On Thu, Jul 10, 2014 at 17:27 -0500, you wrote:

> I think start with 1M and realloc 2x as needed is the way to go after
> all.

Yes. Maybe a bit less than 2x, exponential grows quickly. :)

> I think the only thing to do would be to add an absolute max line
> length of 64M or something to handle the case where someone
> accidentally runs bro-cut against a binary file (like a compressed bro
> log) that just doesn't contain any newlines.

Would be nicer to recognize that differently, like by not finding a
log header; that way we can give a good error message. If such a check
is in place, I wouldn't actually bother with another double-check on
line length; in the unlikely case that the file has a correct header
but totally broken content, I'm sure there are plenty other cases
where bro-cut would fail, and it seems there's not more here that can
happen in addition than running out of memory (which the OS will
catch).