[Bro] Bro Types Not Following Bro Types Documention

[John Omernik]

(Mike I am putting this on list, I replied only to you)

I found https://www.bro.org/sphinx-git/logs/index.html  which is helpful in
that - represents an unset field.  I am still think from a data nerd
perspective, having a character that doesn't fit the type to represent
something is dangerous, however, I can parse the values and replace
programmatically with the information provided, so now I an continue on my
merry way. Thanks for the insight.

That all said... why put anything in a field (as a default) to represent
unset or empty? Are we at risk of evasion? Besides obviously breaking
typing, what about when the type actually accepts the unset character...
what if the user-agent is - or (empty) couldn't that cause downstream
errors?  "You can change the logs to log however you want" is likely the
answer, and correct I can, but shouldn't we try be logical in our approach
so assumptions aren't made on the default material?

On Sat, May 16, 2015 at 4:44 PM, Mike Dopheide <dopheide at gmail.com> wrote:

> John,
>
> The "-" you're seeing in this case isn't meant to be representing a
> count.  The "-" is used in Bro logs to represent a field with missing
> data.  For whatever reason Bro couldn't the bytes sent in this case.
> You'll see -'s more commonly in other logs, like http.log, ssl.log, etc.
>
> -Dop
>
>
>
> On Sat, May 16, 2015 at 3:57 PM, John Omernik <john at omernik.com> wrote:
>
>> I am working doing some automation with Bro, Avro, Kafka and I am a
>> little bit frustrated. (Or I am looking at the wrong documentation, hence
>> my post here, I am very good with being extremely wrong because I am
>> looking at something wrong).
>>
>> Specifically I am looking at the default conn.log. The Type that is
>> specified for some fields such as
>> orig_bytes or resp_bytes is type count
>>
>> Based on the docs I am using here:
>>
>> https://www.bro.org/sphinx/script-reference/builtins.html
>>
>> a count is:
>>
>> count
>>
>> A numeric type representing a 64-bit unsigned integer. A count constant
>> is a string of digits, e.g.1234 or 0. A count can also be written in
>> hexadecimal notation (in which case “0x” must precede the hex digits), e.g.
>> 0xff or 0xABC123.
>>
>> The count type supports the same operators as the int
>> <https://www.bro.org/sphinx/script-reference/builtins.html#type-int> type.
>> A unary plus or minus applied to acount results in an int.
>>
>>
>> This is well and good, however looking at some of the data in my log I
>> see the character "-" as a value.  Based on my reading of a count, that
>> shouldn't exist, a - is not a unsigned integer, nor is it a string of
>> digits whether in base 10 or hexidecimal.
>>
>>
>> Thus my frustration, I'd like to develop some generic bindings to push
>> bro logs into Avro Serialized Kafka messages, but looking at this, I can't
>> even trust the documentation to be accurate? Am I missing something? Is
>> there another documentation reference that more fully represents the data
>> types that would explain why - is a valid integer?
>>
>> _______________________________________________
>> Bro mailing list
>> bro at bro-ids.org
>> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro
>>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20150518/e9b45dce/attachment-0001.html