[Bro-Dev] [JIRA] (BIT-1504) The facility to serialize tables to a log

[Johanna Amann (JIRA)]

    [ https://bro-tracker.atlassian.net/browse/BIT-1504?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=22808#comment-22808 ] 

Johanna Amann commented on BIT-1504:
------------------------------------

Hi,

sorry, it will not be easily possible to support this use-case. Basically, when the logging and input frameworks were first designed, the feature set was cut off at tables -- we wanted to be able to --at a maximum--log arrays of data in a single "line" of log output, no tables; while this might be easy for formats like JSON, for other formats like just ASCII text, etc. it gets complicated quite soon. Also - what do you want to do if someone tries to log a table of a table --- (json once again has not a problem with that, but in a pure ascii format this just gets ugly).

Because of this, the underlying data types that are used to exchange data between the logging framework and the input framework and the main Bro thread do not support tables at all.

The reason why you see support for TYPE_TABLE in some places of the logging and input framework are sets. These are loggable and usable as input - and are internally represented as tables in Bro.

> The facility to serialize tables to a log
> -----------------------------------------
>
>                 Key: BIT-1504
>                 URL: https://bro-tracker.atlassian.net/browse/BIT-1504
>             Project: Bro Issue Tracker
>          Issue Type: New Feature
>          Components: Bro
>            Reporter: Aaron Eppert
>            Priority: High
>
> ```@load base/protocols/http/main
> @load base/protocols/http/utils
> module HTTP;
> redef record Info += {
> 	cookies: table[string] of string &optional &log;
> };
> event http_header(c: connection, is_orig: bool, name: string, value: string)
> {
> 	if ( is_orig && name == "COOKIE" ) {
> 		if ( ! c$http?$cookies ) {
> 			c$http$cookies = table();
> 		}
> 		
> 		local cookie_vec = split_string(value, /;[[:blank:]]*/);
> 		
> 		for (cookie in cookie_vec) {
> 			local kv = split_string(value, /=/);
> 			if (|kv| == 2) {
> 				c$http$cookies[kv[0]] = kv[1];
> 			}
> 		}
> 	}
> }
> ```
> Simple example. The ability to serialize the above to a log file, given it uses simple string indices and values would seem to be straight forward per looking at the Ascii and JSON writers, which appear to support TYPE_TABLE natively. I spent some time looking at how to implement this at the layers above, but the (!t->IsSet()) in SerialTypes.cc's Value::IsCompatibleType(...) is an obvious blocker and I ran out of time to deduce the rest.
> I would assume I am not alone in this want as it would make proper downstream referencing of the resulting KV pairs from the table especially easy to navigate. This is, again, very much the case when using the JSON writer given it should natively serialize into very easily usable KV pair notation.



--
This message was sent by Atlassian JIRA
(v7.0.0-OD-08-005#70107)