[Bro] How information is stored in a set() and table() in bro

[BortolameottiR]

Hi everyone,

Some time ago I have dumped several log files using Bro. I had used the
script in the attachment. Essentially, during the event http_all_headers
I wanted to dump into the log the set of headers and values.

I would like to re-use this dataset I have collected, however this time
I would require the headers to be in the exact order as they are parsed.
This information is not in logs, so I was wondering whether it is
possible: given the script and the logs, to "reverse" the original order
of the headers.

For this purpose, I think I need to know:  1) how data is stored in a
"set [string]", because that's what I use to temporarily store the
values; and 2) how data is stored in "hlist: mime_header_list" which is
a table() in Bro.

Can anyone help me?

I already have a script that capture the headers in order with Bro, but
this would require me to re-capture the data for long period of time. If
I could reverse the process, it would save me quite some time.

Thanks in advance,

Riccardo

-------------- next part --------------
@load /opt/bro/share/bro/base/protocols/http
@load /opt/bro/share/bro/base/protocols/conn

redef record HTTP::Info += {
	## Write in the log ALL header names and their values
	header_values: set[string]	&optional	&log;
	
	## Add the MAC address of origin of the connection
	mac_orig: string	&optional	&log;
};

event bro_init()
	{
		local filter: Log::Filter = [$name="decanter_http", $path="decanter", $include=set("ts", "uid", "id.orig_h", "id.orig_p", "id.resp_h", "id.resp_p", "mac_orig", "method", "uri", "version", "request_body_len", "proxied", "orig_mime_types", "header_values")];
		#filter$interv = 6 hr;
		Log::add_filter(HTTP::LOG, filter);
		Log::remove_filter(HTTP::LOG, "default");
		Log::disable_stream(Conn::LOG);
		Log::disable_stream(Files::LOG);
	}

event http_all_headers (c: connection, is_orig: bool, hlist: mime_header_list)
	{
	if (c?$http && is_orig ==T)
		{
		local header_set : set[string] = set();
		print hlist;
		for (header in hlist)
			{
			print header;
			local concatenate : string;
			concatenate = hlist[header]$name + "||" + hlist[header]$value;
			add header_set[concatenate];  
			}
		c$http$header_values = header_set;
		}
	if (c$orig?$l2_addr)
		{
		c$http$mac_orig = c$orig$l2_addr;
		}
	}