[Bro-Dev] [EXTERNAL] Re: [Bro] Logging VLAN IDs
Thomas, Eric D
edthoma at sandia.gov
Tue Apr 28 11:32:32 PDT 2015
I thought more about your generalized idea and would like to follow up. To
start, adding link-level features to the connection ID hash, while perhaps
useful in some contexts, does not provide us the functionality we desire.
I have an incoming feed of VLAN-tagged traffic (both VLAN and 802.1ah)
with perhaps dozens of different VLANs, and I would like to handle the
connections differently in scripts but also mainly in offline log analysis
depending upon which VLANs the traffic is associated with.
Initially I had proposed simply adding the VLAN Ids to the conn.log file,
but that is certainly too specific of a solution. What are your thoughts
on exposing link-level features at the script layer for connections? For
example, if all observed VLAN tags for a connection were in a set variable
of the script-level Connection record, I could then label my data by
matching VLAN Ids, then process them differently accordingly. Thoughts?
edthoma at sandia.gov
On 4/17/15, 8:55 AM, "Robin Sommer" <robin at icir.org> wrote:
>(Cc'ing bro-dev, I suggest we continue the thread there).
>This sounds generally reasonable, however I think we could take the
>opportunity here to generalize this a bit more for generally including
>link-layer information into connection handling.
>One thing that I didn't quite get form your description is if the
>objective is really just to get the VLAN ID into conn.log, or whether
>you also want to use it for defining what constitutes a connection in
>the first place. The latter would aim at the situations where the same
>IP addresses can appear on different VLANs for independent
>connections. Right now, Bro can't keep them apart, but if we made the
>VLAN part of the connection index into the session table, it would
>treat them separately. Same applies to other link-level features that
>could sometimes be useful to be a part of a connection's ID (like MAC
>With that in mind, some thoughts on generalizing this (note, I not
>sureif you're working from 2.3 or git master. The PktSrc API has
>changed a bit recently, I'll take git as my starting point).
> - One challenge is passing the the VLAN ID through to the various
> packet-related methods. You're suggesting additional parameters,
> which would work. However, these methods are already taking a
> bunch of parameters, and if in the future we wanted to pass
> through further link-layer info, we'd have to add even more. A
> more flexible alternative would be switching to simply passing a
> Packet structure around that encapsulates all the information,
> including what's already there (e.g., timestmap, pcap_hdr,
> payload, etc.). The new PktSrc API already has such a class:
> PktSrc::Packet; from a quick look I think we could elevate that
> to be something passed around more generally, and then extend it
> - For the connections, I would store the VLAN inside the ConnID
> struct, and then modify BuildConnIDHashKey() to take it into
> account. That way, the session table will make it part of its
> index. Same for the script-land conn_id record; that will then
> make script-level tables work that index by conn_id.
> - Extending the ConnID like this could actually be made a run-time
> option: I believe it shouldn't be too difficult to let users
> chose the fields defining a ConnID, so that they can decide if,
> say, they want to VLAN to be in there or not. We could predefine
> a set of potential features to choose from, along with some
> script-land API to pick the set to use, with the current 4-tuple
> being the default. (This could be a 2nd step for later; if the
> first two points above were in place, this extension should
> become mainly a question of finding the right configuration
>I haven't thought this thruogh too carefully, so it's conceivable that
>I'm missing something. But I think it would be really helpful for many
>folks to get more flexibility into the definition of what consitutes a
>connection, with VLANs being a good initial target to support.
>On Tue, Apr 14, 2015 at 16:59 +0000, you wrote:
>> Dear Bro developers,
>> I've been tasked with trying to modify the Bro source code so that
>> conn.log includes the VLAN IDs (including 802.1ah) that have been
>> in packets associated with that connection. I've scoped out a solution,
>> but I want to run it by you first before I start to go for it, in case
>> missing something really big.
>> PktSrc::Process() does processing of VLAN and 802.1ah, but it just skips
>> over them by advancing the data pointer. I will, in addition, store
>> VLAN IDs in a new member of the modified PktSrc class. This gets passed
>> through net_packet_dispatch() and NetSessions::DispatchPacket(). At this
>> point NetSessions::NextPacket() gets called, but since the PktSrc
>> get passed to it, I'd need another way to pass it the VLAN ID. I am
>> considering two options:
>> 1. duplicate NextPacket() adding a new parameter to pass it the VLAN
>> and call that instead, or
>> 2. store the VLAN IDs in the NetSessions class, in DispatchPacket() so
>> it¹s available to NextPacket() and DoNextPacket() <- Is there a reason
>> this wouldn¹t work, e.g. issues with multi-threading/multi-processing?
>> Is there one option that seems better to you?
>> NetSessions::DoNextPacket() is called next and I would also need a
>> modification to pass it VLAN IDs, using one of the options above. In
>> method we finally get access to the appropriate Connection instance, so
>> would store the VLAN IDs in that instance in DoNextPacket().
>> I'd need to modify the Connection class in Conn.h to include a new
>> for tracking VLAN IDs. I'd modify Connection::BuildConnVal() and
>> scripts/base/init-bare.bro's connection record to make the VLAN IDs
>> available to scripts. Lastly, I'd write a script to redef the conn Info
>> structure and handle one or more connection events (perhaps
>> connection_state_remove) to copy the VLAN IDs from the connection record
>> to the Info record.
>> Is there anything I'm missing? Is there a better way to approach this?
>Robin Sommer * ICSI/LBNL * robin at icir.org * www.icir.org/robin
More information about the bro-dev