[Bro] Bro and APCON
josh.guild at morphick.com
Thu Apr 21 17:07:23 PDT 2016
That's awesome that you got it worked out! I'll take a look and see what
the configs are for the APCONs we're dealing with and maybe get to the
bottom of our issue.
On Thu, Apr 21, 2016, 19:58 Dave Crawford <bro at pingtrip.com> wrote:
> Just providing a quick update that think we solved our issue. Our SPANs
> began getting FabricPath packets after the network team upgraded to Nexus
> switches (see my separate thread about workers at 100% CPU).
> Long story short, Apcon has a "Protocol Stripping" configuration screen
> where you can enable stripping of the FP encapsulation layer. After
> enabling that option the "data_before_established" and "inappropriate_FIN"
> messages stopped.
> On Apr 21, 2016, at 12:22 PM, Josh Guild <josh.guild at morphick.com> wrote:
> Yep, in one of the environments, we're getting a ton of
> "possible_split_routing" and "data_before_established" both with and
> without the APCON in the mix.
> Thinking it has to do with how Bro is handling the load balancing to
> On Thu, Apr 21, 2016 at 10:45 AM, Dave Crawford <bro at pingtrip.com> wrote:
>> Thanks Josh,
>> We’re also trying to determine if the Apcon is a red herring since
>> (unfortunately) two changes were made at the same time. While we swapped
>> our Anues for Apcons the network team was also upgrading to Nexus switches.
>> Our weird log started filling with with “data_before_established” and
>> “possible_split_routing” events right after the changes.
>> On Apr 21, 2016, at 10:30 AM, Josh Guild <josh.guild at morphick.com> wrote:
>> Hi Dave,
>> We bypassed the APCON in one of the environments and it helped a little
>> with capture loss (about a 10% drop) and errors in the weird.log.
>> Unfortunately, this was during a weekend so it's tough to say how much of
>> an impact it made. Another network we're in fixed some load balancing
>> issues upstream and this help significantly with loss (though weird.logs
>> remain about where they were). I think the APCON may have been a red
>> herring in this instance but I'd be curious to see how your network looks
>> before and after implementation if you'd like to keep in touch.
>> The main things I've been looking at are capture loss and weird.log
>> errors (specifically HTTP_version_mismatch, SYN_seq_jump,
>> TCP_seq_underflow_or_misorder) these may lean towards traffic being
>> mangled. This presentation is pretty helpful in showing you what to look
>> On Thu, Apr 21, 2016 at 10:08 AM, Dave Crawford <dave at pingtrip.com>
>>> Were you able to solve this issue? We just started swapping out our
>>> current solution with Apcon’s and wondering if we’ll run into the same
>>> On Apr 7, 2016, at 2:39 PM, Josh Guild <josh.guild at morphick.com> wrote:
>>> Hi all,
>>> We have a few deployments that utilize an APCON for traffic aggregation.
>>> We've noticed in these environments that Bro has trouble reassembling the
>>> traffic correctly and there is a significant amount of capture loss (based
>>> on the script). We've tried different hashing algorithms on the APCON to no
>>> Has anyone else seen anything similar to this or have any insight?
>>> Josh Guild
>>> Network Intelligence Analyst
>>> <https://twitter.com/stay_spooky> <https://keybase.io/joshuaguild>
>>> Bro mailing list
>>> bro at bro-ids.org
>> Josh Guild
>> Network Intelligence Analyst
>> <https://twitter.com/stay_spooky> <https://keybase.io/joshuaguild>
> Josh Guild
> Network Intelligence Analyst
> <https://twitter.com/stay_spooky> <https://keybase.io/joshuaguild>
-------------- next part --------------
An HTML attachment was scrubbed...
More information about the Bro