[Bro] Seeking hardware/OS recommendations

[Stephen J. Smoogen]

On 4/7/06, Joncarlo Ruggieri <jruggieri at ucdavis.edu> wrote:
> Hi,
>
> We are currently running Bro on 2 Dell PowerEdge 2650s.
>
> Each has 2 Syskonnect SK-9844 cards.
>
> Each machine is listening to 2 taps (4 interfaces, which represet ingress
> and egress traffic for each of the 2 taps).
>
> The systems are running RedHat Enterprise Linux 4 AS.

There is a long list of things that can be an issue here:

1) What are the hardware IRQ's for each of the network cards
2) What version of libpcap are you using with bro (an alternative
libpcap from http://public.lanl.gov/cpw/ is known to enhance snort and
other tools. We used it at lanl for our test bro system without load
problems)

The 3 things that we did to heighten performance:

Use a 2.6 kernel with large memory buffers.
Use cards that support this (I do not know about the syscon, but the
eepro1000 worked fine)
Make sure that each network card has a dedicated IRQ to it.

We also found that for most hardware.. we needed to have one dedicated
system per tap for network speeds over 500 mbit/s. The hardware person
thought it was limits on the intel hardware architecture. I left
before we got into studying what changes the AMD hardware architecture
would bring to the mix.

More Answers below:


>
> I suppose our questions are:
>
> 1) Which OS should we use - FreeBSD or RedHat?

Whatever floats your boat is my general opinion. I worked for Red Hat
for many years so have a bias towards it.. but I know that the Bro
code seems optimized for FreeBSD.

>
> 2) Can anyone recommend using the Sun Fire X2100s or X4100s?
>

Not I. I have found that the Sun AMD Ultra 20 is really nice, but we
have had problems with the Ultra 40 due to network driver issues.

> 3) Does anyone have advice regarding the Syskonnect SK-9844 or SK-9E92 cards?
>

The main things is to find out how they work with NAPI or the
equivalent of NAPI in *BSD kernels.

> 4) Is it reasonable to assume that the most intensive part of this process
> is the initial collection and analysis by Bro which results in the various
> Bro log files?
>

I found that for most packet captures.. the initial capture was as
much work as the analysis. The gurus set it up to use one box for
captures and then regularly pipe over 2 GB files to the analysis
machines. This allowed the analysis to happen with multiple tools and
run as unpriviledged users

> 5) Are there other hardware or OS recommendations?
>
> I'm sure I omitting something, but this is a good start.
>
> Thanks in advance for your advice!
>

I think the 2650's might still work with some optimization.. at least
as backup systems until your SunFires (or Alienware or whatever) come
in

--
Stephen J Smoogen.
CSIRT/Linux System Administrator