[Bro] Core affinity on AMD Opteron 6276

Ralph Holz ralph.holz.tech at gmail.com
Wed Jan 11 17:04:53 PST 2017


Hi everyone,

I've been told this is the right place to share experience and maybe a
script for our Opteron setup, and get feedback if this is the right thing
to do.

We're trialling Bro on an AMD Opteron 6276 in our campus network and did
not find much useful information on the net on how to configure this
particular setup. It's running fine now and we're waiting for students to
return to uni so we can test under a higher load (they need to watch some
videos...).

Anyway, the 6276 is a strange machine in that AMD markets it as a 64-core
machine - 4 sockets, 16 cores each. However, cores are paired within one
socket, and each pair shares resources and data lines: CPU freq regulation,
FPU, L2+L3, and instruction fetch and decode circuitry (!). This makes it
quite unusual - under Linux, you will find that different methods to count
the cores give you a different number, sometimes 32, sometimes 64.

For our experiments, we chose to use /proc/cpuinfo to determine which cores
are pairs - they should share "physical ID" and "core ID". The attached
hacky script generates a node_cluster.cfg that places 32 workers on 32
cores that should not be paired (logger, manager, proxy are going to sit on
another host).

As far as I can tell, we are not experiencing packet loss at all with the
out-of-the-box scripts loaded, but we're at just 2Gbit load atm, and come
the new semester we'll know more. The moment I add another core, I have one
worker that is experiencing loss, so I am guessing this is the upper limit.

I'd be happy to receive feedback if this is a reasonable setup and the
right thing to do.

Ralph
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20170112/1751bbe9/attachment-0001.html 
-------------- next part --------------
A non-text attachment was scrubbed...
Name: create_config_from_file.py
Type: text/x-python-script
Size: 1392 bytes
Desc: not available
Url : http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20170112/1751bbe9/attachment-0001.bin 


More information about the Bro mailing list