[Bro] So uh...how do you know which pin_cpus to use?

James Lay jlay at slave-tothe-box.net
Tue Oct 18 14:23:08 PDT 2016


On 2016-10-18 15:19, Azoff, Justin S wrote:
>> On Oct 18, 2016, at 5:12 PM, James Lay <jlay at slave-tothe-box.net> 
>> wrote:
>> 
>> On 2016-10-18 14:55, Azoff, Justin S wrote:
>>>> On Oct 18, 2016, at 4:52 PM, Azoff, Justin S <jazoff at illinois.edu>
>>>> wrote:
>>>> 
>>>> 
>>>>> On Oct 18, 2016, at 4:34 PM, James Lay <jlay at slave-tothe-box.net>
>>>>> wrote:
>>>>> 
>>>>> Never really understood this:
>>>>> 
>>>>> "The correct pin_cpus setting to use is dependent on your CPU
>>>>> architecture. Intel and AMD systems enumerate processors in 
>>>>> different
>>>>> ways. Using the wrong pin_cpus setting can cause poor performance."
>>>>> 
>>>>> Is there a magical formula?  Any advice would help thanks.
>>>> 
>>>> The best thing to do is to install the hwloc package and use the
>>>> lstopo or lstopo-no-graphics tool to render a big ascii art image of
>>>> the system.
>>>> 
>>>> 
>>>> on centos7 this works:
>>>> 
>>>> lstopo-no-graphics  --of txt
>>>> 
>>>> 
>>>> You'll get something that looks like this:
>>>> 
>>>> https://www.open-mpi.org/projects/hwloc/lstopo/images/2XeonE5v2+2cuda+1display_v1.11.png
>>>> 
>>>> or
>>>> 
>>>> https://www.open-mpi.org/projects/hwloc/lstopo/images/4Opteron6200.v1.11.png
>>>> 
>>>> The numbers towards the bottom are the cpu ids.  So you can see that
>>>> using something like
>>>> 
>>>> 1,3,5,7,9,11,13,15,17,19,21,23,25
>>>> 
>>>> on an intel cpu would be the worst thing you could do since 21,23,25
>>>> are on the same physical cores as 1,3, and 5
>>> 
>>> Oh, I should add... ".. on that particular system".  On some of our
>>> numa machines the allocation is different and 1,3,5,7,9 would be the
>>> right cpus to use!
>> 
>> Ok cool thanks Justin...so basically I wanna stagger these out so I
>> don't have several processes on the same core ya?
>> 
>> 
> 
> Possibly.. I'd check with what hwloc says.  I think just turning off
> hyper threading makes this even easier since that completely removes
> the possibility of accidentally pinning 2 workers to the same core.

Sweet...thanks Justin...hwloc is a cool app!

James


More information about the Bro mailing list