[Bro] So uh...how do you know which pin_cpus to use?

[Azoff, Justin S]

> On Oct 18, 2016, at 5:12 PM, James Lay <jlay at slave-tothe-box.net> wrote:
> 
> On 2016-10-18 14:55, Azoff, Justin S wrote:
>>> On Oct 18, 2016, at 4:52 PM, Azoff, Justin S <jazoff at illinois.edu> 
>>> wrote:
>>> 
>>> 
>>>> On Oct 18, 2016, at 4:34 PM, James Lay <jlay at slave-tothe-box.net> 
>>>> wrote:
>>>> 
>>>> Never really understood this:
>>>> 
>>>> "The correct pin_cpus setting to use is dependent on your CPU
>>>> architecture. Intel and AMD systems enumerate processors in different
>>>> ways. Using the wrong pin_cpus setting can cause poor performance."
>>>> 
>>>> Is there a magical formula?  Any advice would help thanks.
>>> 
>>> The best thing to do is to install the hwloc package and use the 
>>> lstopo or lstopo-no-graphics tool to render a big ascii art image of 
>>> the system.
>>> 
>>> 
>>> on centos7 this works:
>>> 
>>> lstopo-no-graphics  --of txt
>>> 
>>> 
>>> You'll get something that looks like this:
>>> 
>>> https://www.open-mpi.org/projects/hwloc/lstopo/images/2XeonE5v2+2cuda+1display_v1.11.png
>>> 
>>> or
>>> 
>>> https://www.open-mpi.org/projects/hwloc/lstopo/images/4Opteron6200.v1.11.png
>>> 
>>> The numbers towards the bottom are the cpu ids.  So you can see that 
>>> using something like
>>> 
>>> 1,3,5,7,9,11,13,15,17,19,21,23,25
>>> 
>>> on an intel cpu would be the worst thing you could do since 21,23,25 
>>> are on the same physical cores as 1,3, and 5
>> 
>> Oh, I should add... ".. on that particular system".  On some of our
>> numa machines the allocation is different and 1,3,5,7,9 would be the
>> right cpus to use!
> 
> Ok cool thanks Justin...so basically I wanna stagger these out so I 
> don't have several processes on the same core ya?
> 
> 

Possibly.. I'd check with what hwloc says.  I think just turning off hyper threading makes this even easier since that completely removes the possibility of accidentally pinning 2 workers to the same core.

-- 
- Justin Azoff