Re: [RFC] numactl doesn't work as expected with --phycpubind II

[Cliff Wickman <cpw@xxxxxxx>]

On Fri, Jun 20, 2008 at 06:31:08PM +0530, Sharyathi Nagesh wrote:
> Cliff,Andi
>
> Cliff Wickman wrote:
>
>> You are using version 1.0.3 I assume.  We have 1.0.3-rc1 and 1.0.3-rc2
>> on the download page (ftp://oss.sgi.com/www/projects/libnuma/download).
>> I suggest you send me the final form of your patch and I add it and post
>> the source as 1.0.3.   Sound reasonable?
>
> I am using still older version of numactl, it is 0.9.6. numactl code base 
> 1.0.3 has call to move_page system call and this will cause problems that I 
> have mentioned in previous mail
>
>>>>>> +    cpu_set_t mask;
>>>>>> +    if( numa_sched_getaffinity(0, sizeof(cpu_set_t), &mask) < 0 ){
>>>>>> +        perror("numa_sched_getaffinity:");
> > Ah noticed another problem. Please don't rely on the glibc cpuset_t,
> > there's no guarantee it's big enough.
>
> > In fact libnuma.c already has code for this in set_numa_max_cpu().
> > Please reuse that. Just make sure you don't create a backdoor
> > interface into libnuma.
>
> Andi suggested to used the existing functions to get through the problem, 
> suggested to use 'set_numa_max_cpu', its equivalent in 0.9.6 version is 
> 'number_of_configured_cpus'. On using this in util.c to get the maximum cpu 
> number solves the problem.
>
>  But I was not convinced with the solution, It returns total number of 
> bytes in cpu mask (bit array) this is not what is expected logically.
>  So what I am trying to get is - even the latest code doesn't have any 
> equivalent function which does the work of sysconf(_SC_NPROCESSORS_CONF) 
> correctly.
>  Hence we may have to keep the implementation to return the maximum cpu 
> active, may be pushing it into libnuma.c
>  Please let me know your thoughts
>
> Thanks
> Sharyathi

I agree that the patch from Anton Blanchard fixes the old  0.9.6 version,
up thru version 1.0.2 as well.

These are available in 1.0.3 and beyond.
(from the man page)

numa_num_configured_cpus() returns the number of cpus  in  the system.
This count includes any cpus that are currently disabled. This count is
derived from the cpu numbers in /sys/devices/system/cpu. If the kernel
is configured without /sys (CONFIG_SYSFS=n) then it falls back to using
the number of online cpus.

numa_num_thread_cpus()  returns  the  number  of  cpus that the calling
thread is  allowed  to  use.   This  count  is  derived  from the  map
/proc/self/status,   field   "Cpus_allowed".   Also   see  the bitmask
numa_all_cpus_ptr.

But as Andi suggested, why bother with the bugs in old versions?

If the issue is just that you are running on a kernel that does not
have move_pages(2), what you might do is grab the latest version
(numactl-2.0.2) and patch move_pages (in syscall.c) to return an error.

-Cliff
-- 
Cliff Wickman
Silicon Graphics, Inc.
cpw@xxxxxxx
(651) 683-3824
--
To unsubscribe from this list: send the line "unsubscribe linux-numa" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at  http://vger.kernel.org/majordomo-info.html