Is this Numa Issue ??

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



Dear All,

Greetings!!

We have been noticing an issue with Numa2.0 on SLES11. While issuing java
-version when java initializes some NUMA related things we get several
following messages

map size mismatch; abort
: No such file or directory

With some instrumentation  in Java and Numa we notice that the following is
causing the issue:

These messages are coming off numa API "numa_node_to_cpus"

This api has been changed in numa version 2.0. It has new implementation
(_v2)

numa_node_to_cpus api (man page info - converts  a node number to a bitmask
of CPUs. The user must pass a long enough buffer. If the buffer is not long
enough errno will be set to ERANGE and -1 returned. On success 0 is
returned.)  expects following as arguments as far as user is concerned
node number
Buffer
Size of Buffer

numa_node_to_cpus_v1 (original) processed these arguments fine, however
numa_node_to_cpus_v2 processes these arguments differently to generate
bitmask of CPU.
buffer length passed gets multiplied by 8 and passed to
numa_node_to_cpus(node, struct bitmask* buf) ----> this is v2 signature (2
args)

So for e.g we pass as array of size 512. The bitmask buf would have size of
4096 (512*8).

On the other hand the numa code internally does some calculation to
initialize node cpu mask. This internal calculation is based on sys call "
sched_getaffinity" return value (In my failing scenario its coming out to
be 1024). It returns process's CPU affinity mask.

My system has 4 node and 8 CPUs configured.

Now within the numa_node_to_cpus code there is a check if the size of
buffer passed from the user matches the one returned by the
sched_getaffinity. This check fails and hence we see "map size mismatch:
abort" messages coming out on console.

I would like to understand why there is a difference here. Does numva API
(_v2) expect something different from the user.  For e.g If I match my
buffer size to 1024 (128*8 instead od 512*8) I do not see these messages.
Kindly clarify who is at fault here ....

As per numa man page documentation, there is no restriction on the size of
buffer which should be passed to numa_node_to_cpus api.


Testcase to reproduce the problem

#include <errno.h>
#include <stdio.h>
#include <stdlib.h>
#include <numa.h>

typedef unsigned long BUF[64];

int numa_exit_on_error = 0;

void node_to_cpus(void)
{
        int i;
        BUF cpubuf;
        BUF affinityCPUs;
        int maxnode = numa_max_node();
        printf("available: %d nodes (0-%d)\n", 1+maxnode, maxnode);
        for (i = 0; i <= maxnode; i++) {
                printf("Calling numa_node_to_cpus()\n");
                printf("Size of BUF is : %d \n",sizeof(BUF));
                if ( 0 == numa_node_to_cpus(i, cpubuf, sizeof(BUF)) ) {
                        printf("Calling numa_node_to_cpus() again \n");
                        if ( 0 == numa_node_to_cpus(i, cpubuf, sizeof(BUF))
) {
                        } else {
                                printf("Got < 0 \n");
                                numa_error("numa_node_to_cpu");
                                numa_exit_on_error = 1;
                                exit(numa_exit_on_error);
                        }
                } else {
                        numa_error("numa_node_to_cpu 0");
                        numa_exit_on_error = 1;
                        exit(numa_exit_on_error);
                }
        }
}
int main()
{
        void node_to_cpus();
        if (numa_available() < 0)
        {
            printf("This system does not support NUMA policy\n");
            numa_error("numa_available");
            numa_exit_on_error = 1;
            exit(numa_exit_on_error);
        }
        node_to_cpus();
        return numa_exit_on_error;
}

I would appreciate an early response as this issue is holding off our work.

Many Thanks!!

Warm Regards,
Deepti Kalra

IBM Software Labs,
JTC - Development Team,
1st Floor, Fortius Block FTP, EPIP Whitefield,
Bangalore - 560066
Direct: +91 80 22669287
Email: deepti.kalra@xxxxxxxxxx
****************************************
"The question isn't who is going to let me;
 it's who is going to stop me."

--
To unsubscribe from this list: send the line "unsubscribe linux-numa" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[Index of Archives]     [Linux Kernel]     [Linux USB Devel]     [Video for Linux]     [Linux Audio Users]     [Yosemite News]     [Linux SCSI]     [Devices]

  Powered by Linux