[Numpy-discussion] ULONG not in UINT16, UINT32, UINT64 under 64-bit windows, is this possible?

Olivier Delalleau shish@keba...
Mon Aug 15 08:25:22 CDT 2011


The reason is there can be multiple dtypes (i.e. with different .num)
representing the same kind of data.
Usually in Python this goes unnoticed, because you do not test a dtype
through its .num, instead you use for instance "== 'uint32'", and all works
fine.
However, it can indeed confuse C code in situations like the one you
describe, because of direct comparison of .num.
I guess you have a few options:
- Do not compare .num (I'm not sure what would be the equivalent to "==
'utin32' in C though) => probably slower
- Re-cast your array in the exact dtype you need (in Python you can do this
with .view) => probably cumbersome
- Write a customized comparison function that figures out at initialization
time all dtypes that represent the same data, and then is able to do a fast
comparison based on .num => probably best, but requires more work

Here's some Python code that lists the various scalar dtypes associated to
unique .num in numpy (excerpt slightly modified from code found in Theano --
http://deeplearning.net/software/theano -- BSD license). Call the
"get_numeric_types()" function, and print both the string representation of
the resulting dtypes as well as their .num.

def get_numeric_subclasses(cls=numpy.number, ignore=None):
    """
    Return subclasses of `cls` in the numpy scalar hierarchy.

    We only return subclasses that correspond to unique data types.
    The hierarchy can be seen here:
        http://docs.scipy.org/doc/numpy/reference/arrays.scalars.html
    """
    if ignore is None:
        ignore = []
    rval = []
    dtype = numpy.dtype(cls)
    dtype_num = dtype.num
    if dtype_num not in ignore:
        # Safety check: we should be able to represent 0 with this data
type.
        numpy.array(0, dtype=dtype)
        rval.append(cls)
        ignore.append(dtype_num)
    for sub in cls.__subclasses__():
        rval += [c for c in get_numeric_subclasses(sub, ignore=ignore)]
    return rval


def get_numeric_types():
    """
    Return numpy numeric data types.

    :returns: A list of unique data type objects. Note that multiple data
types
    may share the same string representation, but can be differentiated
through
    their `num` attribute.
    """
    rval = []
    def is_within(cls1, cls2):
        # Return True if scalars defined from `cls1` are within the
hierarchy
        # starting from `cls2`.
        # The third test below is to catch for instance the fact that
        # one can use ``dtype=numpy.number`` and obtain a float64 scalar,
even
        # though `numpy.number` is not under `numpy.floating` in the class
        # hierarchy.
        return (cls1 is cls2 or
                issubclass(cls1, cls2) or
                isinstance(numpy.array([0], dtype=cls1)[0], cls2))
    for cls in get_numeric_subclasses():
        dtype = numpy.dtype(cls)
        rval.append([str(dtype), dtype, dtype.num])
    # We sort it to be deterministic, then remove the string and num
elements.
    return [x[1] for x in sorted(rval, key=str)]


2011/8/15 Pearu Peterson <pearu.peterson@gmail.com>

>
> Hi,
>
> A student of mine using 32-bit numpy 1.5 under 64-bit Windows 7 noticed
> that
> giving a numpy array with dtype=uint32 to an extension module the
> following codelet would fail:
>
> switch(PyArray_TYPE(ARR)) {
>   case PyArray_UINT16: /* do smth */ break;
>   case PyArray_UINT32: /* do smth */ break;
>   case PyArray_UINT64: /* do smth */ break;
>   default: /* raise type error exception */
> }
>
> The same test worked fine under Linux.
>
> Checking the value of PyArray_TYPE(ARR) (=8) showed that it corresponds to
> NPY_ULONG (when counting the items in the enum definition).
>
> Is this situation possible where NPY_ULONG does not correspond to a 16 or
> 32 or 64 bit integer?
> Or does this indicate a bug somewhere for this particular platform?
>
> Thanks,
> Pearu
>
> _______________________________________________
> NumPy-Discussion mailing list
> NumPy-Discussion@scipy.org
> http://mail.scipy.org/mailman/listinfo/numpy-discussion
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.scipy.org/pipermail/numpy-discussion/attachments/20110815/564f1fe8/attachment.html 


More information about the NumPy-Discussion mailing list