Cython: specify typed memoryview as template type for C++ map - Stack Overflow

admin管理员组

文章数量:1395198

I'd like to use C++ typed maps instead of Python dicts, where the map's keys are integers and values are typed Cython memoryviews. A minimal (non)working example is:

# distutils: language=c++

from libcpp.unordered_map cimport unordered_map as Map

cdef Map[int, int[:,:]] P;

where I get the compilation error

test.pyx:5:8: Reference-counted type 'int[:, :]' cannot be used as a template argument

Is there any way to specify objects of this type? I'd like to take full advantage of Cython's speed advantages, and that means using typed containers!

I'd like to use C++ typed maps instead of Python dicts, where the map's keys are integers and values are typed Cython memoryviews. A minimal (non)working example is:

# distutils: language=c++

from libcpp.unordered_map cimport unordered_map as Map

cdef Map[int, int[:,:]] P;

where I get the compilation error

test.pyx:5:8: Reference-counted type 'int[:, :]' cannot be used as a template argument

Is there any way to specify objects of this type? I'd like to take full advantage of Cython's speed advantages, and that means using typed containers!

• types
• cython

Share Improve this question Follow edited Mar 27 at 4:34 InSync 11.1k44 gold badges1818 silver badges5656 bronze badges asked Mar 27 at 2:11 apizzimentiapizzimenti 47722 gold badges88 silver badges2020 bronze badges

Add a comment  | 

1 Answer 1

Sorted by: Reset to default 0

As the error says: "Reference-counted type 'int[:, :]' cannot be used as a template argument", your example does not work because memoryviews are Python objects with reference counting, while C++ templates expect plain C++ types.

Depending on your applications, you first have to decide in which side is the memory managed, Python or C++?

If the memory is managed in C++ and used in Python, one can use a plain C++ container to hold the data, in your example, Map[int, vector[int]]. If one wants to treat the vector in Map as 2D arrays, one can use python memoryviews to view the vectors' underlying pointers, for example:

import numpy as np
from libcpp.vector cimport vector
​
cdef vector[int] a = [1, 2, 3, 4]
cdef int[:] va = <int[:4]>&a[0]
np_arr = np.asarray(va).reshape(2, 2)
print(np_arr)
# OWNDATA is False indicates momeory is not managed by numpy, but C++
print(np_arr.flags)   
# Modify elements in the array will change the viewed vectors.
np_arr[1, 1] *= 10
print(a)

If the memory is managed in Python and used in C++, for example, numpy arrays hold the data and a std::unordered_map holds the underlying data pointers:

import numpy as np
from libcpp.unordered_map cimport unordered_map as Map
​
cdef int[:, ::1] v0 = np.array([1, 2, 3, 4]).reshape(2, 2)
cdef int[:, ::1] v1 = np.array([5, 6, 7, 8]).reshape(2, 2)
​
# Treat the underlying data as 1D arrays
cdef Map[int, int*] P;
​
P[0] = &v0[0, 0]
P[1] = &v1[0, 0]
​
# Modify pointers in the map will change the elements of the underlying numpy arrays
P.at(0)[3] *= 10
print(np.asarray(v0))

In my experience, making it first who is responsible for the memory and exchanging data when necessary is a good practice and also efficient.

Warning: when using the memory through views or pointers, one has to make sure the underlying memory is not released.

本文标签： Cython specify typed memoryview as template type for C mapStack Overflow