pointers - Splitting a MemoryMappedFile into 2 chunks of data in C# - Stack Overflow

admin管理员组

文章数量:1126077

I'm trying to create a MemoryMappedFile with 2 view accessors, each accessing different regions of this shared memory.

This is a simplified version of what I'm doing:

const int block1Size = 20;
const int block2Size = 10;

// create a shared memory region for the 2 blocks of data
MemoryMappedFile sharedFile = MemoryMappedFile.CreateNew("dummy_host_name", block1Size + block2Size);

// create an accessor for each memory block
// first block starts at index 0 and has 20 bytes
MemoryMappedViewAccessor block1Accessor = sharedFile.CreateViewAccessor(0, block1Size);

// second block starts at the and of the first block and has 10 bytes
MemoryMappedViewAccessor block2Accessor = sharedFile.CreateViewAccessor(block1Size, block2Size);

At first, this seems to work as expected, both memory view accessors return different memory addresses:

byte* block1 = null;
byte* block2 = null;

block1Accessor.SafeMemoryMappedViewHandle.AcquirePointer(ref block1);
block2Accessor.SafeMemoryMappedViewHandle.AcquirePointer(ref block2);

Console.WriteLine($"Block1 address: {(long)block1:X}");
Console.WriteLine($"Block2 address: {(long)block2:X}");

Block1 address: 178823B0000  
Block2 address: 178823C0000

But when I try to write into these memory blocks, it seems they both write and read to the same memory address.

For example, when I try to copy a byte sequence into the first memory block, it also copy that sequence to the second memory block:

// data before copying
// first block: 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0
// second block: 0, 0, 0, 0, 0, 0, 0, 0, 0, 0
fixed (byte* sourcePointer = sequenceByteArray)
{
    Buffer.MemoryCopy(sourcePointer, block1, block1size, block1size);
}

// data after copying
// first block: 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20
// second block: 1, 2, 3, 4, 5, 6, 7, 8, 9, 10

When I write to the second memory block, the same issues happens.

I'm not sure what is going on here, the pointers addresses are different, the view accessor creation seems ok (unless I misunderstood the first parameter called offset) I'm out of ideas.

Here is the entire console test code:

using System.IO.MemoryMappedFiles;

const int block1Size = 20;
const int block2Size = 10;

// create a shared memory region for the 2 blocks of data
MemoryMappedFile sharedFile = MemoryMappedFile.CreateNew("dummy_host_name", block1Size + block2Size);

// create an accessor for each memory block
// first block starts at index 0 and has 20 bytes
MemoryMappedViewAccessor block1Accessor = sharedFile.CreateViewAccessor(0, block1Size);

// second block starts at the and of the first block and has 10 bytes
MemoryMappedViewAccessor block2Accessor = sharedFile.CreateViewAccessor(block1Size, block2Size);

unsafe
{
    byte* block1 = null;
    byte* block2 = null;
    // acquire pointers for each block
    block1Accessor.SafeMemoryMappedViewHandle.AcquirePointer(ref block1);
    block2Accessor.SafeMemoryMappedViewHandle.AcquirePointer(ref block2);
    Console.WriteLine($"Block1 address: {(long)block1:X}");
    Console.WriteLine($"Block2 address: {(long)block2:X}");

    // clear garbage data
    Copy(CreateZeros(block1Size), block1);
    Copy(CreateZeros(block2Size), block2);
    PrintData(block1, block2);

    // write a sequence of 1 to 20 into the first memory block
    byte[] firstDataInput = Enumerable.Range(1, block1Size).Select(i => (byte)i).ToArray();
    Copy(firstDataInput, block1);
    PrintData(block1, block2);

    // write a sequence of 21 to 30 into the second memory block
    byte[] secondDataInput = Enumerable.Range(block1Size + 1, block2Size).Select(i => (byte)i).ToArray();
    Copy(secondDataInput, block2);
    PrintData(block1, block2);
}

static unsafe string ToStr(byte* data, int length)
{
    Span<byte> buffer = new Span<byte>(data, length);
    return string.Join(", ", buffer.ToArray());
}

static unsafe void Copy(byte[] source, byte* destination)
{
    fixed (byte* sourcePointer = source)
    {
        Buffer.MemoryCopy(sourcePointer, destination, source.Length, source.Length);
    }
}

static unsafe byte[] CreateZeros(int size)
{
    byte[] zeros = new byte[size];
    Array.Fill<byte>(zeros, 0);
    return zeros;
}

static unsafe void PrintData(byte* block1Data, byte* block2Data)
{
    string firstDataStr = ToStr(block1Data, block1Size);
    string secondDataStr = ToStr(block2Data, block2Size);

    Console.WriteLine("first block: " + firstDataStr);
    Console.WriteLine("second block: " + secondDataStr);
}

I'm trying to create a MemoryMappedFile with 2 view accessors, each accessing different regions of this shared memory.

This is a simplified version of what I'm doing:

const int block1Size = 20;
const int block2Size = 10;

// create a shared memory region for the 2 blocks of data
MemoryMappedFile sharedFile = MemoryMappedFile.CreateNew("dummy_host_name", block1Size + block2Size);

// create an accessor for each memory block
// first block starts at index 0 and has 20 bytes
MemoryMappedViewAccessor block1Accessor = sharedFile.CreateViewAccessor(0, block1Size);

// second block starts at the and of the first block and has 10 bytes
MemoryMappedViewAccessor block2Accessor = sharedFile.CreateViewAccessor(block1Size, block2Size);

At first, this seems to work as expected, both memory view accessors return different memory addresses:

byte* block1 = null;
byte* block2 = null;

block1Accessor.SafeMemoryMappedViewHandle.AcquirePointer(ref block1);
block2Accessor.SafeMemoryMappedViewHandle.AcquirePointer(ref block2);

Console.WriteLine($"Block1 address: {(long)block1:X}");
Console.WriteLine($"Block2 address: {(long)block2:X}");

Block1 address: 178823B0000  
Block2 address: 178823C0000

But when I try to write into these memory blocks, it seems they both write and read to the same memory address.

For example, when I try to copy a byte sequence into the first memory block, it also copy that sequence to the second memory block:

// data before copying
// first block: 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0
// second block: 0, 0, 0, 0, 0, 0, 0, 0, 0, 0
fixed (byte* sourcePointer = sequenceByteArray)
{
    Buffer.MemoryCopy(sourcePointer, block1, block1size, block1size);
}

// data after copying
// first block: 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20
// second block: 1, 2, 3, 4, 5, 6, 7, 8, 9, 10

When I write to the second memory block, the same issues happens.

I'm not sure what is going on here, the pointers addresses are different, the view accessor creation seems ok (unless I misunderstood the first parameter called offset) I'm out of ideas.

Here is the entire console test code:

using System.IO.MemoryMappedFiles;

const int block1Size = 20;
const int block2Size = 10;

// create a shared memory region for the 2 blocks of data
MemoryMappedFile sharedFile = MemoryMappedFile.CreateNew("dummy_host_name", block1Size + block2Size);

// create an accessor for each memory block
// first block starts at index 0 and has 20 bytes
MemoryMappedViewAccessor block1Accessor = sharedFile.CreateViewAccessor(0, block1Size);

// second block starts at the and of the first block and has 10 bytes
MemoryMappedViewAccessor block2Accessor = sharedFile.CreateViewAccessor(block1Size, block2Size);

unsafe
{
    byte* block1 = null;
    byte* block2 = null;
    // acquire pointers for each block
    block1Accessor.SafeMemoryMappedViewHandle.AcquirePointer(ref block1);
    block2Accessor.SafeMemoryMappedViewHandle.AcquirePointer(ref block2);
    Console.WriteLine($"Block1 address: {(long)block1:X}");
    Console.WriteLine($"Block2 address: {(long)block2:X}");

    // clear garbage data
    Copy(CreateZeros(block1Size), block1);
    Copy(CreateZeros(block2Size), block2);
    PrintData(block1, block2);

    // write a sequence of 1 to 20 into the first memory block
    byte[] firstDataInput = Enumerable.Range(1, block1Size).Select(i => (byte)i).ToArray();
    Copy(firstDataInput, block1);
    PrintData(block1, block2);

    // write a sequence of 21 to 30 into the second memory block
    byte[] secondDataInput = Enumerable.Range(block1Size + 1, block2Size).Select(i => (byte)i).ToArray();
    Copy(secondDataInput, block2);
    PrintData(block1, block2);
}

static unsafe string ToStr(byte* data, int length)
{
    Span<byte> buffer = new Span<byte>(data, length);
    return string.Join(", ", buffer.ToArray());
}

static unsafe void Copy(byte[] source, byte* destination)
{
    fixed (byte* sourcePointer = source)
    {
        Buffer.MemoryCopy(sourcePointer, destination, source.Length, source.Length);
    }
}

static unsafe byte[] CreateZeros(int size)
{
    byte[] zeros = new byte[size];
    Array.Fill<byte>(zeros, 0);
    return zeros;
}

static unsafe void PrintData(byte* block1Data, byte* block2Data)
{
    string firstDataStr = ToStr(block1Data, block1Size);
    string secondDataStr = ToStr(block2Data, block2Size);

    Console.WriteLine("first block: " + firstDataStr);
    Console.WriteLine("second block: " + secondDataStr);
}

• c#
• pointers
• ipc
• shared-memory

Share Improve this question Follow edited 2 days ago marc_s 754k183183 gold badges1.4k1.4k silver badges1.5k1.5k bronze badges asked 2 days ago dcidraldcidral 25933 silver badges1818 bronze badges 3

• Why not using Write method of accessor (see code example there)? Each time you access property SafeMemoryMappedViewHandle you get a IDisposable which must be disposed. The next stop is using AcquirePointer method and I don't see where you release it. – Sinatr Commented 2 days ago
• Are you expecting the actions to happens as you write bytes? I'd rather expect them to happens after you dispose the last temporarily created object. And to test successfully written data you must read them normally, as memory mapped files. – Sinatr Commented 2 days ago
• Hi Sinatr, this is just a simplification to ilustrate the issue, the actual code does take care of the issues you mentione. – dcidral Commented 2 days ago

Add a comment  | 

2 Answers 2

Sorted by: Reset to default 4

MemoryMappedViewAccessor.SafeMemoryMappedViewHandle denotes the address of the first virtual memory page the respective part of the file for the view is mapped to. Typically, a page is 4 KB or 64 KB in size (but larger page sizes are a possibility).

Your two view accessors represent parts of the file that are within the same 4 KB region. Therefore, the virtual memory pages you are using here essentially map to the very same 4 KB (or whatever the memory page size is) region in the file, and the block1 and block2 pointers map to the very same file position. (Note how the difference between the block1 and block2 pointer values is 64 KB, which is either the page size your system is using or a multiple of 4 KB pages.)

What you overlooked is MemoryMappedViewAccessor.PointerOffset, which you would need to add to MemoryMappedViewAccessor.SafeMemoryMappedViewHandle address, e.g.:

lock1Accessor.SafeMemoryMappedViewHandle.AcquirePointer(ref block1);
block2Accessor.SafeMemoryMappedViewHandle.AcquirePointer(ref block2);

block1 += block1Accessor.PointerOffset;
block2 += block2Accessor.PointerOffset;

Or work just with a single view accessor and pointer arithmetic, as Sergei's answer demonstrates. This would be much simpler...

With Having two accessor you end up with writing into the same memory handle. if you would like to keep use of pointers - get block1 pointer from the memory handle and manually calculate pointer for block2. In that case your example gives a correct result

using System.IO.MemoryMappedFiles;

const int block1Size = 20;
const int block2Size = 10;
// create a shared memory region for the 2 blocks of data
MemoryMappedFile sharedFile = MemoryMappedFile.CreateNew("dummy_host_name", block1Size + block2Size);

// create an accessor for each memory block
// first block starts at index 0 and has 20 bytes
MemoryMappedViewAccessor block1Accessor = sharedFile.CreateViewAccessor(0, block1Size);

unsafe
{
    byte* block1 = null;
    
    // acquire pointers for each block
    block1Accessor.SafeMemoryMappedViewHandle.AcquirePointer(ref block1);
    byte* block2 = block1 + block1Size;

    Console.WriteLine($"Block1 address: {(long)block1:X}");
    Console.WriteLine($"Block2 address: {(long)block2:X}");

    // clear garbage data
    Copy(CreateZeros(block1Size), block1);
    Copy(CreateZeros(block2Size), block2);

    PrintData(block1, block2);

    // write a sequence of 1 to 20 into the first memory block
    byte[] firstDataInput = Enumerable.Range(1, block1Size).Select(i => (byte)i).ToArray();
    Copy(firstDataInput, block1);

    PrintData(block1, block2);

    // write a sequence of 21 to 30 into the second memory block
    byte[] secondDataInput = Enumerable.Range(block1Size + 1, block2Size).Select(i => (byte)i).ToArray();
    Copy(secondDataInput, block2);
    PrintData(block1, block2);
}

本文标签： pointersSplitting a MemoryMappedFile into 2 chunks of data in CStack Overflow