c# - Microsoft.NET.Sdk.Worker Reading an Azure Eventhub (16 partitions) memory consume problem - Stack Overflow

admin管理员组

文章数量:1346968

I created a C# Worker service who reads an eventhub (16 partitions). Each partition has your own EventHubConsumerClient object. Each azure hub partition receive data every second... so my service working all the time.

My problem is: Worker process increase memory consume all the time... I feel like partitionEvent object isn't removed from memory by Garbage Collector... and i can't clean partitionEvent mannualy (readonly object).

I'm using Worker project, azure eventhub libraries 5.11.6

EventPosition startingPosition;

PartitionProperties properties = await _consumer.GetPartitionPropertiesAsync(partitionId, stoppingToken);
startingPosition = EventPosition.FromSequenceNumber(properties.LastEnqueuedSequenceNumber);

var configHub = new ReadEventOptions
{
    PrefetchCount = 500,
    MaximumWaitTime = TimeSpan.FromMilliseconds(100)
};

int eventCounter = 0;

await foreach (PartitionEvent partitionEvent in _consumer.ReadEventsFromPartitionAsync
(partitionId, startingPosition, configHub, stoppingToken).ConfigureAwait(false))
{
    try
    {
        if (partitionEvent.Data != null)
        {
            ReadOnlyMemory<byte> body = partitionEvent.Data.Body;

            IEnumerable<LPEventHub> listaLPsHub = 
                JsonConvert.DeserializeObject<IEnumerable<LPEventHub>>
                (Encoding.UTF8.GetString(body.Span));

            if (listaLPsHub.Count() > 0)
            {
                eventCounter++;
            }

            if (eventCounter % 20 == 0)
            {
                GC.Collect();
                GC.WaitForPendingFinalizers();
            }
        }
    }
    catch (TaskCanceledException er)
    {
        
    }
    catch (Exception ex)
    {
     
    }
    finally
    {

    }
}

本文标签： cMicrosoftNETSdkWorker Reading an Azure Eventhub (16 partitions) memory consume problemStack Overflow