c# - What is the ackDeadline property on Subscription and ackDeadline on Subscriber in GCP PubSub? - Stack Overflow

admin管理员组

文章数量:1293159

I am trying to understand the difference between the ackDeadline property on the Subscription and the ackDeadline property on the Subscriber. It is clear that the ackDeadline on the Subscription is the time period after which the Subscription will resend the message to the Subscriber. The minimum value is 10s on the Subscription. I fail to understand the semantics of the ackDeadline on the Subscriber. FYI my use-case involves getting the message and putting the messages on a Kafka topic. I don't care if I don't process all the messages. I need to process the latest set of messages and I don't know if it is a stale or latest message without first parsing the messages. I don't want to increase the ackDeadline and want to ignore/discard the message if my Subscriber lags. Thanks!

I tried to research the difference between the ackDeadline on the Subscription and the Subscriber. I am still confused what should the value be on each. My subscription has the following settings:

public static async Task<SubscriberServiceApiClient> CreateSubscription(string CmeProjectId, string Topic, SubscriptionName SubscriptionId)
{
    SubscriberServiceApiClient subscriberService = await SubscriberServiceApiClient.CreateAsync();
    TopicName topicName = TopicName.FromProjectTopic(CmeProjectId, Topic);
    try
    {
        //Please read common subscription properties: 
        var subscriptionRequest = new Subscription
        {
            SubscriptionName = SubscriptionId,
            TopicAsTopicName = topicName,
            //How long are unacked messages stored. Default = 7 days, minimum = 10 minutes.
            MessageRetentionDuration = new Duration
            {
                Seconds = (long)TimeSpan.FromMinutes(10).TotalSeconds
            },
            //How long do Subscriptions survive without any activity. Default value = 31 days, minimum value = 1 day
            ExpirationPolicy = new ExpirationPolicy
            {
                Ttl = Duration.FromTimeSpan(TimeSpan.FromDays(1))
            },
            /*
            Create a deadletter queue to discard messages if message was not ackd after 5 retries.
            Do not attach a subscription to dead letter queue as we want to drop stale messages and not store them.
            MaxDelivery attempts has a minimum value of 5.
            */
            DeadLetterPolicy = new DeadLetterPolicy
            {
                MaxDeliveryAttempts = 5,
                DeadLetterTopic = TopicName.FromProjectTopic(MAREX_PROJECT_ID, "dead-letter-topic").ToString()
            },
            //Process the message exactly once.
            //Messages are not redelivered if message is acked or when message is outstanding ()
            EnableExactlyOnceDelivery = true
        };
        await subscriberService.CreateSubscriptionAsync(subscriptionRequest);
    }
    catch (RpcException e) when (e.Status.StatusCode == StatusCode.AlreadyExists)
    {
        Console.WriteLine("Subscription already exists!");
    }

    return subscriberService;
}

The Subscriber is the following:

public static async Task<SubscriberClient> CreateSubscriber(SubscriptionName subscriptionId)
{
    SubscriberClient subscriber = await new SubscriberClientBuilder
    {
        SubscriptionName = subscriptionId,
        Settings = new SubscriberClient.Settings
        {
            //Lease time before which a message must either be acknowledged or have its lease extended.
            AckDeadline = TimeSpan.FromSeconds(10),

            //Number of outstanding messages after which subscription won't send any messages.
            //Total byte count of all outstanding messages that won't fill up available memory.
            FlowControlSettings = new FlowControlSettings(maxOutstandingElementCount: 50000, maxOutstandingByteCount: 50000 * 560)
        }
    }.BuildAsync();
    return subscriber;
}

I am trying to understand the difference between the ackDeadline property on the Subscription and the ackDeadline property on the Subscriber. It is clear that the ackDeadline on the Subscription is the time period after which the Subscription will resend the message to the Subscriber. The minimum value is 10s on the Subscription. I fail to understand the semantics of the ackDeadline on the Subscriber. FYI my use-case involves getting the message and putting the messages on a Kafka topic. I don't care if I don't process all the messages. I need to process the latest set of messages and I don't know if it is a stale or latest message without first parsing the messages. I don't want to increase the ackDeadline and want to ignore/discard the message if my Subscriber lags. Thanks!

I tried to research the difference between the ackDeadline on the Subscription and the Subscriber. I am still confused what should the value be on each. My subscription has the following settings:

public static async Task<SubscriberServiceApiClient> CreateSubscription(string CmeProjectId, string Topic, SubscriptionName SubscriptionId)
{
    SubscriberServiceApiClient subscriberService = await SubscriberServiceApiClient.CreateAsync();
    TopicName topicName = TopicName.FromProjectTopic(CmeProjectId, Topic);
    try
    {
        //Please read common subscription properties: https://cloud.google/pubsub/docs/subscription-properties
        var subscriptionRequest = new Subscription
        {
            SubscriptionName = SubscriptionId,
            TopicAsTopicName = topicName,
            //How long are unacked messages stored. Default = 7 days, minimum = 10 minutes.
            MessageRetentionDuration = new Duration
            {
                Seconds = (long)TimeSpan.FromMinutes(10).TotalSeconds
            },
            //How long do Subscriptions survive without any activity. Default value = 31 days, minimum value = 1 day
            ExpirationPolicy = new ExpirationPolicy
            {
                Ttl = Duration.FromTimeSpan(TimeSpan.FromDays(1))
            },
            /*
            Create a deadletter queue to discard messages if message was not ackd after 5 retries.
            Do not attach a subscription to dead letter queue as we want to drop stale messages and not store them.
            MaxDelivery attempts has a minimum value of 5.
            */
            DeadLetterPolicy = new DeadLetterPolicy
            {
                MaxDeliveryAttempts = 5,
                DeadLetterTopic = TopicName.FromProjectTopic(MAREX_PROJECT_ID, "dead-letter-topic").ToString()
            },
            //Process the message exactly once.
            //Messages are not redelivered if message is acked or when message is outstanding (https://cloud.google/pubsub/docs/exactly-once-delivery)
            EnableExactlyOnceDelivery = true
        };
        await subscriberService.CreateSubscriptionAsync(subscriptionRequest);
    }
    catch (RpcException e) when (e.Status.StatusCode == StatusCode.AlreadyExists)
    {
        Console.WriteLine("Subscription already exists!");
    }

    return subscriberService;
}

The Subscriber is the following:

public static async Task<SubscriberClient> CreateSubscriber(SubscriptionName subscriptionId)
{
    SubscriberClient subscriber = await new SubscriberClientBuilder
    {
        SubscriptionName = subscriptionId,
        Settings = new SubscriberClient.Settings
        {
            //Lease time before which a message must either be acknowledged or have its lease extended.
            AckDeadline = TimeSpan.FromSeconds(10),

            //Number of outstanding messages after which subscription won't send any messages.
            //Total byte count of all outstanding messages that won't fill up available memory.
            FlowControlSettings = new FlowControlSettings(maxOutstandingElementCount: 50000, maxOutstandingByteCount: 50000 * 560)
        }
    }.BuildAsync();
    return subscriber;
}

• c#
• google-cloud-platform
• google-cloud-pubsub

Share Follow edited Feb 12 at 23:59 Ferry To 1,52122 gold badges3838 silver badges5555 bronze badges asked Feb 12 at 22:26 tall_talestall_tales 322 bronze badges

Add a comment  | 

2 Answers 2

Sorted by: Reset to default 0

When using the client library, the subscription-level acknowledgement deadline is not relevant. It is only used for push subscriptions or for pull subscriptions that are using unary pull. In these cases, the ack deadline represents the time after delivery that the push endpoint or unary pull subscriber has to acknowledge the message. For pull-based subscriptions, one can extend the deadline with a modifyAckDeadline RPC.

The client library uses streaming pull and manages the acknowledgement deadline of messages received by internally sending modifyAckDeadline RPCs. In C#, the behavior of these extensions is governed by three properties:

• MaxTotalAckExtension: The amount of time for which to extend the ack deadline of a message by sending modifyAckDeadline RPCs. Defaults to 60 minutes.
• AckDeadline: The amount of time for which to extend the deadline of the message on each modifyAckDeadline RPC. Defaults to 60 seconds.
• AckExtensionWindow: The amount of time before the current deadline at which to send the next modifyAckDeadline RPC. Defaults to 15 seconds.

These three properties together give you control over the lease extension behavior in the client library. Generally, the defaults work pretty well for most cases but their may be reasons you want to change the values. For example, reducing AckDeadline can reduce the time for a message to be redelivered to another subscriber if the subscriber that originally received the message crashes. You may also want to tune MaxTotalAckExtension based on how long you expect messages to take to be processed. For AckExtensionWindow, you can shrink it if you are confident in your client's ability to acknowledge a message within the initial AckDeadline in order to avoid an extra modifyAckDeadline RPC, but doing so may risk the RPC not getting to the server in time to extend the deadline further if the message has not yet been acknowledged.

There is no notion of "subscriber ackDeadline". Or it's only defined at the library level.

However, a subscriber can ask to extend the subscription ackDeadline, up to 10 minutes, if it need more time to process message, if it has CPU saturation,... Per message and in real time (no need to reconfigure the subscription for this)

本文标签：