PubSub Subscription 'ackDeadline' and/or 'ackDeadlineSeconds'

[lc-chrisbarton]

I am receiving "Error: .google.pubsub.v1.Subscription#ackDeadline is not a field: undefined" when using ackDeadline but switching the options to 'ackDeadlineSeconds' stops the error and reports the correct value in the GCP web console but seems to set the deadline to a very small value because it does not seem to have any effect i.e. I can make repeated calls and receive the same messages.

      let topic = client.pubsub.topic(topic_name)
      let opts = {
        ackDeadline: 60000,
        flowControl: { maxMessages: 5 },
      }
      topic.createSubscription('foo', opts)

but switching the options to the following works:

      let opts = {
        ackDeadlineSeconds: 60,
        flowControl: { maxMessages: 5 },
      }

[callmehiphop]

Oh no! Sorry about that, that's some old documentation that was meant to be removed. Instead of redelivering messages, we modify the ack deadline for you, so that option shouldn't be used.

[lc-chrisbarton]

But the field 'ackDeadline' is also used in the code and hard coded to "this.ackDeadline = 10000;":

https://github.com/GoogleCloudPlatform/google-cloud-node/blob/pubsub-0.14.0/packages/pubsub/src/subscription.js#L446

[lc-chrisbarton]

When you say "Instead of redelivering messages, we modify the ack deadline for you". Are you saying that I don't need to set a deadline on the subscription and if I do (e.g. using the command-line) it won't have an effect because you reset it to a small value when you first connect and then extend it after that? Well I guess you don't reset it to a lower value if the subscription already exists?

[callmehiphop]

That is correct

[lukesneeringer]

@ChrisBartonLC: Some more detail on this. The TL;DR version of it is "You can never have to think about ack deadlines and we will do the right thing."

Basically how it works under the hood is: when the client library receives a message, it automatically sets the ack deadline to 10 seconds (as you observed), and we have a process that runs on a cadence that continues to extend it until you ack or nack. If something bad happens and, say, your application crashes, then obviously we stop extending (because we crashed) and you get the messages again.

It is not quite as simple as firing off a request every 10 seconds. That is the starting value. We have some logic where we time how long it takes you to ack all of your past messages and make the deadline appropriate to that. So, if you ack a all of your messages after a minute, then we will start extending the deadline for a minute. (We set the deadline to the 99th percentile of ack times since you opened the subscription -- and we keep extending it, so in theory you should never get a redelivery based on a client library failure so long as the client does not crash or something similar.) That way if you have longer-running jobs, you will not be likely to get messages redelivered because of (for example) network latency, but if you have short messages, we minimize how long you wait to get the message again if something goes wrong.

If for whatever reason this is a problem for your use case, then we would love to hear details. Assuming you have a common use case, however, you should be able to just never think about having to do any of that work yourself, and only worry about ack and nack.

[lukesneeringer]

I am also going to close this under the assumption that the issue is resolved, but feel free to reopen (or open a new issue) if my assessment is incorrect for whatever reason. :-)

[lc-chrisbarton]

It seems to work fine but FYI your comment "So, if you ack a all of your messages after a minute" in a Cloud Function this code runs for seconds not minutes. I've seen other "google-cloud-node" clients e.g the Spanner one have issues when running in Cloud Functions because of assumptions about usage which don't apply e.g. a lot of the caching logic just gets in the way.

[lc-chrisbarton]

@callmehiphop Thinking about this some more...What would happen if I have 10 messages that couldn't be parsed and for which the ACK deadline would always pass because they caused an exception. Would these messages always be available for consumption after 10 seconds? Would these have the potential to block all message processing if say I only had one worker and that worker was set to process a maximum of 10 messages? I mention this because I was guarding against this by setting the ACK deadline to some time >> larger that the rate at which the messages were being processed.

[callmehiphop]

@ChrisBartonLC In the scenario you described I would suggest calling nack() on any message you did not want to hang around.

[lc-chrisbarton]

Thanks