subscription.on message wasn't received of messagActually unrelated failures, but we should bump this anyway.es after 10 mins or more.

[stephenplusplus]

From @QTGate on September 7, 2017 17:42

• OS: Debian
• Node.js version: 8.4.0
• npm version: 5.4.0
• google-cloud-node version: 0.56.0
• @google-cloud/pubsub version: 0.14.0

Steps to reproduce

const pubsub = require ( '@google-cloud/pubsub' )( connectOption )

const subscription = pubsub.topic ( topicName ).subscription ( subScription )

subscription.on ('message', message => {
    message.ack()
    ....
})

subscription.once('error', err => {
    ....
})

I received all un-received messages when I restart it.
I checked the connectionPool object when it have not received message:
connectionPool.isPaused [false]
connectionPool.isOpen [true]
connectionPool.connections [5]

Thank you.

Copied from original issue: googleapis/google-cloud-node#2598

[stephenplusplus]

From @callmehiphop on September 8, 2017 18:19

Thanks for reporting! We've had multiple reports of similar behavior and we're currently investigating this issue. In the meantime you might want to use pubsub v0.13.x as a temporary workaround.

[stephenplusplus]

From @QTGate on September 8, 2017 23:32

Thank you.
I did a temp code to use the pubsub v0.14.0 like this:

const pubsub = require ( '@google-cloud/pubsub' )( connectOption )

const listenSub = () => {
    let subscription = pubsub.topic ( topicName ).subscription ( subScription )
    subscription.on ( 'message', message => {
        message.ack()
        ....
   })

    subscription.once('error', err => {
        ....
    })

    //        each 10 mins restart
    const tt = setTimeout (() => {
	return subscription.close().then(() => {
	    listenPubSubMaster = null
	    return listenSub ()
    }, 600000 )
}

listenSub()

[stephenplusplus]

From @caseyduquettesc on September 18, 2017 3:20

We see a similar behavior, although it takes a few hours. Eventually the subscription stops processing new messages and we have to restart the subscription. This only started after upgrading to 0.14.

[stephenplusplus]

From @mscottx88 on September 22, 2017 18:52

I too am seeing the same problem. We set up a forever running process on GKE and it seems to go unresponsive after some time. I came to the same conclusion of putting a restarter.

Let's hope this gets resolved soon!

[stephenplusplus]

From @callmehiphop on September 23, 2017 0:4

This is definitely a result of moving towards the new streaming API, I'm working on resolving this issue, but in the mean time I suggest downgrading to 0.13. I'll update this issue once it's been resolved.

[stephenplusplus]

From @pongad on September 23, 2017 10:44

@callmehiphop FYI, we resolved this in Go by adding gRPC keepalive. I'm not sure how the code should look like in Node though. https://code-review.googlesource.com/#/c/gocloud/+/17070/

[stephenplusplus]

From @callmehiphop on September 25, 2017 14:10

@pongad thanks for the update! I believe we just have to set the grpc.keepalive_time_ms option.

[stephenplusplus]

From @callmehiphop on September 25, 2017 14:17

@QTGate @mscottx88 @caseyduquettesc I've opened a PR (#2627) which I believe will fix the issue you are seeing, would any of you mind testing it out? Thanks!

[stephenplusplus]

From @mscottx88 on September 25, 2017 14:18

I'm willing to give it a go, for sure!

On Mon, Sep 25, 2017, 7:17 AM Dave Gramlich notifications@github.com
wrote:

@QTGate https://github.com/qtgate @mscottx88
https://github.com/mscottx88 @caseyduquettesc
https://github.com/caseyduquettesc I've opened a PR (#2627
googleapis/google-cloud-node#2627)
which I believe will fix the issue you are seeing, would any of you mind
testing it out? Thanks!

—
You are receiving this because you were mentioned.

Reply to this email directly, view it on GitHub
googleapis/google-cloud-node#2598 (comment),
or mute the thread
https://github.com/notifications/unsubscribe-auth/AT6MMUjOHHVuq1ZzzlIViODrOF50HIecks5sl7X8gaJpZM4PQKSI
.

[stephenplusplus]

From @mscottx88 on September 25, 2017 22:46

@callmehiphop I applied the two fixed files to my deployment but now I am seeing duplication of message delivery. That is, I am seeing the same message being pulled more than one time.

update Also, it appears that the connection has been lost or something as it is sitting idle again even with messages being published to it.

[stephenplusplus]

From @callmehiphop on September 26, 2017 17:30

@jganetsk I'm noticing that messages that are not acked before a stream is closed will be redelivered when the next stream is opened regardless of whether or not the ack deadline was hit. Is that the expected behavior? If so I'm assuming we need to filter these messages out.

[stephenplusplus]

From @jganetsk on September 26, 2017 17:45

@callmehiphop I don't think you should be filtering any messages out. In general, it is possible to receive duplicates, and it is possible that they come before the ack deadline was hit. If this happens consistently, that's a problem that we should look at. How much time passes between delivery and redelivery? Does it correspond to the default ack deadline?