Gossamer Miss Behavior `Reason: Request refused` and `Reason: Request timeout`

[EclesioMeloJunior]

Issue summary

Substrate nodes are punishing Gossamer since gossamer refuses a substrate block request

2022-09-05 10:25:12.026 DEBUG tokio-runtime-worker sync: Request to peer PeerId("12D3KooWGc4ohzsjYwBQnZdV1CpUNv46JpgCUfL5iGLqW2KfWxm2") failed: Refused.    
2022-09-05 10:25:12.026 TRACE tokio-runtime-worker sub-libp2p: External API => Disconnect(12D3KooWGc4ohzsjYwBQnZdV1CpUNv46JpgCUfL5iGLqW2KfWxm2, SetId(0))    
2022-09-05 10:25:12.026 TRACE tokio-runtime-worker sub-libp2p: PSM <= Dropped(12D3KooWGc4ohzsjYwBQnZdV1CpUNv46JpgCUfL5iGLqW2KfWxm2, SetId(0))    
2022-09-05 10:25:12.026 TRACE tokio-runtime-worker peerset: Dropping 12D3KooWGc4ohzsjYwBQnZdV1CpUNv46JpgCUfL5iGLqW2KfWxm2: -256 to 68    
2022-09-05 10:25:12.026 TRACE tokio-runtime-worker sub-libp2p: External API <= Closed(12D3KooWGc4ohzsjYwBQnZdV1CpUNv46JpgCUfL5iGLqW2KfWxm2, SetId(0))    
2022-09-05 10:25:12.026 TRACE tokio-runtime-worker sub-libp2p: Handler(PeerId("12D3KooWGc4ohzsjYwBQnZdV1CpUNv46JpgCUfL5iGLqW2KfWxm2"), ConnectionId(2)) <= Close(SetId(0))    
2022-09-05 10:25:12.026 DEBUG tokio-runtime-worker sync: 12D3KooWGc4ohzsjYwBQnZdV1CpUNv46JpgCUfL5iGLqW2KfWxm2 disconnected    
2022-09-05 10:25:12.026 TRACE tokio-runtime-worker sync: 0 blocks ready for import    
2022-09-05 10:25:12.027 TRACE tokio-runtime-worker sync: Block imported successfully Some(6) (0x3d60…24ee)    
2022-09-05 10:25:12.029  INFO tokio-runtime-worker substrate: ✨ Imported #6 (0x3d60…24ee)    
2022-09-05 10:25:12.029 TRACE tokio-runtime-worker afg: Polling round 1, state = State { prevote_ghost: Some((0x2d42e1c0b9e689cdf2b51a09621be564171bdde521fdb169fe538f69c0adb97a, 0)), finalized: None, estimate: Some((0x2d42e1c0b9e689cdf2b51a09621be564171bdde521fdb169fe538f69c0adb97a, 0)), completable: false }, step = Some(Precommitted)    
2022-09-05 10:25:12.029 TRACE tokio-runtime-worker sub-libp2p: PSM => Connect(12D3KooWGc4ohzsjYwBQnZdV1CpUNv46JpgCUfL5iGLqW2KfWxm2, SetId(0)): No connection in proper state. Delaying.    
2022-09-05 10:25:12.029 TRACE tokio-runtime-worker peerset: Report 12D3KooWGc4ohzsjYwBQnZdV1CpUNv46JpgCUfL5iGLqW2KfWxm2: -1024 to -956. Reason: Request refused

Other information and links

• Full log here: https://gist.github.com/EclesioMeloJunior/1974a0188c14d97433154fedac48803e

[EclesioMeloJunior]

Additional information: it is possible to see in the Gossamer logs the following error, which might be related to the Substrate punishment

2022-09-05T14:42:52-04:00 DEBUG    cannot create response for request: failed to get start block 0x9126d6f4fb6f58aaa164b9d01d8c392f0788fe2fb1c9e118dabed425db1ff127 for request: Key not found	sync.go:L111	pkg=network