Currently, Pulsar connector periodically queries the admin api to detect partition changes. However, this can cause the pipeline to fail if the admin api fail.
@weixiangchen Reported a admin 502 when running pipeline for longer peirods (30 minutes).
We should discuss whether we want to
- fail the job
- restart the job
- report the exception and go on
when similiar 502 exception happens.